Merge latest QEMU version

[beckus]

I have had a couple discussions regarding merging the latest version of QEMU. I decided to open an issue to facilitate collaboration and avoid duplication of effort.

The last time I tried to merge the latest version of QEMU, it broke the unit tests (I don't recall if there were any other issues). While my original philosophy was to keep the unit tests working, I think it may be time to abandon them and let them break, and instead give priority to finishing the merge.

There was a pull request #16 opened for merging QEMU 2.7 - I did not finish looking at it, but it appeared that it was just a brute force merge that broke QEMU STM32 - i.e. the merge overwrote critical pieces, and the QEMU STM32 machine still needed to be re-incorporated. So, I never finished addressing this pull request. At this point, it is probably better to start a new merge process and close the pull request.

Incidentally, one thing I had hoped to do in tandem with the merge would be to update QEMU STM32 to use the QOM framework: the way I originally built the machine is kind of a hack, and using QOM seems to make things more elegant and scalable, and would also allow the peripherals to be viewed from within the device tree. But, the level of effort of doing this is probably too high (and would introduce risk of breaking things). But, this may be a worthwhile investment for anyone using QEMU for serious purposes, e.g. commercially.

[ryanpatiency]

This is a great project, thank you.

But today when I tried to build it on my ubuntu 18, it complained about g_log...., do you know where I should go first to fix this issue?

[beckus]

Hello, I am glad that you find the project useful.

No, I am not sure offhand where the issue is. What exactly is the error message? Is this Ubuntu 18.04 or 18.10?

[ryanpatiency]

it is Ubuntu 18.04.02 on Wayland

block/vhdx.c:102:21: error: ‘parent_vhdx_guid’ defined but not used [-Werror=unused-const-variable=]
 static const MSGUID parent_vhdx_guid = { .data1 = 0xb04aefb7,
                     ^~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
/tmp/qemu_stm32/rules.mak:31: recipe for target 'block/vhdx.o' failed
make: *** [block/vhdx.o] Error 1

[beckus]

Hello,
I'm not sure exactly what the issue is, but it seems that this is a warning that is being treated as an error. One thing you could try is to tell the compiler to ignore warnings. I don't remember exactly how to do this offhand, but there is a flag that can be set... This does not fix the underlying issue, but maybe will get it to compile.

Sorry for the delay in responding - I've had a lot of deadlines lately. I can try to look at it to see what the error is, but it may be a while before I get to it...

Thank you, Andre

[ryanpatiency]

You are welcome.

…

On Sun, Mar 17, 2019 at 3:03 PM Andre Beckus ***@***.***> wrote: Hello, I'm not sure exactly what the issue is, but it seems that this is a warning that is being treated as an error. One thing you could try is to tell the compiler to ignore warnings. I don't remember exactly how to do this offhand, but there is a flag that can be set... This does not fix the underlying issue, but maybe will get it to compile. Sorry for the delay in responding - I've had a lot of deadlines lately. I can try to look at it to see what the error is, but it may be a while before I get to it... Thank you, Andre — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALFyik3IzKx69oheahQLDgkxw3QMGbCFks5vXejagaJpZM4VcOuH> .

[beckus]

I installed Ubuntu 18.04, and received this error as well. I ran the configure command with the --disable-werror command, and it compiles (though I am not sure if there are any issues with running QEMU). So, to be specific, I ran these commands to compile:
./configure --enable-debug --target-list="arm-softmmu" --disable-werror
make

I see someone reported the error in 2016 here:
https://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/%5Bqemu%5D-error-%27g_mem_set_vtable%27-is-deprecated-4175586499/

I imagine this is an issue that was fixed in QEMU, but we are seeing it because QEMU_STM32 is based on an old version of QEMU - at some point I will try and resolve this by incorporating the latest QEMU version...

[ryanpatiency]

Thank you very much Andre Beckus <[email protected]> 於 2019年3月24日 週日 上午7:16寫道：

…

I installed Ubuntu 18.04, and received this error as well. I ran the configure command with the --disable-werror command, and it compiles (though I am not sure if there are any issues with running QEMU). So, to be specific, I ran these commands to compile: ./configure --enable-debug --target-list="arm-softmmu" --disable-werror make I see someone reported the error in 2016 here: https://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/%5Bqemu%5D-error-%27g_mem_set_vtable%27-is-deprecated-4175586499/ I imagine this is an issue that was fixed in QEMU, but we are seeing it because QEMU_STM32 is based on an old version of QEMU - at some point I will try and resolve this by incorporating the latest QEMU version... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALFyimastbG6SRMLSEQdk5EzA5X8LIjhks5vZrXCgaJpZM4VcOuH> .

[sarim]

I just found this project. Is there any plan to push it to the main qemu project?

[beckus]

No, it would be nice, but I don't have any plans to push it to the main qemu project.

Some reasons for not doing so:

1. This project is still in a state of flux, and I would like the flexibility to change it without going through a patch process.
2. I suspect it will take some time and effort to go through the submission process - time I would rather put into the project itself. I have also made changes to the core framework, which may not be acceptable in the main QEMU branch or which will need to be modified. I don’t see much disadvantage to leaving this as a fork. That being said, there are core components that could benefit other parts of QEMU (i.e. The clock tree objects and unit testing framework) that should be submitted.

[fariouche]

Hello. I started to port to the latest semi. Lot of work and don't know much about semi.
The reason was because of a crash in pendsv when using zephyr blinky sample app.... And hopping that the latest qemu will solve it

[fariouche]

Finished the initial port, it compiles but does not run yet. It's not easy as many things have changed. Qom is more and more used, so had to add more quem objects. The other difficult part is the clock. I'm not familiar with it, ans they have now a clock object. Maybe the best will be to rewrite RCC and clock tree to use the new clock system

[beckus]

Hello @fariouche, thank you for the update, and thank you for working on this. I unfortunately have not updated the QEMU version in a long time, and it only becomes more and more difficult.

I am not surprised that QOM is being used more - I liked the idea and would have started using it if I had more time. I am not familiar with the new clock object, but it does seem like rewriting RCC to use its functionality could make it more reliable.

I am sorry that I am not able to be of more assistance, but I wish you the best in the update.

[fariouche]

No problem @beckus, your work is already a lot, and well, this is open source and free. I will help because I want it ;)
So far, the clocktree is crashing. I'm not familiar with the RCC of the stm32, but I may have to dig inside the datasheet to understand it. But for now, I would like to avoid having to rewrite clocktree as it seems to be a pretty big code to redo and I do not understand it yet.
The biggest change here is that they are not using the 2 externals variables for the clocks, but instead they are using a proper sysclk object. And the init of the armv7m changed too (no more "translated" function equivalent when loading the kernel). The new init function takes a sysclk object too. I hope it will not be too difficult, but yes, this is taking a lot of time (way more than expected lol)

[fariouche]

ok, so new update: I have it working now. Good news, the zephyr blinky sample is now working (the original issue I wanted to solve). I still have one issue with clock PLL flag not being set in one of my examples using STM32 HAL (but simple baremetal blinky sample app is working). I will investigate that next week, but I'm glad to see good progress. :)

[beckus]

Thank you again for the updates. It sounds like the ARMv7M code has been cleaned up in the main QEMU branch - I think that is good, though I realize it causes some difficulty.
Glad to hear that you are making progress and that you were able to get it working with one example. I understand how these things take longer than expected. Best of luck in fixing the remaining issues.

[fariouche]

Hello! So now everything is working fine!
I've created a fork and push all my modifications. Still hadn't launched the qemu tests to see, but I've tested uart,gpio,adc,rcc, timers,flash boot + reset.
I'm not sure that my fork will be merge-able easily from a pull request as the initial branch changed too much compared to qemu, and too many conflicts... so i had to merge only stm32 files and discarded any other commits on qemu (they may or not be relevant, so we'll see)

[beckus]

Hello, thank you for the update, and sorry for the delay.
Very glad to hear. I know I had a lot of trouble with these merges as well, which is one reason I fell behind.
Unfortunately, I believe the unit tests that I wrote for QEMU STM32 will not work properly with newer versions of QEMU. So I believe future tests will need to be all manual, or new unit tests will need to be written.

Thanks for your efforts, and best of luck!

• Andre

[jmfriedt]

I am very concerned with the project since it relies on Python2 which is now being phased out from recent GNU/Linux distributions, meaning it can no longer be compiled on a fresh installation. I was struggling to merge the build scripts of qemu tag 4.2.0 when the transition from Python2 to Python3 seemed to occur -- and failing -- when I started reading the issue list and finding this https://github.com/fariouche/qemu_stm32
It indeed compiles, but similar to executing on the stm32vldiscovery machine of the official qemu a program I have checked to run on the real hardware (linking with libopencm3 for the initialization of the processor), I seem unable to run on neither the F100 nor this F103 emulation. I get stuck with

$ gdb-multiarch test.elf
target remote :1234
load
continue

seems to hang, and there

bt
#0  0x08029f68 in ?? ()
#1  <signal handler called>

or

list
62      void __attribute__ ((weak)) reset_handler(void)
63      {
64              volatile unsigned *src, *dest;
65              funcp_t *fp;
66
67              for (src = &_data_loadaddr, dest = &_data;
68                      dest < &_edata;
69                      src++, dest++) {
70                      *dest = *src;
71              }

which does not seem too good in getting the initialization completed. I am using the examples at https://github.com/jmfriedt/stm32 with the Makefile.stm32f1 configuration.
Any pointer to functional examples to run on the emulator would be welcome, ideally linking with libopencm3 since the ADC emulation was already found to have an error with respect to the test libopencm3 was assessing to check for conversion completion, so I wonder if the initialization sequence could help identify some unexpected clock behaviour.
Thanks.

[beckus]

Hello @jmfriedt,
Thank you for raising this issue and providing details of your troubleshooting. I agree the dependence on Python2 is a big concern.

I never used any functional examples beyond the stm32 p103 demos that I developed, which are not comprehensive and do not use libopencm3.

Unfortunately, due to other commitments, I do not expect to be able to work on this for the foreseeable future. I will add a link to this issue on the main page for general awareness.

Thank you again for your efforts. I'm sorry I'm not of more help.