dev: dockerless qemu workflow

[trevarj]

I've been trying to get qemu to work without docker, but been failing. I'd like to be able to run qemu + gdb to step through the code, since on-board gdb is a huge pain.

I took a look at some of the qemu/ scripts and tried to steal some pieces from them so that I can eventually get to a workflow that allows me to do:

idf.py qemu --flash-file build/flash_image.bin -g --efuse-file main/qemu/qemu_efuse.bin monitor

...but the screen that pops up is just black.

I had built based on configs/sdkconfig_qemu.defaults and created a flash_image.bin as in make-flash-img.bin.

Any help on what I'm missing would be great. Thanks!

[jgriffiths]

Hi @trevarj

@JamieDriver Might be able to help with the qemu side.

We are working on a local/native build variant that you can run under linux directly as a native executable, which allows direct debugging with gdb (among other niceties like being able to run under valgrind/asan etc). Assuming you want to debug the firmware code/logic rather than a specific device you may find that useful.

I will see if I can expedite getting that branch made public (its very beta/WIP at the moment).

[trevarj]

Hi @jgriffiths

We are working on a local/native build variant that you can run under linux directly as a native executable

Oh, this sounds great!

Assuming you want to debug the firmware code/logic rather than a specific device you may find that useful.

Yes, exactly. Just want to debug the generic code and learn my way around a bit.

Thanks a lot!

[JamieDriver]

Yeah, I'm not sure how well idf.py qemu works tbh.

We've only recently moved away from building qemu from source to using the one provided by esp-idf - but as @jgriffiths says we might even move away from that once the native linux build is ready.

Try building the image as in main/qemu/make-flash-img.sh and then running it as in main/qemu/qemu_run.sh - they are setup for the ci but you should be able to extract the bits you need. (I assume you have already done: idf_tools.py install qemu-xtensa)

fyi I have recently had to raise a PR on HWI to use the esp-idf provided qemu - maybe consider some of these steps :
https://github.com/JamieDriver/HWI/blob/a5f9672b47c913de43dc83b5fb188ca7a84c2146/test/setup_environment.sh#L354

The (unchanged) code to run the emulator is here:
https://github.com/JamieDriver/HWI/blob/a5f9672b47c913de43dc83b5fb188ca7a84c2146/test/test_jade.py#L63
(Which should be more-or-less the same command-line as in the main/qemu/ scripts ...

lmk how it goes.

[trevarj]

Nice, this worked. Even can do gdb debugging.
Thanks again!

My steps

#!/bin/sh
# Must be at toplevel of jade/
pkill xtensa-esp-elf-gdb
pkill qemu-system-xtensa
# copy qemu config -- modified to enable logging
cp configs/sdkconfig_qemu_psram_webdisplay_larger_dbg.defaults sdkconfig.defaults

# build
idf.py build

# make flash image
esptool.py --chip esp32 merge_bin --fill-flash-size 4MB -o build/flash_image.bin \
  --flash_mode dio --flash_freq 40m --flash_size 4MB \
  0x9000 build/partition_table/partition-table.bin \
  0xe000 build/ota_data_initial.bin \
  0x1000 build/bootloader/bootloader.bin \
  0x10000 build/jade.bin

# copy efuse
cp main/qemu/qemu_efuse.bin build/qemu_efuse.bin

# set permission
chmod 777 build/qemu_efuse.bin build/flash_image.bin

# monitoring
# idf.py qemu \
#   --flash-file build/flash_image.bin \
#   --efuse-file main/qemu/qemu_efuse.bin \
#   --qemu-extra-args "-nic user,model=open_eth,id=lo0,hostfwd=tcp:0.0.0.0:30122-:30122,hostfwd=tcp:0.0.0.0:30121-:30121" \
#   monitor

# gdb debugging
idf.py qemu \
  --flash-file build/flash_image.bin \
  --efuse-file main/qemu/qemu_efuse.bin \
  --qemu-extra-args "-nic user,model=open_eth,id=lo0,hostfwd=tcp:0.0.0.0:30122-:30122,hostfwd=tcp:0.0.0.0:30121-:30121" \
  --gdb monitor

# open web display
# xdg-open http://localhost:30122 &

# in another terminal...
# idf.py gdb
# (gdb) b

[JamieDriver]

Nice one!

Would you object to my adding the above (with attribution, ofc!) to the wiki ?

[JamieDriver]

fyi there is an additional small code change you can make so the jade logging appears in the qemu window rather than being sent back over the tcp/serial connection. I'll post that as well ...

[trevarj]

Would you object to my adding the above (with attribution, ofc!) to the wiki ?

@JamieDriver no objections, happy to help.