gdb breakpoint "cannot access memory"

[ChoiHanJin]

Hello,

I am currently trying to apply this script with the following materials.

• Linux: v5.17.1
• QEMU: https://gitlab.com/bwidawsk/qemu.git - branch cxl-2.0v4
• mkosi: https://github.com/systemd/mkosi - version 12

My host OS is Fedora 35, and this is my execution command
run_qemu.sh --cxl --cxl-legacy --debug --git-qemu --cxl-debug --gdb

Then it starts QEMU.

/home/hanjin/git/qemu/build/qemu-system-x86_64 -machine q35,accel=kvm,nvdimm=on,cxl=on -m 8192M,slots=4,maxmem=40964M -smp 8,sockets=2,cores=2,threads=2 -display none -nographic -drive if=pflash,format=raw,unit=0,file=OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=OVMF_VARS.fd -debugcon file:uefi_debug.log -global isa-debugcon.iobase=0x402 -drive file=root.img,format=raw,media=disk -kernel ./mkosi.extra/boot/vmlinuz-5.17.1 -initrd mkosi.extra/boot/initramfs-5.17.1.img -append 'selinux=0 audit=0 console=tty0 console=ttyS0 root=/dev/sda2 ignore_loglevel rw cxl_acpi.dyndbg=+fplm cxl_pci.dyndbg=+fplm cxl_core.dyndbg=+fplm cxl_mem.dyndbg=+fplm cxl_port.dyndbg=+fplm cxl_region.dyndbg=+fplm cxl_test.dyndbg=+fplm cxl_mock.dyndbg=+fplm' -device e1000,netdev=net0,mac=52:54:00:12:34:56 -netdev user,id=net0,hostfwd=tcp::10022-:22 -object memory-backend-file,id=cxl-mem1,share=on,mem-path=cxl-window1,size=512M -object memory-backend-file,id=cxl-label1,share=on,mem-path=cxl-label1,size=1K -object memory-backend-file,id=cxl-label2,share=on,mem-path=cxl-label2,size=1K -device 'pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,window-base[0]=0x4c00000000,memdev[0]=cxl-mem1' -device cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0,port=0 -device cxl-rp,id=rp1,bus=cxl.0,addr=1.0,chassis=0,slot=1,port=1 -device cxl-type3,bus=rp0,memdev=cxl-mem1,id=cxl-pmem0,size=256M,lsa=cxl-label1 -device cxl-type3,bus=rp1,memdev=cxl-mem1,id=cxl-pmem1,size=256M,lsa=cxl-label2 -snapshot -gdb tcp::10000 -S -object memory-backend-ram,id=mem0,size=2048M -numa node,nodeid=0,memdev=mem0, -numa cpu,node-id=0,socket-id=0 -object memory-backend-ram,id=mem1,size=2048M -numa node,nodeid=1,memdev=mem1, -numa cpu,node-id=1,socket-id=1 -object memory-backend-ram,id=mem2,size=2048M -numa node,nodeid=2,memdev=mem2, -object memory-backend-ram,id=mem3,size=2048M -numa node,nodeid=3,memdev=mem3, -numa node,nodeid=4, -object memory-backend-file,id=nvmem0,share=on,mem-path=nvdimm-0,size=16384M,align=1G -device nvdimm,memdev=nvmem0,id=nv0,label-size=2M,node=4 -numa node,nodeid=5, -object memory-backend-file,id=nvmem1,share=on,mem-path=nvdimm-1,size=16384M,align=1G -device nvdimm,memdev=nvmem1,id=nv1,label-size=2M,node=5 -numa dist,src=0,dst=0,val=10 -numa dist,src=0,dst=1,val=21 -numa dist,src=0,dst=2,val=12 -numa dist,src=0,dst=3,val=21 -numa dist,src=0,dst=4,val=17 -numa dist,src=0,dst=5,val=28 -numa dist,src=1,dst=1,val=10 -numa dist,src=1,dst=2,val=21 -numa dist,src=1,dst=3,val=12 -numa dist,src=1,dst=4,val=28 -numa dist,src=1,dst=5,val=17 -numa dist,src=2,dst=2,val=10 -numa dist,src=2,dst=3,val=21 -numa dist,src=2,dst=4,val=28 -numa dist,src=2,dst=5,val=28 -numa dist,src=3,dst=3,val=10 -numa dist,src=3,dst=4,val=28 -numa dist,src=3,dst=5,val=28 -numa dist,src=4,dst=4,val=10 -numa dist,src=4,dst=5,val=28 -numa dist,src=5,dst=5,val=10

And then when I start gdb and set break point an error occurs.

$ gdb vmlinux
(gdb) target remote :10000
(gdb) b start_kernel
Breakpoint 1 at 0xffffffff830cae77: file init/main.c, line 928.
(gdb) c
Continuing.
Warning:
Cannot insert breakpoint 1.
Cannot access memory at address 0xffffffff830cae77

Command aborted.

If I turn off the KVM option of QEMU, then gdb can set breakpoint. However QEMU does not work with this warning.

qemu-system-x86_64: warning: This family of AMD CPU doesn't support hyperthreading(2)
Please configure -smp options properly or try enabling topoext feature.

How can I solve this problem?

Thank you for your time.

[sscargal]

What CPU do you have? (lscpu)

If your CPU supports Hyperthreading and Virtualization, you may need to enable it in the BIOS. AMD-V, short for AMD Virtualization, is the virtualization extension provided by AMD CPUs. In some BIOS versions, AMD-V is also called AMD SVM. If this technology is disabled, VM won't open.

If your CPU doesn't support Hyperthreading and Virtualization, then you'll need to change this option -smp 8,sockets=2,cores=2,threads=2 to use a single-socket, cores, and threads.

[ChoiHanJin]

Thank you for the answer.

This is my CPU info.

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         39 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  6
  On-line CPU(s) list:   0-5
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Core(TM) i5-9600K CPU @ 3.70GHz
    CPU family:          6
    Model:               158
    Thread(s) per core:  1
    Core(s) per socket:  6
    Socket(s):           1
    Stepping:            12
    CPU max MHz:         4600.0000
    CPU min MHz:         800.0000
    BogoMIPS:            7399.70
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clf
                         lush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm c
                         onstant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc c
                         puid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 s
                         dbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_
                         timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpc
                         id_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad
                          fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap c
                         lflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts h
                         wp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d arch_capabilities
Virtualization features:
  Virtualization:        VT-x
Caches (sum of all):
  L1d:                   192 KiB (6 instances)
  L1i:                   192 KiB (6 instances)
  L2:                    1.5 MiB (6 instances)
  L3:                    9 MiB (1 instance)
NUMA:
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-5
Vulnerabilities:
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Not affected
  Mds:                   Mitigation; Clear CPU buffers; SMT disabled
  Meltdown:              Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP disabled, RSB fill
                         ing
  Srbds:                 Mitigation; Microcode
  Tsx async abort:       Mitigation; TSX disabled

I set the --preset option to tiny and changed -smp option. However when I turned off the KVM option, QEMU still does not work.

This is my QEMU starting command.

/home/hanjin/git/qemu/build/qemu-system-x86_64 -machine q35,nvdimm=on,cxl=on -m 4096M,slots=2,maxmem=20482M -smp 1,sockets=1,cores=1,threads=1 -display none -nographic -drive if=pflash,format=raw,unit=0,file=OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=OVMF_VARS.fd -debugcon file:uefi_debug.log -global isa-debugcon.iobase=0x402 -drive file=root.img,format=raw,media=disk -kernel ./mkosi.extra/boot/vmlinuz-5.17.1 -initrd mkosi.extra/boot/initramfs-5.17.1.img -append 'selinux=0 audit=0 console=tty0 console=ttyS0 root=/dev/sda2 ignore_loglevel rw cxl_acpi.dyndbg=+fplm cxl_pci.dyndbg=+fplm cxl_core.dyndbg=+fplm cxl_mem.dyndbg=+fplm cxl_port.dyndbg=+fplm cxl_region.dyndbg=+fplm cxl_test.dyndbg=+fplm cxl_mock.dyndbg=+fplm' -device e1000,netdev=net0,mac=52:54:00:12:34:56 -netdev user,id=net0,hostfwd=tcp::10022-:22 -object memory-backend-file,id=cxl-mem1,share=on,mem-path=cxl-window1,size=512M -object memory-backend-file,id=cxl-label1,share=on,mem-path=cxl-label1,size=1K -object memory-backend-file,id=cxl-label2,share=on,mem-path=cxl-label2,size=1K -device 'pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,window-base[0]=0x4c00000000,memdev[0]=cxl-mem1' -device cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0,port=0 -device cxl-rp,id=rp1,bus=cxl.0,addr=1.0,chassis=0,slot=1,port=1 -device cxl-type3,bus=rp0,memdev=cxl-mem1,id=cxl-pmem0,size=256M,lsa=cxl-label1 -device cxl-type3,bus=rp1,memdev=cxl-mem1,id=cxl-pmem1,size=256M,lsa=cxl-label2 -snapshot -object memory-backend-ram,id=mem0,size=2048M -numa node,nodeid=0,memdev=mem0, -numa cpu,node-id=0,socket-id=0 -object memory-backend-ram,id=mem1,size=2048M -numa node,nodeid=1,memdev=mem1, -numa node,nodeid=2, -object memory-backend-file,id=nvmem0,share=on,mem-path=nvdimm-0,size=16384M,align=1G -device nvdimm,memdev=nvmem0,id=nv0,label-size=2M,node=2 -numa dist,src=0,dst=0,val=10 -numa dist,src=0,dst=1,val=12 -numa dist,src=0,dst=2,val=17 -numa dist,src=1,dst=1,val=10 -numa dist,src=1,dst=2,val=28 -numa dist,src=2,dst=2,val=10

[marc-hb]

Very old report, please re-open if you can still reproduce.

[marc-hb]

However when I turned off the KVM option, QEMU still does not work.

How does QEMU fail for you then?

In my case, disabling KVM avoids the "Cannot access memory" error and gdb seems to work fine. Obviously, the system is much slower. QEMU 9.2.2

This page does not mention any incompatibility between KVM and gdb:
https://www.kernel.org/doc/html/v6.14-rc7/process/debugging/gdb-kernel-debugging.html

[marc-hb]

Good news: hardware breakpoints work with KVM!

Bad news: qemu -S still does not work; seems to rely on some "software" breakpoint behind the scenes.

Maybe KVM is simply incompatible with gdb. Some forum posts say so. I could not find anything one way or the other in the official documentation https://qemu-stsquad.readthedocs.io/en/doc-updates/system/gdb.html

Another option is a gdbserver in the kernel itself. Did not try that yet.

[marc-hb]

https://bugzilla.redhat.com/show_bug.cgi?id=839263 seems related.