LLDB: Breakpoints regressed - Web Assembly invoked from WASM C-API in a dynamic library (fix included)

[adv-sw]

What happened : LLDB breakpoints stopped working.
Specific use case example here : https://advance-software.com/develop/#tutorial

Our app invokes Web Asssembly C-API from a plugin (DLL on Windows) - specifically wasmtime runtime.

I've debugged this & the fix is as follows - seems an overly aggressive optimization has been introduced that needs rolling back.

PATCH UPDATED: IsAlive not affecting behaviour so no reason for patch to exclude it.

File: lldb\source\Plugins\JITLoader\GDB\JITLoaderGDB.cpp

Add at the top of the file :
#define WASM_DBG_PATCH 1

a. In JITLoaderGDB::SetJITBreakpoint

#ifndef WASM_DBG_PATCH
if (DidSetJITBreakpoint())
return;
#endif

b. In JITLoaderGDB::ModulesDidLoad

#ifdef WASM_DBG_PATCH
if (m_process->IsAlive())
#else
if (!DidSetJITBreakpoint() && m_process->IsAlive())
#endif

c. Build lldb as normal.

Breakpoints working again 🙂

[llvmbot]

@llvm/issue-subscribers-lldb

Author: Advance Software (adv-sw)

What happened : LLDB breakpoints stopped working. Specific use case example here : https://advance-software.com/develop/#tutorial

Our app invokes Web Asssembly C-API from a plugin (DLL on Windows) - specifically wasmtime runtime.

I've debugged this & the fix is as follows - seems an overly aggressive optimization has been introduced that needs rolling back.

File: lldb\source\Plugins\JITLoader\GDB\JITLoaderGDB.cpp

a. In JITLoaderGDB::SetJITBreakpoint

Comment out :

//if (DidSetJITBreakpoint())
//return;

b. In JITLoaderGDB::ModulesDidLoad

Comment out :

//if (!DidSetJITBreakpoint() && m_process->IsAlive())

c. Build lldb as normal.

Breakpoints working again 🙂

[jimingham]

Thanks for the report.

The fix you suggest only makes sense if more than one library loaded into your program has the __jit_debug_descriptor symbol we are setting a breakpoint on. If that happened, the already set JIT breakpoint might be for the library that's not running your JIT code, and so your loads don't trigger the breakpoint and so aren't scanned for symbols.

But your fix would be wrong, since we would lose track of the first breakpoint we had set when we set the second one. We're just storing one breakpoint ID in this class.

TTTT, I'm not sure why the JIT loader code goes to the trouble of looking up a symbol on each library load, finding the address of that symbol, and setting an address breakpoint there. It always sets the breakpoint if it finds the symbol, and doesn't do anything with the particular module it happens to have found the symbol in.

If, the first time JITLoaderGDB::SetJITBreakpoint sees this symbol, it sets a "by symbol name" breakpoint on that symbol instead, lldb would have handled adding locations when new libraries with this symbol were loaded transparently, and the DidSetJITBreakpoint logic would all be correct.

[adv-sw]

It fixes the problem. There might be a more elegant fix.

You can run our https://advance-software.com/develop/#tutorial test using lldb default to confirm breakpoint is not hit, my fix to confirm breakpoint is hit & whatever more elegant solution you have to confirm this also works.

I'm not deep diving lldb, I have too much work on. Proceed as you wish.

All I need for now is working breakpoints, which I now have & can maintain in an external patch if necessary.

I don't know, this stuff used to work, works with the patch I've provided & broke after some changes in lldb.

I've got what I need, if you need more, dive deeper.

To repeat my repro/test, whatever version of lldb is being tested goes here : https://github.com/adv-sw/wasm_sdk/tree/main/tools

& you'll need to download our full wasm sdk to repeat the test. A simpler test is likely possible but that's what I've got right now.

Test is Windows only for now, though issue may well be cross platform.

[jimingham]

Sure, people that report bugs aren't required to fix them...

But regardless of what's actually going wrong here, if the JITLoader code arranged to set the breakpoint by name on the symbol once (which can even be done before attaching or launching), then no matter when the symbol actually shows up lldb will resolve the breakpoint. So if/when somebody gets around to fixing this (and any other timing problems that could arise around setting this breakpoint) they should still IMHBSEO do it by setting a by name breakpoint...

[adv-sw]

PATCH UPDATED: IsAlive not affecting behaviour so no reason for patch to exclude it.

Update in original post & my comment on timing removed.

[adv-sw]

Also getting odd debugger behaviour - stepping inside an if statement causes knowledge of all variables to be lost.