Debugger support

[trungnt2910]

For debugger support, Haiku uses specific syscalls that opens a magic port with the kernel.

The kernel also installs a "nub thread" to the debugged process.

How exactly this is done is still not documented. However, all of this should technically be possible using ptrace and/or signals.

[trungnt2910]

As of 48e34a0, basic debugging infrastructure has been installed.
Currently, hyclone_server forces the process to spawn a nub thread through the Requests API. This approach is somewhat similar to DarlingHQ's cooperative debugging.

The nub thread currently doesn't do anything yet, neither does the debuggee after being debugged. To trace syscalls, monika would need some refactoring.

Nevertheless, this is still enough to allow strace to exit with a success status code!

[trungnt2910]

As of hcrev232 (or commit e907550), some parts of strace is working!

Specifically, strace -f (fast mode with syscall tracing only and no argument processing) works. The normal mode will need additional support for reading the debuggee's memory (B_DEBUG_MESSAGE_READ_MEMORY).

[trungnt2910]

hcrev234 adds the ability to use strace without -f. This means that strace can properly analyze the parameters passed to each syscalls through the debugger API.

There is still a lot of work to be done, such as thread/process notification creation or signal handling.

[trungnt2910]

As of hcrev238, strace has support for debugging newly spawned teams and threads. However:

• Deadlocks are common, due to the fragile nature of the communication between haiku_loader and hyclone_server.
• Tracing signals is still not supported.

[kenmays]

• Ref: https://www.haiku-os.org/docs/userguide/en/applications/debugger

Have you used/reviewed Haiku's Debugger (i.e 'Debugger -c ') ?? Pros/Cons ??

[trungnt2910]

I've used Haiku's Debugger on native Haiku, but my experience with it is not so good...

As for HyClone, the debugger API is still incomplete. The most important feature that prevents Debugger from working is register reading support (it is not too hard to implement, simply reading the signal context will do the job), and probably thread stopping and resuming (which is also not too hard to implement, as we can simply send a signal to the target thread).