Debugger support for ret-sync protocol to enable integration with Ghidra, IDA, etc

[hjanetzek]

Hello I think it would be great if one could use ret-sync with MAME debugger. This would certainly simplify my workflow to reverse engineer some 80s home organs and sound modules :)

https://github.com/bootleg/ret-sync

Would there be interest to add this to MAME?
I guess an implementation of basic sync/breakpoints should be possible as lua plugin. I might work on this at some point. For now I just wanted bring up the idea and would be happy to hear comments and suggestions.

"Combining static and dynamic binary analysis - ret-sync" by Jean-Christophe Delaunay
https://www.synacktiv.com/ressources/bieresecu1_ret-sync_en.pdf

[rb6502]

That sounds interesting for sure, and emulating home organs and sound modules is certainly of interest :-)

We have support in a Lua plugin for the GDB remote protocol (plugins/gdbstub/) so that would likely be a decent basis for what you want to do.

[hjanetzek]

Thanks @rb6502 that gdbstub looks like a major piece of the puzzle to get me started

[rb6502]

@hjanetzek Did you ever get anywhere with this? Ghidra integration would definitely be interesting to me too :-)

[hjanetzek]

Hey @rb6502 - no I didn't investigate this further. But it's still on my want-to-do list when getting back to my MAME projects.

[hjanetzek]

Ok I'm trying it again. It's apparently not the easiest way to learn Lua in MAME console, while trying to understand the ret-sync gdb python implementation.
What is working is that one cannot start the ret-sync Ghidra server with this address already being used when the socket is opened in Lua before. Though running this in the console crashes MAME instantly.

socket = emu.file("", 7)
socket:open("socket.127.0.0.1:9100")
id = "MAME"
socket:[puts|write]("[notice]{\"type\":\"new_dbg\",\"msg\":\"dbg connect - " .. id .. "\",\"dialect\":\"gdb\"}\n")

what would it take to get the common Lua socket library into MAME?

Edit: nvm almost got it - just the regular socket/core.so: Undefined symbol "lua_gettop" problem now

[hjanetzek]

Getting somewhere :)

[hjanetzek]

Started a WIP branch here hjanetzek/mame#1

https://www.youtube.com/watch?v=VU1SDowXf0U

[dlabi]

@hjanetzek thanks for this, it worked like a charm!