Is gdb or another debugger available that can work with this toolchain?

[greatwolf]

Hi,

This is more of a quick question than an issue. I downloaded the tomatoware toolchain for use with FreshTomato 2022.6 running on Netgear R6700v1. A quick 'hello world' test seems to work; was able to compile, upload to router and run via ssh.

Is there a debugger available that lets me debug programs running on the router? I don't see gdb in ./bin even though the other usual gnu tools seem present. If not how does one go about debugging programs?

Thanks

[lancethepants]

Yes, one you've extracted Tomatoware onto the router gdb is also available, should be in either /mmc/bin, or /mmc/sbin.

[greatwolf]

I think I'm a little confused. I initially downloaded the toolchain from here https://files.lancethepants.com/Toolchains/arm/gcc-11.2%2C%20uclibc-ng-1.0.38%2C%20Linux-2.6.36.4/. That version does not appear to include gdb in there. It also looks like that toolchain does a cross-compile from a desktop linux environment(eg. intel x64) as the host targeting an ARMv7 binary which is what I want.

Now I also looked at https://files.lancethepants.com/Tomatoware/ and that one does have gdb in there. However, best I can tell that doesn't appear to be a cross-compiling toolchain. I get cannot execute binary file: Exec format error if I try to run any of the compiler commands.

I'm guessing that toolchain is using ARMv7 as the host? That's certainly not what I'm looking for since putting the entire toolchain and uploading it to a router is a non-starter -- which I expect to be the case most of the time since a typical router isn't going to have the resources necessary to do such a thing, even modern routers today would be hard pressed to do that.

Apologies, I'm not too familiar with the nomenclature used, like mmc, opt, plexware, entware etc. I'm probably looking in the wrong spots.

[lancethepants]

Here's a toolchain I've compiled for arm/uclibc-ng with included cross-gdb you can try. A couple of things to note with my toolchains.

They are completely different than the toolchain used to build tomato firmware. In order to properly run the resulting binary on the router, you should either statically compile the binary so that it doesn't have any dependencies, or actually install tomatoware on the router so it can use the correct libraries. If your hello world wasn't built statically, then you got lucky that tomato firmware's libc was close enough to run it. Tomato uses uclibc which is very old whereas Tomatoware uses uclibc-ng and there have been a lot breaking changes between the two. uclibc does not guarantee abi stability. So I would encourage you to statically compile your binaries.

If you decide not to statically compile then you need to use LDFLAGS="-Wl,--dynamic-linker=/mmc/lib/ld-uClibc.so.1" so that the program can point to the correct dynamic linker. You would also need to install Tomatoware on the router. I will probably patch this in the toolchain so cross-compiled binaries don't accidently use the tomato firmware system libraries.
https://github.com/lancethepants/tomatoware/blob/master/scripts/environment.sh#L23

While routers are much slower than x86_64 machines they can be surprisingly capable. I started tomatoware while using old slow mipsel based routers. Arm routers are much faster and you usually get at least 2 cores, so you get them both working when compiling. I now use Tomatoware on my rt-ax88u with quad-core arm64 cpu. The c++ compiler is definitely more memory intensive than plain old C, so sometimes I'll only use 3 or 2 cores in those circumstances.

I just compiled openvpn for example on my rt-ax88u (so faster than your router) and it took ~3 minutes. Using ccache the subsequent compile took ~5 seconds. And I didn't have to worry about cross-compiling all the dependencies because all the ones openvpn needs are already included in Tomatoware.

[greatwolf]

That's very informative. Yeah, I actually did statically link with -static when I did my 'hello world' test. It did end up coming out to ~1.3mb because of it. If I do dynamic linking with the runtime though, would it be possible to also upload ld-uClibc.so and any other dependencies onto the router? It's a bit of a Frankenstein approach and I'm unsure if during binary loading it'll find the proper dependencies in the right location.

[lancethepants]

Yes, you should be able to upload just the libraries you need if dynamic linking. I have patched libc (uclibc & musl) to have /mmc as the sysroot, so libraries and the dynamlic linker should be located in /mmc/lib, also remembering to pass -Wl,--dynamic-linker=/mmc/lib/ld-uClibc.so.1 to you compiler invocation.

You may also want to pass -s when compiling to strip binaries for smaller sizes. Of course this removes debugging symbols so skip if you need debugging. The arm/musl version may create smaller binaries, haven't really compared them. Also you may consider upx (included in tomatoware) for binary compression. I usually do upx --lzma binary for max compression. It can regularly reduce the size by half or third depending on the binary.

[lancethepants]

Or you can static link most things leaving just libc and the dynamlic linker left.
edit: which on musl the dynamic linker is just a symlink to the libc.

# g++ hello.cpp -o hello
# ldd ./hello
        /mmc/lib/ld-musl-aarch64.so.1 (0x7f8c673000)
        libstdc++.so.6 => /mmc/usr/lib/libstdc++.so.6 (0x7f8c446000)
        libgcc_s.so.1 => /mmc/lib/libgcc_s.so.1 (0x7f8c421000)
        libc.so => /mmc/lib/ld-musl-aarch64.so.1 (0x7f8c673000)

# g++ -static-libstdc++ -static-libgcc hello.cpp -o hello
# ldd ./hello
        /mmc/lib/ld-musl-aarch64.so.1 (0x7fa7e52000)
        libc.so => /mmc/lib/ld-musl-aarch64.so.1 (0x7fa7e52000)