Debugging Linux Software

This article describes how to debug user-space applications on the Linux on ARC.

Building Tool chain

Refer to README.md file in the toolchain repository to learn how to build toolchain for Linux uClibc target. In most cases you would want to run from the toolhain repository directory:

$ ./build-all.sh --no-elf32

Toolchain will be installed to the ../INSTALL directory.

Running Linux

There are two ways supported by Synopsys to run Linux on ARC cores: on the SystemC model based on nSIM simulator and on the AXS101 SDP.

SystemC Model

Source code of the SystemC model is supplied as an example with the nSIM distribution.

First you need to build model. Refer to our wiki article for details.

By default model already contains a prebuilt Linux image that can be used to run and debug target applications. If you wish to have your own Linux image, then refer to this article for details. If you are using Buildroot make sure that gdbserver is selected in the Target packages / Debugging, profiling and benchmarking.

AXS101 SDP

Follow this article to build Linux kernel. You need to specify axs101 as a defconfig. If you are using Buildroot, then you can do this in the Kernel configuration section, parameter Defconfig name aka BR2_LINUX_KERNEL_DEFCONFIG.

After this follow this article if you are using OpenOCD or this article if you are using Ashling GDB Server. Load <BUILDROOT_OUTPUT>/images/vmlinux file into the target memory as with any baremetal application. Note that if you want to only load vmlinux into the target memory, you can use arc-linux-gdb for this, however if you are going to perform a debugging of the Linux kernel, thne you need to have a baremetal debugger: Metaware Debugger or ARC GNU ELF32 GDB.

After loading image into memory start the core.

To connect to UART terminal of Linux system, use minicom application. Run:

$ sudo minicom -s

That will start minicom in configuration mode. Configure serial port to use serial device /dev/ttyUSB0, bps/par/bits are 115200 8N1 and both hardware and software flow controls are disabled. Save setup as default adn exit from Minicom. Restart minicom without -s and it will connect to the specified serial device. If you want to use another UART device of the target system, then you will need to change path to serial device to the desired value.

Additional Configuration

To ease process of delivering target application into the ARC Linux it is recommended to configure NFS share and mount it on the ARC Linux. Refer to this article for details.

Debugging Applications

It is assumed that one or another way you've copied target application to the target system. Run application on target with gdbserver:

# gdbserver :49101 <application-to-debug> [application arguments]

TCP port number could any port, not occupied by other applications. Then run GDB on the host:

$ arc-linux-gdb <application-to-debug>

Then set sysroot directory path. Sysroot is a "mirror" of the target system file system: it contains copies of the applications and shared libraries installed on the target system. Path to the sysroot directory should be set to allow GDB to step into shared libraries functions. Note that shared libraries and applications on the target system can be stripped from the debug symbols to preserve disk space, while files in the sysroot shouldn't be stripped. In case of Buildroot-generated rootfs sysroot directory can be found under <BUILDROOT_OUTPUT>/staging.

(gdb) set sysroot <SYSROOT_PATH>

Then connect to the remote gdbserver:

(gdb) target remote <TARGET_IP>:49101

You can find <TARGET_IP> via running ifconfig on the target system. TCP port must much the one used when starting up gdbserver. It is important that sysroot should be set before connecting to remote target, otherwise GDB might have issues with stepping into shared libraries functions.

Then you can your debug session as usual. In the simplest case:

(gdb) continue

Debugging Applications with Native GDB

Starting from ARC GNU Toolchain release 2014.08 it is possible to build full GDB to run natively on ARC Linux. You can either select GDB in the Target applications of the Buildroot configuration, or build it as a normal cross-compiled application with ./configure --host=arc-linux-uclibc.

Then you can debug applications the same way you do this on the host system without gdbserver.

When choosing between gdbserver and native GDB please consider pros and cons of native GDB.

Pros:

• Overhead for network communication between GDB and gdbserver is removed, theoretically improving debugging performance.
• Some features are not implemented in the gdbserver. One example is "follow-child" model of "follow-fork" behaviour.
• There is no need for a second host to perform debugging session, since everything is on the target system.

Cons:

• It is required that applications on target system should have debugging symbols (unless you are so hardcore that you don't need them). Debugging symbols, especially in the most verbose case occupy significant disk space. Depending on the type of target hardware this might be or might not be a thing to consider. Usually this can be ignored in case of virtual prototypes, and is hardly a problem with development systems, however disk space is probably very limited on the production systems. Large rootfs size also means increased time required to load rootfs into the target memory.
• Not only debugging symbols will take noticeable disk space, but also GDB will also read them intensively, so if target file system has a low performance, this might be noticeable.
• Support for native GDB has been added much later to the ARC tool chain, hence it is theoretically less stable.
• Host GDB requires more computational power than gdbserver. This might offset all of the gains from exclusion of the networking layer.

In general it is highly dependent on the target system whether gdbserver or native GDB is better and it is up to the software developer to decide what is better in their particular case.