functionalities/gdb_remote_client/TestqOffsets.py test fails on AArch64

[tstellar]

The TestqOffsets.py test is failing on AArch64. I even tried to test with d4eca12, which was the commit that added the test and test still fails, so it looks like it's never been working on the aarch64 system I'm using.

[llvmbot]

@llvm/issue-subscribers-lldb

Author: Tom Stellard (tstellar)

The [TestqOffsets.py](https://github.com/llvm/llvm-project/blob/main/lldb/test/API/functionalities/gdb_remote_client/TestqOffsets.py) test is failing on AArch64. I even tried to test with d4eca12, which was the commit that added the test and test still fails, so it looks like it's never been working on the aarch64 system I'm using.

[tstellar]

Output from the failed test:

FAIL: lldb-api :: functionalities/gdb_remote_client/TestqOffsets.py (522 of 2175)
******************** TEST 'lldb-api :: functionalities/gdb_remote_client/TestqOffsets.py' FAILED ********************
Script:
--
/usr/bin/python3 /home/gha/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/gha/llvm-project/build/./lib --env LLVM_INCLUDE_DIR=/home/gha/llvm-project/build/include --env LLVM_TOOLS_DIR=/home/gha/llvm-project/build/./bin --arch aarch64 --build-dir /home/gha/llvm-project/build/lldb-test-build.noindex --lldb-module-cache-dir /home/gha/llvm-project/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/gha/llvm-project/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/gha/llvm-project/build/./bin/lldb --compiler /home/gha/llvm-project/build/./bin/clang --dsymutil /home/gha/llvm-project/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/gha/llvm-project/build/./bin --lldb-obj-root /home/gha/llvm-project/build/tools/lldb --lldb-libs-dir /home/gha/llvm-project/build/./lib --cmake-build-type Release /home/gha/llvm-project/lldb/test/API/functionalities/gdb_remote_client -p TestqOffsets.py
--
Exit Code: 1

Command Output (stdout):
--
lldb version 22.0.0git (https://github.com/llvm/llvm-project revision 0e3319537972d284e94a183899f15b40a798fd1f)
  clang revision 0e3319537972d284e94a183899f15b40a798fd1f
  llvm revision 0e3319537972d284e94a183899f15b40a798fd1f
Skipping the following test categories: ['libc++', 'msvcstl', 'dsym', 'gmodules', 'debugserver', 'objc']

--
Command Output (stderr):
--
FAIL: LLDB (/home/gha/llvm-project/build/bin/clang-aarch64) :: test (TestqOffsets.TestqOffsets.test)
======================================================================
FAIL: test (TestqOffsets.TestqOffsets.test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/gha/llvm-project/lldb/test/API/functionalities/gdb_remote_client/TestqOffsets.py", line 20, in test
    self.assertEqual(text.GetLoadAddress(target), 0x471000)
AssertionError: 18446744073709551615 != 4657152
Config=aarch64-/home/gha/llvm-project/build/bin/clang
----------------------------------------------------------------------
Ran 1 test in 0.028s

FAILED (failures=1)

[DavidSpickett]

>>> hex(18446744073709551615)
'0xffffffffffffffff'

Yeah, definitely strange :)

I have never seen this fail on our buildbots, I will look through the test for any potential differences.

[DavidSpickett]

First thought is we choose not to send the qOffsets packet, though I don't see any logic in LLDB that would skip it. Please apply this patch to enable logging so we can see if it is sent:

diff --git a/lldb/test/API/functionalities/gdb_remote_client/TestqOffsets.py b/lldb/test/API/functionalities/gdb_remote_client/TestqOffsets.py
index a1adc20e864b..bd6ff8cfee94 100644
--- a/lldb/test/API/functionalities/gdb_remote_client/TestqOffsets.py
+++ b/lldb/test/API/functionalities/gdb_remote_client/TestqOffsets.py
@@ -16,5 +16,8 @@ class TestqOffsets(GDBRemoteTestBase):
         text = target.modules[0].FindSection(".text")
         self.assertEqual(text.GetLoadAddress(target), lldb.LLDB_INVALID_ADDRESS)
 
+        self.runCmd("log enable gdb-remote all")
+        self.runCmd("log enable lldb all")
+
         process = self.connect(target)
         self.assertEqual(text.GetLoadAddress(target), 0x471000)

You can run this test on its own with:

$ ./bin/lldb-dotest -p TestqOffsets.py

If it's not that then perhaps undefined behaviour in handling the packet, which is hard to diagnose from afar. Does /home/gha/ mean this is a GitHub runner? If so, I could try the same thing.

@labath in case they have ideas too.

[labath]

I don't have any ideas. I think it's more likely the issue is environmental (something about that machine is causing lldb do go down a different code path, but I have no idea what could that be. It would be an interesting experiment to change lldb/test/API/functionalities/gdb_remote_client/qOffsets.yaml to use a different architecture than AArch64, and see if it still fails (if it starts succeeding it's likely something environmental).

[tstellar]

I've been able to reproduce this on two different aarch64 machines: AWS m8g and an aarch64 system in our offices. I will try some of the debugging suggestions.