[lldb] MachProcess::GetPlatform fails to detect platform in macOS 15.4

[talkeren]

Hey, there is an issue to debug some binaries since the latest mac version, so I investigated it and found part of the root cause.

It should be reproducible with Chromium's test binaries.

gn gen out/Default
ninja -C out/Default unit_tests

And than this will fail

(lldb) target create out/Default/unit_tests
Current executable set to '/Users/tkeren/chromium/src/out/Default/unit_tests' (arm64).
(lldb) process launch
error: process resume at entry point failed: Resume timed out.
Process 54485 exited with status = -1 (0xffffffff) lost connection

The actual failure happens because qProcessInfo fails due to a timeout. This MachO binary has 500+ load commands, for each one MachProcess::GetMachOInformationFromMemory calls MachProcess::GetDeploymentInfo which calls MachProcess::GetPlatform.

MachProcess::GetPlatform cache the result in m_platform, but it just check if the value is not zero, if MachProcess::GetProcessPlatformViaDYLDSPI fails to return a valid value, it will call it every time, which is what happens now. This function is slow - it takes about ~60ms for each call. so 60ms * 500 * 2 - we are talking about a minute to handle that command, which causes timeout.

I didn't investigate why m_dyld_process_info_get_platform returns 0 (which probably start to happen in macOS 15.4 because this is when the issue started), but I do have a fix for this issue which is just caching the result wether it is valid or not.

[llvmbot]

@llvm/issue-subscribers-lldb

Author: Tal Keren (talkeren)

Hey, there is an issue to debug some binaries since the latest mac version, so I investigated it and found part of the root cause.

It should be reproducible with Chromium's test binaries.

gn gen out/Default
ninja -C out/Default unit_tests

And than this will fail

(lldb) target create out/Default/unit_tests
Current executable set to '/Users/tkeren/chromium/src/out/Default/unit_tests' (arm64).
(lldb) process launch
error: process resume at entry point failed: Resume timed out.
Process 54485 exited with status = -1 (0xffffffff) lost connection

The actual failure happens because qProcessInfo fails due to a timeout. This MachO binary has 500+ load commands, for each one MachProcess::GetMachOInformationFromMemory calls MachProcess::GetDeploymentInfo which calls MachProcess::GetPlatform.

MachProcess::GetPlatform cache the result in m_platform, but it just check if the value is not zero, if MachProcess::GetProcessPlatformViaDYLDSPI fails to return a valid value, it will call it every time, which is what happens now. This function is slow - it takes about ~60ms for each call. so 60ms * 500 * 2 - we are talking about a minute to handle that command, which causes timeout.

I didn't investigate why m_dyld_process_info_get_platform returns 0 (which probably start to happen in macOS 15.4 because this is when the issue started), but I do have a fix for this issue which is just caching the result wether it is valid or not.

[jasonmolenda]

I need to look at the debugserver patch a little more closely tomorrow, thanks for putting that up. I think dyld gets the platform from the LC_BUILD_VERSION load command of the main binary -- otool -hlv out/Default/unit_tests would show it. An older equivalent load command was LC_VERSION_MIN_MACOSX. The patch may be worth taking regardless of the cause, but I am curious about the cause of libdyld's SPI not able to find a valid platform for it.

[talkeren]

I think that it looks fine:

Load command 13
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform MACOS
    minos 11.0
      sdk 15.4
   ntools 1
     tool LLD
  version 21.0

It also tried to build an older version which produced a binary that I was able to debug for sure and it still doesn't work, this is why I think that it is related to 15.4. The same thing happens for other team members so it is not only my machine....

[talkeren]

As a test I created a simple binary that calls _dyld_process_info_get_platform on itself. It does get the. correct value:

(lldb) process launch
Process 34047 launched: '/tmp/a.out' (arm64)
platform: 1 pid: 34047

But the same call from debugserver gets 0:

DNBLogError("platform: %u pid: %u", platform, m_pid);

error: platform: 0 pid: 34047

[jasonmolenda]

hm I wish I understood what was going on here; this is the only report we have of this. Unless debugserver were calling the libdyld SPI incorrectly, I can't imagine why there would be a different for that SPI when used from debugserver versus used on the local task. This is the first report I've seen of this. The only really interesting part of that LC_BUILD_VERSION is the linker used is not the system linker.

[jasonmolenda]

Looking at the loader.h definitions, the tool difference surely can't matter.

struct build_version_command {
    uint32_t    cmd;            /* LC_BUILD_VERSION */
    uint32_t    cmdsize;        /* sizeof(struct build_version_command) plus */
                                /* ntools * sizeof(struct build_tool_version) */
    uint32_t    platform;       /* platform */
    uint32_t    minos;          /* X.Y.Z is encoded in nibbles xxxx.yy.zz */
    uint32_t    sdk;            /* X.Y.Z is encoded in nibbles xxxx.yy.zz */
    uint32_t    ntools;         /* number of tool entries following this */
};

struct build_tool_version {
    uint32_t    tool;           /* enum for the tool */
    uint32_t    version;        /* version number of the tool */
};

[jasonmolenda]

Is there any chance I could get a copy of a binary that behaves this way so I can poke at it a little more? I'd genuinely like to understand what the issue is. I think the fix to debugserver is fine, I'll look more closely today, but I don't want to move past a behavior change like this that I can't explain. The normal way to do this would be to file a Feedback report at http://feedback.apple.com which can receive binaries and track the issue properly in apple's bug reporting system, but I don't know if that would be onerous if you aren't registered with the apple developer website.