~4x unit test performance regression with SDK version 2.6.1

[z2oh]

We recently updated the SDK from version 2.4.0 to 2.6.3 and noticed a large performance regression for our unit test runs in CI (~8.5m on average to ~33m). I bisected SDK versions and found that 2.6.1 was when the slow performance started.

Nothing really stuck out to me as an obvious explanation in the changeset: 2.6.0...2.6.1 , though I'm not much of a Swift programmer.

We're running our unit tests serially (as we have some concurrency issues with state bleeding between tests), with this set of DD SDK envvars:

DD_TEST_RUNNER: 1
DD_ENV: ci
DD_DISABLE_SOURCE_LOCATION: 1
DD_CIVISIBILITY_ITR_ENABLED: 0
DD_CIVISIBILITY_CODE_COVERAGE_ENABLED: 0
DD_DISABLE_NTPCLOCK: 1

The regression shows up locally as well (even more significantly in my local testing, a 6x-10x slowdown). Surprisingly, this is without DD_TEST_RUNNER set.

[ypopovych]

Hello. Do you have warnings in the logs? Because if it happens without DD_TEST_RUNNER seems as it happens on the build or loading stages (SDK will do nothing, only print log if DD_TEST_RUNNER is not set)

[amonshiz]

There does appear to be a step to unshallow a checkout by 1 commit at 2.6.0...2.6.1#diff-9a31fc03c809fe362d1cccf5c951d46075a047ed8b23a2f18b25b2df4ecb8a6eR252-R280

    static func mergeHeadInfo(headSha: String, workspace: String, log: Logger) -> Git? {
        let git = { (cmd: String) -> String? in
            Spawn.output(try: "git -C \"\(workspace)\" \(cmd)", log: log)
        }
        // Unshallow one commit
        guard git("fetch --deepen=1 --update-shallow --filter=blob:none --recurse-submodules=no") != nil else {
            return nil
        }
        guard let message = git("log -n 1 --format=%B \(headSha)") else {
            return nil
        }
        guard let info = git("show -s --format=%an\t%ae\t%at\t%cn\t%ce\t%ct \(headSha)") else {
            return nil
        }
        let fields = info.components(separatedBy: "\t")
        guard fields.count >= 6 else { return nil }
        let auhorDate = UInt(fields[2]).map { Date(timeIntervalSince1970: Double($0)) }?.spanAttribute ?? fields[1]
        let commiterDate = UInt(fields[5]).map { Date(timeIntervalSince1970: Double($0)) }?.spanAttribute ?? fields[5]
        return Git(
            commitHead: .init(sha: headSha,
                              message: message,
                              author: .init(name: fields[0],
                                            email: fields[1],
                                            date: auhorDate),
                              committer: .init(name: fields[3],
                                               email: fields[4],
                                               date: commiterDate))
        )
    }

We are also seeing a performance impact on our tests of about 2x durations. We have tried disabling the ITR and code coverage settings and that doesn't seem to have an impact. Our CI only does a shallow clone of the repo with the following command from the actions/checkout action.

  /usr/bin/git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --filter=blob:none --depth=1 origin +refs/pull/<number>/merge:refs/remotes/pull/<number>/merge

Thus the SDK attempting to deepen that could cause a duration increase. Is the deepen=1 an attempt to get the true head commit in the case that the code being built is a merge commit (ex: GitHub's PR head vs the branch head)?