Code coverage report not matching Xcode test results

[sigsbeym]

I'm using this with some swift packages I've been working on and they are not reporting coverage results that match what I see in my Xcode project when running tests. It looks like it's including coverage for dependency targets, which is reducing my total coverage percentage significantly, and Xcode is not doing this.

When I use swift test --enable-code-coverage via cli I also see this reduced total coverage amount, but when I use xcodebuild the xcresult file coverage totals match what I see in Xcode.

I tried modifying this action to fallback to the xcodebuild commands that are already being used in the action for project/workspace files, but somehow it's also not matching Xcode test coverage. I think there's something with how the results are being parsed out of the xcresult file by the dd sdk testing framework that I can't see in the action logs which is causing this irregularity.

Any help or guidance would be appreciated.

[ypopovych]

Hello @sigsbeym.

Yes. Xcode filters code coverage by target and SDK reports shared.

SDK simply doesn't know which targets to filter, so calculates it for all test process binaries which built with code coverage.

[sigsbeym]

@ypopovych Is there not a way to specify this to the SDK? I know that's not possible with swift cli, but xcodebuild seems to work fine if I supply it with a test plan. It doesn't seem like this is an option with your SDK even when using the xcodebuild command.

[ypopovych]

@sigsbeym Xcode doesn't send test plan info to the test process (only tests to filter). It gathers all info and at the end filters it by itself. That's why swift cli shows different percentage.

Right now we don't have option to filter it.