xccov-to-sonarqube-generic.sh does not account with multiple code branches

[crsantos]

Hi,

AFAIK, the xccov-to-sonarqube-generic.sh is not accounting with multiple code branches in the same line, like ternary operators of nil coalescing fallbacks.

e.g.:

Using this snippet we can see that we have 2 different code paths:

testBool(optionalBool ?? false)

1. optionalBool exists and its value is used
2. optionalBool is nil and we are coalescing to false

From a random export of xcrun xccov view --archive App.xcresult:
(this is not a match from the snippet above, only an example)

/Users/Random/work/app-ios/Sources/Model.swift:
  ...
  59: 0
  60: 0
  61: 2 [
(1, 9, 0)
]
  62: 2
  63: 2
  64: 8 [
(1, 67, 2)
(91, 0, 2)
]
  65: 2
  66: 2
  67: 4 [
(1, 28, 2)
(97, 0, 2)
]
  68: 2 [
(87, 1, 0)
]
  69: 2
  70: 2 [
(77, 2, 1)
]
  71: 2
  72: 2
  73: *
...

Looking at the conversion script, it only cares about:

 69: 2
 70: 2

But there's extra info on the next line that is important to confirm if we have covered all the possible code branches:

70: 2 [
(77, 2, 1)
]

Questions:

1. As of today, is that script the official way of reporting Swift code coverage from an xcresult?
2. If so, shouldn't it support different code branches?

Thanks.

[joe-tingsanchali-sonarsource]

Hello @crsantos ,

Apologies for such a late reply.

I'm about to update the swift-coverage example to show how to use Slather, which might be a better fit for you.

I'll work on trying to reproduce your error.

[crsantos]

Hello @crsantos ,

Apologies for such a late reply.

I'm about to update the swift-coverage example to show how to use Slather, which might be a better fit for you.

I'll work on trying to reproduce your error.

Hi Joe, thanks for getting back to me.

I am still not sure if I need Slather since sonarqube agent will upload the data I need, that specific script just doesn't parse things properly but it can be fixed.

Why would Slather be better? Does it deals with this edge case?

Thanks.

[joe-tingsanchali-sonarsource]

Hi @crsantos ,

Glad to hear from you! Slather is a dedicated application for Swift coverage as opposed to this simple script that someone created years ago. Yes, it does introduce a Ruby dependency, but the setup is easy. Here's a simple GitHub Actions example:

name: Build
on:
  push:
    branches:
      - main
  pull_request:
    types: [opened, synchronize, reopened]
jobs:
  sonarqube:
    name: SonarQube
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v6
        with:
            fetch-depth: 0
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: '3.2' # Not needed with a .ruby-version, .tool-versions or mise.toml
          bundler-cache: true
      - name: Build and test, collect coverage
        run: xcodebuild clean build test -project CodecovDemo.xcodeproj -scheme CodecovDemo -destination 'platform=iOS Simulator,OS=26.1,name=iPhone 17' -enableCodeCoverage YES
      - name: Convert xcresult coverage to SonarQube coverage format
        run:  bundle exec slather coverage --sonarqube-xml --scheme CodecovDemo ./CodecovDemo.xcodeproj
      - name: SonarQube Scan
        uses: SonarSource/sonarqube-scan-action@v7
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
        with:
          args: >
            -X
            -Dsonar.sources=CodecovDemo
            -Dsonar.tests=CodecovDemoTests,CodecovDemoUITests
            -Dsonar.inclusions=**/*.swift
            -Dsonar.exclusions=vendor/**
            -Dsonar.coverage.exclusions=**/GeneratedAssetSymbols.swift
            -Dsonar.coverageReportPaths=sonarqube-generic-coverage.xml

In any case, I have an improved script that you can try in my feature branch: https://github.com/SonarSource/sonar-scanning-examples/tree/feature/JT-improve-swift-coverage

Please try Slather and try the improved script here and let me know what works for you.

[crsantos]

Hi @crsantos ,

Glad to hear from you! Slather is a dedicated application for Swift coverage as opposed to this simple script that someone created years ago. Yes, it does introduce a Ruby dependency, but the setup is easy. Here's a simple GitHub Actions example:

...
In any case, I have an improved script that you can try in my feature branch: https://github.com/SonarSource/sonar-scanning-examples/tree/feature/JT-improve-swift-coverage

Please try Slather and try the improved script here and let me know what works for you.

Thanks Joe.
Nice ✨ , I can see your branch 🙏

I was not aware that slather already does sonarqube output format.
I will give it a spin to confirm partial branch covering works too ⌛

How do we do it as of today?

We have been running an improved version of this repo's script xccov-to-sonarqube-generic.sh, without any issues for months, but this is pure bash, horrible to maintain. This was a discussion here and soon I realized this was not a priority for Sonarqube, so I had to roll my sleeves and make it work.

I can share my updated script later, since we use it on our test pipelines without any issue for the past months. I cannot guarantee the quality of bash code, though.

[joe-tingsanchali-sonarsource]

Haha, thanks, I'm glad you noticed a bit of ✨ magic.

Yes, slather looks pretty good, just pass --sonarqube-xml flag to get SonarQube coverage format. See various options and output formats here.

Example:

slather coverage --sonarqube-xml --scheme <schemeName> --build-directory <buildDirectory>  path/to/project.xcodeproj

but this is pure bash, horrible to maintain.
...
I can share my updated script later, since we use it on our test pipelines without any issue for the past months. I cannot guarantee the quality of bash code, though.

Yes, exactly why I want to move away from our own xccov-to-sonarqube-generic.sh script.
Yes, please share your script if you would like or submit a PR and I'll try to incorporate it here.

[joe-tingsanchali-sonarsource]

Hi @crsantos , #218 has been merged into master. Please fetch/pull the latest changes and check if the xccov-to-sonarqube-generic.sh handles multiple code branches in your Xcode/Swift coverage. Please also consider using Slather, which I've documented in https://github.com/SonarSource/sonar-scanning-examples/tree/master/swift-coverage#using-slather-highly-recommended.

[crsantos]

Please also consider using Slather, which I've documented in https://github.com/SonarSource/sonar-scanning-examples/tree/master/swift-coverage#using-slather-highly-recommended.

Thanks for keeping this updated, Joe 🙇

I have followed your advice and I am currently testing slather instead of our custom bash script based on the original one 🙏

I'll let you know how it went after I validate slather generated xml on:

1. PR jobs
2. in our daily job that keeps the new code baseline updated on sonarqube

⌛

[crsantos]

Hi again Joe,

I confirmed and Slather does not support branch coverage 😢

Looking at the Sonarqube xml output you can confirm that it doesn't support coveredBranches or branchesToCover:

e.g.:

<lineToCover lineNumber="68" covered="true" branchesToCover="2" coveredBranches="1"/>

I could not even find any reference to those keywords in the repo.

Given that, slather is not an option. I will give it a spin to your updated bash script that looks way better than mine. ⌛

[joe-tingsanchali-sonarsource]

I confirmed and Slather does not support branch coverage 😢
...
Given that, slather is not an option. I will give it a spin to your updated bash script that looks way better than mine. ⌛

Hi! Good to hear from you. That's too bad. Ok, let me know if my script works. Maybe I'll work on keeping this script alive then.

[crsantos]

Hey, first of all thank you so much for following up on this. Someone cared about iOS. 🙇

I have a few insights I would like to share with Sonar community, because I think my script is not 100% correct, and this one is also not correct, lets dig in and you all can give your 2 cents after it.

---

I have tried the updated script in this repo vs my custom one and I am finding different results, I have an idea why, and I remember handling a specific case in the implementation.

I would like to show you a repro so we can understand where is the difference.

Code used

This is sample code, ignore quality, I have tried to make sure all the branches were covered in this case:

Sample new code

extension MyReducer.State {
    /// Temporary function to test branch coverage tracking.
    /// This function exercises multiple branching patterns, assume `items` is a list inside this struct, it's irrelevant to coverage:
    /// - if/else branches
    /// - nil-coalescing operator
    /// - ternary operator
    func testBranchCoverage(searchEnabled: Bool, maxItems: Int?) -> String {
        // Branch 1: if/else
        let status = if searchEnabled {
            "search-enabled"
        } else {
            "search-disabled"
        }

        // Branch 2: nil-coalescing
        let limit = maxItems ?? 10

        // Branch 3: ternary
        let description = limit > 5 ? "large-list" : "small-list"

        // Branch 4: guard with optional
        guard items.count > 0 else {
            return "empty-\(status)"
        }

        return "\(status)-\(description)-\(limit)"
    }
}

Sample Test

// Test Branch Coverage (TEMPORARY - for coverage script testing)

func testBranchCoverage_searchEnabled_withMaxItems() {
    let state = testStore(insert: .default(allowSelection: true)).state // irrelevant for test
    let result = state.testBranchCoverage(searchEnabled: true, maxItems: 15)
    XCTAssertEqual(result, "search-enabled-large-list-15")
}

func testBranchCoverage_searchDisabled_withoutMaxItems() {
    let state = testStore(insert: .default(allowSelection: true)).state
    let result = state.testBranchCoverage(searchEnabled: false, maxItems: nil)
    XCTAssertEqual(result, "search-disabled-large-list-10")
}

func testBranchCoverage_searchEnabled_withSmallMaxItems() {
    let state = testStore(insert: .default(allowSelection: true)).state
    let result = state.testBranchCoverage(searchEnabled: true, maxItems: 3)
    XCTAssertEqual(result, "search-enabled-small-list-3")
}

func testBranchCoverage_emptyList() {
    let state = testStore(items: [], insert: .default(allowSelection: true)).state
    let result = state.testBranchCoverage(searchEnabled: true, maxItems: 10)
    XCTAssertEqual(result, "empty-search-enabled")
}

How it looks on Sonarcloud

Repo script 87.8%	My custom script 100%

What is happening here

From what I can remember while I was implementing this, there was a mismatch of what xccov returns and what actually is covered in these cases, and I had to add 1 in some cases, when covered.

Let's see, e.g.: Line 339 (guard statement)

                                  👇
        guard items.count > 0 else {

339: 4 [
(36, 0, 1)
]

My understanding of this xccov format is the following, (I didn't find any official docs on it, feel free to link any):

• Starting at column 36 (before {)
• till the end of line (0 means EOL)
• it had 1 hit

My script: is assuming 2 branches, 2 are covered
Repo script: is assuming 2 branches, only 1 is covered

In fact: guard was fully covered:

• Tests 1, 2, 3: guard passes (items.count > 0)
• Test 4: guard fails (items.count = 0), and early return

Another example, Line 326:

        let status = if searchEnabled {

326: 4 [
(39, 0, 3)
]

• Starting at column 39 (before {)
• till the end of line (0 means EOL)
• it had 3 hits (3 tests using searchEnabled == true)

My script: is assuming 2 branches, 2 are covered
Repo script: is assuming 2 branches, only 1 is covered

I would say this line has 2 parts: the one before { and what is past that.

❓ But does this mean 2 branches? That is what confuses me. Anyway, this line should be covered 100%, with one of the following:

• <lineToCover lineNumber="326" covered="true" branchesToCover="2" coveredBranches="2"/>
• <lineToCover lineNumber="326" covered="true" branchesToCover="1" coveredBranches="1"/>
• <lineToCover lineNumber="326" covered="true" />

Here are a few more graphical examples

After this analysis, I can conclude my script is accounting with all the branches, but I am not sure if I am not inflating the branch count.

What do you think?

[vshikulo-tbc]

Hi @crsantos, I have the same issue with updated script. Could you please share your version?