sf apex get test command returns 0% apex test coverage

[kez-sophos]

Summary

sf apex get test command returns 0 totalCovered and 0 coveredPercent

Steps To Reproduce

1. Authenticate to a scratch org
2. Run apex test / tests

sf apex run test --json --result-format json --suite-names TEST_SUITE_NAME --target-org ORG --output-dir tests_run_scratch/ --wait 1

3, Get apex test result

sf apex get test --json --code-coverage --result-format json --target-org ORG --output-dir tests_run_scratch/ --test-run-id TEST_ID

can retrieve test id by running the query in the test org (AsyncApexJobId): SELECT Status, AsyncApexJobId FROM ApexTestRunResult

Expected result

apex get test command returns accurate line and percent, test coverage for apex classes

Actual result

apex get test command returns 0 line and percent, test coverage for apex classes

System Information

cmd.exe

{
  "architecture": "win32-x64",
  "cliVersion": "@salesforce/cli/2.50.6",
  "nodeVersion": "node-v20.15.0",
  "osVersion": "Windows_NT 10.0.22631",
  "rootPath": "C:\\Users\\<USER>\\AppData\\Local\\sf\\client\\2.50.6-6e9d7f7",
  "shell": "cmd.exe",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.1.6 (core)",
    "@oclif/plugin-commands 4.0.6 (core)",
    "@oclif/plugin-help 6.2.5 (core)",
    "@oclif/plugin-not-found 3.2.10 (core)",
    "@oclif/plugin-plugins 5.3.4 (core)",
    "@oclif/plugin-search 1.2.2 (core)",
    "@oclif/plugin-update 4.4.7 (core)",
    "@oclif/plugin-version 2.2.6 (core)",
    "@oclif/plugin-warn-if-update-available 3.1.8 (core)",
    "@oclif/plugin-which 3.2.6 (core)",
    "@salesforce/cli 2.50.6 (core)",
    "apex 3.2.4 (core)",
    "auth 3.6.33 (core)",
    "community 3.2.22 (user) published 30 days ago (Sat Jun 22 2024)",
    "data 3.5.5 (core)",
    "deploy-retrieve 3.9.13 (core)",
    "dev 2.3.4 (user) published 37 days ago (Fri Jun 14 2024) (latest is 2.3.6)",
    "info 3.3.17 (core)",
    "limits 3.3.17 (core)",
    "marketplace 1.2.17 (core)",
    "org 4.3.6 (user) published 1 days ago (Sun Jul 21 2024)",
    "packaging 2.7.0 (core)",
    "schema 3.3.19 (core)",
    "settings 2.3.8 (core)",
    "sobject 1.4.19 (core)",
    "source 3.5.3 (core)",
    "telemetry 3.6.1 (core)",
    "templates 56.3.2 (core)",
    "trust 3.7.13 (core)",
    "user 3.5.17 (core)",
    "sfdx-git-delta 5.30.0 (user) published 243 days ago (Tue Nov 21 2023) (latest is 5.40.2)"
  ]
}

Additional information

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[mshanemc]

Hey, my friends on Apex looked at this and you need to use --code-coverage on the run test command. Otherwise, the tests don't compute coverage, and that may be why you're getting all 0s.

[github-actions]

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.

[kez-sophos]

Hi @mshanemc,

Adding the --code-coverage flag to the apex run test command changes the return behavior of the command.

Running the command WITHOUT the code coverage flag returns the test run id, if the execution of the unit tests takes longer than the wait time.

sf apex run test --json --result-format json --suite-names sophos_service_tests --target-org <TARGET_ORG> --output-dir tests_run_scratch/ --wait 1

Returns:
{
"status": 0,
"result": {
"testRunId": "707Ox00000TyNv7"
},
"warnings": []
}

whilst running the command WITH the code coverage flag returns an error if the execution of the unit tests takes longer than the wait time.

sf apex run test --code-coverage --json --result-format json --suite-names sophos_service_tests --target-org <TARGET_ORG> --output-dir tests_run_scratch/ --wait -1

Returns:
{
"code": 1,
"context": "Test",
"commandName": "Test",
"message": "Cannot specify code coverage with a TestRunId result",
"name": "Error",
"status": 1,
"stack": "Error: Cannot specify code coverage with a TestRunId result\n at TestService.writeResultFiles
"exitCode": 1,
"warnings": []
}

What we expect is for the apex run command with code coverage flag returns the test run id as well if the apex test execution takes longer than the wait time.

[iowillhoit]

Hey @kez-sophos, sorry for the delay on this. I was able to replicate that error. After some testing I noticed that if you don't use the --output-dir flag you do get a testRunId back when it times out.

So you could:

• Run the sf apex run test --code-coverage [...] without --wait
• Get the testRunId back
• Run with sf apex get test --test-run-id THE_ID and use the --output-dir flag

I am checking with another team about this behavior.

[git2gus]

This issue has been linked to a new work item: W-16441400

[iowillhoit]

I've got a ticket created to handle the combination of these flags better. In the meantime, use the workaround above. Thanks!