Apex Testing: test-result.md report, "Poorly Covered Tests" section - purpose unclear; allow removing the section via config

[AndrewStopchenko-SO]

Problem 1 - "Poorly Covered Tests" section ("Test Quality Warnings" sub-header), struggling to understand the coverage intent (action to improve?)

When working in a project with many Apex classes and Apex Test classes, the "Poorly Covered Tests" section does not make much sense, because it displays a per-test-method coverage % across total apex class lines count, and the section is quite noisy - e.g. in one of the projects I received 573 poorly covered tests, but I have no idea how this information helps me or what it suggests me to do about it.

How this works now

Out of curiosity, I created a small Apex class with 3 dummy methods, and a Test class to invoke each of those methods.

Sample Apex class:

public with sharing class Coverage1 {

    public static Account[] method1() {
        Account[] recs = new Account[] {};
        recs.add(new Account());
        recs.add(new Account());
        recs.add(new Account());
        return recs;
    }

    public static Contact[] method2() {
        Contact[] recs = new Contact[] {};
        recs.add(new Contact());
        recs.add(new Contact());
        return recs;
    }

    public static Task[] method3() {
        Task[] recs = new Task[] {};
        recs.add(new Task());
        recs.add(new Task());
        recs.add(new Task());
        recs.add(new Task());
        recs.add(new Task());
        return recs;
    }
}

Sample Apex Test class:

@isTest
private class Coverage1_Test {

    @isTest
    static void test_method1() {
        Assert.isNotNull(Coverage1.method1());
    }

    @isTest
    static void test_method2() {
        Assert.isNotNull(Coverage1.method2());
    }

    @isTest
    static void test_method3() {
        Assert.isNotNull(Coverage1.method3());
    }
}

Then, I ran all tests in the Coverage1_Test. The resulting test-result-{testRunId}.md file preview shows the following:

So, the "Poorly Covered Tests" section shows each test method of the test class, and the coverage % next is what appears to be the formula:

coveragePercent = testMethodCoveredLinesCount / classTotalLinesCount

Okay, but - what does this coverage % value tell me about my unit tests, or how to improve them? If the low coverage % of a single test method implies that the test method is "bad" (though, I tend to disagree with this), how should one fix that?

Could be opinionated, but I usually write Apex test methods so that a testMethod focuses on validating the logic and edge cases of a single being-tested class method. Sometimes, even multiple testMethods may aim at covering some edge cases or "highload" scenarios for a single being-tested class method. But with the coveragePercent statistic above, it suggests that the testMethod should aim at covering as many tested class's lines as possible - the better the coverage % will be then, right?

In its current form, this section's information looks excessive and redundant; it grows exponentially as we include more Apex classes, or if we include more ApexTests in the test run.

Suggestions

1. If possible, can this section be optionally rendered, e.g. using some configuration flag?
2. Alternatively, can this section be updated to show some actionable information instead?
   • a nice option might be to list here poorly-covered classes only (<75%). Yes, this information is currently available in the "Code Coverage by Class" section, but, that section is the last in this report (we have to scroll past all other sections to see that), and it lists all Apex classes (if I'm not mistaken ordered by coverage% ascending), including the properly-covered ones too.
     • If adjusted to only show the poorly-covered Apex classes, it might be a good idea to sort them by ClassName (as opposed to "Code Coverage by Class" sorted by coverage %), making it faster & easier to scan diagonally.

Problem 2 - "Test Results with Coverage" section, ClassBeingTested shows TestClassName, not the being-tested ApexClassName (as expected).

Please pay attention to the "Test Results with Coverage" section output.

I was hoping that this section provides a breakdown of the coverage % for some specific being-tested ApexClassName as it is achieved by some particular testClass-testMethod test method's execution (e.g. if some ClassA method1, method2, method3 were invoked by ClassATest.testMethod1() and ClassATest.testMethod2() methods, we'd know how those individual ClassATest methods contributed to that ClassA coverage).

So, my understanding was that this section tries to reflect the functionality of DevConsole, where we:

• run tests
• open some individual tested Class
• in the upper-left corner, select some coverage dropdown option - overall coverage % by all tests; coverage % by individual testClass-testMethod runs.

At least that was my (apparently wrong) understanding, when looking at the report for test run in a real-world project (500+ tests), where the resulting data table is really long and "data-intensive".

But in this single-test-class example above, the "Class Being Tested" table's column shows ... the TestClassName values.

But, why?.. Honestly, I struggle to understand what exactly the number means here - like, out of totalCoveredLinesByTestClass, what percentage of them were covered thanks to some particular testMethod of that testClass?..

In other words, this seems like the same problem as Problem 1 - the table displays the same kind of coverage information as the "Poorly Covered Tests", just in a fancier way ("Pass"/"Fail" info; runtime milliseconds).

Summary

When we run unit tests, the final objective is to know the coverage of how many lines of the actual logic (ApexControllers/services/utility classes) was achieved with this test run, or what tests have failed.
Right now, we receive the coverage of the testMethod covered lines, across the TestClass total covered lines.

---

Salesforce Extension Version in VS Code: 66.3.2

Salesforce CLI Version: @salesforce/cli/2.128.5 win32-x64 node-v22.22.1

OS and version: Windows_NT x64 10.0.19045

VS Code version: 1.113.0