Skip to content

Add failure summary section to console output for easier debugging #1355

@tienquocbui

Description

@tienquocbui

When running a test suite with multiple failures, it can be difficult to quickly locate and review all the failures, especially when:

  • The test suite is large with many tests
  • Output is verbose with passing tests interspersed with failures
  • Failures are scattered throughout the output
  • Multiple expectations fail within the same test
    Currently, users need to scroll through potentially hundreds or thousands of lines of output to find each failure individually. This makes debugging and triaging test failures more time-consuming than necessary.

Example current output:

✓ Test "successful test 1" passed (0.001s)
✓ Test "successful test 2" passed (0.001s)
✗ Test "failing test" recorded an issue at MyTests.swift:42:5
  Expectation failed: (actual → 5) == (expected → 3)
✓ Test "successful test 3" passed (0.001s)
... (hundreds more tests)
✗ Test "another failure" recorded an issue at OtherTests.swift:89:7
  Expectation failed: (value → "wrong") == ("correct")
... (more tests)

Proposed solution:
Add a failure summary section at the end of the test run that aggregates all failures in one place for quick review. This summary would appear after the standard test results but before the final statistics.

Example proposed output:

✓ Test "successful test 1" passed (0.001s)
✓ Test "successful test 2" passed (0.001s)
✗ Test "failing test" recorded an issue at MyTests.swift:42:5
  Expectation failed: (actual → 5) == (expected → 3)
✓ Test "successful test 3" passed (0.001s)
... (hundreds more tests)

FAILED TEST SUMMARY (2 failures)

[1/2] MyModule / MyTests / failing test
  ✗ Expectation failed: (actual → 5) == (expected → 3)
  at MyTests.swift:42:5

[2/2] OtherModule / OtherTests / another failure
  ✗ Expectation failed: (value → "wrong") == ("correct")
  at OtherTests.swift:89:7

Test run completed in 10.5s (pass: xxx, fail: x, skip: x)

Related work:
This builds on the experimental AdvancedConsoleOutputRecorder work but integrates the failure summary feature directly into the existing ConsoleOutputRecorder to provide immediate value without waiting for the full hierarchical display feature.

This feature has been discussed with @stmontgomery who suggested bringing the failure summary to the existing console recorder rather than waiting for the experimental advanced recorder to be fully ready. The goal is to provide immediate value to users struggling to find failures in large test suites.

Metadata

Metadata

Assignees

Labels

command-line experience⌨️ enhancements to the command line interfaceenhancementNew feature or request

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions