Skip to content

Conversation

adamchalmers
Copy link
Contributor

@adamchalmers adamchalmers commented Jul 23, 2025

The CLI tests in test_main were annoying so here's some problems and fixes:

  • Before this PR, a single failing test would stop all the other tests. So you'd have to keep rerunning to fix a lot of little failures.
    • Fix: Now instead it'll run all tests, collecting failures and printing them all at the end.
  • Before this PR, if the 39th test failed, rerunning the CLI tests would make you run all the preceding 38 tests before failing. Which made rerunning to see if you fixed the test quite slow.
    • Fix: Allow ZOO_CLI_TEST_NAME var. If it's set, only tests whose name contain that value will be run. This means if you know

I tried to just split the test up into many tests, which would have let us use nextest to get all these features. But unfortunately I wasn't able to get it to work. Here's my attempt.

@adamchalmers adamchalmers requested a review from jessfraz as a code owner July 23, 2025 21:53
@adamchalmers adamchalmers marked this pull request as draft July 23, 2025 21:53
Copy link

codecov bot commented Jul 23, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 64.23%. Comparing base (653fc32) to head (7930cf5).
Report is 2 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1305       +/-   ##
===========================================
- Coverage   74.82%   64.23%   -10.59%     
===========================================
  Files          37       37               
  Lines        5629     5629               
===========================================
- Hits         4212     3616      -596     
- Misses       1417     2013      +596     
Flag Coverage Δ
unittests 64.23% <ø> (-10.59%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@adamchalmers adamchalmers force-pushed the achalmers/different-test-style branch from 3db3d39 to 7930cf5 Compare July 23, 2025 22:07
@adamchalmers adamchalmers marked this pull request as ready for review July 23, 2025 22:09
@adamchalmers adamchalmers merged commit 6406b66 into main Jul 23, 2025
15 of 16 checks passed
@adamchalmers adamchalmers deleted the achalmers/different-test-style branch July 23, 2025 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants