Skip to content

Conversation

@paultiq
Copy link
Owner

@paultiq paultiq commented Sep 19, 2025

This arose from [discussion]:

CI is currently running tests serially in a deterministic order, with some order dependencies due to lack of isolation of both the default connection & test files.

This PR improves the tests by:

  • Randomizing their order: to surface any order dependencies
  • Enabling parallel tests: to reduce total test time
  • Using unique path fixtures or table names to eliminate conflicts across concurrent tests
  • misc: move one slow test case to tests/slow, and rework test_query_interruption.

Added Plugins

pytest-xdist

Disabled by default. This PR adds -n 2 to packaging_wheels to run two parallel tests at a time.

Comments:

  • GH Runners (often) have 4 cores, and I've had luck with -n auto to let xdist use the number of processors
  • Since DuckDB will parallelize the work, this will increase CPU contention, especially for performance intensive tests. There may be some tests where parallelizing is not desired - for those, the solution is to run tests in two steps - once with xdist once without.
pytest-randomly

Enabled by default. Randomizes the order of tests.

Comments:

  • Random order is good for surfacing test dependencies and assumptions
  • This identified a number of test dependencies, especially around use of a "dirty" default connection or reusing file names

@paultiq paultiq closed this Sep 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant