Should we have a higher number in the pytest -n auto ? auto currently assigns to 4 workers

[ryx2]

Question

The 'all-extras' tests take around 2mins and have ~2000 tests inside. I think We should pursue ~12-20 workers to speed this up since the compute required for each is so low.

Looking at your situation with ~2000 pytest jobs that are each lightweight, you're right that -n auto (which gives you 4 workers on GitHub Actions) is likely suboptimal. Here's how to think about the optimal number:

Key Factors for Optimization

1. GitHub Actions typically provides 2-core machines for standard runners (4 logical cores with hyperthreading)
2. Your tests are I/O or setup bound (minimal compute) - this means you can oversubscribe CPU cores
3. Overhead costs: Each worker has startup time and memory overhead

Optimal Range

For lightweight tests on GitHub Actions, the sweet spot is usually between 8-20 workers. Here's why:

• 8-12 workers: Conservative oversubscription (2-3x physical cores)
• 16-20 workers: Aggressive but often effective for I/O-bound tests
• Beyond 20: Diminishing returns due to coordination overhead

Testing Strategy

I'd recommend benchmarking with these values:

# In your GitHub Actions workflow
- name: Run tests
  run: pytest -n 12  # Start here

Or make it configurable:

env:
  PYTEST_WORKERS: 12  # Easy to adjust

steps:
  - run: pytest -n ${{ env.PYTEST_WORKERS }}

Quick Optimization Test

Try running locally with different values:

time pytest -n 4   # Current baseline
time pytest -n 8
time pytest -n 12
time pytest -n 16
time pytest -n 20

Additional Optimizations

Looking at the pydantic-ai repo, you might also consider:

1. pytest-split: Distribute tests across multiple GitHub Actions jobs running in parallel
2. Test ordering: Run fast tests first with pytest-order or pytest-randomly
3. Caching: Make sure you're caching pip dependencies and any test fixtures

For your specific case with minimal compute per test, I'd start with -n 12 and adjust based on results. You should see significant improvement over the current 4 workers, potentially cutting your time from 2 minutes to under 1 minute.

Additional Context

No response

[Kludex]

Feel free to open a PR to improve it.

[github-actions]

This issue is stale, and will be closed in 3 days if no reply is received.