Different test order #701
Description
Python Version
3.12.11
pytest Version
8.4.2
Package Version
4.0.1
Description
Hey!
I have a strange thing going on (or I don't quite understand how this plugin works).
In github CI I have this order of tests
============================= test session starts ==============================
platform linux -- Python 3.12.11, pytest-8.4.2, pluggy-1.6.0
Using --randomly-seed=439041728
django: version: 5.2.6, settings: server.settings (from ini)
rootdir: /code
configfile: setup.cfg
plugins: anyio-4.11.0, randomly-4.0.1, Faker-10.0.0, django-test-migrations-1.5.0, timeout-2.4.0, django-4.11.1, cov-7.0.0, django-fakery-4.1.3, httpx-0.35.0, mock-3.15.1, hypothesis-6.140.2
timeout: 10.0s
timeout method: signal
timeout func_only: False
collected 93 items
tests/test_server/test_urls.py ....... [ 7%]
tests/test_apps/test_tilda/test_repos/test_project.py .. [ 9%]
tests/test_server/test_admin.py .................................... [ 48%]
tests/test_apps/test_tgbot/test_entrypoints/test_curator.py ........... [ 60%]
server/apps/tgbot/states/helper.py . [ 61%]
tests/test_apps/test_tgbot/test_storage.py .......... [ 72%]
tests/test_server/test_common/test_transaction_atomic.py . [ 73%]
tests/test_apps/test_tgbot/test_models.py . [ 74%]
tests/test_apps/test_tilda/test_repos/test_project_participant.py ..... [ 79%]
tests/test_apps/test_llm/test_http.py . [ 80%]
tests/test_apps/test_tilda/test_migrations/test_project_data.py .. [ 82%]
tests/test_apps/test_tgbot/test_views.py . [ 83%]
tests/test_apps/test_notificate/test_service.py FF..F. [ 90%]
tests/test_apps/test_tilda/test_models.py ... [ 93%]
tests/test_apps/test_tgbot/test_migrations/test_statemessage_data.py .. [ 95%]
tests/test_server/test_logging.py . [ 96%]
tests/test_apps/test_tgbot/test_repos.py ... [100%]
I run the same tests locally, in the same docker container with the same key, and get a different test order.
================================ test session starts =================================
platform linux -- Python 3.12.11, pytest-8.4.2, pluggy-1.6.0
django: version: 5.2.6, settings: server.settings (from ini)
Using --randomly-seed=439041728
rootdir: /code
configfile: setup.cfg
plugins: anyio-4.11.0, Faker-10.0.0, django-fakery-4.1.3, django-test-migrations-1.5.0, hypothesis-6.140.2, cov-7.0.0, django-4.11.1, mock-3.15.1, timeout-2.4.0, randomly-4.0.1, httpx-0.35.0
timeout: 10.0s
timeout method: signal
timeout func_only: False
collected 93 items
tests/test_server/test_urls.py ... [ 3%]
tests/test_apps/test_tilda/test_repos/test_project.py .. [ 5%]
tests/test_server/test_admin.py .................................... [ 44%]
tests/test_apps/test_tgbot/test_entrypoints/test_curator.py ........... [ 55%]
tests/test_apps/test_tgbot/test_storage.py .......... [ 66%]
tests/test_apps/test_tgbot/test_models.py . [ 67%]
tests/test_apps/test_tilda/test_repos/test_project_participant.py ..... [ 73%]
tests/test_apps/test_tgbot/test_views.py . [ 74%]
tests/test_apps/test_notificate/test_service.py ...... [ 80%]
tests/test_apps/test_tilda/test_models.py ... [ 83%]
tests/test_apps/test_tgbot/test_repos.py ... [ 87%]
tests/test_apps/test_tilda/test_migrations/test_project_data.py .. [ 89%]
tests/test_apps/test_tgbot/test_migrations/test_statemessage_data.py .. [ 91%]
tests/test_server/test_urls.py .... [ 95%]
server/apps/tgbot/states/helper.py . [ 96%]
tests/test_server/test_common/test_transaction_atomic.py . [ 97%]
tests/test_apps/test_llm/test_http.py . [ 98%]
tests/test_server/test_logging.py . [100%]
Is this the norm or a bug?