🔨⬆️ Add support for batch-updating dependencies by prefix (e.g., pytest*) + ✅ tests using uvloop

[pcrespov]

What do these changes do?

• 🔨 Introduced a new recipe to update dependencies whose names start with a given prefix.
  • Useful for batch-updating related packages (e.g., all pytest* or pydantic* libraries).
• ⬆️✅ Usage & upgrade:
  • Navigate to requirements/tools and run make help for instructions.
  • ⬆️ Example executed in this PR: Upgrade all pytest* libraries across the repo using:

    make req-all startswith=pytest

  • ✅ fixes removal of event_loop in pytest-asyncio. Followed migration instructions.
• Screenshot:
  
• ✅ tests using uvloop where used in production

Changes

In the tables below, you can see that all pytest* library versions have been unified across the entire repository. Additionally, some updates were made to celery-library to correct previously incorrect requirements.

• #packages before ~ 34
• #packages after ~ 36

#	name	before	after	upgrade	count	packages
1	aiohttp	3.12.12	3.12.13		1	tests🧪
2	aiosignal	1.3.2	1.4.0	minor	1	tests🧪
3	attrs	25.1.0	25.3.0	minor	1	tests🧪
4	certifi	2025.1.31	2025.6.15	minor	1	tests🧪
5	charset-normalizer	3.4.1	3.4.2		1	tests🧪
6	coverage	7.6.12	7.9.2	minor	1	tests🧪
7	frozenlist	1.5.0	1.7.0	minor	1	tests🧪
8	iniconfig	2.0.0	2.1.0	minor	1	tests🧪
9	jsonschema	4.23.0	4.24.0	minor	1	tests🧪
10	jsonschema-specifications	2024.10.1	2025.4.1	MAJOR	1	tests🧪
11	lazy-object-proxy	1.10.0	1.11.0	minor	1	tests🧪
12	more-itertools	10.6.0	10.7.0	minor	1	tests🧪
13	multidict	6.1.0	6.6.3	minor	1	tests🧪
14	openapi-core	0.19.4	0.19.5		1	tests🧪
15	openapi-spec-validator	0.7.1	0.7.2		1	tests🧪
16	packaging	24.2	25.0	MAJOR	1	tests🧪
17	pluggy	1.5.0	1.6.0	minor	1	tests🧪
18	propcache	0.3.0	0.3.2		1	tests🧪
19	psutil	6.1.1	7.0.0	MAJOR	2	storage⬆️🧪
20	pygments	2.19.1	2.18.0,2.19.2,2.17.2,2.19.1		2	models-library🔧 public-api🧪
21	pytest	8.3.5	8.4.1	minor	37	agent🧪 api-server🧪 autoscaling🧪 aws-library🧪 catalog🧪 celery-library🧪 clusters-keeper🧪 common-library🧪 dask-sidecar🧪 dask-task-models-library🧪 datcore-adapter🧪 director-v2🧪 director🧪 docker-api-proxy🧪 dynamic-scheduler🧪 dynamic-sidecar🧪 e2e-playwright🧪🧪 environment-setup🧪 invitations🧪 migration🧪 models-library🧪 notifications-library🧪 notifications🧪 payments🧪 postgres-database🧪 public-api🧪 resource-usage-tracker🧪 service-integration🧪🧪 service-library🧪 settings-library🧪 simcore-sdk🧪 storage🧪 swarm-deploy🧪 tests🧪 web🧪
22	pytest-asyncio	0.26.0	1.0.0	MAJOR	32	agent🧪 api-server🧪 autoscaling🧪 aws-library🧪 catalog🧪 celery-library🧪 clusters-keeper🧪 common-library🧪 dask-sidecar🧪 dask-task-models-library🧪 datcore-adapter🧪 director-v2🧪 director🧪 docker-api-proxy🧪 dynamic-scheduler🧪 dynamic-sidecar🧪 environment-setup🧪 invitations🧪 migration🧪 models-library🧪 notifications-library🧪 notifications🧪 payments🧪 postgres-database🧪 public-api🧪 resource-usage-tracker🧪 service-library🧪 simcore-sdk🧪 storage🧪 swarm-deploy🧪 tests🧪 web🧪
23	pytest-celery	1.1.3	1.2.0	minor	1	storage🧪
24	pytest-cov	6.0.0, 6.1.1	6.2.1	minor	31	agent🧪 api-server🧪 autoscaling🧪 aws-library🧪 catalog🧪 celery-library🧪 clusters-keeper🧪 common-library🧪 dask-sidecar🧪 dask-task-models-library🧪 datcore-adapter🧪 director-v2🧪 director🧪 docker-api-proxy🧪 dynamic-scheduler🧪 dynamic-sidecar🧪 invitations🧪 migration🧪 models-library🧪 notifications-library🧪 notifications🧪 payments🧪 postgres-database🧪 resource-usage-tracker🧪 service-integration🧪 service-library🧪 settings-library🧪 simcore-sdk🧪 storage🧪 tests🧪 web🧪
25	pytest-docker	3.2.0, 3.2.1	3.2.3		8	api-server🧪 catalog🧪 director-v2🧪 director🧪 migration🧪 postgres-database🧪 service-library🧪 web🧪
26	pytest-docker-tools	3.1.3	3.1.9		1	storage🧪
27	pytest-mock	3.14.0	3.14.1		28	agent🧪 api-server🧪 autoscaling🧪 aws-library🧪 catalog🧪 celery-library🧪 clusters-keeper🧪 common-library🧪 dask-sidecar🧪 dask-task-models-library🧪 datcore-adapter🧪 director-v2🧪 director🧪 docker-api-proxy🧪 dynamic-scheduler🧪 dynamic-sidecar🧪 migration🧪 models-library🧪 notifications-library🧪 notifications🧪 payments🧪 resource-usage-tracker🧪 service-library🧪 settings-library🧪 simcore-sdk🧪 storage🧪 swarm-deploy🧪 web🧪
28	pytest-xdist	3.6.1	3.8.0	minor	5	datcore-adapter🧪 director-v2🧪 service-library🧪 simcore-sdk🧪 web🧪
29	requests	2.32.3	2.32.4		1	tests🧪
30	rpds-py	0.23.1	0.26.0	minor	1	tests🧪
31	termcolor	2.5.0	3.1.0	MAJOR	1	tests🧪
32	urllib3	2.3.0	2.5.0	minor	1	tests🧪
33	werkzeug	3.1.3	3.1.1	🔥 downgrade	1	tests🧪
34	yarl	1.18.3	1.20.1	minor	1	tests🧪

Legend:

• ⬆️ base dependency (only services because packages are floating)
• 🧪 test dependency
• 🔧 tool dependency

Repo-wide overview of libraries

• #reqs files parsed: 107

#	name	versions-base	versions-test	versions-tool
260	pytest	8.4.1	8.4.1
261	pytest-aiohttp		1.1.0
262	pytest-asyncio		1.0.0
263	pytest-base-url		2.1.0
264	pytest-benchmark		5.1.0
265	pytest-celery		1.2.0
266	pytest-cov		6.2.1
267	pytest-docker		3.2.3
268	pytest-docker-tools		3.1.9
269	pytest-html		4.1.1
270	pytest-icdiff		0.9
271	pytest-instafail		0.5.0
272	pytest-localftpserver		1.3.2
273	pytest-metadata		3.1.1
274	pytest-mock		3.14.1
275	pytest-playwright		0.7.0
276	pytest-runner		6.0.1
277	pytest-sugar		1.0.0
278	pytest-xdist		3.8.0

Related issue/s

• Maintenance in sync with @sanderegg
• Part of the split from ⬆️ 🔒️🔨 Maintenance: repo-wide test/tools upgrades and new batch-upgrades scripts for otel and pydantic #7973

How to test

cd requirements/tools
make shell
$ make help
...
$ make -n reqs startswith=pytest

Dev-ops

None

[matusdrobuliak66]

nice improvement thanks!

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.25%. Comparing base (f47c538) to head (31ed953).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8014      +/-   ##
==========================================
+ Coverage   87.90%   88.25%   +0.35%     
==========================================
  Files        1855     1855              
  Lines       71517    71517              
  Branches     1258     1258              
==========================================
+ Hits        62865    63117     +252     
+ Misses       8288     8036     -252     
  Partials      364      364              

Flag	Coverage Δ
integrationtests	64.25% <ø> (+0.08%)	⬆️
unittests	86.87% <100.00%> (+0.32%)	⬆️

Components	Coverage Δ
api	76.84% <ø> (ø)
pkg_aws_library	93.93% <ø> (ø)
pkg_celery_library	87.15% <ø> (ø)
pkg_dask_task_models_library	79.62% <ø> (ø)
pkg_models_library	93.27% <ø> (ø)
pkg_notifications_library	85.26% <ø> (ø)
pkg_postgres_database	88.19% <ø> (ø)
pkg_service_integration	69.92% <ø> (ø)
pkg_service_library	71.37% <100.00%> (-0.25%)	⬇️
pkg_settings_library	90.64% <ø> (ø)
pkg_simcore_sdk	85.05% <ø> (+0.05%)	⬆️
agent	96.29% <ø> (ø)
api_server	92.64% <ø> (ø)
autoscaling	96.03% <ø> (ø)
catalog	92.58% <ø> (+0.37%)	⬆️
clusters_keeper	99.13% <ø> (ø)
dask_sidecar	92.35% <ø> (ø)
datcore_adapter	97.94% <ø> (ø)
director	76.91% <ø> (+0.09%)	⬆️
director_v2	90.95% <ø> (ø)
dynamic_scheduler	96.69% <ø> (ø)
dynamic_sidecar	90.09% <ø> (ø)
efs_guardian	89.65% <ø> (ø)
invitations	93.60% <ø> (ø)
payments	92.57% <ø> (ø)
resource_usage_tracker	92.52% <ø> (+3.52%)	⬆️
storage	86.78% <ø> (+0.50%)	⬆️
webclient	∅ <ø> (∅)
webserver	88.63% <ø> (+0.87%)	⬆️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f47c538...31ed953. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pcrespov]

@giancarloromeo can you please folow up and investigate why we have two versions of pytest-celery?

[giancarloromeo]

@giancarloromeo can you please folow up and investigate why we have two versions of pytest-celery?

This seems the reason why pytest-celery remained to the previous in storage compared to the one in celery-library:

services/storage/requirements on  add-notifications-service [!?] via 🐍 v3.11.12 (.venv) 
❯ make reqs upgrade=pytest-celery==1.2.0 
cd ..; \
uv pip compile --upgrade-package "pytest-celery==1.2.0" \
        --no-header \
        --output-file requirements/_test.txt requirements/_test.in
  × No solution found when resolving dependencies:
  ╰─▶ Because pytest-celery==1.2.0 depends on psutil>=7.0.0 and psutil==6.1.1, we can conclude that pytest-celery==1.2.0 cannot be
      used.
      And because you require pytest-celery==1.2.0, we can conclude that your requirements are unsatisfiable.

[sanderegg]

Thanks!

[bisgaard-itis]

Nice! Thanks a lot for the effort

[pcrespov]

@mergify queue

[mergify]

queue

🟠 Waiting for conditions to match

• -closed [📌 queue requirement]
• any of: [🔀 queue conditions]
  • all of: [📌 queue conditions of queue default]
    • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
    • #approved-reviews-by >= 2 [🛡 GitHub branch protection]
    • #approved-reviews-by>=2
    • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
    • #changes-requested-reviews-by=0
    • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
    • #review-threads-unresolved=0
    • -conflict
    • -draft
    • base=master
    • label!=🤖-do-not-merge
    • label=🤖-automerge
    • any of: [🛡 GitHub branch protection]
      • check-skipped = deploy to dockerhub
      • check-neutral = deploy to dockerhub
      • check-success = deploy to dockerhub
    • any of: [🛡 GitHub branch protection]
      • check-success = system-tests
      • check-neutral = system-tests
      • check-skipped = system-tests
    • any of: [🛡 GitHub branch protection]
      • check-success = unit-tests
      • check-neutral = unit-tests
      • check-skipped = unit-tests
    • any of: [🛡 GitHub branch protection]
      • check-success = check OAS' are up to date
      • check-neutral = check OAS' are up to date
      • check-skipped = check OAS' are up to date
    • any of: [🛡 GitHub branch protection]
      • check-success = integration-tests
      • check-neutral = integration-tests
      • check-skipped = integration-tests
    • any of: [🛡 GitHub branch protection]
      • check-success = build-test-images (frontend) / build-test-images
      • check-neutral = build-test-images (frontend) / build-test-images
      • check-skipped = build-test-images (frontend) / build-test-images
• -conflict [📌 queue requirement]
• -draft [📌 queue requirement]
• any of: [📌 queue -> configuration change requirements]
  • -mergify-configuration-changed
  • check-success = Configuration changed

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud