feat: route UploadFinisher to dedicated queue

[drazisil-codecov]

Summary

This PR separates UploadFinisher tasks from UploadProcessor tasks to prevent lightweight finishing operations from being blocked by heavy processing work.

Problem

During high upload volume, UploadFinisher tasks were getting queued behind UploadProcessor tasks, causing delays of up to 19 minutes for notification delivery. This is problematic because:

• UploadProcessor tasks are heavy (parsing coverage files, can take minutes)
• UploadFinisher tasks are lightweight (mark upload complete, trigger notifications)
• Both were sharing the same queue (uploads / enterprise_uploads)

Solution

Add an explicit route for upload_finisher_task_name that:

• Routes to a dedicated upload_finisher queue when configured
• Falls back to the uploads queue for backward compatibility (no config = no change)
• Enterprise tasks route to enterprise_upload_finisher via the existing router

Key Changes

# Get the upload queue for backward-compatible fallback
_upload_queue = get_config("setup", "tasks", "upload", "queue", default=task_default_queue)

# UploadFinisher gets its own queue
upload_finisher_task_name: {
    "queue": get_config("setup", "tasks", "upload_finisher", "queue", default=_upload_queue)
}

Deployment

This change is safe to deploy independently:

• Without k8s config: falls back to current uploads queue (no behavior change)
• With k8s config (codecov/k8s-v2#xxx): activates dedicated queue routing

Recommended deployment order:

1. Deploy this PR first (safe fallback)
2. Deploy k8s-v2 config changes (activates new routing)

Testing

• Updated test_celery_config.py to include new route key
• Added test_upload_finisher_has_separate_queue in test_router.py to verify:
  • Non-enterprise: upload_finisher → upload_finisher queue
  • Non-enterprise: upload → uploads queue
  • Enterprise: upload_finisher → enterprise_upload_finisher queue
  • Enterprise: upload → enterprise_uploads queue

---

Note

Separates lightweight UploadFinisher tasks from other upload tasks to avoid queue contention, with safe fallback behavior.

• Adds explicit route for upload_finisher_task_name in BaseCeleryConfig.task_routes; resolves queue via setup.tasks.upload_finisher.queue, falling back to the configured upload queue (_upload_queue) if unset
• Keeps other upload tasks under app.tasks.upload.* routing
• Updates unit tests:
  • test_celery_config.py: includes app.tasks.upload.UploadFinisher in expected routes
  • test_router.py: verifies non-enterprise and enterprise routing for UploadFinisher vs other upload tasks, including dedicated enterprise_upload_finisher

^{Written by Cursor Bugbot for commit cb93891. This will update automatically on new commits. Configure here.}

[sentry]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.90%. Comparing base (367d3f1) to head (cb93891).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #634   +/-   ##
=======================================
  Coverage   93.90%   93.90%           
=======================================
  Files        1286     1286           
  Lines       46802    46803    +1     
  Branches     1517     1517           
=======================================
+ Hits        43951    43952    +1     
  Misses       2542     2542           
  Partials      309      309           

Flag	Coverage Δ
apiunit	96.56% <ø> (ø)
sharedintegration	38.87% <100.00%> (+<0.01%)	⬆️
sharedunit	88.80% <100.00%> (+<0.01%)	⬆️
workerintegration	59.10% <ø> (ø)
workerunit	91.30% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[codecov-notifications]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

CodSpeed Performance Report

Merging this PR will not alter performance

_{Comparing jb/upload-finisher-separate-queue (cb93891) with main (aca0c54)¹}

Summary

✅ 9 untouched benchmarks

Footnotes

1. No successful run was found on main (367d3f1) during the generation of this report, so aca0c54 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩