[Improvement] – Add --queued flag to dvc exp push

[BALOGUN-DAVID]

[Improvement] – Add --queued flag to dvc exp push

Description

Context

Previously, dvc exp push could only push completed experiments (resolved via experiment names, revisions, or --all-commits). Queued experiments — which are stored internally as Git stash commits managed by the Celery task queue — had no mechanism to be pushed to a remote Git repository. This meant that users working across multiple machines or collaborating in teams could not transfer their pending experiment queue to a shared remote, forcing them to re-queue experiments manually on each machine.

Approach

This change introduces a --queued flag to dvc exp push that pushes all currently queued experiments to a Git remote using temporary refs. The implementation spans 4 files (+104 lines):

• dvc/repo/experiments/refs.py — Added a new CELERY_QUEUE constant (refs/exps/celery/queue) to define the ref namespace for queued experiment refs.

• dvc/repo/experiments/push.py (core logic) —

  • Added a queued: bool parameter to the push() function, which early-returns to the new _push_queued() helper when True.
  • New _push_queued() function that:
    1. Iterates all queued entries via repo.experiments.celery_queue.iter_queued()
    2. Creates temporary Git refs (refs/exps/celery/queue/<stash_rev>) for each entry's stash commit
    3. Pushes all refs to the remote using push_refspecs() with progress reporting
    4. Cleans up local temporary refs in a finally block to ensure no ref leakage
    5. Returns a {"queued": [...]} result dict with human-readable experiment names

• dvc/commands/experiments/push.py (CLI layer) —

  • Added --queued argparse flag (action="store_true")
  • When --queued is set, the command dispatches to the queued push path, displays pushed experiment names using rich-styled output, and bypasses all other push logic (cache upload, rev selection, Studio notification)

• tests/unit/command/test_experiments.py — Added test_experiments_push_queued verifying CLI argument parsing and correct forwarding to the push() function.

Impact

• Functionality: Users can now run dvc exp push <remote> --queued to push all pending queued experiments to a Git remote. Supports --force to overwrite existing remote refs.
• Developer Experience: Temporary refs are automatically cleaned up after push, preventing ref pollution in the local repository. The implementation follows the established patterns used by _push() for consistency.
• Robustness: Gracefully handles the empty-queue case ("No queued experiments to push."), and uses finally-based cleanup to guarantee temp ref removal even on push failure.

---

Tests

• [ x] Existing tests updated or refactored

Unit Tests Summary

Test Function	File	Description
test_experiments_push_queued	tests/unit/command/test_experiments.py	Verifies --queued --force CLI args are parsed correctly and forwarded to push() with queued=True, force=True. Validates return code 0 with mocked {"queued": ["exp-1", "exp-2"]} result.

Visual Evidence

Run Tests

python -m pytest tests/unit/command/test_experiments.py::test_experiments_push_queued -xvs
python -m pytest tests/unit/command/test_experiments.py -xvs

---

Diff Summary

 dvc/commands/experiments/push.py       | 25 +++++++++++++++
 dvc/repo/experiments/push.py           | 53 ++++++++++++++++++++++++++++++--
 dvc/repo/experiments/refs.py           |  1 +
 tests/unit/command/test_experiments.py | 27 ++++++++++++++++
 4 files changed, 104 insertions(+), 2 deletions(-)

• [ x] ❗ I have followed the Contributing to DVC checklist.

• 📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

[codecov]

Codecov Report

❌ Patch coverage is 98.37398% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.00%. Comparing base (2431ec6) to head (9babeba).
⚠️ Report is 194 commits behind head on main.

Files with missing lines	Patch %	Lines
dvc/repo/experiments/push.py	92.59%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #11009      +/-   ##
==========================================
+ Coverage   90.68%   91.00%   +0.32%     
==========================================
  Files         504      506       +2     
  Lines       39795    41224    +1429     
  Branches     3141     3264     +123     
==========================================
+ Hits        36087    37515    +1428     
- Misses       3042     3070      +28     
+ Partials      666      639      -27     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[BALOGUN-DAVID]

Hi @skshetry, I’ve just opened this PR to address #10876 . When you have a moment, I would really appreciate your review and feedback on the implementation. Thank you!