Skip to content

feat: download artifacts matching glob patterns after each batch#126

Draft
jsk11235 wants to merge 7 commits intomainfrom
worktree-download-files
Draft

feat: download artifacts matching glob patterns after each batch#126
jsk11235 wants to merge 7 commits intomainfrom
worktree-download-files

Conversation

@jsk11235
Copy link
Collaborator

Summary

  • Adds download_globs config field to [report] section — a list of glob patterns for files to download from sandboxes after each batch execution
  • After each batch, runs find -path in the sandbox to resolve globs (RT1), then downloads all matched files in a single sandbox.download() call (RT2)
  • Downloaded files stored at {output_dir}/{sandbox_id}/{batch_id}/ preserving relative directory structure
  • Best-effort: download failures are logged as warnings, never fail a batch

Config example

[report]
download_globs = ["artifact-output/**", "*.xml", "coverage/*"]

Files changed

  • src/config/schema.rsdownload_globs: Vec<String> on ReportConfig with #[serde(default)]
  • src/orchestrator/runner.rsbuild_find_command(), try_download_artifacts(), new fields + builders
  • src/orchestrator/spawn.rs — wire batch_idx, download_globs, output_dir into runner
  • examples/tests/test_artifacts.py — pytest tests that produce JSON, log, CSV, XML artifacts
  • offload-pytest-artifacts.toml — e2e config exercising the feature

Test plan

  • cargo fmt --check passes
  • cargo clippy --all-targets --all-features passes
  • cargo nextest run passes (134 tests including 6 new)
  • E2E: ./target/release/offload -c offload-pytest-artifacts.toml --verbose run downloads 4 artifact files per sandbox to test-results-artifacts/{sandbox_id}/{batch_id}/artifact-output/
  • Test on Modal provider (remote sandbox)

🤖 Generated with Claude Code

jacobkirmayer-imbue and others added 7 commits March 20, 2026 10:45
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add try_download_artifacts() to TestRunner which:
1. Runs `find -path` in the sandbox to resolve configured glob patterns
2. Downloads all matched files in a single sandbox.download() call
3. Stores files under {output_dir}/{sandbox_id}/{batch_id}/

This is best-effort — download failures are logged but never fail a batch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants