build: Optimize fuzzer artifact upload/download in scheduled.yml#16767
Open
pratikpugalia wants to merge 1 commit intofacebookincubator:mainfrom
Open
build: Optimize fuzzer artifact upload/download in scheduled.yml#16767pratikpugalia wants to merge 1 commit intofacebookincubator:mainfrom
pratikpugalia wants to merge 1 commit intofacebookincubator:mainfrom
Conversation
Summary: The compile job in `scheduled.yml` builds 14 fuzzer binaries and uploads each as a separate artifact (~500-700 MB unstripped debug C++ binaries). This results in 14 sequential `upload-artifact` calls totaling ~15-30 min of upload time, plus redundant download overhead across 19 downstream fuzzer jobs. This diff optimizes artifact handling in two ways: 1. **Strip debug symbols**: After the build, `strip -o` copies each binary to a staging directory without debug info, reducing individual binary sizes from ~500-700 MB to ~50-70 MB each (~1 GB total). 2. **Bundle into a single artifact**: Replaces 14 individual `upload-artifact` steps with one upload of the staging directory, and updates all 19 downstream fuzzer jobs to download this single `fuzzer-binaries` artifact. Estimated impact: - Upload time: ~15-30 min → ~2-3 min (1 upload vs 14 sequential) - Download size per fuzzer job: ~500-700 MB → ~1 GB shared bundle (stripped) - Net wall-clock savings: ~15-25 min per workflow run The `signatures` artifact is unaffected. All binary filenames used by `chmod +x` and execution steps remain unchanged. Differential Revision: D96555567
meta-cla
bot
added
the
CLA Signed
|
@pratikpugalia has exported this pull request. If you are a Meta employee, you can view the originating Diff in D96555567. |
✅ Deploy Preview for meta-velox canceled.
|
pratikpugalia
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
The compile job in
scheduled.ymlbuilds 14 fuzzer binaries and uploads each asa separate artifact (~500-700 MB unstripped debug C++ binaries). This results in
14 sequential
upload-artifactcalls totaling ~15-30 min of upload time, plusredundant download overhead across 19 downstream fuzzer jobs.
This diff optimizes artifact handling in two ways:
Strip debug symbols: After the build,
strip -ocopies each binary to astaging directory without debug info, reducing individual binary sizes from
~500-700 MB to ~50-70 MB each (~1 GB total).
Bundle into a single artifact: Replaces 14 individual
upload-artifactsteps with one upload of the staging directory, and updates all 19 downstream
fuzzer jobs to download this single
fuzzer-binariesartifact.Estimated impact:
The
signaturesartifact is unaffected. All binary filenames used bychmod +xand execution steps remain unchanged.
Differential Revision: D96555567