Skip to content

indexer: Push image saved from built container#13441

Merged
oliverchang merged 13 commits intomasterfrom
save-indexer-image
Jun 20, 2025
Merged

indexer: Push image saved from built container#13441
oliverchang merged 13 commits intomasterfrom
save-indexer-image

Conversation

@oliverchang
Copy link
Copy Markdown
Collaborator

No description provided.

@oliverchang
Copy link
Copy Markdown
Collaborator Author

/gcbrun trial_build.py expat --fuzzing-engines libfuzzer --sanitizers indexer

1 similar comment
@oliverchang
Copy link
Copy Markdown
Collaborator Author

/gcbrun trial_build.py expat --fuzzing-engines libfuzzer --sanitizers indexer

@oliverchang
Copy link
Copy Markdown
Collaborator Author

@jonathanmetzman this duplicates with the pushing you added recently, so I removed it for now. WDYT?

Comment thread infra/build/functions/build_project.py Outdated


def _indexer_built_image_name(name: str):
return f'us-central1-docker.pkg.dev/oss-fuzz/indexer/{name}'
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm.. I wonder if we can use docker image save and docker image load to upload a tar to GCS (using signed URLs) instead of pushing, which comes with the pre-existing problems.

Need to confirm that this works as expected though and saves env vars + entrypoints.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've thought of this approach as well. I would bet it works but has some slightly worse performance in downloading and uploading.
I think it would be a good idea to make a cloud function that triggers on write to the bucket that can upload it to the right docker registry. This way the images can be used as normal.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea! I've added a TODO here.

@jonathanmetzman
Copy link
Copy Markdown
Contributor

@jonathanmetzman this duplicates with the pushing you added recently, so I removed it for now. WDYT?

It seems fine as use of the images has been deprioritized for now. But I wonder if this technique is inferior for reproducing historical bugs? My (slight) preference is to do both.

@oliverchang
Copy link
Copy Markdown
Collaborator Author

oliverchang commented Jun 17, 2025

TODO for myself:

  • Ensure we save the same cflags etc used during indexing so the same ones are used for recompiling (when we use replay_build.sh).
  • Make it easy to regenerate indexes (from existing CDB).
  • Inject default replay_build.sh if it doesn't exist.

@oliverchang oliverchang requested review from DonggeLiu and removed request for jonathanmetzman June 17, 2025 10:21
@oliverchang
Copy link
Copy Markdown
Collaborator Author

/gcbrun trial_build.py expat skcms --fuzzing-engines libfuzzer --sanitizers indexer

@oliverchang
Copy link
Copy Markdown
Collaborator Author

/gcbrun trial_build.py c c++ --fuzzing-engines libfuzzer --sanitizers indexer

@oliverchang
Copy link
Copy Markdown
Collaborator Author

/gcbrun trial_build.py c c++ --fuzzing-engines libfuzzer --sanitizers indexer

@oliverchang oliverchang merged commit 20a387d into master Jun 20, 2025
18 of 19 checks passed
@oliverchang oliverchang deleted the save-indexer-image branch June 20, 2025 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants