Skip to content

Optimize Cross Repo-Concurrency in Purge Logic #467

@estebanreyl

Description

@estebanreyl

Optimize Cross Repo-Concurrency in Purge Logic

Currently, when performing a purge across multiple repositories, each repository is processed sequentially:

for repoName, tagRegex := range tagFilters {

While there is parallelism within each repo (controlled by the --concurrency flag), there is no concurrency across repositories. This limits the overall efficiency of the purge.

Why This Matters:

Read and delete operations have different costs: Read operations (e.g. fetching tags or manifests) are generally faster and less throttled than delete operations. By processing multiple repositories in parallel, we can overlap read operations in one repo with delete operations in another, significantly reducing total execution time.

This change will substantially alter the output behavior, especially in terms of timing and interleaving of logs.
It may require more granular concurrency flags to separately control intra-repo and inter-repo concurrency and maybe even read vs delete concurrency. Backward compatibility and user expectations around output format and ordering should be carefully evaluated.

Next Steps:

This issue is being filed as a follow-up to #462, which intentionally avoids this change to maintain consistent output. Once that PR is merged, we can explore this optimization in a dedicated follow-up.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions