-
Notifications
You must be signed in to change notification settings - Fork 46
Description
Optimize Cross Repo-Concurrency in Purge Logic
Currently, when performing a purge across multiple repositories, each repository is processed sequentially:
for repoName, tagRegex := range tagFilters {
While there is parallelism within each repo (controlled by the --concurrency flag), there is no concurrency across repositories. This limits the overall efficiency of the purge.
Why This Matters:
Read and delete operations have different costs: Read operations (e.g. fetching tags or manifests) are generally faster and less throttled than delete operations. By processing multiple repositories in parallel, we can overlap read operations in one repo with delete operations in another, significantly reducing total execution time.
This change will substantially alter the output behavior, especially in terms of timing and interleaving of logs.
It may require more granular concurrency flags to separately control intra-repo and inter-repo concurrency and maybe even read vs delete concurrency. Backward compatibility and user expectations around output format and ordering should be carefully evaluated.
Next Steps:
This issue is being filed as a follow-up to #462, which intentionally avoids this change to maintain consistent output. Once that PR is merged, we can explore this optimization in a dedicated follow-up.