Skip to content

Commit 12da30f

Browse files
author
Eidmantas Ivanauskas
committed
docs: add engineering notes and new flags (scheduler, batching, registry retries)
1 parent 7f1e72d commit 12da30f

File tree

1 file changed

+28
-0
lines changed

1 file changed

+28
-0
lines changed

README.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,3 +111,31 @@ see how we can make this happen.
111111

112112
There is [an open proposal](https://github.com/argoproj/argo-cd/issues/7385) to migrate this project into the `argoproj` org (out
113113
of the `argoproj-labs` org) and include it in the installation of Argo CD.
114+
115+
## Engineering notes (recent changes)
116+
117+
- Registry client hardening
118+
- HTTP transport reuse per registry with sane timeouts (keep-alives, capped TLS and response header timeouts) to cut connection churn under load.
119+
- Singleflight-style deduplication and jittered retries (tags, manifests) with per-attempt deadlines to avoid thundering herds and reduce /jwt/auth pressure.
120+
121+
- Git write-back throughput
122+
- Per-repo serialization to eliminate races in monorepos, plus a batched writer that coalesces multiple intents into a single commit/push per repo/branch.
123+
- Multi-branch grouping: intents for different write branches never mix; each branch flushes independently.
124+
- Logs reflect queued writes: look for "Queuing N parameter update(s) … (git write pending)" and the subsequent commit/push logs.
125+
126+
- Scheduling and fairness
127+
- Optional scheduler flags to prioritize apps: `--schedule` (default|lru|fail-first), `--cooldown` (deprioritize recently successful apps), and `--per-repo-cap` (cap updates per repo per cycle).
128+
- Goal: prevent a hot monorepo from starving others while keeping high concurrency.
129+
130+
- Operational guidance
131+
- Concurrency: set `--max-concurrency` roughly ≥ number of active repos; monorepos serialize on their writer, others proceed in parallel.
132+
- Registry RPS: tune `limit` in `registries.conf` (e.g., 30–50 RPS) and monitor latency/429s.
133+
- Monorepos: prefer per-app write branches or rely on batching to reduce fetch/commit/push churn.
134+
135+
Flags added
136+
- `--schedule` (env: IMAGE_UPDATER_SCHEDULE): default|lru|fail-first
137+
- `--cooldown` (env: IMAGE_UPDATER_COOLDOWN): duration (e.g., 30s)
138+
- `--per-repo-cap` (env: IMAGE_UPDATER_PER_REPO_CAP): integer (0 = unlimited)
139+
140+
Notes
141+
- For tests or legacy behavior, set `GIT_BATCH_DISABLE=true` to perform immediate (non-batched) write-back.

0 commit comments

Comments
 (0)