Skip to content

Commit e798fb8

Browse files
committed
docs: add V3 catalog architecture section to README
- Document granular catalog structure with atomic writes - Add example catalog directory structure - List key V3 benefits (no race conditions, format versioning) - Reference V3 scripts and implementation plan - Update architecture section with V3 patterns
1 parent 001aba4 commit e798fb8

File tree

1 file changed

+34
-0
lines changed

1 file changed

+34
-0
lines changed

README.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,40 @@ The "Magic" happens in the `core/` engine:
262262
3. **Templater:** Uses **Jinja2** (with auto-escape enabled) to render `.spec` files and `Dockerfiles`.
263263
4. **Publisher:** A parallelized Matrix CI builds all targets and updates the YUM repository.
264264

265+
### V3 Catalog Architecture
266+
267+
Monitoring Hub uses a **granular catalog structure** with **atomic writes** to eliminate race conditions in parallel builds:
268+
269+
```
270+
catalog/
271+
├── node_exporter/
272+
│ ├── rpm_amd64_el9.json # Atomic: 1 job = 1 file
273+
│ ├── rpm_amd64_el10.json
274+
│ ├── rpm_arm64_el9.json
275+
│ ├── deb_amd64_ubuntu-22.04.json
276+
│ ├── deb_amd64_debian-12.json
277+
│ ├── docker.json
278+
│ └── metadata.json # Aggregated (generated on-demand)
279+
└── catalog.json # Global index (backward compatible)
280+
```
281+
282+
**Key Benefits:**
283+
284+
* **No Race Conditions:** Each GitHub Actions job writes exactly 1 file - no concurrent writes to the same file
285+
* **Format Versioning:** All files have `"format_version": "3.0"` for future compatibility
286+
* **On-Demand Aggregation:** Granular files are aggregated into `metadata.json` at read-time by the portal
287+
* **Atomic Publishing:** Individual artifact failures don't corrupt the entire catalog
288+
* **Faster Rebuilds:** Only changed artifacts are regenerated
289+
290+
**V3 Scripts:**
291+
292+
* `generate_artifact_metadata.py` - Generate atomic artifact JSON files (RPM/DEB/Docker)
293+
* `aggregate_catalog_metadata.py` - Aggregate granular files into metadata.json
294+
* `publish_artifact_metadata.sh` - Publish atomic files to gh-pages
295+
* `site_generator_v2.py` - Portal generation with V3 catalog support
296+
297+
See [docs/architecture/refactoring-v2-plan.md](docs/architecture/refactoring-v2-plan.md) for complete V3 implementation details.
298+
265299
## <img src=".github/icons/package-cyan.svg" width="24" height="24" style="vertical-align: bottom;"> Distribution
266300

267301
### YUM Repository (RPM)

0 commit comments

Comments
 (0)