Skip to content

Commit 981317d

Browse files
authored
Merge pull request #8 from techofourown/codex/catalog-index-publication-20260317
fix: publish contract-indexed catalog rows
2 parents e3c35d8 + 05b819e commit 981317d

File tree

7 files changed

+268
-1
lines changed

7 files changed

+268
-1
lines changed

.github/workflows/ci.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ jobs:
1515
- uses: oras-project/setup-oras@v1
1616
- name: Validate script syntax
1717
run: bash -n scripts/render-catalog-bundle.sh scripts/check-catalog-bundle-smoke.sh scripts/check-image-refs-exist.sh scripts/check-publish-workflow.sh
18+
- name: Validate catalog-row helper syntax
19+
run: python3 -m py_compile scripts/render-catalog-rows.py
1820
- name: Validate publish workflow invariants
1921
run: bash scripts/check-publish-workflow.sh
2022
- name: Validate and render catalog bundle

.github/workflows/publish-catalog-bundle.yml

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@ on:
99
- refresh-from-upstream-image-publish
1010
workflow_dispatch:
1111

12+
concurrency:
13+
group: publish-application-catalog-${{ github.repository }}
14+
cancel-in-progress: false
15+
1216
jobs:
1317
publish:
1418
runs-on: ubuntu-latest
@@ -26,15 +30,51 @@ jobs:
2630
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
2731
- name: Publish bundle
2832
run: |
33+
set -euo pipefail
2934
REF="ghcr.io/techofourown/sw-ourbox-catalog-hello-world:latest"
35+
CATALOG_REF="ghcr.io/techofourown/sw-ourbox-catalog-hello-world:catalog-amd64"
36+
CATALOG_ARTIFACT_TYPE="application/vnd.techofourown.ourbox.application-catalog.catalog.v1"
3037
IMMUTABLE_TAG="sha-${GITHUB_SHA}-run-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
3138
IMMUTABLE_REF="ghcr.io/techofourown/sw-ourbox-catalog-hello-world:${IMMUTABLE_TAG}"
39+
VERSION_TAG="main-${GITHUB_SHA::12}"
40+
CREATED="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
3241
oras push \
3342
--artifact-type application/vnd.techofourown.ourbox.application-catalog.v1.tar+gzip \
3443
"${IMMUTABLE_REF}" \
3544
dist/application-catalog-bundle.tar.gz:application/vnd.techofourown.ourbox.application-catalog.v1.tar+gzip
3645
DIGEST="$(oras resolve "${IMMUTABLE_REF}")"
46+
EXISTING_CATALOG=""
47+
rm -rf dist/catalog-existing
48+
if oras pull "${CATALOG_REF}" -o dist/catalog-existing >"${RUNNER_TEMP}/catalog-pull.out" 2>"${RUNNER_TEMP}/catalog-pull.err"; then
49+
EXISTING_CATALOG="$(find dist/catalog-existing -maxdepth 4 -type f -name 'catalog.tsv' | head -n 1 || true)"
50+
elif grep -Eiq 'MANIFEST_UNKNOWN|NAME_UNKNOWN|not found|404' "${RUNNER_TEMP}/catalog-pull.err"; then
51+
echo "No existing catalog index found at ${CATALOG_REF}; creating a new catalog.tsv"
52+
else
53+
echo "Failed to pull existing catalog index ${CATALOG_REF}" >&2
54+
cat "${RUNNER_TEMP}/catalog-pull.err" >&2
55+
exit 1
56+
fi
57+
python3 scripts/render-catalog-rows.py \
58+
--catalog-json catalog/catalog.json \
59+
--profile-env catalog/profile.env \
60+
--images-lock dist/images.lock.json \
61+
--existing-catalog "${EXISTING_CATALOG}" \
62+
--out-catalog dist/catalog.tsv \
63+
--channel stable \
64+
--tag "${IMMUTABLE_TAG}" \
65+
--created "${CREATED}" \
66+
--version "${VERSION_TAG}" \
67+
--revision "${GITHUB_SHA}" \
68+
--arch amd64 \
69+
--artifact-digest "${DIGEST}" \
70+
--pinned-ref "ghcr.io/techofourown/sw-ourbox-catalog-hello-world@${DIGEST}"
71+
oras push \
72+
--artifact-type "${CATALOG_ARTIFACT_TYPE}" \
73+
"${CATALOG_REF}" \
74+
dist/catalog.tsv:text/tab-separated-values
3775
oras tag "${IMMUTABLE_REF}" latest >/dev/null
76+
oras tag "${IMMUTABLE_REF}" stable >/dev/null
77+
oras tag "${IMMUTABLE_REF}" "${VERSION_TAG}" >/dev/null
3878
LATEST_DIGEST="$(oras resolve "${REF}")"
3979
[[ "${LATEST_DIGEST}" == "${DIGEST}" ]] || {
4080
echo "latest tag did not resolve to the published immutable digest" >&2
@@ -71,5 +111,6 @@ jobs:
71111
path: |
72112
dist/application-catalog-bundle.tar.gz
73113
dist/application-catalog-bundle.tar.gz.sha256
114+
dist/catalog.tsv
74115
dist/images.lock.json
75116
dist/catalog-bundle.publish-record.json

README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,11 @@ selected by the installer and expanded into a concrete application set.
1919
- `ghcr.io/techofourown/sw-ourbox-apps-hello-world/hello-world:latest`
2020
- `ghcr.io/techofourown/sw-ourbox-apps-chat/ourbox-chat:latest`
2121

22+
This repo also publishes installer-browsable catalog rows at
23+
`ghcr.io/techofourown/sw-ourbox-catalog-hello-world:catalog-amd64`. Those rows
24+
let host-side installers resolve the newest stable bundle whose
25+
`OURBOX_PLATFORM_CONTRACT_DIGEST` matches the selected OS payload contract.
26+
2227
## Repository layout
2328

2429
- [catalog/catalog.json](/techofourown/sw-ourbox-catalog-hello-world/catalog/catalog.json)

catalog/catalog.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
"id": "hello-world",
1414
"app_uid": "techofourown/hello-world",
1515
"display_name": "Hello World",
16-
"description": "A tiny hello-world application used to prove multi-repository catalog consumption.",
16+
"description": "Small hello-world app sourced from a second sw-ourbox-apps repo.",
1717
"renderer": "hello-world",
1818
"service_name": "hello-world",
1919
"service_port": 80,

scripts/check-catalog-bundle-smoke.sh

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ need_cmd python3
1717
need_cmd tar
1818
need_cmd sha256sum
1919

20+
python3 -m py_compile "${ROOT}/scripts/render-catalog-rows.py"
21+
2022
python3 - <<'PY' "${ROOT}/catalog/catalog.json" "${ROOT}/catalog/image-sources.json" "${ROOT}/catalog/profile.env"
2123
import json
2224
import re
@@ -179,4 +181,46 @@ for image in images_lock["images"]:
179181
raise SystemExit(f"generated image lock used_by mismatch for {image['name']}")
180182
PY
181183

184+
python3 "${ROOT}/scripts/render-catalog-rows.py" \
185+
--catalog-json "${ROOT}/catalog/catalog.json" \
186+
--profile-env "${ROOT}/catalog/profile.env" \
187+
--images-lock "${ROOT}/dist/images.lock.json" \
188+
--out-catalog "${TMP_ROOT}/catalog.tsv" \
189+
--channel stable \
190+
--tag "sha-test-run-1" \
191+
--created "2026-03-17T00:00:00Z" \
192+
--version "main-deadbeefcafe" \
193+
--revision "deadbeefcafedeadbeefcafedeadbeefcafedead" \
194+
--arch amd64 \
195+
--artifact-digest "sha256:1111111111111111111111111111111111111111111111111111111111111111" \
196+
--pinned-ref "ghcr.io/example/sw-ourbox-catalog-hello-world@sha256:1111111111111111111111111111111111111111111111111111111111111111"
197+
198+
python3 - <<'PY' "${TMP_ROOT}/catalog.tsv"
199+
import csv
200+
import sys
201+
from pathlib import Path
202+
203+
rows = list(csv.DictReader(Path(sys.argv[1]).open("r", encoding="utf-8"), delimiter="\t"))
204+
if len(rows) != 1:
205+
raise SystemExit(f"expected one rendered catalog row, got {len(rows)}")
206+
row = rows[0]
207+
expected = {
208+
"channel": "stable",
209+
"tag": "sha-test-run-1",
210+
"created": "2026-03-17T00:00:00Z",
211+
"version": "main-deadbeefcafe",
212+
"revision": "deadbeefcafedeadbeefcafedeadbeefcafedead",
213+
"arch": "amd64",
214+
"platform_contract_digest": "sha256:636af2d46d04b086366e97184d4e257d6c6e7dc75f070758d032cdd3cd4ff976",
215+
"platform_profile": "hello-world",
216+
"artifact_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111",
217+
"pinned_ref": "ghcr.io/example/sw-ourbox-catalog-hello-world@sha256:1111111111111111111111111111111111111111111111111111111111111111",
218+
}
219+
for key, expected_value in expected.items():
220+
if row.get(key) != expected_value:
221+
raise SystemExit(f"unexpected {key}: {row.get(key)!r}")
222+
if len(str(row.get("platform_images_lock_sha256", ""))) != 64:
223+
raise SystemExit(f"unexpected platform_images_lock_sha256: {row.get('platform_images_lock_sha256')!r}")
224+
PY
225+
182226
printf '[%s] catalog bundle smoke passed\n' "$(date -Is)"

scripts/check-publish-workflow.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,16 @@ workflow = Path(sys.argv[1]).read_text(encoding="utf-8")
1212
lines = {line.strip() for line in workflow.splitlines()}
1313
required = [
1414
'IMMUTABLE_TAG="sha-${GITHUB_SHA}-run-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"',
15+
'VERSION_TAG="main-${GITHUB_SHA::12}"',
1516
'"${IMMUTABLE_REF}" \\',
1617
'DIGEST="$(oras resolve "${IMMUTABLE_REF}")"',
1718
'oras tag "${IMMUTABLE_REF}" latest >/dev/null',
19+
'oras tag "${IMMUTABLE_REF}" stable >/dev/null',
20+
'oras tag "${IMMUTABLE_REF}" "${VERSION_TAG}" >/dev/null',
1821
'LATEST_DIGEST="$(oras resolve "${REF}")"',
1922
'[[ "${LATEST_DIGEST}" == "${DIGEST}" ]] || {',
23+
'python3 scripts/render-catalog-rows.py \\',
24+
'dist/catalog.tsv:text/tab-separated-values',
2025
]
2126
banned = [
2227
'DIGEST="$(oras resolve "${REF}")"',

scripts/render-catalog-rows.py

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
#!/usr/bin/env python3
2+
from __future__ import annotations
3+
4+
import argparse
5+
import csv
6+
import hashlib
7+
import json
8+
import re
9+
from pathlib import Path
10+
11+
PINNED_REF_RE = re.compile(r"^[^\s]+@sha256:[0-9a-f]{64}$")
12+
DIGEST_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
13+
TIMESTAMP_RE = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$")
14+
HEADER = [
15+
"channel",
16+
"tag",
17+
"created",
18+
"version",
19+
"revision",
20+
"arch",
21+
"platform_contract_digest",
22+
"platform_profile",
23+
"platform_images_lock_sha256",
24+
"artifact_digest",
25+
"pinned_ref",
26+
]
27+
28+
29+
def load_env(path: Path) -> dict[str, str]:
30+
data: dict[str, str] = {}
31+
for raw_line in path.read_text(encoding="utf-8").splitlines():
32+
line = raw_line.strip()
33+
if not line or line.startswith("#"):
34+
continue
35+
key, value = line.split("=", 1)
36+
data[key] = value
37+
return data
38+
39+
40+
def sha256_file(path: Path) -> str:
41+
return hashlib.sha256(path.read_bytes()).hexdigest()
42+
43+
44+
def load_existing_rows(path: Path | None) -> list[dict[str, str]]:
45+
if path is None or not path.is_file():
46+
return []
47+
with path.open("r", encoding="utf-8", newline="") as handle:
48+
reader = csv.DictReader(handle, delimiter="\t")
49+
if reader.fieldnames != HEADER:
50+
raise SystemExit(f"{path} does not declare the expected catalog.tsv header")
51+
return [{key: str(value or "").strip() for key, value in row.items()} for row in reader]
52+
53+
54+
def render_row(
55+
*,
56+
channel: str,
57+
immutable_tag: str,
58+
created: str,
59+
version: str,
60+
revision: str,
61+
arch: str,
62+
platform_contract_digest: str,
63+
platform_profile: str,
64+
images_lock_sha256: str,
65+
artifact_digest: str,
66+
pinned_ref: str,
67+
) -> dict[str, str]:
68+
if not channel:
69+
raise SystemExit("channel must be non-empty")
70+
if not immutable_tag:
71+
raise SystemExit("tag must be non-empty")
72+
if not TIMESTAMP_RE.fullmatch(created):
73+
raise SystemExit(f"created must be an RFC3339 UTC timestamp with Z suffix: {created!r}")
74+
if not version:
75+
raise SystemExit("version must be non-empty")
76+
if not revision:
77+
raise SystemExit("revision must be non-empty")
78+
if not arch:
79+
raise SystemExit("arch must be non-empty")
80+
if not DIGEST_RE.fullmatch(platform_contract_digest):
81+
raise SystemExit("platform contract digest must be sha256-pinned")
82+
if not platform_profile:
83+
raise SystemExit("platform profile must be non-empty")
84+
if not re.fullmatch(r"[0-9a-f]{64}", images_lock_sha256):
85+
raise SystemExit("images lock sha256 must be a plain 64-character hex digest")
86+
if not DIGEST_RE.fullmatch(artifact_digest):
87+
raise SystemExit("artifact digest must be sha256-pinned")
88+
if not PINNED_REF_RE.fullmatch(pinned_ref):
89+
raise SystemExit("pinned ref must be digest-pinned")
90+
91+
return {
92+
"channel": channel,
93+
"tag": immutable_tag,
94+
"created": created,
95+
"version": version,
96+
"revision": revision,
97+
"arch": arch,
98+
"platform_contract_digest": platform_contract_digest,
99+
"platform_profile": platform_profile,
100+
"platform_images_lock_sha256": images_lock_sha256,
101+
"artifact_digest": artifact_digest,
102+
"pinned_ref": pinned_ref,
103+
}
104+
105+
106+
def main() -> int:
107+
parser = argparse.ArgumentParser(description="Render installer-browsable catalog rows for an application catalog repo.")
108+
parser.add_argument("--catalog-json", required=True)
109+
parser.add_argument("--profile-env", required=True)
110+
parser.add_argument("--images-lock", required=True)
111+
parser.add_argument("--existing-catalog")
112+
parser.add_argument("--out-catalog", required=True)
113+
parser.add_argument("--channel", default="stable")
114+
parser.add_argument("--tag", required=True)
115+
parser.add_argument("--created", required=True)
116+
parser.add_argument("--version", required=True)
117+
parser.add_argument("--revision", required=True)
118+
parser.add_argument("--arch", required=True)
119+
parser.add_argument("--artifact-digest", required=True)
120+
parser.add_argument("--pinned-ref", required=True)
121+
args = parser.parse_args()
122+
123+
catalog_json = Path(args.catalog_json).resolve()
124+
profile_env = Path(args.profile_env).resolve()
125+
images_lock = Path(args.images_lock).resolve()
126+
existing_catalog = Path(args.existing_catalog).resolve() if args.existing_catalog else None
127+
out_catalog = Path(args.out_catalog).resolve()
128+
129+
catalog = json.loads(catalog_json.read_text(encoding="utf-8"))
130+
profile = load_env(profile_env)
131+
catalog_id = str(catalog.get("catalog_id", "")).strip()
132+
if not catalog_id:
133+
raise SystemExit(f"{catalog_json} must declare catalog_id")
134+
if str(profile.get("OURBOX_APPLICATION_CATALOG_ID", "")).strip() != catalog_id:
135+
raise SystemExit(f"{profile_env} OURBOX_APPLICATION_CATALOG_ID does not match {catalog_id!r}")
136+
137+
existing_rows = load_existing_rows(existing_catalog)
138+
new_row = render_row(
139+
channel=str(args.channel).strip(),
140+
immutable_tag=str(args.tag).strip(),
141+
created=str(args.created).strip(),
142+
version=str(args.version).strip(),
143+
revision=str(args.revision).strip(),
144+
arch=str(args.arch).strip(),
145+
platform_contract_digest=str(profile.get("OURBOX_PLATFORM_CONTRACT_DIGEST", "")).strip(),
146+
platform_profile=catalog_id,
147+
images_lock_sha256=sha256_file(images_lock),
148+
artifact_digest=str(args.artifact_digest).strip(),
149+
pinned_ref=str(args.pinned_ref).strip(),
150+
)
151+
152+
merged_rows = [
153+
row
154+
for row in existing_rows
155+
if not (row.get("channel") == new_row["channel"] and row.get("tag") == new_row["tag"])
156+
]
157+
merged_rows.append(new_row)
158+
merged_rows.sort(key=lambda row: row["created"], reverse=True)
159+
160+
out_catalog.parent.mkdir(parents=True, exist_ok=True)
161+
with out_catalog.open("w", encoding="utf-8", newline="") as handle:
162+
writer = csv.DictWriter(handle, fieldnames=HEADER, delimiter="\t", lineterminator="\n")
163+
writer.writeheader()
164+
for row in merged_rows:
165+
writer.writerow(row)
166+
return 0
167+
168+
169+
if __name__ == "__main__":
170+
raise SystemExit(main())

0 commit comments

Comments
 (0)