Skip to content

Commit 5db947b

Browse files
author
jaehyuk.hyun
committed
docs(media-flip): record the staging flip, and drop two invented issue numbers
Staging is done, so the section that said "NOT run from here" is now the record of what was actually run. Open question #3 — which of R2's two public surfaces staging had on — is answered: no r2.dev URL, one custom domain, disabled rather than removed so rollback stays a one-field PUT instead of a DNS change. The before/after is a real object, not a 404 probe. A missing key answers 404 whether the bucket is open or shut, so the first probe proved nothing; the measurement that counts is 200 with 12856 bytes becoming 401. Also written down, because both nearly undid the flip and neither is obvious from the code: - R2_PUBLIC_URL has to be RELATIVE. The absolute form reaches the gate and looks right while re-creating exactly the rows the cleanup deleted. The warning was already in this file and got read too late. - Editing Cloud Run alone does not survive a deploy: deploy.yml uses --set-env-vars and sources the value from a GitHub secret, so the next deploy silently reverts it. And the secret is the _NEW-suffixed one — the unsuffixed name belongs to the old project. The order staging used (truncate, flip, then fix the URL) was wrong and survived only because the database was empty. That is stated plainly so production does not inherit it. The #83 / #81 references throughout were never real — the repo has only #4 and #5. They are replaced with the checklist items that actually block.
1 parent c832735 commit 5db947b

1 file changed

Lines changed: 83 additions & 16 deletions

File tree

docs/design/media-bucket-flip-checklist.md

Lines changed: 83 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -263,11 +263,76 @@ a running stack, `docker exec proofport-minio mc anonymous set download local/<b
263263
same command, forward direction. Effectively instant; a same-host container ACL flip, no
264264
propagation delay.
265265

266-
### The real R2 equivalent — NOT run from here
266+
### The real R2 equivalent — STAGING IS DONE (2026-08-23)
267+
268+
Open question #3 is answered. Staging's surfaces, read from the API rather than guessed:
269+
270+
| Surface | Staging (`openstoa-stg`) | Action taken |
271+
| --- | --- | --- |
272+
| `r2.dev` dev URL | not enabled | none needed |
273+
| custom domain | `stg-cdn.zkproofport.app` | set `enabled: false` |
274+
275+
Measured, not assumed — with a REAL object, not a 404 probe. A missing key returns 404
276+
whether the bucket is open or shut, so probing one proves nothing:
277+
278+
```
279+
GET https://stg-cdn.zkproofport.app/topics/<id>/chat/<sender>/<blob>.bin
280+
before: 200, 12856 bytes # anonymous, no credentials, ciphertext readable
281+
after: 401
282+
```
283+
284+
**Disabled, not removed.** `wrangler r2 bucket domain remove` also deletes the CNAME it
285+
created, which makes rollback a DNS change. The REST API takes `{"enabled": false}` on the
286+
same domain and is a one-field flip back:
287+
288+
```bash
289+
curl -X PUT "https://api.cloudflare.com/client/v4/accounts/$CF_ACCOUNT/r2/buckets/openstoa-stg/domains/custom/stg-cdn.zkproofport.app" \
290+
-H "Authorization: Bearer $CLOUDFLARE_TOKEN" -H 'Content-Type: application/json' \
291+
--data '{"enabled":false}' # "enabled":true to roll back
292+
```
293+
294+
#### Two things that nearly undid the flip
295+
296+
**1. `R2_PUBLIC_URL` must be RELATIVE, and the first attempt made it absolute.**
297+
`https://stg-community.zkproofport.app/api/media` reaches the gate and looks correct, but it
298+
re-creates the very rows the cleanup deleted — every upload mints an absolute URL again. The
299+
warning is already above under "Deleting the rows is not durable"; it was read too late. The
300+
wanted value is `/api/media`, and `media-route.test.ts` ("RELATIVE R2_PUBLIC_URL (M-6)") is
301+
what guarantees the gate does not care about URL shape.
302+
303+
**2. Changing Cloud Run alone does not survive a deploy.** `deploy.yml:270` uses
304+
`--set-env-vars` (not `--update-env-vars`) and sources `R2_PUBLIC_URL` from a GitHub secret,
305+
so the next deploy overwrites any manual `gcloud run services update`. Both must change:
306+
307+
```bash
308+
echo -n "/api/media" | gh secret set STAGING_R2_PUBLIC_URL_NEW # the durable one
309+
gcloud run services update proofport-community-staging --region us-central1 \
310+
--update-env-vars "R2_PUBLIC_URL=/api/media" # takes effect now
311+
```
312+
313+
The `_NEW` suffix is not optional: `deploy.yml:81-87` appends it whenever
314+
`target_project = masselabs`, so `STAGING_R2_PUBLIC_URL` (no suffix) belongs to the OLD
315+
project and editing it changes nothing.
316+
317+
#### Order matters, and staging got away with the wrong one
318+
319+
The correct order is: **relative `R2_PUBLIC_URL` -> redeploy -> delete rows -> flip the
320+
bucket.** Staging ran truncate -> flip -> fix the URL, which is wrong, and was harmless only
321+
because the database had just been emptied so there were no rows left to re-break.
322+
Production has real rows and will not be so forgiving.
323+
324+
#### Chat attachments were never at risk
325+
326+
Worth stating because it is the thing that looks most likely to break: chat media is read
327+
through `/api/topics/{topicId}/chat/media?key=...` (see `ChatPanel.tsx` and
328+
`packages/mobile/src/api/openstoaClient.ts`), never through `R2_PUBLIC_URL`. That variable
329+
only ever addressed plaintext post/topic/profile images. Closing the public domain cannot
330+
darken chat.
331+
332+
### Finding the surfaces (for production)
267333

268334
An R2 bucket has **two independent** public-read surfaces, and closing one does not close
269-
the other. Find out which are on before flipping anything — staging's is genuinely unknown
270-
(open question #3 in `media-bucket-privatisation.md` never got resolved):
335+
the other. Find out which are on before flipping anything:
271336

272337
```bash
273338
# Staging bucket is `openstoa-stg`; production is `openstoa-prod`
@@ -288,7 +353,9 @@ npx wrangler r2 bucket domain remove openstoa-stg --domain <domain from the list
288353
```
289354

290355
Production is the same two commands against `openstoa-prod` — but the production flip is
291-
separately blocked on #83#81/M-7 (see items 3/4), so it is not simply "run these now".
356+
separately blocked on the edge work in items 3/4 (CDN cache rule, rate limit), so it is
357+
not simply "run these now". (Earlier drafts cited issues #83 and #81 here — neither exists;
358+
the repo's only open issues are #4 and #5. The blockers are the checklist items themselves.)
292359

293360
Both commands accept `--force`/`-y` to skip the confirmation prompt; do not use it — the prompt
294361
names the bucket it is about to change. Removing a custom domain **also deletes the CNAME
@@ -324,16 +391,16 @@ items, both marked optional (`(선택)`) and both **unchecked**:
324391
- [ ] OpenStoa 정적 자산만 캐싱
325392
```
326393

327-
Neither is applied. **Status: task #81/M-7 has not started at the Cloudflare config
394+
Neither is applied. **Status: the item-4 edge work has not started at the Cloudflare config
328395
layer** — confirmed by reading the checklist state, not inferred from the todo tracker
329396
alone. Today, `/api/media` relies entirely on Cloudflare's *default* cache behavior, which
330397
— per the citation already in `media-bucket-privatisation.md`'s Sources — generally does
331398
not cache API-shaped paths without an explicit Cache Rule.
332399

333-
**A real conflict to flag for whoever builds #81, found by reading both docs together**:
400+
**A real conflict to flag for whoever builds the edge cache rule (item 4), found by reading both docs together**:
334401
the one *documented* (optional, unapplied) rule above would **bypass all of `/api/*`**
335402
if it's ever applied literally as written, it would also bypass `/api/media`, the opposite
336-
of what #81 needs. Cloudflare evaluates Cache Rules in priority order, so `/api/media`
403+
of what the edge cache rule needs. Cloudflare evaluates Cache Rules in priority order, so `/api/media`
337404
needs either its own higher-priority rule or an explicit carve-out from the blanket
338405
`/api/*` bypass — not a detail to discover after both rules are live and disagreeing.
339406

@@ -345,7 +412,7 @@ curl -sI https://<app-host>/api/media/<a genuinely public key> | grep -i cf-cach
345412

346413
Request it twice. First request from a cold edge node is typically `MISS` or `EXPIRED`;
347414
the second (from the same edge PoP) should show `HIT`. `DYNAMIC` or `BYPASS` on both means
348-
nothing is being cached at all — the state #81 needs to fix.
415+
nothing is being cached at all — the state the edge cache rule needs to fix.
349416

350417
**The check that must not be got wrong — private objects must NEVER show `HIT`:**
351418

@@ -360,33 +427,33 @@ The M-5 route already sets `Cache-Control: private` for exactly these objects
360427
(`src/app/api/media/[...key]/route.ts`), and Cloudflare's documented default honors that —
361428
but a Cache Rule can override it (an "Eligible for cache: All" + "Edge TTL: Override
362429
origin" rule ignores the origin's own `Cache-Control` header entirely). **This has to be
363-
re-verified against whatever Cache Rule #81 actually ships**, not assumed safe because the
430+
re-verified against whatever Cache Rule item 4 actually ships**, not assumed safe because the
364431
default was safe before a rule existed.
365432

366433
**Status: UNKNOWN FROM HERE for live edge behavior** (no Cloudflare in front of
367434
`localhost` — this cannot be exercised from local dev at all, by construction). **VERIFIED
368435
FROM READING that no rule exists yet** — which makes the "private is never cached" property
369436
trivially true today (nothing is cached, so nothing can leak) and the "public is cached"
370-
property false (task not started). Both need re-checking once #81 ships a real rule —
437+
property false (task not started). Both need re-checking once item 4 ships a real rule —
371438
passing the first check the day the rule is deployed does not mean it stays true after the
372439
next edit to that rule.
373440

374441
## 4. Rate limiting
375442

376-
This is the EDGE rate limit, and it is #81/M-7 — noted as a dependency here, not duplicated.
443+
This is the EDGE rate limit, and it is item 4 — noted as a dependency here, not duplicated.
377444

378445
**Correction to an earlier draft of this doc**, which said "no rate-limit code exists on
379446
`/api/media` today (verified by grep)". That was true when written and is now false:
380447
`722436e` ("feat(media): cap how fast one caller can pull images") added
381448
`src/lib/mediaRateLimit.ts`, and the route calls it —
382449
`checkMediaReadRateLimit(identity)` → 429 with `Retry-After`
383450
(`src/app/api/media/[...key]/route.ts:236`). So the APPLICATION-level cap now exists; what
384-
remains for #81 is the edge-level one, in front of the origin, which is a different layer
451+
remains for item 4 is the edge-level one, in front of the origin, which is a different layer
385452
solving a different half of the problem (an edge limit spares the origin the request
386453
entirely; the in-app limit still costs a Cloud Run invocation to answer 429).
387454

388455
Item 4's staging/production verdicts below are unchanged by this — staging is N/A by the
389-
product decision, and production is blocked on #83#81/M-7 regardless.
456+
product decision, and production is blocked on the item 3/4 edge work regardless.
390457

391458
**What breaks without it, once private:** every image load that isn't an edge-cache `HIT`
392459
(item 3) reaches the app and runs the M-5 gate's DB query (topic visibility / membership
@@ -429,8 +496,8 @@ depended on bucket access either way, flip or no flip).
429496
|---|---|---|---|---|
430497
| 1 | No absolute media URLs stored | **Satisfied** — 54 + 30 found (84 posts), deleted, re-query returns 0 | Unknown from here — run the script's dry run | Unknown from here — run the script's dry run |
431498
| 2 | Storage layer itself denies anonymous reads | **Satisfied** — MinIO now `private` by default; anonymous direct fetch 200 → 403, gate still 200, verified live | Not done — run the wrangler commands in item 2 | Not done, and blocked on 3/4 anyway |
432-
| 3 | CDN cache exists, public cacheable, private never cached | **N/A** — no Cloudflare in front of localhost | **N/A by decision** — edge layer is production-only (see note below) | Unsatisfied — blocked on #83 -> #81/M-7 |
433-
| 4 | Rate limit in front of `/api/media` | In-app cap **exists** (`722436e`); edge cap N/A — no edge in front of localhost | **N/A by decision** — edge layer is production-only (see note below) | Unsatisfied — blocked on #83 -> #81/M-7 |
499+
| 3 | CDN cache exists, public cacheable, private never cached | **N/A** — no Cloudflare in front of localhost | **N/A by decision** — edge layer is production-only (see note below) | Unsatisfied — blocked on items 3/4 |
500+
| 4 | Rate limit in front of `/api/media` | In-app cap **exists** (`722436e`); edge cap N/A — no edge in front of localhost | **N/A by decision** — edge layer is production-only (see note below) | Unsatisfied — blocked on items 3/4 |
434501
| 5 | Rollback path known and its cost is understood | **Documented above** — cheap, not instant for real R2 | Same mechanism, unverified access to actually exercise it from here | Same mechanism, unverified access to actually exercise it from here |
435502

436503
### The edge layer is PRODUCTION-ONLY, by an explicit product decision
@@ -442,7 +509,7 @@ in front of production is precisely what makes it possible to attach these there
442509
there.
443510

444511
Read the table accordingly: **staging's bucket flip does not wait on 3 or 4.** Only the
445-
production flip is blocked, on #83 (production GCLB) -> #81/M-7 (CDN). Do not re-report
512+
production flip is blocked, on the production GCLB and the CDN cache rule (items 3/4). Do not re-report
446513
staging as "unsatisfied" on these two rows; that reading has cost the project the same
447514
conversation more than once.
448515

0 commit comments

Comments
 (0)