Skip to content

Commit 559013e

Browse files
adamdotdevinadamdotdevin
committed
docs: perf plans
1 parent 8e3ab4a commit 559013e

File tree

6 files changed

+966
-0
lines changed

6 files changed

+966
-0
lines changed

‎specs/01-persist-payload-limits.md‎

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
## Payload limits
2+
3+
Prevent blocking storage writes and runaway persisted size
4+
5+
---
6+
7+
### Summary
8+
9+
Large payloads (base64 images, terminal buffers) are currently persisted inside key-value stores:
10+
11+
- web: `localStorage` (sync, blocks the main thread)
12+
- desktop: Tauri Store-backed async storage files (still expensive when values are huge)
13+
14+
We’ll introduce size-aware persistence policies plus a dedicated “blob store” for large/binary data (IndexedDB on web; separate files on desktop). Prompt/history state will persist only lightweight references to blobs and load them on demand.
15+
16+
---
17+
18+
### Goals
19+
20+
- Stop persisting image `dataUrl` blobs inside web `localStorage`
21+
- Stop persisting image `dataUrl` blobs inside desktop store `.dat` files
22+
- Store image payloads out-of-band (blob store) and load lazily when needed (e.g. when restoring a history item)
23+
- Prevent terminal buffer persistence from exceeding safe size limits
24+
- Keep persistence behavior predictable across web (sync) and desktop (async)
25+
- Provide escape hatches via flags and per-key size caps
26+
27+
---
28+
29+
### Non-goals
30+
31+
- Cross-device sync of images or terminal buffers
32+
- Lossless persistence of full terminal scrollback on web
33+
- Perfect blob deduplication or a complex reference-counting system on day one
34+
35+
---
36+
37+
### Current state
38+
39+
- `packages/app/src/utils/persist.ts` uses `localStorage` (sync) on web and async storage only on desktop.
40+
- Desktop storage is implemented via `@tauri-apps/plugin-store` and writes to named `.dat` files (see `packages/desktop/src/index.tsx`). Large values bloat these files and increase flush costs.
41+
- Prompt history persists under `Persist.global("prompt-history")` (`packages/app/src/components/prompt-input.tsx`) and can include image parts (`dataUrl`).
42+
- Prompt draft persistence uses `packages/app/src/context/prompt.tsx` and can also include image parts (`dataUrl`).
43+
- Terminal buffer is serialized in `packages/app/src/components/terminal.tsx` and persisted in `packages/app/src/context/terminal.tsx`.
44+
45+
---
46+
47+
### Proposed approach
48+
49+
#### 1) Add per-key persistence policies (KV store guardrails)
50+
51+
In `packages/app/src/utils/persist.ts`, add policy hooks for each persisted key:
52+
53+
- `warnBytes` (soft warning threshold)
54+
- `maxBytes` (hard cap)
55+
- `transformIn` / `transformOut` for lossy persistence (e.g. strip or refactor fields)
56+
- `onOversize` strategy: `drop`, `truncate`, or `migrateToBlobRef`
57+
58+
This protects both:
59+
60+
- web (`localStorage` is sync)
61+
- desktop (async, but still expensive to store/flush giant values)
62+
63+
#### 2) Add a dedicated blob store for large data
64+
65+
Introduce a small blob-store abstraction used by the app layer:
66+
67+
- web backend: IndexedDB (store `Blob` values keyed by `id`)
68+
- desktop backend: filesystem directory under the app data directory (store one file per blob)
69+
70+
Store _references_ to blobs inside the persisted JSON instead of the blob contents.
71+
72+
#### 3) Persist image parts as references (not base64 payloads)
73+
74+
Update the prompt image model so the in-memory shape can still use a `dataUrl` for UI, but the persisted representation is reference-based.
75+
76+
Suggested approach:
77+
78+
- Keep `ImageAttachmentPart` with:
79+
- required: `id`, `filename`, `mime`
80+
- optional/ephemeral: `dataUrl?: string`
81+
- new: `blobID?: string` (or `ref: string`)
82+
83+
Persistence rules:
84+
85+
- When writing persisted prompt/history state:
86+
- ensure each image part is stored in blob store (`blobID`)
87+
- persist only metadata + `blobID` (no `dataUrl`)
88+
- When reading persisted prompt/history state:
89+
- do not eagerly load blob payloads
90+
- hydrate `dataUrl` only when needed:
91+
- when applying a history entry into the editor
92+
- before submission (ensure all image parts have usable `dataUrl`)
93+
- when rendering an attachment preview, if required
94+
95+
---
96+
97+
### Phased implementation steps
98+
99+
1. Add guardrails in `persist.ts`
100+
101+
- Implement size estimation in `packages/app/src/utils/persist.ts` using `TextEncoder` byte length on JSON strings.
102+
- Add a policy registry keyed by persist name (e.g. `"prompt-history"`, `"prompt"`, `"terminal"`).
103+
- Add a feature flag (e.g. `persist.payloadLimits`) to enable enforcement gradually.
104+
105+
2. Add blob-store abstraction + platform hooks
106+
107+
- Add a new app-level module (e.g. `packages/app/src/utils/blob.ts`) defining:
108+
- `put(id, bytes|Blob)`
109+
- `get(id)`
110+
- `remove(id)`
111+
- Extend the `Platform` interface (`packages/app/src/context/platform.tsx`) with optional blob methods, or provide a default web implementation and override on desktop:
112+
- web: implement via IndexedDB
113+
- desktop: implement via filesystem files (requires adding a Tauri fs plugin or `invoke` wrappers)
114+
115+
3. Update prompt history + prompt draft persistence to use blob refs
116+
117+
- Update prompt/history serialization paths to ensure image parts are stored as blob refs:
118+
- Prompt history: `packages/app/src/components/prompt-input.tsx`
119+
- Prompt draft: `packages/app/src/context/prompt.tsx`
120+
- Ensure “apply history prompt” hydrates image blobs only when applying the prompt (not during background load).
121+
122+
4. One-time migration for existing persisted base64 images
123+
124+
- On read, detect legacy persisted image parts that include `dataUrl`.
125+
- If a `dataUrl` is found:
126+
- write it into the blob store (convert dataUrl → bytes)
127+
- replace persisted payload with `{ blobID, filename, mime, id }` only
128+
- re-save the reduced version
129+
- If migration fails (missing permissions, quota, etc.), fall back to:
130+
- keep the prompt entry but drop the image payload and mark as unavailable
131+
132+
5. Fix terminal persistence (bounded snapshot)
133+
134+
- In `packages/app/src/context/terminal.tsx`, persist only:
135+
- last `maxLines` and/or
136+
- last `maxBytes` of combined text
137+
- In `packages/app/src/components/terminal.tsx`, keep the full in-memory buffer unchanged.
138+
139+
6. Add basic blob lifecycle cleanup
140+
To avoid “blob directory grows forever”, add one of:
141+
142+
- TTL-based cleanup: store `lastAccessed` per blob and delete blobs older than N days
143+
- Reference scan cleanup: periodically scan prompt-history + prompt drafts, build a set of referenced `blobID`s, and delete unreferenced blobs
144+
145+
Start with TTL-based cleanup (simpler, fewer cross-store dependencies), then consider scan-based cleanup if needed.
146+
147+
---
148+
149+
### Data migration / backward compatibility
150+
151+
- KV store data:
152+
- policies should be tolerant of missing fields (e.g. `dataUrl` missing)
153+
- Image parts:
154+
- treat missing `dataUrl` as “not hydrated yet”
155+
- treat missing `blobID` (legacy) as “not persisted” or “needs migration”
156+
- Desktop:
157+
- blob files should be namespaced (e.g. `opencode/blobs/<blobID>`) to avoid collisions
158+
159+
---
160+
161+
### Risk + mitigations
162+
163+
- Risk: blob store is unavailable (IndexedDB disabled, desktop fs permissions).
164+
- Mitigation: keep base state functional; persist prompts without image payloads and show a clear placeholder.
165+
- Risk: lazy hydration introduces edge cases when submitting.
166+
- Mitigation: add a pre-submit “ensure images hydrated” step; if hydration fails, block submission with a clear error or submit without images.
167+
- Risk: dataUrl→bytes conversion cost during migration.
168+
- Mitigation: migrate incrementally (only when reading an entry) and/or use `requestIdleCallback` on web.
169+
- Risk: blob cleanup deletes blobs still needed.
170+
- Mitigation: TTL default should be conservative; scan-based cleanup should only delete blobs unreferenced by current persisted state.
171+
172+
---
173+
174+
### Validation plan
175+
176+
- Unit-level:
177+
- size estimation + policy enforcement in `persist.ts`
178+
- blob store put/get/remove round trips (web + desktop backends)
179+
- Manual scenarios:
180+
- attach multiple images, reload, and confirm:
181+
- KV store files do not balloon
182+
- images can be restored when selecting history items
183+
- open terminal with large output and confirm reload restores bounded snapshot quickly
184+
- confirm prompt draft persistence still works in `packages/app/src/context/prompt.tsx`
185+
186+
---
187+
188+
### Rollout plan
189+
190+
- Phase 1: ship with `persist.payloadLimits` off; log oversize detections in dev.
191+
- Phase 2: enable image blob refs behind `persist.imageBlobs` (web + desktop).
192+
- Phase 3: enable terminal truncation and enforce hard caps for known hot keys.
193+
- Phase 4: enable blob cleanup behind `persist.blobGc` (TTL first).
194+
- Provide quick kill switches by disabling each flag independently.
195+
196+
---
197+
198+
### Open questions
199+
200+
- What should the canonical persisted image schema be (`blobID` field name, placeholder shape, etc.)?
201+
- Desktop implementation detail:
202+
- add `@tauri-apps/plugin-fs` vs custom `invoke()` commands for blob read/write?
203+
- where should blob files live (appDataDir) and what retention policy is acceptable?
204+
- Web implementation detail:
205+
- do we store `Blob` directly in IndexedDB, or store base64 strings?
206+
- Should prompt-history images be retained indefinitely, or only for the last `MAX_HISTORY` entries?

‎specs/02-cache-eviction.md‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
## Cache eviction
2+
3+
Add explicit bounds for long-lived in-memory state
4+
5+
---
6+
7+
### Summary
8+
9+
Several in-memory caches grow without limits during long sessions. We’ll introduce explicit eviction (LRU + TTL + size caps) for sessions/messages/file contents and global per-directory sync stores.
10+
11+
---
12+
13+
### Goals
14+
15+
- Prevent unbounded memory growth from caches that survive navigation
16+
- Add consistent eviction primitives shared across contexts
17+
- Keep UI responsive under heavy usage (many sessions, large files)
18+
19+
---
20+
21+
### Non-goals
22+
23+
- Perfect cache hit rates or prefetch strategies
24+
- Changing server APIs or adding background jobs
25+
- Persisting caches for offline use
26+
27+
---
28+
29+
### Current state
30+
31+
- Global sync uses per-directory child stores without eviction in `packages/app/src/context/global-sync.tsx`.
32+
- File contents cached in `packages/app/src/context/file.tsx` with no cap.
33+
- Session-heavy pages include `packages/app/src/pages/session.tsx` and `packages/app/src/pages/layout.tsx`.
34+
35+
---
36+
37+
### Proposed approach
38+
39+
- Introduce a shared cache utility that supports:
40+
- `maxEntries`, `maxBytes` (approx), and `ttlMs`
41+
- LRU ordering with explicit `touch(key)` on access
42+
- deterministic `evict()` and `clear()` APIs
43+
- Apply the utility to:
44+
- global-sync per-directory child stores (cap number of directories kept “hot”)
45+
- file contents cache (cap by entries + bytes, with TTL)
46+
- session/message caches (cap by session count, and optionally message count)
47+
- Add feature flags per cache domain to allow partial rollout (e.g. `cache.eviction.files`).
48+
49+
---
50+
51+
### Phased implementation steps
52+
53+
1. Add a generic cache helper
54+
55+
- Create `packages/app/src/utils/cache.ts` with a small, dependency-free LRU+TTL.
56+
- Keep it framework-agnostic and usable from Solid contexts.
57+
58+
Sketch:
59+
60+
```ts
61+
type CacheOpts = {
62+
maxEntries: number
63+
ttlMs?: number
64+
maxBytes?: number
65+
sizeOf?: (value: unknown) => number
66+
}
67+
68+
function createLruCache<T>(opts: CacheOpts) {
69+
// get, set, delete, clear, evictExpired, stats
70+
}
71+
```
72+
73+
2. Apply eviction to file contents
74+
75+
- In `packages/app/src/context/file.tsx`:
76+
- wrap the existing file-content map in the LRU helper
77+
- approximate size via `TextEncoder` length of content strings
78+
- evict on `set` and periodically via `requestIdleCallback` when available
79+
- Add a small TTL (e.g. 10–30 minutes) to discard stale contents.
80+
81+
3. Apply eviction to global-sync child stores
82+
83+
- In `packages/app/src/context/global-sync.tsx`:
84+
- track child stores by directory key in an LRU with `maxEntries`
85+
- call a `dispose()` hook on eviction to release subscriptions and listeners
86+
- Ensure “currently active directory” is always `touch()`’d to avoid surprise evictions.
87+
88+
4. Apply eviction to session/message caches
89+
90+
- Identify the session/message caching touchpoints used by `packages/app/src/pages/session.tsx`.
91+
- Add caps that reflect UI needs (e.g. last 10–20 sessions kept, last N messages per session if cached).
92+
93+
5. Add developer tooling
94+
95+
- Add a debug-only stats readout (console or dev panel) for cache sizes and eviction counts.
96+
- Add a one-click “clear caches” action for troubleshooting.
97+
98+
---
99+
100+
### Data migration / backward compatibility
101+
102+
- No persisted schema changes are required since this targets in-memory caches.
103+
- If any cache is currently mirrored into persistence, keep keys stable and only change in-memory retention.
104+
105+
---
106+
107+
### Risk + mitigations
108+
109+
- Risk: evicting content still needed causes extra refetches and flicker.
110+
- Mitigation: always pin “active” entities and evict least-recently-used first.
111+
- Risk: disposing global-sync child stores could leak listeners if not cleaned up correctly.
112+
- Mitigation: require an explicit `dispose()` contract and add dev assertions for listener counts.
113+
- Risk: approximate byte sizing is imprecise.
114+
- Mitigation: combine entry caps with byte caps and keep thresholds conservative.
115+
116+
---
117+
118+
### Validation plan
119+
120+
- Add tests for `createLruCache` covering TTL expiry, LRU ordering, and eviction triggers.
121+
- Manual scenarios:
122+
- open many files and confirm memory stabilizes and UI remains responsive
123+
- switch across many directories and confirm global-sync does not continuously grow
124+
- long session navigation loop and confirm caches plateau
125+
126+
---
127+
128+
### Rollout plan
129+
130+
- Land cache utility first with flags default off.
131+
- Enable file cache eviction first (lowest behavioral risk).
132+
- Enable global-sync eviction next with conservative caps and strong logging in dev.
133+
- Enable session/message eviction last after observing real usage patterns.
134+
135+
---
136+
137+
### Open questions
138+
139+
- What are the current session/message cache structures and their ownership boundaries?
140+
- Which child stores in `global-sync.tsx` have resources that must be disposed explicitly?
141+
- What caps are acceptable for typical workflows (files open, directories visited, sessions viewed)?

0 commit comments

Comments
 (0)