Skip to content

Commit 543a0d9

Browse files
authored
Improve terminal display parity calibration (#206)
1 parent 743f5db commit 543a0d9

42 files changed

Lines changed: 3254 additions & 258 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
77
## [Unreleased]
88

99
### 🚀 Added
10+
- Terminal: add Desktop/Web display consistency calibration with shared reference setup, device-local compensation, diagnostics, and a real parity profiling script. (#206)
1011
- Space Explorer: preview and open VS Code-built-in audio/video files (`mp3`, `wav`, `wave`, `ogg`, `oga`, `mp4`, `webm`) as playable document windows. (#204)
1112
- Agent: system notifications now fire when agents finish work and return to standby. (#198)
1213
- Settings: left-sidebar search helps users locate settings and jump directly to the matching section. (#192)

docs/terminal/MULTI_CLIENT_ARCHITECTURE.md

Lines changed: 54 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
> Status: Canonical technical direction
44
> Scope: terminal and agent nodes rendered across Desktop, Web UI, and future Mobile clients
5-
> Last updated: 2026-04-28
5+
> Last updated: 2026-04-30
66
77
Verification workflow:
88

@@ -141,6 +141,59 @@ OpenCove uses an appearance profile as the terminal equivalent of a shared displ
141141

142142
The profile can be implemented with CSS variables in clients, similar to a `rem` system, but it is not the truth. The truth remains integer terminal cells: `cols x rows`.
143143

144+
Desktop and Web UI terminal clients also share a stable terminal pixel-snap DPR. xterm rounds cell
145+
metrics through its effective device pixel ratio, so a 1x browser and a 2x Electron window can
146+
derive different cell widths from the same font setting. OpenCove normalizes this renderer-local DPR
147+
for terminal measurement; it is display-only and must not grant geometry authority to a renderer.
148+
149+
Display calibration is a diagnostic layer over this profile. The profiler may sweep
150+
`fontSize`/`lineHeight`/`letterSpacing` candidates in a client and compare xterm/FitAddon proposed
151+
geometry plus cell metrics against the Desktop baseline. Calibration output is a recommended
152+
appearance candidate, not a runtime authority: it must not call `pty.resize`, mutate worker
153+
canonical geometry, or replace the explicit appearance commit path.
154+
155+
## User Display Alignment Workflow
156+
157+
OpenCove exposes display alignment as a user-controlled Settings workflow, not an automatic hidden
158+
resize policy:
159+
160+
1. Automatic reference setup is enabled by default. The first online client for the current
161+
terminal appearance profile records the shared reference if none exists.
162+
2. Go to `Settings -> General -> Terminal Display Consistency`.
163+
3. Turn off `Set Reference Automatically` when you do not want first-client reference capture. This
164+
does not delete existing references or local client calibration.
165+
4. Keep `Apply Calibration Automatically` on when you want a saved device adjustment to be applied
166+
automatically. Turn it off to compare the raw terminal font settings.
167+
5. Choose `Use This Device as Target` only when you want to replace the automatic reference. This
168+
stores the shared target cell metrics with the existing terminal appearance profile.
169+
6. Open another client, then choose `Calibrate This Device`. The client sweeps local display
170+
compensation candidates and stores the best local match in that client only.
171+
7. If the result is not visually acceptable, adjust the shared terminal font family/size, set a new
172+
reference, then calibrate the other clients again.
173+
8. Use `Clear Device Adjustment` to remove local compensation, or `Copy Diagnostics` when reporting a
174+
parity issue.
175+
176+
This workflow follows the same owner boundary as the runtime architecture:
177+
178+
- the shared reference is a persisted user preference
179+
- the automatic reference setup toggle is a persisted user preference and defaults to enabled
180+
- the automatic calibration compensation toggle is a persisted user preference and defaults to
181+
enabled
182+
- the client calibration is local storage scoped to the current terminal appearance profile and
183+
active shared reference
184+
- the automatic first-client reference is captured from a real mounted terminal xterm/FitAddon
185+
instance, not from a synthetic hidden terminal
186+
- enabled local compensation may change xterm `fontSize`, `lineHeight`, and `letterSpacing`
187+
- local compensation may trigger local FitAddon measurement
188+
- local compensation must not resize the PTY or update worker canonical `cols/rows`
189+
- user-facing calibration results are shown as match quality (`Exact`, `Close`, `Needs adjustment`);
190+
raw engineering scores stay in diagnostics only
191+
192+
The target is identical terminal cell metrics when the clients can support them. When exact parity is
193+
not possible because of platform font rendering, the fallback is explicit and inspectable: users pick
194+
the closest visual candidate, keep the canonical terminal geometry stable, and attach diagnostics to
195+
the bug report instead of letting each renderer silently fight for size authority.
196+
144197
## Multi-Client Policy
145198

146199
- First interactive client may be controller.

docs/terminal/VERIFICATION_AND_RECORDS_PLAN.md

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
> Status: Active working plan
44
> Scope: terminal and agent nodes across Desktop, Web UI, and future Mobile clients
5-
> Last updated: 2026-04-28
5+
> Last updated: 2026-04-30
66
77
## Purpose
88

@@ -88,6 +88,7 @@ Use for user-visible promises:
8888
- `cmd+w` close and reopen
8989
- Codex/OpenCode live output while another client attaches
9090
- resize stress under Desktop/Web interaction
91+
- Desktop/Web display parity profiling with calibration candidate sweep
9192

9293
Goal:
9394

@@ -214,6 +215,7 @@ These scenarios should remain easy to rerun and should never be left unowned:
214215
6. Another client opens while Codex or OpenCode is actively streaming output.
215216
7. WebGL or canvas degradation rebuilds the local renderer without killing interactivity.
216217
8. Hidden or backgrounded client resumes and converges through resync.
218+
9. Display calibration detects the best client-local appearance candidate without changing PTY geometry.
217219

218220
Each item should have at least one durable asset:
219221

@@ -247,6 +249,7 @@ Current high-value script bundle:
247249
- `OPENCOVE_REPRO_ITERATIONS=1 OPENCOVE_REPRO_CLOSE_MODE=cold-restart ELECTRON_RUN_AS_NODE=1 pnpm exec electron scripts/debug-repro-restored-agent-input.mjs`
248250
- `ELECTRON_RUN_AS_NODE=1 OPENCOVE_PROFILE_AGENT_COUNT=12 OPENCOVE_PROFILE_PROVIDER=codex ./node_modules/.bin/electron scripts/profile-agent-restore-startup.mjs`
249251
- `ELECTRON_RUN_AS_NODE=1 OPENCOVE_PROFILE_AGENT_COUNT=20 OPENCOVE_PROFILE_PROVIDER=codex ./node_modules/.bin/electron scripts/profile-agent-restore-startup.mjs`
252+
- `OPENCOVE_PROFILE_WEB_HEADFUL=1 OPENCOVE_PROFILE_WEB_DEVICE_SCALE_FACTORS=1,2 OPENCOVE_PROFILE_ASSERT_PARITY=1 pnpm profile:terminal:display-parity`
250253
- `OPENCOVE_E2E_SKIP_BUILD=1 node scripts/test-e2e-web-canvas.mjs -- tests/e2e-web-canvas/workerWebCanvas.spec.ts --grep "reconnects terminal sessions after a page reload|allows controlling a shared terminal session from multiple web clients"`
251254
- `OPENCOVE_E2E_SKIP_BUILD=1 node scripts/test-e2e-web-canvas.mjs -- tests/e2e-web-canvas/workerWebCanvas.agent-resume.spec.ts tests/e2e-web-canvas/workerWebCanvas.view-state.spec.ts`
252255

@@ -259,6 +262,24 @@ Latest verified on `2026-04-29` for bulk Agent startup restore profiling:
259262
- 20-Agent result after WebGL budgeting: `all-runtime-sessions-bound=2312ms`, `all-terminal-outputs-visible=6878ms`, `init=20`, `renderer-health-recover=0`, renderer split `8 webgl / 12 dom`.
260263
- Diagnostic comparison: the prior 20-Agent run created `28` terminal init cycles, `8` mixed WebGL/DOM nodes, and `3` renderer-health recoveries; the budgeted run removes that self-induced renderer churn.
261264

265+
Latest verified on `2026-04-30` for Desktop/Web UI terminal display parity and user calibration:
266+
267+
- `pnpm build`
268+
- `pnpm check`
269+
- `pnpm lint`
270+
- `pnpm test -- --run tests/unit/contexts/terminalDisplayReferenceAutoCapture.spec.tsx tests/unit/contexts/terminalNode.appearance-sync.spec.tsx tests/unit/contexts/terminalDisplayCalibration.spec.ts tests/unit/contexts/settingsPanel.spec.tsx tests/unit/contexts/settingsPanel.terminalDisplay.spec.tsx tests/unit/scripts/terminal-display-calibration.spec.ts tests/unit/contexts/terminalNode.effectiveDevicePixelRatio.spec.ts`
271+
- `OPENCOVE_PROFILE_WEB_HEADFUL=1 OPENCOVE_PROFILE_WEB_DEVICE_SCALE_FACTORS=1,2 OPENCOVE_PROFILE_ASSERT_PARITY=1 pnpm profile:terminal:display-parity`
272+
- `OPENCOVE_PROFILE_KEEP_USER_DATA=1 OPENCOVE_PROFILE_WEB_HEADFUL=1 OPENCOVE_PROFILE_WEB_DEVICE_SCALE_FACTORS=1,2 OPENCOVE_PROFILE_ASSERT_PARITY=1 pnpm profile:terminal:display-parity`
273+
- Result: Desktop and Web UI DPR 1/2 all converged to `81x24`, `cssCellWidthDelta=0`, `cssCellHeightDelta=0`, and `effectiveDprDelta=0`.
274+
- Calibration result: Web DPR 1 and Web DPR 2 each swept `39` appearance candidates and selected `{ fontSize: 13, lineHeight: 1, letterSpacing: 0 }` with score `0`.
275+
- Owner invariant result: local display compensation updates xterm display options and local FitAddon sizing, while shared font-size changes remain on the explicit `appearance_commit` geometry path.
276+
- First-client default reference result: with `OPENCOVE_PROFILE_KEEP_USER_DATA=1`, the persisted
277+
`terminalDisplayReference` was automatically captured from Desktop as `81x24`, cell `7.5x15`,
278+
`effectiveDpr=2`.
279+
- UX result: automatic reference setup and automatic calibration compensation are separate
280+
user-controlled settings and both default to enabled; normal UI shows match quality instead of the
281+
raw calibration score, while diagnostics still include the score.
282+
262283
Latest verified on `2026-04-25` for the current Desktop restore/hydration slice:
263284

264285
- `pnpm exec tsc -p tsconfig.json --noEmit`

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@
7878
"test:e2e:web-canvas": "node scripts/test-e2e-web-canvas.mjs",
7979
"test:e2e:web-shell": "node scripts/test-e2e-web-shell.mjs",
8080
"test:terminal:presentation": "node scripts/test-terminal-presentation-contract.mjs",
81+
"profile:terminal:display-parity": "node scripts/profile-terminal-display-parity.mjs",
8182
"lint": "oxlint .",
8283
"lint:fix": "oxlint --fix .",
8384
"opencove": "node src/app/cli/opencove.mjs",
Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
const DEFAULT_LINE_HEIGHTS = [1, 1.05, 1.1]
2+
const DEFAULT_LETTER_SPACINGS = [0]
3+
const DEFAULT_FONT_SIZE_RADIUS = 1.5
4+
const DEFAULT_FONT_SIZE_STEP = 0.25
5+
6+
function uniqueSortedNumbers(values) {
7+
return [
8+
...new Set(values.filter(value => Number.isFinite(value)).map(value => round(value, 3))),
9+
].sort((left, right) => left - right)
10+
}
11+
12+
function round(value, decimals = 2) {
13+
const factor = 10 ** decimals
14+
return Math.round(value * factor) / factor
15+
}
16+
17+
function buildCenteredRange(center, radius, step) {
18+
if (!Number.isFinite(center) || center <= 0) {
19+
return []
20+
}
21+
22+
const safeRadius = Number.isFinite(radius) && radius >= 0 ? radius : DEFAULT_FONT_SIZE_RADIUS
23+
const safeStep = Number.isFinite(step) && step > 0 ? step : DEFAULT_FONT_SIZE_STEP
24+
const start = center - safeRadius
25+
const end = center + safeRadius
26+
const values = []
27+
28+
for (let value = start; value <= end + safeStep / 2; value += safeStep) {
29+
if (value > 0) {
30+
values.push(value)
31+
}
32+
}
33+
34+
return uniqueSortedNumbers(values)
35+
}
36+
37+
function readPositiveNumber(value) {
38+
return typeof value === 'number' && Number.isFinite(value) && value > 0 ? value : null
39+
}
40+
41+
function readFiniteNumber(value) {
42+
return typeof value === 'number' && Number.isFinite(value) ? value : null
43+
}
44+
45+
function createDelta(left, right) {
46+
if (left === null || right === null) {
47+
return null
48+
}
49+
50+
return round(left - right, 4)
51+
}
52+
53+
function normalizeMeasurement(input, geometrySource) {
54+
const size = geometrySource === 'proposed' ? input.proposedGeometry : input.size
55+
const fallbackSize = geometrySource === 'proposed' ? input.size : input.proposedGeometry
56+
const resolvedSize = size ?? fallbackSize ?? null
57+
const renderMetrics = input.renderMetrics ?? {}
58+
59+
return {
60+
cols: readPositiveNumber(resolvedSize?.cols),
61+
rows: readPositiveNumber(resolvedSize?.rows),
62+
cssCellWidth: readPositiveNumber(renderMetrics.cssCellWidth),
63+
cssCellHeight: readPositiveNumber(renderMetrics.cssCellHeight),
64+
effectiveDpr: readPositiveNumber(renderMetrics.effectiveDpr),
65+
}
66+
}
67+
68+
function scoreDeltas(deltas) {
69+
const missingPenalty = 1_000_000
70+
const colsPenalty = deltas.cols === null ? missingPenalty : Math.abs(deltas.cols) * 1_000
71+
const rowsPenalty = deltas.rows === null ? missingPenalty : Math.abs(deltas.rows) * 1_000
72+
const cellWidthPenalty =
73+
deltas.cssCellWidth === null ? missingPenalty : Math.abs(deltas.cssCellWidth) * 100
74+
const cellHeightPenalty =
75+
deltas.cssCellHeight === null ? missingPenalty : Math.abs(deltas.cssCellHeight) * 100
76+
return round(colsPenalty + rowsPenalty + cellWidthPenalty + cellHeightPenalty, 4)
77+
}
78+
79+
function createPreferenceDistance(candidate, preferredCandidate) {
80+
if (!preferredCandidate) {
81+
return 0
82+
}
83+
84+
const fontSizeDistance = Math.abs(
85+
(readFiniteNumber(candidate?.fontSize) ?? 0) -
86+
(readFiniteNumber(preferredCandidate.fontSize) ?? 0),
87+
)
88+
const lineHeightDistance = Math.abs(
89+
(readFiniteNumber(candidate?.lineHeight) ?? 0) -
90+
(readFiniteNumber(preferredCandidate.lineHeight) ?? 0),
91+
)
92+
const letterSpacingDistance = Math.abs(
93+
(readFiniteNumber(candidate?.letterSpacing) ?? 0) -
94+
(readFiniteNumber(preferredCandidate.letterSpacing) ?? 0),
95+
)
96+
97+
return round(fontSizeDistance + lineHeightDistance * 10 + letterSpacingDistance, 4)
98+
}
99+
100+
function readPreferenceDistance(result) {
101+
return readFiniteNumber(result.preferenceDistance) ?? 0
102+
}
103+
104+
export function buildTerminalDisplayCalibrationCandidates({
105+
baseFontSize,
106+
fontSizes,
107+
lineHeights,
108+
letterSpacings,
109+
fontSizeRadius = DEFAULT_FONT_SIZE_RADIUS,
110+
fontSizeStep = DEFAULT_FONT_SIZE_STEP,
111+
} = {}) {
112+
const resolvedFontSizes =
113+
Array.isArray(fontSizes) && fontSizes.length > 0
114+
? uniqueSortedNumbers(fontSizes)
115+
: buildCenteredRange(baseFontSize, fontSizeRadius, fontSizeStep)
116+
const resolvedLineHeights =
117+
Array.isArray(lineHeights) && lineHeights.length > 0
118+
? uniqueSortedNumbers(lineHeights)
119+
: DEFAULT_LINE_HEIGHTS
120+
const resolvedLetterSpacings =
121+
Array.isArray(letterSpacings) && letterSpacings.length > 0
122+
? uniqueSortedNumbers(letterSpacings)
123+
: DEFAULT_LETTER_SPACINGS
124+
const candidates = []
125+
126+
for (const fontSize of resolvedFontSizes) {
127+
for (const lineHeight of resolvedLineHeights) {
128+
for (const letterSpacing of resolvedLetterSpacings) {
129+
candidates.push({ fontSize, lineHeight, letterSpacing })
130+
}
131+
}
132+
}
133+
134+
return candidates
135+
}
136+
137+
export function scoreTerminalDisplayCalibrationCandidate({
138+
targetMetrics,
139+
candidateMetrics,
140+
candidate,
141+
preferredCandidate,
142+
}) {
143+
const target = normalizeMeasurement(targetMetrics, 'size')
144+
const measurement = normalizeMeasurement(candidateMetrics, 'proposed')
145+
const deltas = {
146+
cols: createDelta(measurement.cols, target.cols),
147+
rows: createDelta(measurement.rows, target.rows),
148+
cssCellWidth: createDelta(measurement.cssCellWidth, target.cssCellWidth),
149+
cssCellHeight: createDelta(measurement.cssCellHeight, target.cssCellHeight),
150+
effectiveDpr: createDelta(measurement.effectiveDpr, target.effectiveDpr),
151+
}
152+
const score = scoreDeltas(deltas)
153+
154+
return {
155+
candidate,
156+
score,
157+
preferenceDistance: createPreferenceDistance(candidate, preferredCandidate),
158+
target,
159+
measurement,
160+
deltas,
161+
exactGeometry: deltas.cols === 0 && deltas.rows === 0,
162+
exactCellMetrics: deltas.cssCellWidth === 0 && deltas.cssCellHeight === 0,
163+
}
164+
}
165+
166+
export function rankTerminalDisplayCalibrationCandidates(results) {
167+
return [...results].sort((left, right) => {
168+
if (left.score !== right.score) {
169+
return left.score - right.score
170+
}
171+
172+
if (readPreferenceDistance(left) !== readPreferenceDistance(right)) {
173+
return readPreferenceDistance(left) - readPreferenceDistance(right)
174+
}
175+
176+
const leftFontSize = readFiniteNumber(left.candidate?.fontSize) ?? Number.POSITIVE_INFINITY
177+
const rightFontSize = readFiniteNumber(right.candidate?.fontSize) ?? Number.POSITIVE_INFINITY
178+
return leftFontSize - rightFontSize
179+
})
180+
}
181+
182+
export function summarizeTerminalDisplayCalibration(results, limit = 5) {
183+
const ranked = rankTerminalDisplayCalibrationCandidates(results)
184+
const best = ranked[0] ?? null
185+
return {
186+
best,
187+
topCandidates: ranked.slice(0, limit),
188+
candidateCount: results.length,
189+
}
190+
}
191+
192+
export function compareTerminalDisplayMetrics(desktop, web) {
193+
const desktopRender = desktop.renderMetrics ?? {}
194+
const webRender = web.renderMetrics ?? {}
195+
return {
196+
colsDelta: (web.size?.cols ?? 0) - (desktop.size?.cols ?? 0),
197+
rowsDelta: (web.size?.rows ?? 0) - (desktop.size?.rows ?? 0),
198+
cssCellWidthDelta: (webRender.cssCellWidth ?? 0) - (desktopRender.cssCellWidth ?? 0),
199+
cssCellHeightDelta: (webRender.cssCellHeight ?? 0) - (desktopRender.cssCellHeight ?? 0),
200+
effectiveDprDelta: (webRender.effectiveDpr ?? 0) - (desktopRender.effectiveDpr ?? 0),
201+
}
202+
}
203+
204+
export function isTerminalDisplayParity(comparison) {
205+
return (
206+
comparison.colsDelta === 0 &&
207+
comparison.rowsDelta === 0 &&
208+
Math.abs(comparison.cssCellWidthDelta) <= 0.05 &&
209+
Math.abs(comparison.cssCellHeightDelta) <= 0.05
210+
)
211+
}

0 commit comments

Comments
 (0)