Skip to content

Commit 057ab5d

Browse files
committed
test: cover the codex effort values that killed agents at config load
Two regressions from the 2026-08-17 incident, where a task dispatched at `effort:max` spawned `codex -c model_reasoning_effort=max` and codex rejected it while loading its config — before the prompt was read, so all three retries died identically. - agentCliSpawning: assert the CLI builder clamps max/ultra to xhigh for codex. The TUI builder already had this assertion; the CLI builder is the one that actually shipped the bad argv. - agentErrorAnalysis: classify the pathless "Error loading config.toml: unknown variant `max`" wording codex emits for a `-c` override rejection (the existing test used the file:line:col form), and assert the failure blocks on the first attempt instead of burning all three retries.
1 parent 4678635 commit 057ab5d

2 files changed

Lines changed: 35 additions & 0 deletions

File tree

server/services/agentCliSpawning.test.js

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -394,6 +394,20 @@ describe('buildCliSpawnConfig', () => {
394394
expect(config.args).toContain('model_reasoning_effort=xhigh');
395395
});
396396

397+
// Regression for the 2026-08-17 incident: codex's config enum stops at
398+
// `xhigh`, so a task dispatched at `effort:max` used to spawn
399+
// `codex -c model_reasoning_effort=max`, which codex rejects while LOADING
400+
// its config — before the prompt is read, so every retry died identically.
401+
// The TUI builder has the twin of this assertion; the CLI builder is the one
402+
// that actually shipped the bad argv.
403+
it('clamps max/ultra down to xhigh for codex rather than emitting a value it rejects', () => {
404+
for (const effort of ['max', 'ultra']) {
405+
const config = buildCliSpawnConfig({ id: 'codex', command: 'codex' }, 'gpt-5.4', {}, { effort });
406+
expect(config.args).toContain('model_reasoning_effort=xhigh');
407+
expect(config.args.join(' ')).not.toContain(`model_reasoning_effort=${effort}`);
408+
}
409+
});
410+
397411
it('adds --effort for claude', () => {
398412
const config = buildCliSpawnConfig({ id: 'claude-code', command: 'claude' }, 'claude-opus-4-8', {}, { effort: 'high' });
399413
expect(config.args[config.args.indexOf('--effort') + 1]).toBe('high');

server/services/agentErrorAnalysis.test.js

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,27 @@ describe('analyzeAgentFailure — ERROR_PATTERNS classification', () => {
129129
expect(analysis.message).not.toContain('/home/x/');
130130
});
131131

132+
// Same incident, the OTHER source: PortOS's own `-c model_reasoning_effort=max`
133+
// override. Codex reports an override rejection with the same "Error loading
134+
// config.toml" lead but NO file:line:col prefix, so this wording is what the
135+
// three failing runs on 2026-08-17 actually recorded — verified by running
136+
// `codex exec -c model_reasoning_effort=max -`. Classifying it (not falling
137+
// through to `unknown`) is what keeps it a Tier 1 config fix instead of an
138+
// escalated investigation. `resolveCliEffort` is what stops PortOS emitting
139+
// the bad value in the first place (server/lib/providerModels.js).
140+
it('classifies a codex -c override rejection (no file path in the message)', () => {
141+
const line = 'Error loading config.toml: unknown variant `max`, expected one of `none`, `minimal`, `low`, `medium`, `high`, `xhigh`\n in `model_reasoning_effort`';
142+
const analysis = analyzeAgentFailure(withLead(line), { id: 't' }, 'gpt-5.6-sol');
143+
expect(analysis.category).toBe('cli-config-invalid');
144+
expect(analysis.rejectedConfigKey).toBe('model_reasoning_effort');
145+
expect(analysis.rejectedConfigValue).toBe('max');
146+
// A CLI that dies at config load dies identically every time — the run must
147+
// block on the FIRST failure rather than burn all three retries on it.
148+
const decision = resolveFailedTaskDecision({ id: 't', metadata: {} }, analysis);
149+
expect(decision.status).toBe('blocked');
150+
expect(decision.metadataUpdates.blockedCategory).toBe('cli-config-invalid');
151+
});
152+
132153
it('classifies a config file that fails to load at all', () => {
133154
const analysis = analyzeAgentFailure(withLead('Error loading config.toml: expected a value after the equals sign'), { id: 't' }, 'x');
134155
expect(analysis.category).toBe('cli-config-invalid');

0 commit comments

Comments
 (0)