Skip to content

Commit 2fbcd36

Browse files
authored
🤖 Add integration tests for stream error recovery (no amnesia) (#333)
## Summary Adds integration test to verify that stream error recovery preserves context (no amnesia bug). ## Changes - **Debug IPC for testing**: Added `DEBUG_TRIGGER_STREAM_ERROR` IPC channel - **StreamManager debug method**: `debugTriggerStreamError()` triggers artificial stream errors that follow the same code path as real errors - **Integration test**: Single error + resume scenario verifies context preservation via **structured markers** ## Test Design **Structured-marker approach** for precise validation: **Test Flow:** 1. Generate unique nonce for test run (random 10-char identifier) 2. Model counts 1-100 using structured format: `${nonce}-<n>: <word>` (e.g. `ai7qcnc20g-1: one`) 3. Collect stream deltas until ≥10 complete markers detected 4. Trigger artificial network error mid-stream 5. Resume stream and wait for completion 6. Verify final message has **both** properties: - **(a) Prefix preservation**: Starts with exact pre-error streamed text - **(b) Exact continuation**: Contains next sequential marker (${nonce}-11) shortly after prefix **Validation:** - Pre-error content captured from stream-delta events (user-visible data path) - Stable prefix truncated to last complete marker line (no partial markers) - Assertions directly prove both amnesia-prevention properties - No coupling to internal storage formats or metadata **Why this approach:** - **Precise**: Detects exact continuation (not just "some work done") - **Unambiguous**: Random nonce makes false positives virtually impossible - **Robust**: Structured format less likely to confuse model than natural language - **Fast**: Haiku 4.5 completes in ~18-21 seconds ## Bug Fix Also fixed event collection bug in `collectStreamUntil`: properly track consumed deltas to avoid returning the same event multiple times. Previous logic returned first matching event on every poll, causing duplicate processing. ## Related Follow-up to #331 which fixed the amnesia bug by preserving accumulated parts on error. ## Test Results ✅ Test passes reliably in ~18-21 seconds ✅ Validates **exact** prefix preservation and continuation ✅ No flaky failures from timing issues ✅ Integration tests pass: 1 passed, 1 total _Generated with `cmux`_
1 parent d363128 commit 2fbcd36

File tree

4 files changed

+411
-0
lines changed

4 files changed

+411
-0
lines changed

src/constants/ipc-constants.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,9 @@ export const IPC_CHANNELS = {
3939
// Window channels
4040
WINDOW_SET_TITLE: "window:setTitle",
4141

42+
// Debug channels (for testing only)
43+
DEBUG_TRIGGER_STREAM_ERROR: "debug:triggerStreamError",
44+
4245
// Dynamic channel prefixes
4346
WORKSPACE_CHAT_PREFIX: "workspace:chat:",
4447
WORKSPACE_METADATA: "workspace:metadata",

src/services/ipcMain.ts

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -855,6 +855,25 @@ export class IpcMain {
855855
log.error(`Failed to open terminal: ${message}`);
856856
}
857857
});
858+
859+
// Debug IPC - only for testing
860+
ipcMain.handle(
861+
IPC_CHANNELS.DEBUG_TRIGGER_STREAM_ERROR,
862+
(_event, workspaceId: string, errorMessage: string) => {
863+
try {
864+
// eslint-disable-next-line @typescript-eslint/dot-notation -- accessing private member for testing
865+
const triggered = this.aiService["streamManager"].debugTriggerStreamError(
866+
workspaceId,
867+
errorMessage
868+
);
869+
return { success: triggered };
870+
} catch (error) {
871+
const message = error instanceof Error ? error.message : String(error);
872+
log.error(`Failed to trigger stream error: ${message}`);
873+
return { success: false, error: message };
874+
}
875+
}
876+
);
858877
}
859878

860879
/**

src/services/streamManager.ts

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1271,4 +1271,62 @@ export class StreamManager extends EventEmitter {
12711271
this.emitPartAsEvent(typedWorkspaceId, streamInfo.messageId, part);
12721272
}
12731273
}
1274+
1275+
/**
1276+
* DEBUG ONLY: Trigger an artificial stream error for testing
1277+
* This method allows integration tests to simulate stream errors without
1278+
* mocking the AI SDK or network layer. It triggers the same error handling
1279+
* path as genuine stream errors by aborting the stream and manually triggering
1280+
* the error event (since abort alone doesn't throw, it just sets a flag).
1281+
*/
1282+
debugTriggerStreamError(workspaceId: string, errorMessage: string): boolean {
1283+
const typedWorkspaceId = workspaceId as WorkspaceId;
1284+
const streamInfo = this.workspaceStreams.get(typedWorkspaceId);
1285+
1286+
// Only trigger error if stream is actively running
1287+
if (
1288+
!streamInfo ||
1289+
(streamInfo.state !== StreamState.STARTING && streamInfo.state !== StreamState.STREAMING)
1290+
) {
1291+
return false;
1292+
}
1293+
1294+
// Abort the stream first
1295+
streamInfo.abortController.abort(new Error(errorMessage));
1296+
1297+
// Update streamInfo metadata with error (so subsequent flushes preserve it)
1298+
streamInfo.initialMetadata = {
1299+
...streamInfo.initialMetadata,
1300+
error: errorMessage,
1301+
errorType: "network",
1302+
};
1303+
1304+
// Write error state to partial.json (same as real error handling)
1305+
const errorPartialMessage: CmuxMessage = {
1306+
id: streamInfo.messageId,
1307+
role: "assistant",
1308+
metadata: {
1309+
historySequence: streamInfo.historySequence,
1310+
timestamp: streamInfo.startTime,
1311+
model: streamInfo.model,
1312+
partial: true,
1313+
error: errorMessage,
1314+
errorType: "network", // Test errors are network-like
1315+
...streamInfo.initialMetadata,
1316+
},
1317+
parts: streamInfo.parts,
1318+
};
1319+
void this.partialService.writePartial(workspaceId, errorPartialMessage);
1320+
1321+
// Emit error event (same as real error handling)
1322+
this.emit("error", {
1323+
type: "error",
1324+
workspaceId,
1325+
messageId: streamInfo.messageId,
1326+
error: errorMessage,
1327+
errorType: "network",
1328+
} as ErrorEvent);
1329+
1330+
return true;
1331+
}
12741332
}

0 commit comments

Comments
 (0)