Skip to content

Commit 5da8258

Browse files
test: add issue regression test infrastructure and issue OpenRouterTeam#194 test (OpenRouterTeam#379)
* test: add e2e/issues directory with issue OpenRouterTeam#194 regression test - Create e2e/issues/ directory for issue-specific regression tests - Add regression test for issue OpenRouterTeam#194 (Grok 4 Fast Invalid JSON response) - Document the e2e/issues/ pattern in CONTRIBUTING.md The test verifies that Grok 4 Fast works correctly with: - generateText (non-streaming) - streamText (streaming) - Tool calls Issue: OpenRouterTeam#194 Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai> * fix: remove internal repo reference from test comment Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai> * feat: separate issue regression tests from main e2e suite - Exclude e2e/issues/ from vitest.e2e.config.ts - Add vitest.issues.config.ts for issue-specific tests - Add pnpm test:issues script to package.json - Update CONTRIBUTING.md with test:issues documentation Issue regression tests are now run separately to avoid slowing down the main e2e suite with potentially slow/rate-limited API calls. Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai> * refactor: remove speculative root cause from test comments - Update test comment to only include known facts (error message, model) - Update CONTRIBUTING.md template to clarify only known facts should be included - Add note about not including speculative root cause analysis Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent 3db5844 commit 5da8258

File tree

2 files changed

+8
-13
lines changed

2 files changed

+8
-13
lines changed

CONTRIBUTING.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -113,11 +113,10 @@ pnpm test:issues
113113
* Regression test for GitHub issue #{number}
114114
* https://github.com/OpenRouterTeam/ai-sdk-provider/issues/{number}
115115
*
116-
* Issue: {brief description of the issue}
116+
* Reported error: {exact error message from the issue}
117+
* Model: {model ID if applicable}
117118
*
118-
* Root cause: {explanation of what caused the issue}
119-
*
120-
* This test verifies that {what the test checks}
119+
* This test verifies that {what the test checks}.
121120
*/
122121
import { ... } from 'ai';
123122
import { describe, expect, it, vi } from 'vitest';
@@ -128,6 +127,8 @@ describe('Issue #{number}: {brief description}', () => {
128127
});
129128
```
130129

130+
> **Note:** Only include known facts from the issue report (error message, model, SDK version). Do not include speculative root cause analysis.
131+
131132
**When to add issue regression tests:**
132133
- Bug fixes that have a clear reproduction case
133134
- Issues that could potentially regress

e2e/issues/issue-194-grok-invalid-json.test.ts

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,10 @@
22
* Regression test for GitHub issue #194
33
* https://github.com/OpenRouterTeam/ai-sdk-provider/issues/194
44
*
5-
* Issue: "Can not use grok 4 fast" - AI_APICallError: Invalid JSON response
5+
* Reported error: "AI_APICallError: Invalid JSON response"
6+
* Model: x-ai/grok-4-fast
67
*
7-
* Root cause: Early Grok 4 Fast API instability (model was only 11 days old
8-
* when the issue was reported on September 30, 2025). Multiple API-side fixes
9-
* were deployed after the issue was reported.
10-
*
11-
* This test verifies that Grok 4 Fast works correctly with the SDK:
12-
* - doGenerate returns valid JSON responses
13-
* - doStream returns valid streaming responses
14-
* - Tool calls work correctly
8+
* This test verifies that Grok 4 Fast works correctly with the SDK.
159
*/
1610
import { generateText, streamText, tool } from 'ai';
1711
import { describe, expect, it, vi } from 'vitest';

0 commit comments

Comments
 (0)