test: add issue regression test infrastructure and issue #194 test#379
Merged
robert-j-y merged 5 commits intomainfrom Jan 28, 2026
Merged
test: add issue regression test infrastructure and issue #194 test#379robert-j-y merged 5 commits intomainfrom
robert-j-y merged 5 commits intomainfrom
Conversation
- Create e2e/issues/ directory for issue-specific regression tests - Add regression test for issue #194 (Grok 4 Fast Invalid JSON response) - Document the e2e/issues/ pattern in CONTRIBUTING.md The test verifies that Grok 4 Fast works correctly with: - generateText (non-streaming) - streamText (streaming) - Tool calls Issue: #194 Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
- Exclude e2e/issues/ from vitest.e2e.config.ts - Add vitest.issues.config.ts for issue-specific tests - Add pnpm test:issues script to package.json - Update CONTRIBUTING.md with test:issues documentation Issue regression tests are now run separately to avoid slowing down the main e2e suite with potentially slow/rate-limited API calls. Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
- Update test comment to only include known facts (error message, model) - Update CONTRIBUTING.md template to clarify only known facts should be included - Add note about not including speculative root cause analysis Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
robert-j-y
changed the title
Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
kesavan-byte
pushed a commit
to osm-API/ai-sdk-provider
that referenced
this pull request
Feb 13, 2026
…am#194 test (OpenRouterTeam#379) * test: add e2e/issues directory with issue OpenRouterTeam#194 regression test - Create e2e/issues/ directory for issue-specific regression tests - Add regression test for issue OpenRouterTeam#194 (Grok 4 Fast Invalid JSON response) - Document the e2e/issues/ pattern in CONTRIBUTING.md The test verifies that Grok 4 Fast works correctly with: - generateText (non-streaming) - streamText (streaming) - Tool calls Issue: OpenRouterTeam#194 Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai> * fix: remove internal repo reference from test comment Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai> * feat: separate issue regression tests from main e2e suite - Exclude e2e/issues/ from vitest.e2e.config.ts - Add vitest.issues.config.ts for issue-specific tests - Add pnpm test:issues script to package.json - Update CONTRIBUTING.md with test:issues documentation Issue regression tests are now run separately to avoid slowing down the main e2e suite with potentially slow/rate-limited API calls. Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai> * refactor: remove speculative root cause from test comments - Update test comment to only include known facts (error message, model) - Update CONTRIBUTING.md template to clarify only known facts should be included - Add note about not including speculative root cause analysis Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds infrastructure for issue-specific regression tests and creates the first regression test for issue #194 (Grok 4 Fast Invalid JSON response).
Changes:
e2e/issues/directory for issue regression tests, separated from the main e2e suitevitest.issues.config.tsandpnpm test:issuesscript to run issue tests independentlypnpm test:e2eto keep the main suite fastgenerateText,streamText, and tool calls withx-ai/grok-4-fastWhy separate issue tests?
Issue regression tests may hit different models/APIs that could be slow or rate-limited, so they run separately from the main e2e suite.
Updates since last revision:
Checklist
pnpm stylecheckandpnpm typecheckpnpm testand all tests passChangeset
pnpm changesetto create a changeset fileEmpty changeset used since this is a test-only change that doesn't affect the public API.
Link to Devin run: https://app.devin.ai/sessions/ead0af897ddc4763992859f1447e8ec8
Requested by: Robert Yeakel (@robert-j-y)