Skip to content

test: add regression test for issue #199 (PDF processing with OpenAI models)#374

Merged
robert-j-y merged 5 commits intomainfrom
devin/1769563139-issue-199-regression-test
Jan 28, 2026
Merged

test: add regression test for issue #199 (PDF processing with OpenAI models)#374
robert-j-y merged 5 commits intomainfrom
devin/1769563139-issue-199-regression-test

Conversation

@robert-j-y
Copy link
Contributor

@robert-j-y robert-j-y commented Jan 28, 2026

Description

Adds a regression test for issue #199 (PDF Processing Fails on GPT-4.1 and GPT-5 Models via OpenRouter) to the e2e/issues/ directory, following the pattern established in PR #369.

Test coverage:

  • PDF processing with openai/gpt-4.1 (exact model from issue report)
  • PDF processing with openai/gpt-5 (exact model from issue report)
  • Uses base64-encoded Bitcoin whitepaper PDF as the test document

Test file documentation:

  • Includes only known facts from the issue (reported error, error details from comments)
  • No root cause speculation - keeps comments factual

Human Review Checklist

  • Confirm the external PDF URL (bitcoin.org/bitcoin.pdf) is acceptable for CI reliability
  • Verify the 120s timeout is appropriate for PDF processing tests with gpt-4.1 and gpt-5

Checklist

  • I have run pnpm stylecheck and pnpm typecheck
  • I have run pnpm test and all tests pass
  • I have added tests for my changes (if applicable)
  • I have updated documentation (if applicable) - N/A

Changeset

  • I have run pnpm changeset to create a changeset file

Note: Empty changeset used since this is a test-only change that doesn't need a release.


Link to Devin run: https://app.devin.ai/sessions/440e9e282978425b8ddb9c54bfe9b787
Requested by: Robert Yeakel (@robert-j-y)

devin-ai-integration bot and others added 5 commits January 28, 2026 01:21
…models)

Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
@robert-j-y robert-j-y merged commit 1182d6d into main Jan 28, 2026
2 checks passed
@robert-j-y robert-j-y deleted the devin/1769563139-issue-199-regression-test branch January 28, 2026 02:26
kesavan-byte pushed a commit to osm-API/ai-sdk-provider that referenced this pull request Feb 13, 2026
…g with OpenAI models) (OpenRouterTeam#374)

* test: add regression test for issue OpenRouterTeam#199 (PDF processing with OpenAI models)

Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>

* fix: use exact model (gpt-4.1) from issue report in regression test

Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>

* test: add gpt-5 test case to match issue report exactly

Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>

* docs: remove root cause speculation, keep only known facts from issue

Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>

* chore: use empty changeset for test-only change

Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments