feat: extract thinking blocks from LLM responses#91
Conversation
|
Thanks for your first pull request! We are excited to have you contribute. |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the LLM response handling by introducing a dedicated mechanism to capture and expose the 'thinking' or reasoning content that some models provide. This allows applications to access the model's internal thought process, which can be valuable for debugging, understanding, or presenting more detailed responses, without altering the primary content output. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated approval for solo maintainer project
This PR has passed all automated quality gates:
- ✅ Static analysis (PHPStan)
- ✅ Code style (PHP-CS-Fixer)
- ✅ Unit & functional tests
- ✅ Security scanning
- ✅ Dependency review
See SECURITY_CONTROLS.md for compensating controls documentation.
github-actions
bot
added
tests
There was a problem hiding this comment.
Pull request overview
Adds first-class support for “thinking/reasoning” content returned by some LLMs, by extracting <think>...</think> blocks (and Claude native thinking blocks) into a dedicated CompletionResponse::$thinking field while keeping content clean.
Changes:
- Extend
CompletionResponsewith?string $thinkingandhasThinking(). - Add
AbstractProvider::extractThinkingBlocks()and wire it into OpenAI/OpenRouter/Gemini providers; ClaudeProvider additionally handles nativethinkingblocks. - Add/extend unit tests covering thinking extraction behavior across providers and the domain model.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
Classes/Domain/Model/CompletionResponse.php |
Adds thinking property and hasThinking() helper. |
Classes/Provider/AbstractProvider.php |
Adds extractThinkingBlocks() helper and passes thinking through createCompletionResponse(). |
Classes/Provider/ClaudeProvider.php |
Extracts native Claude thinking blocks + inline <think> tags into CompletionResponse::$thinking. |
Classes/Provider/OpenAiProvider.php |
Extracts inline <think> tags from OpenAI chat completion content. |
Classes/Provider/OpenRouterProvider.php |
Extracts inline <think> tags from OpenRouter content for chat + tools. |
Classes/Provider/GeminiProvider.php |
Extracts inline <think> tags from Gemini content for chat + tools. |
Tests/Unit/Domain/Model/CompletionResponseTest.php |
Adds tests for thinking constructor/default and hasThinking() behavior. |
Tests/Unit/Provider/AbstractProviderMutationTest.php |
Adds mutation-killing tests for extractThinkingBlocks(). |
Tests/Unit/Provider/ClaudeProviderTest.php |
Adds tests for native + inline thinking extraction, including tools flow. |
Tests/Unit/Provider/OpenAiProviderTest.php |
Adds tests for inline <think> extraction for OpenAI chat completions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new feature to extract <think> blocks from LLM responses, which is a valuable addition for models that provide reasoning. The implementation is well-structured, adding a thinking property to CompletionResponse and a helper extractThinkingBlocks in AbstractProvider. The changes are consistently applied across various providers like Claude, OpenAI, and Gemini. The test coverage for the new functionality is comprehensive and well-written.
I have one suggestion for improvement. In ClaudeProvider, there's an opportunity to reduce code duplication by extracting the response processing logic into a shared helper method. This is a medium-severity suggestion aimed at improving maintainability.
CybotTM
force-pushed
the
feat/extract-thinking-blocks
branch
from
04947c9 to
7459a76
Compare
There was a problem hiding this comment.
Automated approval for solo maintainer project
This PR has passed all automated quality gates:
- ✅ Static analysis (PHPStan)
- ✅ Code style (PHP-CS-Fixer)
- ✅ Unit & functional tests
- ✅ Security scanning
- ✅ Dependency review
See SECURITY_CONTROLS.md for compensating controls documentation.
There was a problem hiding this comment.
Automated approval for solo maintainer project
This PR has passed all automated quality gates:
- ✅ Static analysis (PHPStan)
- ✅ Code style (PHP-CS-Fixer)
- ✅ Unit & functional tests
- ✅ Security scanning
- ✅ Dependency review
See SECURITY_CONTROLS.md for compensating controls documentation.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Automated approval for solo maintainer project
This PR has passed all automated quality gates:
- ✅ Static analysis (PHPStan)
- ✅ Code style (PHP-CS-Fixer)
- ✅ Unit & functional tests
- ✅ Security scanning
- ✅ Dependency review
See SECURITY_CONTROLS.md for compensating controls documentation.
Add `thinking` property to CompletionResponse for models that emit reasoning content (DeepSeek, Qwen via OpenRouter, Claude extended thinking). The property separates thinking from output so consumers get clean content without trailing </think> artifacts. - Add ?string $thinking and hasThinking() to CompletionResponse - Add extractThinkingBlocks() helper to AbstractProvider - ClaudeProvider: extract native thinking blocks + inline <think> tags - OpenAI/OpenRouter/Gemini: extract inline <think> tags from content - Add comprehensive tests for all extraction paths Signed-off-by: Sebastian Mendel <info@sebastianmendel.de>
- Use trim() in hasThinking() for consistency with providers - Replace <think> tags with space to prevent word-gluing - Use array+implode for Claude native thinking blocks - Add word-gluing prevention test Signed-off-by: Sebastian Mendel <info@sebastianmendel.de>
Use /[ \t]+/ instead of /\s+/ to only normalize horizontal whitespace, preserving newlines and markdown formatting in completion content. Signed-off-by: Sebastian Mendel <info@sebastianmendel.de>
CybotTM
force-pushed
the
feat/extract-thinking-blocks
branch
from
8c8cd6a to
9fb582a
Compare
There was a problem hiding this comment.
Automated approval for solo maintainer project
This PR has passed all automated quality gates:
- ✅ Static analysis (PHPStan)
- ✅ Code style (PHP-CS-Fixer)
- ✅ Unit & functional tests
- ✅ Security scanning
- ✅ Dependency review
See SECURITY_CONTROLS.md for compensating controls documentation.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #91 +/- ##
============================================
+ Coverage 92.44% 92.48% +0.03%
- Complexity 1928 1937 +9
============================================
Files 77 77
Lines 6814 6849 +35
============================================
+ Hits 6299 6334 +35
Misses 515 515
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Summary
?string $thinkingproperty andhasThinking()toCompletionResponsefor models that emit reasoning content (DeepSeek, Qwen via OpenRouter, Claude extended thinking)extractThinkingBlocks()helper toAbstractProvider— strips<think>...</think>tags from content, returns clean text + separated thinkingthinkingcontent blocks (API-level) and inline<think>tags<think>tags from content strings$thinkingdefaults tonull, no breaking changesTest plan
CompletionResponse::hasThinking()— 5 new tests (present, null, empty string)extractThinkingBlocks()— 8 new tests (no tags, single, multiple, multiline, empty, case-insensitive, whitespace, trimming)