-
Notifications
You must be signed in to change notification settings - Fork 277
fix: don't set reasoning effort for non-reasoning models #97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: don't set reasoning effort for non-reasoning models #97
Conversation
Signed-off-by: Huamin Chen <[email protected]>
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes a regression introduced when reasoning mode support was added, specifically ensuring that reasoning effort is not set for models that don't support reasoning features.
- Refactored the reasoning mode implementation from hardcoded model family detection to a configuration-driven approach
- Added model reasoning configurations to define which models support reasoning and their specific syntax requirements
- Updated test coverage to verify that unknown/unsupported models don't receive reasoning fields
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
config/config.yaml |
Adds model reasoning configurations defining supported models and their reasoning syntax |
src/semantic-router/pkg/config/config.go |
Implements config types and pattern matching logic for model reasoning configurations |
src/semantic-router/pkg/extproc/reason_mode_selector.go |
Refactors reasoning logic to use config-driven approach instead of hardcoded model detection |
src/semantic-router/pkg/extproc/reason_mode_selector_test.go |
Updates tests to use new config-driven approach and adds comprehensive test coverage |
src/semantic-router/pkg/extproc/reasoning_integration_test.go |
Updates integration tests to verify proper behavior for unknown models and config-driven reasoning |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Signed-off-by: Huamin Chen <[email protected]>
Signed-off-by: Huamin Chen <[email protected]>
|
let us stop the cycles round reasoning string match、pattern etc works for a better choice. I think we should have better approach to take care of the auto reasoning logics, the modelname can be very dynamic since that is set in vllm startup options, like i can set dsv31 as the modelname for deepseek-v3.1. we should have the fixed groups/family names for models like we have supported:
deepseek: gpt-oss: qwen the modelname can be very flexible, like for deepseek family, we can set (ds-v3,self-hosted/ds,official/deepseek, anything we want, since that is used in modelname in request body) we just care about what the model reasoning family is. if it is empty, we just think it is not supported hybrid reasoning. |
Signed-off-by: Huamin Chen <[email protected]>
|
@Xunzhuo model family is a good idea. It is added to the new config now. |
|
review comments addressed, merging it so I can add other PRs using this new config. |

What type of PR is this?
If the model doesn't support reasoning, don't set the reasoning effort in API call.
What this PR does / why we need it:
This is a regression when reasoning mode support is added
Which issue(s) this PR fixes:
Fixes #
Release Notes: Yes/No