docs: reasoning quickstart #110
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
|
can refer to it: https://docusaurus.io/zh-CN/docs/next/migration/v3#common-mdx-problems |
tao12345666333
force-pushed
the
docs/reasoning-quickstart
branch
from
6668536 to
927f1bc
Compare
rootfs
reviewed
Sep 10, 2025
rootfs
reviewed
Sep 10, 2025
rootfs
reviewed
Sep 10, 2025
rootfs
reviewed
Sep 10, 2025
rootfs
reviewed
Sep 10, 2025
| - model: qwen3-7b | ||
| score: 0.8 | ||
|
|
||
| - name: general |
There was a problem hiding this comment.
at the moment, all the categories must be from mmlu-pro. There is no general there. You can create an issue to support generic, free style categories and we can map the mmlu-pro categories to them.
There was a problem hiding this comment.
OK, #119 for tracking
rootfs
reviewed
Sep 10, 2025
|
@tao12345666333 thanks for writing this up! Some nit but most looks good to me. |
aeft
reviewed
Sep 11, 2025
| - A model only gets reasoning fields if it has a model_config.<MODEL>.reasoning_family that maps to a reasoning_families entry. | ||
| - DeepSeek/Qwen3 (chat_template_kwargs): the router injects chat_template_kwargs only when reasoning is enabled. When disabled, no chat_template_kwargs are added. | ||
| - GPT/GPT-OSS (reasoning_effort): when reasoning is enabled, the router sets reasoning_effort based on the category (fallback to default_reasoning_effort). When reasoning is disabled, if the request already contains reasoning_effort and the model’s family type is reasoning_effort, the router preserves the original value; otherwise it is absent. | ||
| - For more stable classification, you can add category descriptions in config and keep them semantically distinctive. |
There was a problem hiding this comment.
@rootfs It seems like OpenAIRouter.CategoryDescriptions Category.ReasoningDescription Category.Description aren't used in the code (i.e., don't affect performance) currently. Will we use it in the future?
There was a problem hiding this comment.
nice catch! the reasoning description is no-op, information only.
There was a problem hiding this comment.
I revised the description here and added clarification. 75f43ea#diff-5146355cf5a7882f89a4402aef8230aa9362971d09260549d6c5902f034fa6aaR91
Comment on lines
+111
to
+106
| Option B: Docker Compose | ||
| - docker compose up -d | ||
| - Exposes Envoy at http://localhost:8801 (proxying /v1/* to backends via the router) |
There was a problem hiding this comment.
Do you think #73 (review) can be satisfied here?
tao12345666333
force-pushed
the
docs/reasoning-quickstart
branch
from
df58016 to
39f95a1
Compare
Signed-off-by: Jintao Zhang <[email protected]>
Signed-off-by: Jintao Zhang <[email protected]>
tao12345666333
force-pushed
the
docs/reasoning-quickstart
branch
from
39f95a1 to
75f43ea
Compare
Signed-off-by: Jintao Zhang <[email protected]>
tao12345666333
force-pushed
the
docs/reasoning-quickstart
branch
from
75f43ea to
df27568
Compare
rootfs
approved these changes
Sep 12, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
docs: reasoning quickstart
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #51
Release Notes: Yes/No