-
Notifications
You must be signed in to change notification settings - Fork 273
feat: add system prompt toggle endpoint #301
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Huamin Chen <[email protected]>
Signed-off-by: Huamin Chen <[email protected]>
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Adds runtime management of category-specific system prompts, including enable/disable and injection mode control, exposed via optional API endpoints.
- Introduces system prompt configuration endpoints guarded by a CLI flag.
- Adds per-category enable flag and injection mode ("replace" vs "insert") and modifies injection logic accordingly.
- Adds tests for endpoint security and functionality plus CLI integration.
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/semantic-router/pkg/services/classification.go | Adds getters/setters to expose and update router config at runtime. |
| src/semantic-router/pkg/extproc/request_handler.go | Adds injection mode support and switches to using live global config; adds extensive logging. |
| src/semantic-router/pkg/config/config.go | Adds category-level system prompt enable flag, mode handling, and lookup helpers. |
| src/semantic-router/pkg/api/server.go | Adds flag-controlled system prompt management endpoints and handlers. |
| src/semantic-router/pkg/api/server_test.go | Introduces tests for security gating and functional updates of system prompt settings. |
| src/semantic-router/cmd/main.go | Adds CLI flag and passes it into server startup. |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Signed-off-by: Huamin Chen <[email protected]>
Signed-off-by: Huamin Chen <[email protected]>
Signed-off-by: Huamin Chen <[email protected]>
Signed-off-by: Huamin Chen <[email protected]>
Signed-off-by: Huamin Chen <[email protected]>
Signed-off-by: Huamin Chen <[email protected]>
|
merging it for now, need it for next PR. |
|
overall LGTM |
* feat: add system prompt toggle endpoint Signed-off-by: Huamin Chen <[email protected]> * add cli option to explicitly enable the prompt toggle Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * adding system prompt endpoint option to makefile target Signed-off-by: Huamin Chen <[email protected]> * update doc Signed-off-by: Huamin Chen <[email protected]> * address review comment Signed-off-by: Huamin Chen <[email protected]> --------- Signed-off-by: Huamin Chen <[email protected]> Signed-off-by: liuhy <[email protected]>
* feat: add system prompt toggle endpoint Signed-off-by: Huamin Chen <[email protected]> * add cli option to explicitly enable the prompt toggle Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * adding system prompt endpoint option to makefile target Signed-off-by: Huamin Chen <[email protected]> * update doc Signed-off-by: Huamin Chen <[email protected]> * address review comment Signed-off-by: Huamin Chen <[email protected]> --------- Signed-off-by: Huamin Chen <[email protected]> Signed-off-by: liuhy <[email protected]>
* feat: add system prompt toggle endpoint Signed-off-by: Huamin Chen <[email protected]> * add cli option to explicitly enable the prompt toggle Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * adding system prompt endpoint option to makefile target Signed-off-by: Huamin Chen <[email protected]> * update doc Signed-off-by: Huamin Chen <[email protected]> * address review comment Signed-off-by: Huamin Chen <[email protected]> --------- Signed-off-by: Huamin Chen <[email protected]> Signed-off-by: liuhy <[email protected]>
* feat: add system prompt toggle endpoint Signed-off-by: Huamin Chen <[email protected]> * add cli option to explicitly enable the prompt toggle Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * fix test failure Signed-off-by: Huamin Chen <[email protected]> * adding system prompt endpoint option to makefile target Signed-off-by: Huamin Chen <[email protected]> * update doc Signed-off-by: Huamin Chen <[email protected]> * address review comment Signed-off-by: Huamin Chen <[email protected]> --------- Signed-off-by: Huamin Chen <[email protected]> Signed-off-by: liuhy <[email protected]>

What type of PR is this?
Add an endpoint to set system prompt injection to allow runtime management. This endpoint is only turned on with explicit command line option
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Release Notes: Yes/No