-
Notifications
You must be signed in to change notification settings - Fork 255
feat: implement batch classification API #24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: implement batch classification API #24
Conversation
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
|
@OneZero-Y thanks for contributing again! Would you please run |
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
@OneZero-Y can you share your local test with batch classify and potentially add this to the doc too? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on it, please update the classify API docs.
e933caa to
289d231
Compare
Signed-off-by: OneZero-Y <[email protected]>
Signed-off-by: OneZero-Y <[email protected]>
Signed-off-by: OneZero-Y <[email protected]>
289d231 to
76e3916
Compare
What type of PR is this?
Feature - implement batch classification API
What this PR does / why we need it:
This PR implements batch classification API functionality, resolving the API completeness issue where
POST /api/v1/classify/batchreturned HTTP 501 "Not Implemented" error.Changes:
Configuration Examples:
Small batch request (sequential processing):
{ "texts": ["solve math equation", "write business plan", "chemistry experiment"] }Large batch request (concurrent processing):
{ "texts": [ "solve differential equation", "business strategy analysis", "chemistry reaction", "physics calculation", "market research", "mathematical modeling", "financial planning", "scientific experiment" ], "options": {"return_probabilities": true} }Configuration Examples:
Which issue(s) this PR fixes:
Fixes: implement batch classification API