-
-
Notifications
You must be signed in to change notification settings - Fork 10.6k
[Model] Add ApertusToolParser #26307
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces the ApertusToolParser
for handling tool calls with Apertus models. The implementation covers both non-streaming and streaming responses. While the overall structure is good, I've found a critical bug in the streaming logic. The state of previously parsed tool calls is not consistently updated, which leads to dropping the initial parts of streamed tool arguments. I've provided a code suggestion to fix this issue. Once addressed, this will be a solid addition.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Signed-off-by: swan.blanc <[email protected]>
Signed-off-by: swan.blanc <[email protected]>
Signed-off-by: swan.blanc <[email protected]>
Signed-off-by: swan.blanc <[email protected]>
Signed-off-by: swan.blanc <[email protected]>
Signed-off-by: swan.blanc <[email protected]>
866c589
to
4ac6419
Compare
Hi 🤗 @aarnphm / @chaunceyjiang my PR is ready, thanks in advance for your help ! |
Hello the Team !
Purpose
Add Apertus tool parser class
Fix Apertus jinja template to accept both tool call format (chat/completion and responses)
Test Plan
Test on
swiss-ai/Apertus-70B-Instruct-2509
with 4xL40s with the base imagedocker.io/vllm/vllm-openai:v0.10.2
With and without streaming
Testing code:
Test Result
Return the proper tool
Essential Elements of an Effective PR Description Checklist
supported_models.md
andexamples
for a new model.