Skip to content

Conversation

@Xunzhuo
Copy link
Member

@Xunzhuo Xunzhuo commented Sep 29, 2025

What type of PR is this?

feat: support /v1/models in direct response

@netlify
Copy link

netlify bot commented Sep 29, 2025

Deploy Preview for vllm-semantic-router ready!

Name Link
🔨 Latest commit ce9c4f9
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/68dbbadd3bca670008eed1d8
😎 Deploy Preview https://deploy-preview-283--vllm-semantic-router.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions
Copy link

github-actions bot commented Sep 29, 2025

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 src

Owners: @rootfs, @Xunzhuo, @wangchen615
Files changed:

  • src/semantic-router/pkg/extproc/models_endpoint_test.go
  • src/semantic-router/pkg/extproc/request_handler.go

vLLM

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

@rootfs
Copy link
Collaborator

rootfs commented Sep 29, 2025

@Xunzhuo can you review this solution too?
#186 (comment)

@Xunzhuo
Copy link
Member Author

Xunzhuo commented Sep 29, 2025

This need additional xDS config generation which is not standard for InferencePool implementation, so that using direct response. Otherwise we need every Gateway implementation to inject one more cluster to support our models api.

@rootfs
Copy link
Collaborator

rootfs commented Sep 29, 2025

@Xunzhuo sounds good. Can you run precommit? It is ready to go.
cc @JaredforReal

@JaredforReal
Copy link
Collaborator

Sounds good to me

@Xunzhuo
Copy link
Member Author

Xunzhuo commented Sep 30, 2025

tested and move this forward

@Xunzhuo Xunzhuo merged commit f982534 into main Sep 30, 2025
9 checks passed
Aias00 pushed a commit to Aias00/semantic-router that referenced this pull request Oct 4, 2025
Aias00 pushed a commit to Aias00/semantic-router that referenced this pull request Oct 4, 2025
@Xunzhuo Xunzhuo deleted the feat-models branch October 7, 2025 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants