vLLM Semantic Router WG Initialization 👋

[Xunzhuo]

WG Initialization 🌍

We are looking for interests around vLLM Semantic Router project and separate it into different WGs.

Please comment if you are interested in one or more.

vLLM Semantic Router Community WG ⛰️

This section is about setting WG around this project, to gather focus on specify areas.

Community Promotion 🔝

We are grateful for any contributions, and if you show consistent contributions to the above specify area, you will be promoting as its maintainer after votes from maintainer team, and you will be invited to semantic-router-maintainer group, and granted WRITE access to this repo.

Online Pages

Work Groups: https://vllm-semantic-router.com/community/work-groups

[ftarlaci]

Interested in contributing to RouterCore and possibly Environment WGs.

[Xunzhuo]

Thank you @ftarlaci, welcome! We are setting up milestones, will separate specific issues after that : )

[LYZJU2019]

Interested in area/networking WG.

[rootfs]

@Xunzhuo thanks for the idea! Since this project has many research opportunities, we should also consider how to make that happen too.

[Xunzhuo]

@rootfs Sounds good, added that.

[ZhengKai91]

Interested in contributing to RouterCore

[Xunzhuo]

Cool, welcome @ZhengKai91

[ngoiyaeric]

I need to use Semantic router within a month to route into a plethora of similar but slightly different models. Would I have to fine tune prompts into labels for this classifier, I think the setup isn't very straight forward. would also like to contribute, lmk

[rootfs]

@ngoiyaeric can you elaborate your use case? In the current implementation, if you want to set up multiple backend models, you can set the model name in your request to "auto", and rank the models on their performance wrt categories in the config.yaml file. This is a bit manual atm. We are planning to move this process into a more automated workflow.

[Peterren]

interested in RouterCore / Bench / UserExperience

[Xunzhuo]

Welcome @Peterren! We are creating roadmap for v0.1, feel free to checkout anything interesting and fun.

[rahulgurnani]

I am interested in contributing to the project. After reading through the roadmap, I can start with helping with the networking task:
"ExtProc header/body mutation consistent with LLM-d/Envoy AI Gateway filter chains (documented setup for each)".

I would love to contribute to other areas as well.

Thanks!

[Xunzhuo]

Welcome and thanks @rahulgurnani

[srampal]

Hello @Xunzhuo I am interested in contributing in multiple areas including Networking, Routing logic and other areas. I have also proposed some work in K8S GIE working group recently related to ExtProc Modular architecture, Envoy ExtProc integration for AI gateways, "ExtProc header/body mutation consistent with LLM-d/Envoy AI Gateway filter chains (documented setup for each)".