Skip to content

Commit 74f2ea0

Browse files
committed
project: add citation and slack channel
Signed-off-by: bitliu <[email protected]>
1 parent 4407fd8 commit 74f2ea0

File tree

1 file changed

+35
-15
lines changed

1 file changed

+35
-15
lines changed

β€ŽREADME.mdβ€Ž

Lines changed: 35 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -2,22 +2,24 @@
22

33
<img src="website/static/img/repo.png" alt="vLLM Semantic Router"/>
44

5-
[![Documentation](https://img.shields.io/badge/docs-read%20the%20docs-blue)](https://llm-semantic-router.readthedocs.io/en/latest/)
5+
[![Documentation](https://img.shields.io/badge/docs-read%20the%20docs-blue)](https://vllm-semantic-router.com)
66
[![Hugging Face](https://img.shields.io/badge/πŸ€—%20Hugging%20Face-Community-yellow)](https://huggingface.co/LLM-Semantic-Router)
77
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
88
[![Crates.io](https://img.shields.io/crates/v/candle-semantic-router.svg)](https://crates.io/crates/candle-semantic-router)
99

10-
**πŸ“š [Complete Documentation](https://llm-semantic-router.readthedocs.io/en/latest/) | πŸš€ [Quick Start](https://llm-semantic-router.readthedocs.io/en/latest/getting-started/quick-start/) | πŸ—οΈ [Architecture](https://llm-semantic-router.readthedocs.io/en/latest/architecture/system-architecture/) | πŸ“– [API Reference](https://llm-semantic-router.readthedocs.io/en/latest/api/router/)**
10+
**πŸ“š [Complete Documentation](https://vllm-semantic-router.com) | πŸš€ [Quick Start](https://vllm-semantic-router.com/docs/getting-started/installation) | πŸ—οΈ [Architecture](https://vllm-semantic-router.com/docs/architecture/system-architecture/) | πŸ“– [API Reference](https://vllm-semantic-router.com/docs/api/router/)**
1111

1212
![](./website/static/img/code.png)
1313

1414
</div>
1515

16-
## Overview
16+
## Innovations ✨
1717

1818
![](./website/static/img/architecture.png)
1919

20-
### Auto-Reasoning and Auto-Selection of Models
20+
### Intelligent Routing 🧠
21+
22+
#### Auto-Reasoning and Auto-Selection of Models
2123

2224
An **Mixture-of-Models** (MoM) router that intelligently directs OpenAI API requests to the most suitable models from a defined pool based on **Semantic Understanding** of the request's intent (Complexity, Task, Tools).
2325

@@ -33,31 +35,49 @@ The screenshot below shows the LLM Router dashboard in Grafana.
3335

3436
The router is implemented in two ways: Golang (with Rust FFI based on Candle) and Python. Benchmarking will be conducted to determine the best implementation.
3537

36-
### Auto-Selection of Tools
38+
#### Auto-Selection of Tools
3739

3840
Select the tools to use based on the prompt, avoiding the use of tools that are not relevant to the prompt so as to reduce the number of prompt tokens and improve tool selection accuracy by the LLM.
3941

40-
### PII detection
42+
### Enterprise Security πŸ”’
43+
44+
#### PII detection
4145

4246
Detect PII in the prompt, avoiding sending PII to the LLM so as to protect the privacy of the user.
4347

44-
### Prompt guard
48+
#### Prompt guard
4549

4650
Detect if the prompt is a jailbreak prompt, avoiding sending jailbreak prompts to the LLM so as to prevent the LLM from misbehaving.
4751

48-
### Semantic Caching
52+
### Similarity Caching ⚑️
4953

5054
Cache the semantic representation of the prompt so as to reduce the number of prompt tokens and improve the overall inference latency.
5155

52-
## πŸ“– Documentation
56+
## Documentation πŸ“–
5357

5458
For comprehensive documentation including detailed setup instructions, architecture guides, and API references, visit:
5559

56-
**πŸ‘‰ [Complete Documentation at Read the Docs](https://llm-semantic-router.readthedocs.io/en/latest/)**
60+
**πŸ‘‰ [Complete Documentation at Read the Docs](https://vllm-semantic-router.com/)**
5761

5862
The documentation includes:
59-
- **[Installation Guide](https://llm-semantic-router.readthedocs.io/en/latest/getting-started/installation/)** - Complete setup instructions
60-
- **[Quick Start](https://llm-semantic-router.readthedocs.io/en/latest/getting-started/quick-start/)** - Get running in 5 minutes
61-
- **[System Architecture](https://llm-semantic-router.readthedocs.io/en/latest/architecture/system-architecture/)** - Technical deep dive
62-
- **[Model Training](https://llm-semantic-router.readthedocs.io/en/latest/training/training-overview/)** - How classification models work
63-
- **[API Reference](https://llm-semantic-router.readthedocs.io/en/latest/api/router/)** - Complete API documentation
63+
- **[Installation Guide](https://vllm-semantic-router.com/docs/getting-started/installation/)** - Complete setup instructions
64+
- **[System Architecture](https://vllm-semantic-router.com/docs/architecture/system-architecture/)** - Technical deep dive
65+
- **[Model Training](https://vllm-semantic-router.com/docs/training/training-overview/)** - How classification models work
66+
- **[API Reference](https://vllm-semantic-router.com/docs/api/router/)** - Complete API documentation
67+
68+
## Community πŸ‘‹
69+
70+
For questions, feedback, or to contribute, please join `#semantic-router` channel in vLLM Slack.
71+
72+
## Citation
73+
74+
If you find Semantic Router helpful in your research or projects, please consider citing it:
75+
76+
```
77+
@misc{semanticrouter2025,
78+
title={vLLM Semantic Router},
79+
author={vLLM Semantic Router Team},
80+
year={2025},
81+
howpublished={\url{https://github.com/vllm-project/semantic-router}},
82+
}
83+
```

0 commit comments

Comments
Β (0)