File tree Expand file tree Collapse file tree 2 files changed +7
-1
lines changed
Expand file tree Collapse file tree 2 files changed +7
-1
lines changed Original file line number Diff line number Diff line change @@ -42,7 +42,7 @@ These recipes demonstrate aggregated or disaggregated serving:
4242| ** [ DeepSeek-R1] ( deepseek-r1/sglang/disagg-16gpu/ ) ** | SGLang | Disagg WideEP | 32x H200 | ✅ | ❌ | TP=16, multi-node. Use ` model-download-sglang.yaml ` | ❌ |
4343| ** [ DeepSeek-R1] ( deepseek-r1/trtllm/disagg/wide_ep/gb200/ ) ** | TensorRT-LLM | Disagg WideEP (GB200) | 36x GB200 | ✅ | ✅ | Multi-node: 8 decode + 1 prefill nodes | ❌ |
4444| ** [ DeepSeek-R1] ( deepseek-r1/ ) ** | vLLM | Disagg DEP16 | 32x H200 | ✅ | ❌ | Multi-node, data-expert parallel | ❌ |
45- | ** [ Kimi-K2.5] ( kimi-k2.5/ ) ** | TensorRT-LLM | Aggregated | 8x B200 | ✅ | ❌ | MoE model, TP8×EP8, reasoning + tool calling | ❌ |
45+ | ** [ Kimi-K2.5] ( kimi-k2.5/ ) ** 🚧 | TensorRT-LLM | Aggregated | 8x B200 | ✅ | ❌ | Experimental — MoE model, TP8×EP8, reasoning + tool calling | ❌ |
4646
4747** Legend:**
4848- ** Deployment** : ✅ = Complete ` deploy.yaml ` manifest available
Original file line number Diff line number Diff line change 11# Kimi-K2.5 Recipes
22
3+ > 🚧 ** Work-in-Progress — Experimental Recipe**
4+ >
5+ > The TensorRT-LLM Python package used for Dynamo's TRT-LLM integration does not yet include
6+ > native Kimi K2.5 support. This recipe is an ** experimental** effort to bring
7+ > Kimi K2.5 to Dynamo ahead of upstream availability. It needs to patch the container image on top of released dynamo image.
8+
39Deployment recipe for ** Kimi-K2.5** using TensorRT-LLM with Dynamo's KV-aware routing.
410
511## Available Configurations
You can’t perform that action at this time.
0 commit comments