Skip to content

Commit 686b374

Browse files
BihanBihan  Ranapeterschmidt85
authored
Add TensorrRT-LLM Example (#2444)
* Add TensorrRT-LLM Example Add TensorrRT-LLM Deepseek Example Update ChatCompletionsChunk for pytorch backend Add task to build tensorrt-llm docker image Update Tensorrt-LLM Example Drop model_proxy changes Resolve Review Comments Resolve Review Comments Update with hf download instead of git clone Update README.md with hf download * [Examples] Review of the TensorRT-LLM example * Fix serve-r1.dstack.yml in README --------- Co-authored-by: Bihan Rana <[email protected]> Co-authored-by: peterschmidt85 <[email protected]>
1 parent a39e946 commit 686b374

File tree

9 files changed

+553
-5
lines changed

9 files changed

+553
-5
lines changed

docs/examples.md

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,15 @@ hide:
1414

1515
## Deployment
1616
<div class="tx-landing__highlights_grid">
17+
<a href="/examples/deployment/sglang"
18+
class="feature-cell">
19+
<h3>
20+
SGLang
21+
</h3>
22+
<p>
23+
Deploy DeepSeek distilled models with SGLang
24+
</p>
25+
</a>
1726
<a href="/examples/deployment/vllm"
1827
class="feature-cell">
1928
<h3>
@@ -38,17 +47,17 @@ hide:
3847
NIM
3948
</h3>
4049
<p>
41-
Deploy DeepSeek R1 Distill Llama 8B with NIM
50+
Deploy a DeepSeek distilled model with NIM
4251
</p>
4352
</a>
44-
<a href="/examples/deployment/sglang"
53+
<a href="/examples/deployment/trtllm"
4554
class="feature-cell">
4655
<h3>
47-
SGLang
56+
TensorRT-LLM
4857
</h3>
4958
<p>
50-
Deploy DeepSeek-R1-Distill-Llama 8B & 70B with SGLang
51-
</p>
59+
Deploy DeepSeek R1 and its distilled version with TensorRT-LLM
60+
</p>
5261
</a>
5362
</div>
5463

docs/examples/deployment/trtllm/index.md

Whitespace-only changes.

0 commit comments

Comments
 (0)