Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Commit 8f4b312

Browse files
authored
Update distributed.md
1 parent 66dd025 commit 8f4b312

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

docs/distributed.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,8 @@ python3 torchchat.py generate llama3.1 --distributed --tp 2 --pp 2 --prompt "
3737
[skip default]: end
3838
-->
3939

40-
## CHat with Distributed torchchat Inference
40+
## Chat with Distributed torchchat Inference
4141

42-
### Chat
4342
This mode allows you to chat with an LLM in an interactive fashion with distributed Inference. The following example uses 4 GPUs:
4443

4544
[skip default]: begin
@@ -108,7 +107,9 @@ curl http://127.0.0.1:5000/v1/chat/completions \
108107

109108
[skip default]: end
110109

110+
<!--
111111
[shell default]: kill ${server_pid}
112+
-->
112113

113114
</details>
114115

0 commit comments

Comments
 (0)