Skip to content

Commit 4d95b43

Browse files
vmoenscursoragent
andcommitted
[Docs] Add SGLang backend documentation
Update LLM documentation with SGLang support: llms_modules.rst: - Add SGLangWrapper to Wrappers section - Add AsyncSGLang to Wrappers section llms.rst: - Update Key Components to mention SGLang - Add SGLang Quick Example alongside vLLM example - Add SGLang weight sync classes to autosummary: - SGLangWeightSyncScheme - SGLangWeightSender - SGLangCollectiveTransport - get_sglang_model_metadata Example shows both managed and external server modes: - AsyncSGLang.from_pretrained() for managed server - AsyncSGLang.connect() for external server ghstack-source-id: f6b93fa Co-authored-by: Cursor <cursoragent@cursor.com> ghstack-source-id: f6b93fa Pull-Request: #3435
1 parent 9fce21e commit 4d95b43

File tree

2 files changed

+29
-1
lines changed

2 files changed

+29
-1
lines changed

docs/source/reference/llms.rst

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,14 +12,16 @@ Key Components
1212
--------------
1313

1414
1. **Data Structures**: History class for conversation management, structured output classes
15-
2. **LLM Wrappers**: Unified interfaces for Transformers, vLLM, and AsyncVLLM
15+
2. **LLM Wrappers**: Unified interfaces for Transformers, vLLM, SGLang, and async variants
1616
3. **Environments**: ChatEnv, task-specific environments, and transforms
1717
4. **Collectors**: LLMCollector and RayLLMCollector for data collection
1818
5. **Objectives**: GRPOLoss, SFTLoss for training
1919

2020
Quick Example
2121
-------------
2222

23+
**Using vLLM backend:**
24+
2325
.. code-block:: python
2426
2527
from torchrl.modules.llm import vLLMWrapper, AsyncVLLM
@@ -36,6 +38,26 @@ Quick Example
3638
# Create collector
3739
collector = LLMCollector(env, policy, dialog_turns_per_batch=256)
3840
41+
**Using SGLang backend:**
42+
43+
.. code-block:: python
44+
45+
from torchrl.modules.llm import SGLangWrapper, AsyncSGLang
46+
from torchrl.envs.llm import ChatEnv
47+
from torchrl.collectors.llm import LLMCollector
48+
49+
# Create SGLang engine (connects to server or launches managed server)
50+
engine = AsyncSGLang.from_pretrained("Qwen/Qwen2.5-7B", tp_size=2)
51+
# Or connect to existing server:
52+
# engine = AsyncSGLang.connect("http://localhost:30000")
53+
policy = SGLangWrapper(engine, tokenizer=tokenizer, input_mode="history")
54+
55+
# Create environment
56+
env = ChatEnv(tokenizer=tokenizer)
57+
58+
# Create collector
59+
collector = LLMCollector(env, policy, dialog_turns_per_batch=256)
60+
3961
.. warning:: The LLM API is still under development and may change in the future.
4062
Feedback, issues and PRs are welcome!
4163

@@ -85,6 +107,10 @@ transform, or a boolean to the collector constructor.
85107
VLLMDoubleBufferWeightReceiver
86108
VLLMDoubleBufferTransport
87109
get_model_metadata
110+
SGLangWeightSyncScheme
111+
SGLangWeightSender
112+
SGLangCollectiveTransport
113+
get_sglang_model_metadata
88114

89115
Legacy Weight Updaters (Deprecated)
90116
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

docs/source/reference/llms_modules.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,10 @@ Wrappers
1818
LLMWrapperBase
1919
TransformersWrapper
2020
vLLMWrapper
21+
SGLangWrapper
2122
RemoteTransformersWrapper
2223
AsyncVLLM
24+
AsyncSGLang
2325

2426
Data Structure Classes
2527
----------------------

0 commit comments

Comments
 (0)