Skip to content

Commit 55c244c

Browse files
Fsdp credit (#270)
* update with miles * adding FSDP in miles * delete link * add ack * folding details * solve miles * fix complie * fix up FSDP --------- Co-authored-by: zhaochenyang20 <[email protected]>
1 parent 3bc8aa5 commit 55c244c

File tree

1 file changed

+15
-1
lines changed

1 file changed

+15
-1
lines changed

blog/2025-12-03-miles-fsdp.md

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: "Support FSDP2 as A Training Backend for Miles"
2+
title: "Power Up FSDP2 as a Flexible Training Backend for Miles"
33
author: "SGLang RL Team, Miles Team"
44
date: "December 3, 2025"
55
previewImg: /images/blog/miles-fsdp/2_fsdp_train.png
@@ -9,6 +9,18 @@ previewImg: /images/blog/miles-fsdp/2_fsdp_train.png
99
>
1010
> **We have added FSDP to [Miles](https://github.com/radixark/miles) as a more flexible training framework and have aligned it with Megatron. FSDP supports architecture-innovative models such as Qwen3-Next more flexibly and helps us further support VLM RL.**
1111
12+
SGLang RL Team and the Miles community have conducted some interesting explorations around RL training stability and acceleration:
13+
14+
[Aligning the SGLang and FSDP backends](https://github.com/radixark/miles/tree/main/examples/true_on_policy) for **strictly zero KL divergence**
15+
16+
[**Speculative Decoding**](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/main/rlhf/slime/spec/readme-en.md) with online SFT for the draft model
17+
18+
[Unified FP8](https://lmsys.org/blog/2025-11-25-fp8-rl/): Moving Beyond Mixed Precision for Stable and Accelerated MoE RL
19+
20+
Building on this, we now share a new progress that seeks the best adaptbility and usability to new model architectures, enable FSDP2 a more flexible training backend for Miles.
21+
22+
This work is jointly completed by the **SGLang RL Team and Miles Team**. Special thanks to **DataCrunch, AtlasCloud and EigenAI** for compute sponsorship.
23+
1224
## Background
1325

1426
### What is FSDP?
@@ -279,6 +291,8 @@ We sincerely thank the AtlasCloud and DataCrunch for their computing support.
279291

280292
Linkedin: Lancert
281293

294+
In the same time, our amazing members, Chengxi Li and Huapeng Zhou are looking for new work opportunities, welcome to contact them.
295+
282296
<details>
283297
<summary>Engineering Implementation Details</summary>
284298

0 commit comments

Comments
 (0)