Skip to content

Use monarch's distributed setup utility and colocate vLLM with its workers #409

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 22 commits into from
Oct 15, 2025
Merged

Use monarch's distributed setup utility and colocate vLLM with its workers #409

merged 22 commits into from
Oct 15, 2025

Conversation

allenwang28
Copy link
Contributor

@allenwang28 allenwang28 commented Oct 14, 2025 •
edited
Loading

Addresses and closes both #171 and closes out #144

This is needed for both enabling torchtitan / RLTrainer across multiple nodes and colocates the vLLM policy with its workers.

Example run: https://wandb.ai/cabernet-team/grpo-training/runs/snehzq5o

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 14, 2025
@allenwang28 allenwang28 changed the title Uses Use monarch's distributed setup utility and colocate vLLM with its workers Oct 14, 2025
@casteryh casteryh mentioned this pull request Oct 15, 2025
Closed
allenwang28 and others added 16 commits October 15, 2025 02:04
@allenwang28
vllm colocation works
571750f
@init27 @allenwang28
Zero to Forge Tutorials (meta-pytorch#300)
d90249b 
Co-authored-by: Allen Wang <[email protected]>
@Jack-Khuu @allenwang28
Flatten GRPO main: Group and Episode (meta-pytorch#400)
25ebf86
@casteryh @allenwang28
Remove use_dcp option in qwen3_8b (meta-pytorch#415)
8062bf0
@allenwang28
rebase
3d40ed4
@DNXie @allenwang28
Auto-track and globally shut down all Forge actors and services (meta…
8ae6743 
…-pytorch#357)
@JenniferWang @allenwang28
Add CI badges to README.md (meta-pytorch#412)
9ce0f9d
@JenniferWang @allenwang28
[EZ] remove extraneous @pytest.mark.asyncio (meta-pytorch#418)
5c07503
@JenniferWang @allenwang28
Fix policy update test (meta-pytorch#365)
f580316
@ebsmothers @allenwang28
mark vLLM policy integration tests as async (meta-pytorch#416)
a3bf0f7
@felipemello1 @allenwang28
Metric Logging updates 4/N - better actor name (meta-pytorch#351)
0496edb 
Co-authored-by: Felipe Mello <[email protected]>
@casteryh @allenwang28
CI badge should point to main branch (meta-pytorch#422)
d2130ae
@joecummings @allenwang28
Policy cleaner launch / setup (meta-pytorch#401)
ffacf70
@joecummings @allenwang28
Rename service / actor spawning stuff (meta-pytorch#423)
be486ef
@pbontrager @allenwang28
Configurable ReplayBuffer (meta-pytorch#410)
7f35b26
@allenwang28
merge
c96400d
@joecummings joecummings linked an issue Oct 15, 2025 that may be closed by this pull request
Utilize setup_env_for_distributed in Trainer and RefModel instead of manually creating Torchrun env variables #171
Closed
joecummings
joecummings reviewed Oct 15, 2025
View reviewed changes
@allenwang28 allenwang28 marked this pull request as ready for review October 15, 2025 22:02
@allenwang28 allenwang28 mentioned this pull request Oct 15, 2025
Merged
joecummings
joecummings approved these changes Oct 15, 2025
View reviewed changes
Copy link
Member

@joecummings joecummings left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For posterity: can you link to a running WandB project?

@allenwang28 allenwang28 merged commit aa59857 into meta-pytorch:main Oct 15, 2025
9 checks passed
@allenwang28 allenwang28 deleted the titan_setup branch October 15, 2025 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joecummings joecummings joecummings approved these changes

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Utilize setup_env_for_distributed in Trainer and RefModel instead of manually creating Torchrun env variables

10 participants

@allenwang28 @joecummings @init27 @Jack-Khuu @casteryh @DNXie @JenniferWang @ebsmothers @felipemello1 @pbontrager