Skip to content

[DONT LAND] Full Dtensor fully_shard + Local Map + FlexAttention#2621

Open
fegin wants to merge 2 commits intogh/fegin/101/basefrom
gh/fegin/101/head
Open

[DONT LAND] Full Dtensor fully_shard + Local Map + FlexAttention#2621
fegin wants to merge 2 commits intogh/fegin/101/basefrom
gh/fegin/101/head

Conversation

@fegin
Copy link
Contributor

@fegin fegin commented Mar 18, 2026

Stack from ghstack (oldest at bottom):

./run_train.sh --module qwen3 --config qwen3_debugmodel_flex --training.full-dtensor --training.steps 100 --compile.enable
./run_train.sh --module llama3 --config llama3_debugmodel_flex_attn --training.full-dtensor --training.steps 100 --compile.enable

[ghstack-poisoned]
fegin added a commit that referenced this pull request Mar 18, 2026
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Mar 18, 2026
[ghstack-poisoned]
fegin added a commit that referenced this pull request Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/8gpu CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant