Skip to content

Pull requests: bigscience-workshop/Megatron-DeepSpeed

New pull request New
47 Open 215 Closed
47 Open 215 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Startup: add argument-consistency checks & summary table (Fixes #124)
#409 opened Jun 20, 2025 by MagellaX Loading…
1
fix(training): correct rank-zero log messages, Print total model size once at startup (rank-0) – Fixes #123
#408 opened Jun 20, 2025 by MagellaX Loading…
Bump black from 21.4b0 to 24.3.0 dependencies Pull requests that update a dependency file
#402 opened Mar 20, 2024 by dependabot bot Loading…
Add xPos embeddings
#370 opened Mar 7, 2023 by janEbert Loading…
Fix various small problems
#367 opened Feb 28, 2023 by janEbert Loading…
Bloom model training with AML
#365 opened Feb 21, 2023 by savitamittal1 Loading…
Add UL2 data sampling and pretraining
#358 opened Dec 13, 2022 by janEbert Loading…
5
Add FlashAttention
#357 opened Dec 12, 2022 by NouamaneTazi Loading…
3
Distill BLOOM - tentative 2
#354 opened Oct 11, 2022 by younesbelkada • Draft
4
Enable rocm-support
#353 opened Oct 7, 2022 by luukkonenr Loading…
Encoding checkpoint reshaping guide
#349 opened Sep 20, 2022 by tjruwase • Draft
4
Add multiple evaluation compat
#336 opened Aug 30, 2022 by Muennighoff Loading…
[checkpoints] replace bf16 with fp32 checkpoint weights
#327 opened Aug 10, 2022 by stas00 Loading…
5
Prefix LM Eval
#313 opened Jul 16, 2022 by Muennighoff Loading…
4
Add Bitfit
#311 opened Jul 10, 2022 by Muennighoff Loading…
Tool for CKPT averaging
#310 opened Jul 10, 2022 by Muennighoff Loading…
Enable loading ckpt for t0 finetuning
#309 opened Jul 10, 2022 by Muennighoff Loading…
32
[WIP] Hack my way to get OPT running
#301 opened Jul 4, 2022 by thomasw21 • Draft
[MLM] Train script for non causal decoder
#300 opened Jul 4, 2022 by thomasw21 • Draft
a branch combining layer-norm-auto-sync and ds_ckpt_reshape
#292 opened Jun 29, 2022 by stas00 Loading…
BigScience Eval Harness
#291 opened Jun 29, 2022 by Muennighoff Loading…
No-ZeRO reshaping
#289 opened Jun 23, 2022 by Muennighoff Loading…
WIP: Shared t5 code
#286 opened Jun 21, 2022 by thomasw21 Loading…
2 of 4 tasks
[WIP] add debug utils
#275 opened Mar 28, 2022 by stas00 Loading…
Sync 4 layer norms - bf16, fp32, optimizer states on restart
#274 opened Mar 28, 2022 by tjruwase Loading…
ProTip! What’s not been updated in a month: updated:<2025-09-30.