Skip to content

Commit 689d61c

Browse files
committed
Add Deepspeed Zero 3 MiCS support (Issues #20378)
1 parent 2bb1e4a commit 689d61c

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/lightning/pytorch/strategies/deepspeed.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -521,7 +521,7 @@ def model_sharded_context(self) -> Generator[None, None, None]:
521521
import deepspeed
522522

523523
self._init_config_if_needed()
524-
524+
assert self.config is not None
525525
# If detect 'mics_shard_size'>0 in config['zero_optimization'], alter to use deepspeed.zero.MiCS_Init()
526526
# https://deepspeed.readthedocs.io/en/latest/zero3.html#mics-configurations
527527
#! default deepspeed 0.9.0 is not compatible

0 commit comments

Comments
 (0)