Skip to content

Commit ad0e1fd

Browse files
fix when params_have_main_grad (bigscience-workshop#272)
1 parent efd714b commit ad0e1fd

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

megatron/optimizer/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ def get_megatron_optimizer(model,
113113

114114
# Determine whether the params have main-grad field.
115115
params_have_main_grad = False
116-
if args.DDP_impl == 'local':
116+
if args.use_contiguous_buffers_in_local_ddp:
117117
params_have_main_grad = True
118118

119119
# Mixed precision optimizer.

0 commit comments

Comments
 (0)