Skip to content

Commit 06e26f7

Browse files
committed
Hopefully this fixes MLM
1 parent 0b00762 commit 06e26f7

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

megatron/data/mlm_dataset.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -308,7 +308,7 @@ def __init__(
308308
if hasattr(args, "encoder_seq_length") and args.encoder_seq_length is not None:
309309
# T5 style
310310
assert self.inputs_length == args.encoder_seq_length
311-
assert self.targets_length == args.decoder_seq_length
311+
assert self.targets_length == args.decoder_seq_length + 1
312312
else:
313313
assert self.inputs_length + self.targets_length == args.seq_length
314314

0 commit comments

Comments
 (0)