Skip to content

Commit f275189

Browse files
committed
fix fine grained callable test
Signed-off-by: Jimmy Zhang <[email protected]>
1 parent 2a1ccfa commit f275189

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

megatron/core/models/gpt/fine_grained_callables.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -431,7 +431,9 @@ def submodule_combine_forward(
431431
"""
432432
residual = node.layer_state.residual
433433

434-
output = layer.mlp.combine(output, shared_expert_output)
434+
output = layer.mlp.combine(output)
435+
output = layer.mlp.postprocess(output, shared_expert_output)
436+
435437
mlp_output_with_bias = (output, None)
436438

437439
with layer.bias_dropout_add_exec_handler():

0 commit comments

Comments
 (0)