Skip to content

Commit 653ad38

Browse files
authored
add te fused cross entropy argument (#182)
1 parent 4a69804 commit 653ad38

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

primus/configs/models/megatron/language_model.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ attention_softmax_in_fp32: false
6868
# fusion
6969
bias_gelu_fusion: true
7070
cross_entropy_loss_fusion: False
71+
cross_entropy_fusion_impl: "native" # "native", "te"
7172
bias_swiglu_fusion: true
7273
masked_softmax_fusion: true
7374
no_persist_layer_norm: false

0 commit comments

Comments
 (0)