Commit b06ba6d
committed
test_mla_helix: Add reduce_output=False to disable NCCL all-reduce
The MLA's o_proj layer performs an all-reduce by default, which fails
in the MPI worker context because NCCL groups aren't initialized.
Setting reduce_output=False disables this all-reduce operation.1 parent 57fd9a3 commit b06ba6d
1 file changed
+1
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
632 | 632 | | |
633 | 633 | | |
634 | 634 | | |
| 635 | + | |
635 | 636 | | |
636 | 637 | | |
637 | 638 | | |
| |||
0 commit comments