Skip to content

Commit 411d6e1

Browse files
committed
unwaive test, a previous PR may have resolved hang
Signed-off-by: Ludwig Schneider <[email protected]>
1 parent 17b443a commit 411d6e1

File tree

2 files changed

+3
-3
lines changed

2 files changed

+3
-3
lines changed

tensorrt_llm/_torch/custom_ops/torch_custom_ops.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1690,7 +1690,8 @@ def get_valid_tactics(
16901690
**kwargs,
16911691
) -> List[int]:
16921692
valid_strategies = [
1693-
AllReduceStrategy.NCCL_SYMMETRIC.value,
1693+
# TODO: NCCL_SYMMETRIC will cause hang during tuning process
1694+
# AllReduceStrategy.NCCL_SYMMETRIC.value,
16941695
AllReduceStrategy.NCCL.value,
16951696
]
16961697
# Fallback in allreduceOp is set to NCCL_SYMMETRIC as default
@@ -1719,7 +1720,7 @@ def forward(
17191720
input, residual, norm_weight, scale, bias, workspace = inputs
17201721
if tactic == -1:
17211722
# TODO: Use NCCL instead of NCCL_SYMMETRIC to avoid hanging during tuning process
1722-
tactic = AllReduceStrategy.NCCL_SYMMETRIC.value
1723+
tactic = AllReduceStrategy.NCCL.value
17231724

17241725
return torch.ops.trtllm.allreduce(
17251726
input,

tests/integration/test_lists/waives.txt

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -269,7 +269,6 @@ full:sm89/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_
269269
full:sm89/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_tp_pp_symmetric[MMLU-tp2pp2] SKIP (https://nvbugs/5596337)
270270
accuracy/test_llm_api_pytorch.py::TestQwen3_235B_A22B::test_nvfp4[latency_moe_trtllm] SKIP (https://nvbugs/5721672)
271271
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_fp8_4gpus[tp4-fp8kv=True-attn_backend=FLASHINFER-torch_compile=True] SKIP (https://nvbugs/5741304)
272-
unittest/_torch/multi_gpu/test_allreduce.py::test_allreduce_fusion_patterns[2-residual_rms_norm_out_quant_fp8-hidden:7168-seqlen:8192] SKIP (https://nvbugs/5741392)
273272
unittest/executor/test_rpc.py::TestRpcCorrectness::test_incremental_task_async SKIP (https://nvbugs/5741476)
274273
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_fp8_4gpus[pp4-fp8kv=True-attn_backend=TRTLLM-torch_compile=False] SKIP (https://nvbugs/5740377)
275274
accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B::test_nvfp4[dep4_latency_moe_cutlass-torch_compile=False] SKIP (https://nvbugs/5740377)

0 commit comments

Comments
 (0)