Skip to content

Commit e77e6b2

Browse files
committed
Add more environment variables for NCCL
1 parent 3f0bdb9 commit e77e6b2

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

docs/software/communication/nccl.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,20 @@ While the container engine sets these automatically when using the NCCL hook, th
1818

1919
```bash
2020
export NCCL_NET="AWS Libfabric" # (1)!
21+
export NCCL_CROSS_NIC="1"
22+
export NCCL_NET_FORCE_FLUSH="1"
2123
export NCCL_NET_GDR_LEVEL=PHB # (2)!
2224
export FI_CXI_DEFAULT_CQ_SIZE=131072 # (3)!
2325
export FI_CXI_DEFAULT_TX_SIZE=32768
2426
export FI_CXI_DISABLE_HOST_REGISTER=1
27+
export FI_CXI_RDZV_EAGER_SIZE="0"
28+
export FI_CXI_RDZV_GET_MIN="0"
29+
export FI_CXI_RDZV_THRESHOLD="0"
30+
export FI_CXI_RX_MATCH_MODE="hardware"
2531
export FI_CXI_RX_MATCH_MODE=software
32+
export FI_CXI_SAFE_DEVMEM_COPY_THRESHOLD="16777216"
33+
export FI_MR_CACHE_MAX_COUNT="524288"
34+
export FI_MR_CACHE_MAX_SIZE="-1"
2635
export FI_MR_CACHE_MONITOR=userfaultfd
2736
export MPICH_GPU_SUPPORT_ENABLED=0 # (4)!
2837
```

0 commit comments

Comments
 (0)