Skip to content

Commit c4e9cb6

Browse files
lukasgdboeschf
andauthored
Update docs/software/ml/pytorch.md
Co-authored-by: boeschf <[email protected]>
1 parent 42082ce commit c4e9cb6

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/software/ml/pytorch.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -184,7 +184,7 @@ For further details on execution logic, job monitoring and data management, plea
184184
#SBATCH -C thp_never&nvidia_vboost_enabled
185185
```
186186

187-
* The argument `--ddp-bucket-size` controls the level of grouping of many small data-parallel communications into bigger ones and setting it to a high value such as can improve throughput (model-dependent, e.g. `10000000000`).
187+
* The argument `--ddp-bucket-size` controls the level of grouping of many small data-parallel communications into bigger ones and setting it to a high value can improve throughput (model-dependent, e.g. `10000000000`).
188188

189189
* If in doubt about communication performance with NCCL at scale, use [nccl-tests](https://github.com/NVIDIA/nccl-tests) with the relevant communication patterns to check if scaling behavior can be reproduced.
190190

0 commit comments

Comments
 (0)