File tree Expand file tree Collapse file tree 1 file changed +14
-0
lines changed Expand file tree Collapse file tree 1 file changed +14
-0
lines changed Original file line number Diff line number Diff line change @@ -301,3 +301,17 @@ We recommend using `bf16` to avoid scaling and desynchronization issues with hiv
301
301
2 . ` torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate... `
302
302
A possible culprit is that your ` --per-device-train-batch-size ` is too high.
303
303
Try a smaller value.
304
+
305
+ # Citation
306
+ If you use OpenDiloco for your research, please cite our [ paper] ( https://arxiv.org/abs/2407.07852 ) :
307
+ ``` bibtex
308
+ @misc{jaghouar2024opendiloco,
309
+ title={OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training},
310
+ author={Sami Jaghouar and Jack Min Ong and Johannes Hagemann},
311
+ year={2024},
312
+ eprint={2407.07852},
313
+ archivePrefix={arXiv},
314
+ primaryClass={cs.LG},
315
+ url={https://arxiv.org/abs/2407.07852},
316
+ }
317
+ ```
You can’t perform that action at this time.
0 commit comments