Skip to content

Commit 9b5c3ad

Browse files
committed
fix example
1 parent e088694 commit 9b5c3ad

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

docs/source-pytorch/data/alternatives.rst

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,14 +99,19 @@ The webdataset library contains a small wrapper (``WebLoader``) that adds a flui
9999
import lightning as L
100100
import webdataset as wds
101101
102-
dataset = wds.WebDataset(urls)
102+
dataset = wds.WebDataset(
103+
urls,
104+
# needed for multi-gpu or multi-node training
105+
workersplitter=wds.shardlists.split_by_worker,
106+
nodesplitter=wds.shardlists.split_by_node,
107+
)
103108
train_dataloader = wds.WebLoader(dataset)
104109
105110
model = ...
106111
trainer = L.Trainer()
107112
trainer.fit(model, train_dataloader)
108113
109-
You can find a complete example `here <https://github.com/webdataset/webdataset-lightning>`__.
114+
You can find a complete example `here <https://github.com/webdataset/webdataset-lightning>`__.
110115

111116
NVIDIA DALI
112117
^^^^^^^^^^^

0 commit comments

Comments
 (0)