Skip to content

Commit 09db9b9

Browse files
Update torchrec_intro_tutorial.py
1 parent b0f4cf1 commit 09db9b9

File tree

1 file changed

+6
-6
lines changed

1 file changed

+6
-6
lines changed

intermediate_source/torchrec_intro_tutorial.py

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -200,7 +200,7 @@
200200
# In order to train models with massive embedding tables, sharding these
201201
# tables across GPUs is required, which then introduces a whole new set of
202202
# problems and opportunities in parallelism and optimization. Luckily, we have
203-
# the TorchRec library that has encountered, consolidated, and addressed
203+
# the TorchRec library <https://docs.pytorch.org/torchrec/overview.html>`__ that has encountered, consolidated, and addressed
204204
# many of these concerns. TorchRec serves as a **library that provides
205205
# primitives for large scale distributed embeddings**.
206206
#
@@ -496,11 +496,11 @@
496496
#
497497
# * **The module sharder**: This class exposes a ``shard`` API
498498
# that handles sharding a TorchRec Module, producing a sharded module.
499-
# * For ``EmbeddingBagCollection``, the sharder is `EmbeddingBagCollectionSharder <https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.embeddingbag.EmbeddingBagCollectionSharder>`__
499+
# * For ``EmbeddingBagCollection``, the sharder is `EmbeddingBagCollectionSharder `
500500
# * **Sharded module**: This class is a sharded variant of a TorchRec module.
501501
# It has the same input/output as a the regular TorchRec module, but much
502502
# more optimized and works in a distributed environment.
503-
# * For ``EmbeddingBagCollection``, the sharded variant is `ShardedEmbeddingBagCollection <https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.embeddingbag.ShardedEmbeddingBagCollection>`__
503+
# * For ``EmbeddingBagCollection``, the sharded variant is `ShardedEmbeddingBagCollection`
504504
#
505505
# Every TorchRec module has an unsharded and sharded variant.
506506
#
@@ -619,7 +619,7 @@
619619
# Remember that TorchRec is a highly optimized library for distributed
620620
# embeddings. A concept that TorchRec introduces to enable higher
621621
# performance for training on GPU is a
622-
# `LazyAwaitable <https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.types.LazyAwaitable>`__.
622+
# `LazyAwaitable `.
623623
# You will see ``LazyAwaitable`` types as outputs of various sharded
624624
# TorchRec modules. All a ``LazyAwaitable`` type does is delay calculating some
625625
# result as long as possible, and it does it by acting like an async type.
@@ -693,7 +693,7 @@ def _wait_impl(self) -> torch.Tensor:
693693
# order for distribution of gradients. ``input_dist``, ``lookup``, and
694694
# ``output_dist`` all depend on the sharding scheme. Since we sharded in a
695695
# table-wise fashion, these APIs are modules that are constructed by
696-
# `TwPooledEmbeddingSharding <https://pytorch.org/torchrec/torchrec.distributed.sharding.html#torchrec.distributed.sharding.tw_sharding.TwPooledEmbeddingSharding>`__.
696+
# `TwPooledEmbeddingSharding`.
697697
#
698698

699699
sharded_ebc
@@ -742,7 +742,7 @@ def _wait_impl(self) -> torch.Tensor:
742742
# ``EmbeddingBagCollection`` to generate a
743743
# ``ShardedEmbeddingBagCollection`` module. This workflow is fine, but
744744
# typically when implementing model parallel,
745-
# `DistributedModelParallel <https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.model_parallel.DistributedModelParallel>`__
745+
# `DistributedModelParallel`
746746
# (DMP) is used as the standard interface. When wrapping your model (in
747747
# our case ``ebc``), with DMP, the following will occur:
748748
#

0 commit comments

Comments
 (0)