Skip to content

Commit e0eb76a

Browse files
authored
Fix links in TorchRec tutorial (#1835)
Use [subsitution references](https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#substitution-definitions) to style links to TorchRec and PyTorch documentation. Mark shell script codeblock as shell, rather than python.
1 parent 1f0b6d9 commit e0eb76a

File tree

1 file changed

+27
-26
lines changed

1 file changed

+27
-26
lines changed

intermediate_source/torchrec_tutorial.rst

Lines changed: 27 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@ Introduction to Torchrec
22
====================================================
33

44
.. tip::
5-
To get the most of this tutorial, we suggest using this
6-
`Colab Version <https://colab.research.google.com/github/pytorch/torchrec/blob/main/Torchrec_Introduction.ipynb>`__.
5+
To get the most of this tutorial, we suggest using this
6+
`Colab Version <https://colab.research.google.com/github/pytorch/torchrec/blob/main/Torchrec_Introduction.ipynb>`__.
77
This will allow you to experiment with the information presented below.
88

99
Frequently, when building recommendation systems, we want to represent
@@ -14,7 +14,7 @@ entities grow, the size of the embedding tables can exceed a single
1414
GPU’s memory. A common practice is to shard the embedding table across
1515
devices, a type of model parallelism. To that end, **torchRec introduces
1616
its primary API
17-
called**\ ```DistributedModelParallel`` <https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.model_parallel.DistributedModelParallel>`__\ **,
17+
called** |DistributedModelParallel|_ **,
1818
or DMP. Like pytorch’s DistributedDataParallel, DMP wraps a model to
1919
enable distributed training.**
2020

@@ -28,21 +28,19 @@ We highly recommend CUDA when using torchRec. If using CUDA:
2828
- cuda >= 11.0
2929

3030

31-
.. code:: python
31+
.. code:: shell
3232
33-
install pytorch with cudatoolkit 11.3
33+
# install pytorch with cudatoolkit 11.3
3434
conda install pytorch cudatoolkit=11.3 -c pytorch-nightly -y
35-
install torchrec
35+
# install torchrec
3636
pip3 install torchrec-nightly
3737
3838
3939
**Overview**
4040
------------
4141

42-
This tutorial will cover three pieces of torchRec - the ``nn.module``
43-
```EmbeddingBagCollection`` <https://pytorch.org/torchrec/torchrec.modules.html#torchrec.modules.embedding_modules.EmbeddingBagCollection>`__, the
44-
```DistributedModelParallel`` <https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.model_parallel.DistributedModelParallel>`__ API, and
45-
the datastructure ```KeyedJaggedTensor`` <https://pytorch.org/torchrec/torchrec.sparse.html#torchrec.sparse.jagged_tensor.JaggedTensor>`__.
42+
This tutorial will cover three pieces of torchRec - the ``nn.module`` |EmbeddingBagCollection|_, the |DistributedModelParallel|_ API, and
43+
the datastructure |KeyedJaggedTensor|_.
4644

4745

4846
Distributed Setup
@@ -77,16 +75,11 @@ GPU.
7775
From EmbeddingBag to EmbeddingBagCollection
7876
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7977

80-
Pytorch represents embeddings through
81-
```torch.nn.Embedding`` <https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html>`__
82-
and
83-
```torch.nn.EmbeddingBag`` <https://pytorch.org/docs/stable/generated/torch.nn.EmbeddingBag.html>`__.
78+
Pytorch represents embeddings through |torch.nn.Embedding|_ and |torch.nn.EmbeddingBag|_.
8479
EmbeddingBag is a pooled version of Embedding.
8580

8681
TorchRec extends these modules by creating collections of embeddings. We
87-
will use
88-
```EmbeddingBagCollection`` <https://pytorch.org/torchrec/torchrec.modules.html#torchrec.modules.embedding_modules.EmbeddingBagCollection>`__
89-
to represent a group of EmbeddingBags.
82+
will use |EmbeddingBagCollection|_ to represent a group of EmbeddingBags.
9083

9184
Here, we create an EmbeddingBagCollection (EBC) with two embedding bags.
9285
Each table, ``product_table`` and ``user_table``, is represented by 64
@@ -119,9 +112,7 @@ on device “meta”. This will tell EBC to not allocate memory yet.
119112
DistributedModelParallel
120113
~~~~~~~~~~~~~~~~~~~~~~~~
121114

122-
Now, we’re ready to wrap our model with
123-
```DistributedModelParallel`` <https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.model_parallel.DistributedModelParallel>`__
124-
(DMP). Instantiating DMP will:
115+
Now, we’re ready to wrap our model with |DistributedModelParallel|_ (DMP). Instantiating DMP will:
125116

126117
1. Decide how to shard the model. DMP will collect the available
127118
‘sharders’ and come up with a ‘plan’ of the optimal way to shard the
@@ -142,10 +133,7 @@ torchRec will place both on the single GPU.
142133
Query vanilla nn.EmbeddingBag with input and offsets
143134
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
144135

145-
We query
146-
```nn.Embedding`` <https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html>`__
147-
and
148-
```nn.EmbeddingBag`` <https://pytorch.org/docs/stable/generated/torch.nn.EmbeddingBag.html>`__
136+
We query |nn.Embedding|_ and |nn.EmbeddingBag|_
149137
with ``input`` and ``offsets``. Input is a 1-D tensor containing the
150138
lookup values. Offsets is a 1-D tensor where the sequence is a
151139
cumulative sum of the number of values to pool per example.
@@ -174,8 +162,7 @@ Representing minibatches with KeyedJaggedTensor
174162
We need an efficient representation of multiple examples of an arbitrary
175163
number of entity IDs per feature per example. In order to enable this
176164
“jagged” representation, we use the torchRec datastructure
177-
```KeyedJaggedTensor`` <https://pytorch.org/torchrec/torchrec.sparse.html#torchrec.sparse.jagged_tensor.JaggedTensor>`__
178-
(KJT).
165+
|KeyedJaggedTensor|_ (KJT).
179166

180167
Let’s take a look at **how to lookup a collection of two embedding
181168
bags**, “product” and “user”. Assume the minibatch is made up of three
@@ -235,3 +222,17 @@ example, which includes multinode training on the criteo terabyte
235222
dataset, using Meta’s `DLRM <https://arxiv.org/abs/1906.00091>`__.
236223

237224

225+
.. |DistributedModelParallel| replace:: ``DistributedModelParallel``
226+
.. _DistributedModelParallel: https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.model_parallel.DistributedModelParallel
227+
.. |EmbeddingBagCollection| replace:: ``EmbeddingBagCollection``
228+
.. _EmbeddingBagCollection: https://pytorch.org/torchrec/torchrec.modules.html#torchrec.modules.embedding_modules.EmbeddingBagCollection
229+
.. |KeyedJaggedTensor| replace:: ``KeyedJaggedTensor``
230+
.. _KeyedJaggedTensor: https://pytorch.org/torchrec/torchrec.sparse.html#torchrec.sparse.jagged_tensor.JaggedTensor
231+
.. |torch.nn.Embedding| replace:: ``torch.nn.Embedding``
232+
.. _torch.nn.Embedding: https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html
233+
.. |torch.nn.EmbeddingBag| replace:: ``torch.nn.EmbeddingBag``
234+
.. _torch.nn.EmbeddingBag: https://pytorch.org/docs/stable/generated/torch.nn.EmbeddingBag.html
235+
.. |nn.Embedding| replace:: ``nn.Embedding``
236+
.. _nn.Embedding: https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html
237+
.. |nn.EmbeddingBag| replace:: ``nn.EmbeddingBag``
238+
.. _nn.EmbeddingBag: https://pytorch.org/docs/stable/generated/torch.nn.EmbeddingBag.html

0 commit comments

Comments
 (0)