Merge branch 'main' into varlen_tutorial

Author: svekars

.github/workflows/build-tutorials-nightly.yml

@@ -15,7 +15,7 @@ name: Build tutorials (nightly/test)
 # download the binaries in .jenkins/build.sh.
 on:
   # Only main branch for now. Uncomment the below line to enable it on PRs.
-  # pull_request:
+  pull_request:
 
   # Comment out the below line to disable on the main branch
   push:

beginner_source/understanding_leaf_vs_nonleaf_tutorial.py

@@ -265,8 +265,10 @@
 #
 #    Computational graph after backward pass
 #
-# If you call ``retain_grad()`` on a non-leaf node, it results in a no-op.
-# If we call ``retain_grad()`` on a node that has ``requires_grad=False``,
+# If you call ``retain_grad()`` on a leaf tensor, it results in a no-op
+# since leaf tensors already retain their gradients by default (when
+# ``requires_grad=True``).
+# If we call ``retain_grad()`` on a tensor that has ``requires_grad=False``,
 # PyTorch actually throws an error, since it can’t store the gradient if
 # it is never calculated.
 #
@@ -298,13 +300,13 @@
 # +----------------+------------------------+------------------------+---------------------------------------------------+-------------------------------------+
 # |  ``is_leaf``   |   ``requires_grad``    |   ``retains_grad``     |  ``require_grad()``                               |   ``retain_grad()``                 |
 # +================+========================+========================+===================================================+=====================================+
-# | ``True``       | ``False``              | ``False``              | sets ``requires_grad`` to ``True`` or ``False``   | no-op                               |
+# | ``True``       | ``False``              | ``False``              | sets ``requires_grad`` to ``True`` or ``False``   | throws error                        |
 # +----------------+------------------------+------------------------+---------------------------------------------------+-------------------------------------+
-# | ``True``       | ``True``               | ``False``              | sets ``requires_grad`` to ``True`` or ``False``   | no-op                               |
+# | ``True``       | ``True``               | ``False``              | sets ``requires_grad`` to ``True`` or ``False``   | no-op (already retains)             |
 # +----------------+------------------------+------------------------+---------------------------------------------------+-------------------------------------+
 # | ``False``      | ``True``               | ``False``              | no-op                                             | sets ``retains_grad`` to ``True``   |
 # +----------------+------------------------+------------------------+---------------------------------------------------+-------------------------------------+
-# | ``False``      | ``True``               | ``True``               | no-op                                             | no-op                               |
+# | ``False``      | ``True``               | ``True``               | no-op                                             | no-op (already retains)             |
 # +----------------+------------------------+------------------------+---------------------------------------------------+-------------------------------------+
 #
 

intermediate_source/torch_compile_tutorial.py

@@ -163,7 +163,7 @@ def timed(fn):
     result = fn()
     end.record()
     torch.cuda.synchronize()
-    return result, start.elapsed_time(end) / 1024
+    return result, start.elapsed_time(end) / 1000
 
 
 inp = torch.randn(4096, 4096).cuda()

unstable_source/flight_recorder_tutorial.rst

@@ -44,7 +44,7 @@ Flight Recorder consists of two core parts:
 
 - The collection portion: when enabled, information about collectives is recorded in an in-memory circular buffer. Upon job timeout, or on demand, the in-memory buffer can be retrieved or dumped to file.
 
-- An analyzer script is available in the `tools/flight_recorder <https://github.com/pytorch/pytorch/tree/main/tools/flight_recorder>`__ directory (details below).
+- An analyzer script is available in the `torch/distributed/flight_recorder <https://github.com/pytorch/pytorch/tree/main/torch/distributed/flight_recorder>`__ directory (details below).
    The analyzer script runs known heuristics using the collected data and attempts to automatically identify the underlying issue that caused the job to stall.
 
 Enabling Flight Recorder
@@ -169,7 +169,7 @@ The contents of a Flight Recorder ``unpickled`` file are shown below:
 Analyzing Flight Recorder Dumps
 -------------------------------
 
-We have convenient scripts available in `pytorch/tools/flight_recorder` directory for analyzing captured
+We have convenient scripts available in `pytorch/torch/distributed/flight_recorder <https://github.com/pytorch/pytorch/tree/main/torch/distributed/flight_recorder>`__ directory for analyzing captured
 data.
 
 To run the convenience script, follow these steps:
@@ -300,5 +300,5 @@ Conclusion
 In this tutorial, we have learned about a new PyTorch diagnostic tool called Flight Recorder.
 We have discussed how to enable Flight Recorder to collect diagnostic data from a machine.
 Additionally, we explored how to analyze the data captured from the Flight Recorder using a
-convenience script located in the `tools/flight_recorder <https://github.com/pytorch/pytorch/tree/main/tools/flight_recorder>`__
+convenience script located in the `torch/distributed/flight_recorder <https://github.com/pytorch/pytorch/tree/main/torch/distributed/flight_recorder>`__
 directory of the PyTorch repository.