documentation: SageMaker model parallel library v1.10.0 documentation (#3237)

* archive doc for past versions

* fix indexing

* add new smp cpu memory apis

* add new params

* add dynamic scale params, add reference

* minor fix

* minor fixes

* rm temp methods

* add new checkpoint save/load functions, doc improvement

* pass doc8

* Trigger Build

* archive doc for past versions

* fix indexing

* add new smp cpu memory apis

* add new params

* add dynamic scale params, add reference

* minor fix

* minor fixes

* rm temp methods

* add new checkpoint save/load functions, doc improvement

* pass doc8

* Trigger Build

* remove dist word embedding option

Co-authored-by: Shreya Pandit <[email protected]>

Author: mchoi8739

doc/api/training/smd_model_parallel_general.rst

@@ -178,6 +178,16 @@ PyTorch-specific Parameters
     - 1
     - The number of devices over which the tensor parallel modules will be distributed.
       If ``tensor_parallel_degree`` is greater than 1, then ``ddp`` must be set to ``True``.
+  * - ``fp16`` (**smdistributed-modelparallel**>=v1.10)
+    - bool
+    - ``False``
+    - To run FP16 training, add ``"fp16"'": True`` to the smp configuration.
+      Other APIs remain the same between FP16 and FP32.
+      If ``fp16`` is enabled and when user calls ``smp.DistributedModel``,
+      the model will be wrapped with ``FP16_Module``, which converts the model
+      to FP16 dtype and deals with forward pass in FP16.
+      If ``fp16`` is enabled and when user calls ``smp.DistributedOptimizer``,
+      the optimizer will be wrapped with ``FP16_Optimizer``.
   * - ``fp16_params`` (**smdistributed-modelparallel**>=v1.6)
     - bool
     - ``False``

doc/api/training/smp_versions/archives.rst

@@ -3,6 +3,7 @@
 .. toctree::
     :maxdepth: 1
 
+    v1_9_0.rst
     v1_6_0.rst
     v1_5_0.rst
     v1_4_0.rst

doc/api/training/smp_versions/latest.rst

@@ -10,7 +10,7 @@ depending on which version of the library you need to use.
 To use the library, reference the
 **Common API** documentation alongside the framework specific API documentation.
 
-Version 1.7.0, 1.8.0, 1.8.1, 1.9.0 (Latest)
+Version 1.10.0 (Latest)
 ===========================================
 
 To use the library, reference the Common API documentation alongside the framework specific API documentation.