adds format utils

Author: LucasLLC

recipes_source/distributed_checkpoint_recipe.rst

@@ -320,6 +320,37 @@ the intent is to save or load in "non-distributed" style, meaning entirely in th
         run_checkpoint_load_example()
 
 
+Formats
+----------
+One drawback not yet mentioned is that DCP saves checkpoints in a format which is inherently different then those generated using torch.save.
+Since this can be an issue when users wish to share models with users used to the torch.save format, or in general just want to add format flexibility
+to their applications. For this case, we provide the `format_utils` module in `torch.distributed.checkpoint.format_utils`.
+
+A command line utility is provided for the users convenience, which follows the following format:
+`python -m torch.distributed.checkpoint.format_utils -m <checkpoint location> <location to write formats to> <mode>` where mode is one of `torch_to_dcp` or `dcp_to_torch`.
+
+Alternatively, methods are also provided for users who may wish to convert checkpoints directly.
+
+
+.. code-block:: python
+
+    import os
+
+    import torch
+    import torch.distributed.checkpoint as DCP
+    from torch.distributed.checkpoint.format_utils import dcp_to_torch_save, torch_save_to_dcp
+
+    CHECKPOINT_DIR = "checkpoint"
+    TORCH_SAVE_CHECKPOINT_DIR = "torch_save_checkpoint.pth"
+
+    # convert dcp model to torch.save (assumes checkpoint was generated as above)
+    dcp_to_torch_save(CHECKPOINT_DIR, TORCH_SAVE_CHECKPOINT_DIR)
+
+    # converts the torch.save model back to DCP
+    dcp_to_torch_save(TORCH_SAVE_CHECKPOINT_DIR, f"{CHECKPOINT_DIR}_new")
+
+
+
 Conclusion
 ----------
 In conclusion, we have learned how to use DCP's :func:`save` and :func:`load` APIs, as well as how they are different form :func:`torch.save` and :func:`torch.load`.