Automated tutorials push

Author: pytorchbot

_downloads/0bd6b9a8e47e1d64e4d20ef356a6095d/onnx_registry_tutorial.ipynb

@@ -54,60 +54,8 @@
     "In this tutorial, we will cover three scenarios that require extending\n",
     "the ONNX registry with custom operators:\n",
     "\n",
-    "-   Unsupported ATen operators\n",
     "-   Custom operators with existing ONNX Runtime support\n",
-    "-   Custom operators without ONNX Runtime support\n",
-    "\n",
-    "Unsupported ATen operators\n",
-    "==========================\n",
-    "\n",
-    "Although the ONNX exporter team does their best efforts to support all\n",
-    "ATen operators, some of them might not be supported yet. In this\n",
-    "section, we will demonstrate how you can add unsupported ATen operators\n",
-    "to the ONNX Registry.\n",
-    "\n",
-    "<div style=\"background-color: #54c7ec; color: #fff; font-weight: 700; padding-left: 10px; padding-top: 5px; padding-bottom: 5px\"><strong>NOTE:</strong></div>\n",
-    "\n",
-    "<div style=\"background-color: #f3f4f7; padding-left: 10px; padding-top: 10px; padding-bottom: 10px; padding-right: 10px\">\n",
-    "\n",
-    "<p>The steps to implement unsupported ATen operators are the same to replace the implementation of an existingATen operator with a custom implementation.Because we don't actually have an unsupported ATen operator to use in this tutorial, we are going to leveragethis and replace the implementation of <code>aten::add.Tensor</code> with a custom implementation the same way we wouldif the operator was not present in the ONNX Registry.</p>\n",
-    "\n",
-    "</div>\n",
-    "\n",
-    "When a model cannot be exported to ONNX due to an unsupported operator,\n",
-    "the ONNX exporter will show an error message similar to:\n",
-    "\n",
-    "``` {.python}\n",
-    "RuntimeErrorWithDiagnostic: Unsupported FX nodes: {'call_function': ['aten.add.Tensor']}.\n",
-    "```\n",
-    "\n",
-    "The error message indicates that the fully qualified name of unsupported\n",
-    "ATen operator is `aten::add.Tensor`. The fully qualified name of an\n",
-    "operator is composed of the namespace, operator name, and overload\n",
-    "following the format `namespace::operator_name.overload`.\n",
-    "\n",
-    "To add support for an unsupported ATen operator or to replace the\n",
-    "implementation for an existing one, we need:\n",
-    "\n",
-    "-   The fully qualified name of the ATen operator (e.g.\n",
-    "    `aten::add.Tensor`). This information is always present in the error\n",
-    "    message as show above.\n",
-    "-   The implementation of the operator using [ONNX\n",
-    "    Script](https://github.com/microsoft/onnxscript). ONNX Script is a\n",
-    "    prerequisite for this tutorial. Please make sure you have read the\n",
-    "    [ONNX Script\n",
-    "    tutorial](https://github.com/microsoft/onnxscript/blob/main/docs/tutorial/index.md)\n",
-    "    before proceeding.\n",
-    "\n",
-    "Because `aten::add.Tensor` is already supported by the ONNX Registry, we\n",
-    "will demonstrate how to replace it with a custom implementation, but\n",
-    "keep in mind that the same steps apply to support new unsupported ATen\n",
-    "operators.\n",
-    "\n",
-    "This is possible because the `OnnxRegistry`{.interpreted-text\n",
-    "role=\"class\"} allows users to override an operator registration. We will\n",
-    "override the registration of `aten::add.Tensor` with our custom\n",
-    "implementation and verify it exists.\n"
+    "-   Custom operators without ONNX Runtime support\n"
    ]
   },
   {
@@ -121,123 +69,7 @@
     "import torch\n",
     "import onnxruntime\n",
     "import onnxscript\n",
-    "from onnxscript import opset18  # opset 18 is the latest (and only) supported version for now\n",
-    "\n",
-    "class Model(torch.nn.Module):\n",
-    "    def forward(self, input_x, input_y):\n",
-    "        return torch.ops.aten.add(input_x, input_y)  # generates a aten::add.Tensor node\n",
-    "\n",
-    "input_add_x = torch.randn(3, 4)\n",
-    "input_add_y = torch.randn(3, 4)\n",
-    "aten_add_model = Model()\n",
-    "\n",
-    "\n",
-    "# Now we create a ONNX Script function that implements ``aten::add.Tensor``.\n",
-    "# The function name (e.g. ``custom_aten_add``) is displayed in the ONNX graph, so we recommend to use intuitive names.\n",
-    "custom_aten = onnxscript.values.Opset(domain=\"custom.aten\", version=1)\n",
-    "\n",
-    "# NOTE: The function signature must match the signature of the unsupported ATen operator.\n",
-    "# https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/native_functions.yaml\n",
-    "# NOTE: All attributes must be annotated with type hints.\n",
-    "@onnxscript.script(custom_aten)\n",
-    "def custom_aten_add(input_x, input_y, alpha: float = 1.0):\n",
-    "    input_y = opset18.Mul(input_y, alpha)\n",
-    "    return opset18.Add(input_x, input_y)\n",
-    "\n",
-    "\n",
-    "# Now we have everything we need to support unsupported ATen operators.\n",
-    "# Let's register the ``custom_aten_add`` function to ONNX registry, and export the model to ONNX again.\n",
-    "onnx_registry = torch.onnx.OnnxRegistry()\n",
-    "onnx_registry.register_op(\n",
-    "    namespace=\"aten\", op_name=\"add\", overload=\"Tensor\", function=custom_aten_add\n",
-    "    )\n",
-    "print(f\"aten::add.Tensor is supported by ONNX registry: \\\n",
-    "      {onnx_registry.is_registered_op(namespace='aten', op_name='add', overload='Tensor')}\"\n",
-    "      )\n",
-    "export_options = torch.onnx.ExportOptions(onnx_registry=onnx_registry)\n",
-    "onnx_program = torch.onnx.dynamo_export(\n",
-    "    aten_add_model, input_add_x, input_add_y, export_options=export_options\n",
-    "    )"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Now let\\'s inspect the model and verify the model has a\n",
-    "`custom_aten_add` instead of `aten::add.Tensor`. The graph has one graph\n",
-    "node for `custom_aten_add`, and inside of it there are four function\n",
-    "nodes, one for each operator, and one for constant attribute.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "collapsed": false
-   },
-   "outputs": [],
-   "source": [
-    "# graph node domain is the custom domain we registered\n",
-    "assert onnx_program.model_proto.graph.node[0].domain == \"custom.aten\"\n",
-    "assert len(onnx_program.model_proto.graph.node) == 1\n",
-    "# graph node name is the function name\n",
-    "assert onnx_program.model_proto.graph.node[0].op_type == \"custom_aten_add\"\n",
-    "# function node domain is empty because we use standard ONNX operators\n",
-    "assert {node.domain for node in onnx_program.model_proto.functions[0].node} == {\"\"}\n",
-    "# function node name is the standard ONNX operator name\n",
-    "assert {node.op_type for node in onnx_program.model_proto.functions[0].node} == {\"Add\", \"Mul\", \"Constant\"}"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "This is how `custom_aten_add_model` looks in the ONNX graph using\n",
-    "Netron:\n",
-    "\n",
-    "![image](https://pytorch.org/tutorials/_static/img/onnx/custom_aten_add_model.png){.align-center\n",
-    "width=\"70.0%\"}\n",
-    "\n",
-    "Inside the `custom_aten_add` function, we can see the three ONNX nodes\n",
-    "we used in the function (`CastLike`, `Add`, and `Mul`), and one\n",
-    "`Constant` attribute:\n",
-    "\n",
-    "![image](https://pytorch.org/tutorials/_static/img/onnx/custom_aten_add_function.png){.align-center\n",
-    "width=\"70.0%\"}\n",
-    "\n",
-    "This was all that we needed to register the new ATen operator into the\n",
-    "ONNX Registry. As an additional step, we can use ONNX Runtime to run the\n",
-    "model, and compare the results with PyTorch.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "collapsed": false
-   },
-   "outputs": [],
-   "source": [
-    "# Use ONNX Runtime to run the model, and compare the results with PyTorch\n",
-    "onnx_program.save(\"./custom_add_model.onnx\")\n",
-    "ort_session = onnxruntime.InferenceSession(\n",
-    "    \"./custom_add_model.onnx\", providers=['CPUExecutionProvider']\n",
-    "    )\n",
-    "\n",
-    "def to_numpy(tensor):\n",
-    "    return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()\n",
-    "\n",
-    "onnx_input = onnx_program.adapt_torch_inputs_to_onnx(input_add_x, input_add_y)\n",
-    "onnxruntime_input = {k.name: to_numpy(v) for k, v in zip(ort_session.get_inputs(), onnx_input)}\n",
-    "onnxruntime_outputs = ort_session.run(None, onnxruntime_input)\n",
-    "\n",
-    "torch_outputs = aten_add_model(input_add_x, input_add_y)\n",
-    "torch_outputs = onnx_program.adapt_torch_outputs_to_onnx(torch_outputs)\n",
-    "\n",
-    "assert len(torch_outputs) == len(onnxruntime_outputs)\n",
-    "for torch_output, onnxruntime_output in zip(torch_outputs, onnxruntime_outputs):\n",
-    "    torch.testing.assert_close(torch_output, torch.tensor(onnxruntime_output))"
+    "from onnxscript import opset18  # opset 18 is the latest (and only) supported version for now"
    ]
   },
   {
@@ -385,12 +217,11 @@
     "def to_numpy(tensor):\n",
     "    return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()\n",
     "\n",
-    "onnx_input = onnx_program.adapt_torch_inputs_to_onnx(input_gelu_x)\n",
+    "onnx_input = [input_gelu_x]\n",
     "onnxruntime_input = {k.name: to_numpy(v) for k, v in zip(ort_session.get_inputs(), onnx_input)}\n",
-    "onnxruntime_outputs = ort_session.run(None, onnxruntime_input)\n",
+    "onnxruntime_outputs = ort_session.run(None, onnxruntime_input)[0]\n",
     "\n",
     "torch_outputs = aten_gelu_model(input_gelu_x)\n",
-    "torch_outputs = onnx_program.adapt_torch_outputs_to_onnx(torch_outputs)\n",
     "\n",
     "assert len(torch_outputs) == len(onnxruntime_outputs)\n",
     "for torch_output, onnxruntime_output in zip(torch_outputs, onnxruntime_outputs):\n",
@@ -525,27 +356,20 @@
    "outputs": [],
    "source": [
     "assert onnx_program.model_proto.graph.node[0].domain == \"test.customop\"\n",
-    "assert onnx_program.model_proto.graph.node[0].op_type == \"custom_addandround\"\n",
-    "assert onnx_program.model_proto.functions[0].node[0].domain == \"test.customop\"\n",
-    "assert onnx_program.model_proto.functions[0].node[0].op_type == \"CustomOpOne\"\n",
-    "assert onnx_program.model_proto.functions[0].node[1].domain == \"test.customop\"\n",
-    "assert onnx_program.model_proto.functions[0].node[1].op_type == \"CustomOpTwo\""
+    "assert onnx_program.model_proto.graph.node[0].op_type == \"CustomOpOne\"\n",
+    "assert onnx_program.model_proto.graph.node[1].domain == \"test.customop\"\n",
+    "assert onnx_program.model_proto.graph.node[1].op_type == \"CustomOpTwo\""
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "This is how `custom_addandround_model` ONNX graph looks using Netron:\n",
-    "\n",
-    "![image](https://pytorch.org/tutorials/_static/img/onnx/custom_addandround_model.png){.align-center\n",
-    "width=\"70.0%\"}\n",
-    "\n",
-    "Inside the `custom_addandround` function, we can see the two custom\n",
-    "operators we used in the function (`CustomOpOne`, and `CustomOpTwo`),\n",
-    "and they are from module `test.customop`:\n",
+    "This is how `custom_addandround_model` ONNX graph looks using Netron. We\n",
+    "can see the two custom operators we used in the function (`CustomOpOne`,\n",
+    "and `CustomOpTwo`), and they are from module `test.customop`:\n",
     "\n",
-    "![image](https://pytorch.org/tutorials/_static/img/onnx/custom_addandround_function.png)\n",
+    "![image](https://pytorch.org/tutorials/_static/img/onnx/custom_addandround.png)\n",
     "\n",
     "Custom Ops Registration in ONNX Runtime\n",
     "=======================================\n",

_downloads/1c960eb430ba8694b1655bb03904dac2/export_simple_model_to_onnx_tutorial.py

@@ -7,7 +7,7 @@
 Export a PyTorch model to ONNX
 ==============================
 
-**Author**: `Thiago Crepaldi <https://github.com/thiagocrepaldi>`_
+**Author**: `Ti-Tai Wang <https://github.com/titaiwangms>`_ and `Xavier Dupré <https://github.com/xadupre>`_
 
 .. note::
     As of PyTorch 2.1, there are two versions of ONNX Exporter.
@@ -127,7 +127,7 @@ def forward(self, x):
 # Once Netron is open, we can drag and drop our ``my_image_classifier.onnx`` file into the browser or select it after
 # clicking the **Open model** button.
 #
-# .. image:: ../../_static/img/onnx/image_clossifier_onnx_modelon_netron_web_ui.png
+# .. image:: ../../_static/img/onnx/image_classifier_onnx_model_on_netron_web_ui.png
 #   :width: 50%
 #
 #
@@ -155,7 +155,7 @@ def forward(self, x):
 
 import onnxruntime
 
-onnx_input = onnx_program.adapt_torch_inputs_to_onnx(torch_input)
+onnx_input = [torch_input]
 print(f"Input length: {len(onnx_input)}")
 print(f"Sample input: {onnx_input}")
 
@@ -166,7 +166,8 @@ def to_numpy(tensor):
 
 onnxruntime_input = {k.name: to_numpy(v) for k, v in zip(ort_session.get_inputs(), onnx_input)}
 
-onnxruntime_outputs = ort_session.run(None, onnxruntime_input)
+# onnxruntime returns a list of outputs
+onnxruntime_outputs = ort_session.run(None, onnxruntime_input)[0]
 
 ####################################################################
 # 7. Compare the PyTorch results with the ones from the ONNX Runtime
@@ -179,7 +180,6 @@ def to_numpy(tensor):
 # Before comparing the results, we need to convert the PyTorch's output to match ONNX's format.
 
 torch_outputs = torch_model(torch_input)
-torch_outputs = onnx_program.adapt_torch_outputs_to_onnx(torch_outputs)
 
 assert len(torch_outputs) == len(onnxruntime_outputs)
 for torch_output, onnxruntime_output in zip(torch_outputs, onnxruntime_outputs):

_downloads/33f8140bedc02273a55c752fe79058e5/intro_onnx.ipynb

@@ -24,7 +24,8 @@
     "Introduction to ONNX\n",
     "====================\n",
     "\n",
-    "Authors: [Thiago Crepaldi](https://github.com/thiagocrepaldi),\n",
+    "Authors: [Ti-Tai Wang](https://github.com/titaiwangms) and [Xavier\n",
+    "Dupré](https://github.com/xadupre)\n",
     "\n",
     "[Open Neural Network eXchange (ONNX)](https://onnx.ai/) is an open\n",
     "standard format for representing machine learning models. The\n",

_downloads/8dd55b9d6d32d45fae2642c7ffbf454e/export_simple_model_to_onnx_tutorial.ipynb

@@ -24,7 +24,8 @@
     "Export a PyTorch model to ONNX\n",
     "==============================\n",
     "\n",
-    "**Author**: [Thiago Crepaldi](https://github.com/thiagocrepaldi)\n",
+    "**Author**: [Ti-Tai Wang](https://github.com/titaiwangms) and [Xavier\n",
+    "Dupré](https://github.com/xadupre)\n",
     "\n",
     "<div style=\"background-color: #54c7ec; color: #fff; font-weight: 700; padding-left: 10px; padding-top: 5px; padding-bottom: 5px\"><strong>NOTE:</strong></div>\n",
     "\n",
@@ -212,7 +213,7 @@
     "file into the browser or select it after clicking the **Open model**\n",
     "button.\n",
     "\n",
-    "![image](https://pytorch.org/tutorials/_static/img/onnx/image_clossifier_onnx_modelon_netron_web_ui.png){width=\"50.0%\"}\n",
+    "![image](https://pytorch.org/tutorials/_static/img/onnx/image_classifier_onnx_model_on_netron_web_ui.png){width=\"50.0%\"}\n",
     "\n",
     "And that is it! We have successfully exported our PyTorch model to ONNX\n",
     "format and visualized it with Netron.\n",
@@ -255,7 +256,7 @@
    "source": [
     "import onnxruntime\n",
     "\n",
-    "onnx_input = onnx_program.adapt_torch_inputs_to_onnx(torch_input)\n",
+    "onnx_input = [torch_input]\n",
     "print(f\"Input length: {len(onnx_input)}\")\n",
     "print(f\"Sample input: {onnx_input}\")\n",
     "\n",
@@ -266,7 +267,8 @@
     "\n",
     "onnxruntime_input = {k.name: to_numpy(v) for k, v in zip(ort_session.get_inputs(), onnx_input)}\n",
     "\n",
-    "onnxruntime_outputs = ort_session.run(None, onnxruntime_input)"
+    "# onnxruntime returns a list of outputs\n",
+    "onnxruntime_outputs = ort_session.run(None, onnxruntime_input)[0]"
    ]
   },
   {
@@ -294,7 +296,6 @@
    "outputs": [],
    "source": [
     "torch_outputs = torch_model(torch_input)\n",
-    "torch_outputs = onnx_program.adapt_torch_outputs_to_onnx(torch_outputs)\n",
     "\n",
     "assert len(torch_outputs) == len(onnxruntime_outputs)\n",
     "for torch_output, onnxruntime_output in zip(torch_outputs, onnxruntime_outputs):\n",