|
| 1 | +# Visualize a Model using ModelExplorer |
| 2 | + |
| 3 | +The [visualization_utils.py](../../devtools/visualization/visualization_utils.py) contains functions for |
| 4 | +visualizing ExecuTorch models as computational graphs using the `ModelExplorer` utility. |
| 5 | + |
| 6 | +## Installation |
| 7 | + |
| 8 | +To install the `ModelExplorer` and its dependencies, run: |
| 9 | + |
| 10 | +``` |
| 11 | +./devtools/install_requirements.sh |
| 12 | +``` |
| 13 | + |
| 14 | +## Visualize a model |
| 15 | + |
| 16 | +The function `visualize()` takes an `ExportedProgram` and launches a `ModelExplorer` server instance. A browser tab will |
| 17 | +open, containing the visualization. |
| 18 | + |
| 19 | +The operations in the graph will be grouped together into collapsable nodes, based on which `nn.Module` instances they |
| 20 | +originate from (see **<a href="#fig1">Figure 1</a>**). These nodes can be expanded by clicking the button in their top |
| 21 | +left corner, as shown |
| 22 | +in **<a href="#fig2">Figure 2</a>**. The model can contain an entire hierarchy of collapsable nodes, reflecting its |
| 23 | +original _PyTorch_ |
| 24 | +implementation (see **<a href="#fig3">Figure 3</a>**). |
| 25 | + |
| 26 | +<figure id="fig1"> |
| 27 | + <img src="./_static/img/visualization/1.png" width="1000" alt=""> |
| 28 | + <figcaption><b>Figure 1</b>: Model visualization collapsed into a single node representing the original module. </figcaption> |
| 29 | +</figure> |
| 30 | + |
| 31 | +<figure id="fig2"> |
| 32 | + <img src="./_static/img/visualization/2.png" width="1000" alt=""> |
| 33 | + <figcaption><b>Figure 2</b>: Button to expand a node. </figcaption> |
| 34 | +</figure> |
| 35 | + |
| 36 | +<figure id="fig3"> |
| 37 | + <img src="./_static/img/visualization/3.png" width="1000" alt=""> |
| 38 | + <figcaption><b>Figure 3</b>: Hierarchy of expandable nodes. </figcaption> |
| 39 | +</figure> |
| 40 | + |
| 41 | +The **Model Explorer GUI** provides a button in the top left corner of the screen (see **<a href="#fig4">Figure 4 |
| 42 | +</a>**), |
| 43 | +which expands all the nested expandable nodes. The result will display all the low-level operations, surrounded by |
| 44 | +rectangles which indicate their membership to specific `nn.Module` instances. |
| 45 | + |
| 46 | +<figure id="fig4"> |
| 47 | + <img src="./_static/img/visualization/4.png" width="1000" alt=""> |
| 48 | + <figcaption><b>Figure 4</b>: Expand all nodes. </figcaption> |
| 49 | +</figure> |
| 50 | + |
| 51 | + |
| 52 | +Sometimes, it is not ideal to view the model like this. Focusing on visualizing the origin of the final nodes can make |
| 53 | +it harder to see the flow of data in the graph. For this purpose, a button in the top left corner can flatten all the |
| 54 | +layers (expandable nodes), effectively hiding the original `nn.Module` instances and just displaying the model as a |
| 55 | +computational graph (see **<a href="#fig5">Figure 5</a>**). |
| 56 | + |
| 57 | +<figure id="fig5"> |
| 58 | + <img src="./_static/img/visualization/5.png" width="1000" alt=""> |
| 59 | + <figcaption><b>Figure 5</b>: Flatten the model to a simple computational graph. </figcaption> |
| 60 | +</figure> |
| 61 | + |
| 62 | +--- |
| 63 | + |
| 64 | +# Visualize a Model with Highlighted QDQ Clusters and Partitions |
| 65 | + |
| 66 | +The [visualization_utils.py](../../devtools/visualization/visualization_utils.py) contains the function |
| 67 | +`visualize_with_clusters()` which takes an `ExportedProgram` and visualizes it using the `ModelExplorer` utility. |
| 68 | +It groups QDQ clusters and individual partitions together to improve readability. Example usage is available |
| 69 | +in [examples/nxp/aot_neutron_compile.py](../../examples/nxp/aot_neutron_compile.py). |
| 70 | + |
| 71 | +An example of the visualization is shown in **<a href="#fig6">Figure 6</a>.** |
| 72 | +<figure id="fig6"> |
| 73 | + <img src="./_static/img/visualization/visualize_with_clusters_example.png" width="1000" alt=""> |
| 74 | + <figcaption><b>Figure 6</b>: Example of the QDQ cluster and partition highlighting visualization.</figcaption> |
| 75 | +</figure> |
| 76 | + |
| 77 | +## Usage |
| 78 | + |
| 79 | +There are two main use cases for the visualization: |
| 80 | + |
| 81 | +### 1. Launching the `ModelExplorer` and Visualizing the Model Immediately |
| 82 | + |
| 83 | +Call: |
| 84 | + |
| 85 | +```python |
| 86 | +visualize_with_clusters(exported_program) |
| 87 | +``` |
| 88 | + |
| 89 | +This starts a `ModelExplorer` server and opens a browser tab with the visualization. |
| 90 | + |
| 91 | +By default, each call starts a new server instance and opens a new browser tab. |
| 92 | +To reuse an existing server, set the `reuse_server` parameter to `True`. |
| 93 | + |
| 94 | +Starting the server is **blocking**, so the rest of your script will not run. |
| 95 | + |
| 96 | +### 2. Storing a Serialized Graph and Visualizing Later (Non-blocking) |
| 97 | + |
| 98 | +To save the visualization to a JSON file, call: |
| 99 | + |
| 100 | +```python |
| 101 | +visualize_with_clusters(exported_program, "my_model.json") |
| 102 | +``` |
| 103 | + |
| 104 | +This just saves the visualization in the file, and it does **not** start the `ModelExplorer` server. You can then open |
| 105 | +the file in the `ModelExplorer` GUI at any point. To launch the server, run: |
| 106 | + |
| 107 | +```bash |
| 108 | + model-explorer [model-file-json] |
| 109 | +``` |
| 110 | + |
| 111 | +If the `model-file-json` is provided, the `ModelExplorer` will open the model visualization. Otherwise, the |
| 112 | +`ModelBuilder` GUI home page will appear. In that case, click **Select from your computer**, choose the JSON file, |
| 113 | +and then click **View selected models** to display the graph. |
| 114 | + |
| 115 | +--- |
| 116 | + |
| 117 | +## Styling the Graph |
| 118 | + |
| 119 | +`visualize_with_clusters()` supports custom grouping of nodes into QDQ clusters and partitions. |
| 120 | + |
| 121 | +You can pass the following optional parameters: |
| 122 | + |
| 123 | +- `get_node_partition_name` |
| 124 | +- `get_node_qdq_cluster_name` |
| 125 | + |
| 126 | +These are functions that take a node and return a string identifying the partition or cluster it belongs to. |
| 127 | +Nodes with the same partition/cluster string will be grouped together and labeled accordingly in the visualization. |
| 128 | + |
| 129 | +### Load a predefined style for QDQ cluster and partition highlighting. |
| 130 | + |
| 131 | +A color style for the QDQ cluster and partition highlighting is already provided |
| 132 | +in [devtools/visualization/model_explorer_styles/cluster_highlight_style.json](../../devtools/visualization/model_explorer_styles/cluster_highlight_style.json). |
| 133 | +To load it follow these steps: |
| 134 | + |
| 135 | +1. Click the **palette icon** in the top-right corner of the `ModelExplorer` interface. |
| 136 | +2. Click **Import rules**. |
| 137 | +3. Select |
| 138 | + the [cluster_highlight_style.json](../../devtools/visualization/model_explorer_styles/cluster_highlight_style.json) |
| 139 | + file to apply predefined styles that highlight each partition in a different color. |
| 140 | + |
| 141 | +<figure id="fig7"> |
| 142 | + <img src="./_static/img/visualization/6.png" width="1000" alt=""> |
| 143 | + <figcaption><b>Figure 7</b>: Add custom color styling to the graph. </figcaption> |
| 144 | +</figure> |
0 commit comments