Skip to content

Commit ba0d121

Browse files
committed
NXP backend: Add documentation for graph visualization.
1 parent eb41e0e commit ba0d121

File tree

9 files changed

+145
-1
lines changed

9 files changed

+145
-1
lines changed
125 KB
Loading
148 KB
Loading
171 KB
Loading
173 KB
Loading
76.6 KB
Loading
119 KB
Loading
186 KB
Loading

docs/source/devtools-overview.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ The ExecuTorch Developer Tools support the following features:
1717
- **Debugging** - Intermediate outputs and output quality analysis
1818
- **Numerical Discrepancy Detection** - Operator-level numerical discrepancy detection between AOT and runtime intermediate outputs to streamline numerical debugging and validation.
1919
- **Memory Allocation Insights** - Visualize how memory is planned, where all the live tensors are at any point in time
20-
- **Visualization** - Coming soon
20+
- **Visualization** - Visualize the model as a computational graph
2121

2222
## Fundamental components of the Developer Tools
2323

docs/source/visualize.md

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# Visualize a Model using ModelExplorer
2+
3+
The [visualization_utils.py](../../devtools/visualization/visualization_utils.py) contains functions for
4+
visualizing ExecuTorch models as computational graphs using the `ModelExplorer` utility.
5+
6+
## Installation
7+
8+
To install the `ModelExplorer` and its dependencies, run:
9+
10+
```
11+
./devtools/install_requirements.sh
12+
```
13+
14+
## Visualize a model
15+
16+
The function `visualize()` takes an `ExportedProgram` and launches a `ModelExplorer` server instance. A browser tab will
17+
open, containing the visualization.
18+
19+
The operations in the graph will be grouped together into collapsable nodes, based on which `nn.Module` instances they
20+
originate from (see **<a href="#fig1">Figure 1</a>**). These nodes can be expanded by clicking the button in their top
21+
left corner, as shown
22+
in **<a href="#fig2">Figure 2</a>**. The model can contain an entire hierarchy of collapsable nodes, reflecting its
23+
original _PyTorch_
24+
implementation (see **<a href="#fig3">Figure 3</a>**).
25+
26+
<figure id="fig1">
27+
<img src="./_static/img/visualization/1.png" width="1000" alt="">
28+
<figcaption><b>Figure 1</b>: Model visualization collapsed into a single node representing the original module. </figcaption>
29+
</figure>
30+
31+
<figure id="fig2">
32+
<img src="./_static/img/visualization/2.png" width="1000" alt="">
33+
<figcaption><b>Figure 2</b>: Button to expand a node. </figcaption>
34+
</figure>
35+
36+
<figure id="fig3">
37+
<img src="./_static/img/visualization/3.png" width="1000" alt="">
38+
<figcaption><b>Figure 3</b>: Hierarchy of expandable nodes. </figcaption>
39+
</figure>
40+
41+
The **Model Explorer GUI** provides a button in the top left corner of the screen (see **<a href="#fig4">Figure 4
42+
</a>**),
43+
which expands all the nested expandable nodes. The result will display all the low-level operations, surrounded by
44+
rectangles which indicate their membership to specific `nn.Module` instances.
45+
46+
<figure id="fig4">
47+
<img src="./_static/img/visualization/4.png" width="1000" alt="">
48+
<figcaption><b>Figure 4</b>: Expand all nodes. </figcaption>
49+
</figure>
50+
51+
52+
Sometimes, it is not ideal to view the model like this. Focusing on visualizing the origin of the final nodes can make
53+
it harder to see the flow of data in the graph. For this purpose, a button in the top left corner can flatten all the
54+
layers (expandable nodes), effectively hiding the original `nn.Module` instances and just displaying the model as a
55+
computational graph (see **<a href="#fig5">Figure 5</a>**).
56+
57+
<figure id="fig5">
58+
<img src="./_static/img/visualization/5.png" width="1000" alt="">
59+
<figcaption><b>Figure 5</b>: Flatten the model to a simple computational graph. </figcaption>
60+
</figure>
61+
62+
---
63+
64+
# Visualize a Model with Highlighted QDQ Clusters and Partitions
65+
66+
The [visualization_utils.py](../../devtools/visualization/visualization_utils.py) contains the function
67+
`visualize_with_clusters()` which takes an `ExportedProgram` and visualizes it using the `ModelExplorer` utility.
68+
It groups QDQ clusters and individual partitions together to improve readability. Example usage is available
69+
in [examples/nxp/aot_neutron_compile.py](../../examples/nxp/aot_neutron_compile.py).
70+
71+
An example of the visualization is shown in **<a href="#fig6">Figure 6</a>.**
72+
<figure id="fig6">
73+
<img src="./_static/img/visualization/visualize_with_clusters_example.png" width="1000" alt="">
74+
<figcaption><b>Figure 6</b>: Example of the QDQ cluster and partition highlighting visualization.</figcaption>
75+
</figure>
76+
77+
## Usage
78+
79+
There are two main use cases for the visualization:
80+
81+
### 1. Launching the `ModelExplorer` and Visualizing the Model Immediately
82+
83+
Call:
84+
85+
```python
86+
visualize_with_clusters(exported_program)
87+
```
88+
89+
This starts a `ModelExplorer` server and opens a browser tab with the visualization.
90+
91+
By default, each call starts a new server instance and opens a new browser tab.
92+
To reuse an existing server, set the `reuse_server` parameter to `True`.
93+
94+
Starting the server is **blocking**, so the rest of your script will not run.
95+
96+
### 2. Storing a Serialized Graph and Visualizing Later (Non-blocking)
97+
98+
To save the visualization to a JSON file, call:
99+
100+
```python
101+
visualize_with_clusters(exported_program, "my_model.json")
102+
```
103+
104+
This just saves the visualization in the file, and it does **not** start the `ModelExplorer` server. You can then open
105+
the file in the `ModelExplorer` GUI at any point. To launch the server, run:
106+
107+
```bash
108+
model-explorer [model-file-json]
109+
```
110+
111+
If the `model-file-json` is provided, the `ModelExplorer` will open the model visualization. Otherwise, the
112+
`ModelBuilder` GUI home page will appear. In that case, click **Select from your computer**, choose the JSON file,
113+
and then click **View selected models** to display the graph.
114+
115+
---
116+
117+
## Styling the Graph
118+
119+
`visualize_with_clusters()` supports custom grouping of nodes into QDQ clusters and partitions.
120+
121+
You can pass the following optional parameters:
122+
123+
- `get_node_partition_name`
124+
- `get_node_qdq_cluster_name`
125+
126+
These are functions that take a node and return a string identifying the partition or cluster it belongs to.
127+
Nodes with the same partition/cluster string will be grouped together and labeled accordingly in the visualization.
128+
129+
### Load a predefined style for QDQ cluster and partition highlighting.
130+
131+
A color style for the QDQ cluster and partition highlighting is already provided
132+
in [devtools/visualization/model_explorer_styles/cluster_highlight_style.json](../../devtools/visualization/model_explorer_styles/cluster_highlight_style.json).
133+
To load it follow these steps:
134+
135+
1. Click the **palette icon** in the top-right corner of the `ModelExplorer` interface.
136+
2. Click **Import rules**.
137+
3. Select
138+
the [cluster_highlight_style.json](../../devtools/visualization/model_explorer_styles/cluster_highlight_style.json)
139+
file to apply predefined styles that highlight each partition in a different color.
140+
141+
<figure id="fig7">
142+
<img src="./_static/img/visualization/6.png" width="1000" alt="">
143+
<figcaption><b>Figure 7</b>: Add custom color styling to the graph. </figcaption>
144+
</figure>

0 commit comments

Comments
 (0)