Skip to content

Commit dd865a6

Browse files
committed
Update XNNPACK docs to use to_edge_transform_and_lower API
1 parent ca2ac54 commit dd865a6

File tree

1 file changed

+39
-21
lines changed

1 file changed

+39
-21
lines changed

docs/source/tutorial-xnnpack-delegate-lowering.md

Lines changed: 39 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -25,38 +25,24 @@ import torchvision.models as models
2525
from torch.export import export, ExportedProgram
2626
from torchvision.models.mobilenetv2 import MobileNet_V2_Weights
2727
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
28-
from executorch.exir import EdgeProgramManager, ExecutorchProgramManager, to_edge
29-
from executorch.exir.backend.backend_api import to_backend
28+
from executorch.exir import EdgeProgramManager, ExecutorchProgramManager, to_edge_transform_and_lower
3029

3130

3231
mobilenet_v2 = models.mobilenetv2.mobilenet_v2(weights=MobileNet_V2_Weights.DEFAULT).eval()
3332
sample_inputs = (torch.randn(1, 3, 224, 224), )
3433

3534
exported_program: ExportedProgram = export(mobilenet_v2, sample_inputs)
36-
edge: EdgeProgramManager = to_edge(exported_program)
37-
38-
edge = edge.to_backend(XnnpackPartitioner())
35+
edge: EdgeProgramManager = to_edge_transform_and_lower(
36+
exported_program,
37+
partitioner=[XnnpackPartitioner()],
38+
)
3939
```
4040

4141
We will go through this example with the [MobileNetV2](https://pytorch.org/hub/pytorch_vision_mobilenet_v2/) pretrained model downloaded from the TorchVision library. The flow of lowering a model starts after exporting the model `to_edge`. We call the `to_backend` api with the `XnnpackPartitioner`. The partitioner identifies the subgraphs suitable for XNNPACK backend delegate to consume. Afterwards, the identified subgraphs will be serialized with the XNNPACK Delegate flatbuffer schema and each subgraph will be replaced with a call to the XNNPACK Delegate.
4242

4343
```python
4444
>>> print(edge.exported_program().graph_module)
45-
GraphModule(
46-
(lowered_module_0): LoweredBackendModule()
47-
(lowered_module_1): LoweredBackendModule()
48-
)
4945

50-
def forward(self, arg314_1):
51-
lowered_module_0 = self.lowered_module_0
52-
executorch_call_delegate = torch.ops.higher_order.executorch_call_delegate(lowered_module_0, arg314_1); lowered_module_0 = arg314_1 = None
53-
getitem = executorch_call_delegate[0]; executorch_call_delegate = None
54-
aten_view_copy_default = executorch_exir_dialects_edge__ops_aten_view_copy_default(getitem, [1, 1280]); getitem = None
55-
aten_clone_default = executorch_exir_dialects_edge__ops_aten_clone_default(aten_view_copy_default); aten_view_copy_default = None
56-
lowered_module_1 = self.lowered_module_1
57-
executorch_call_delegate_1 = torch.ops.higher_order.executorch_call_delegate(lowered_module_1, aten_clone_default); lowered_module_1 = aten_clone_default = None
58-
getitem_1 = executorch_call_delegate_1[0]; executorch_call_delegate_1 = None
59-
return (getitem_1,)
6046
```
6147

6248
We print the graph after lowering above to show the new nodes that were inserted to call the XNNPACK Delegate. The subgraphs which are being delegated to XNNPACK are the first argument at each call site. It can be observed that the majority of `convolution-relu-add` blocks and `linear` blocks were able to be delegated to XNNPACK. We can also see the operators which were not able to be lowered to the XNNPACK delegate, such as `clone` and `view_copy`.
@@ -149,6 +135,23 @@ mkdir cmake-out
149135
cmake \
150136
-DCMAKE_INSTALL_PREFIX=cmake-out \
151137
-DCMAKE_BUILD_TYPE=Release \
138+
dule(
139+
(lowered_module_0): LoweredBackendModule()
140+
(lowered_module_1): LoweredBackendModule()
141+
)
142+
143+
144+
145+
def forward(self, b_features_0_1_num_batches_tracked, b_getattr_l__self___features___1___conv_0_1_num_batches_tracked, b_getattr_l__self___features___1___conv_2_num_batches_tracked, b_getattr_l__self___features___2___conv_0_1_num_batches_tracked, b_getattr_l__self___features___2___conv_1_1_num_batches_tracked, b_getattr_l__self___features___2___conv_3_num_batches_tracked, b_getattr_l__self___features___3___conv_0_1_num_batches_tracked, b_getattr_l__self___features___3___conv_1_1_num_batches_tracked, b_getattr_l__self___features___3___conv_3_num_batches_tracked, b_getattr_l__self___features___4___conv_0_1_num_batches_tracked, b_getattr_l__self___features___4___conv_1_1_num_batches_tracked, b_getattr_l__self___features___4___conv_3_num_batches_tracked, b_getattr_l__self___features___5___conv_0_1_num_batches_tracked, b_getattr_l__self___features___5___conv_1_1_num_batches_tracked, b_getattr_l__self___features___5___conv_3_num_batches_tracked, b_getattr_l__self___features___6___conv_0_1_num_batches_tracked, b_getattr_l__self___features___6___conv_1_1_num_batches_tracked, b_getattr_l__self___features___6___conv_3_num_batches_tracked, b_getattr_l__self___features___7___conv_0_1_num_batches_tracked, b_getattr_l__self___features___7___conv_1_1_num_batches_tracked, b_getattr_l__self___features___7___conv_3_num_batches_tracked, b_getattr_l__self___features___8___conv_0_1_num_batches_tracked, b_getattr_l__self___features___8___conv_1_1_num_batches_tracked, b_getattr_l__self___features___8___conv_3_num_batches_tracked, b_getattr_l__self___features___9___conv_0_1_num_batches_tracked, b_getattr_l__self___features___9___conv_1_1_num_batches_tracked, b_getattr_l__self___features___9___conv_3_num_batches_tracked, b_getattr_l__self___features___10___conv_0_1_num_batches_tracked, b_getattr_l__self___features___10___conv_1_1_num_batches_tracked, b_getattr_l__self___features___10___conv_3_num_batches_tracked, b_getattr_l__self___features___11___conv_0_1_num_batches_tracked, b_getattr_l__self___features___11___conv_1_1_num_batches_tracked, b_getattr_l__self___features___11___conv_3_num_batches_tracked, b_getattr_l__self___features___12___conv_0_1_num_batches_tracked, b_getattr_l__self___features___12___conv_1_1_num_batches_tracked, b_getattr_l__self___features___12___conv_3_num_batches_tracked, b_getattr_l__self___features___13___conv_0_1_num_batches_tracked, b_getattr_l__self___features___13___conv_1_1_num_batches_tracked, b_getattr_l__self___features___13___conv_3_num_batches_tracked, b_getattr_l__self___features___14___conv_0_1_num_batches_tracked, b_getattr_l__self___features___14___conv_1_1_num_batches_tracked, b_getattr_l__self___features___14___conv_3_num_batches_tracked, b_getattr_l__self___features___15___conv_0_1_num_batches_tracked, b_getattr_l__self___features___15___conv_1_1_num_batches_tracked, b_getattr_l__self___features___15___conv_3_num_batches_tracked, b_getattr_l__self___features___16___conv_0_1_num_batches_tracked, b_getattr_l__self___features___16___conv_1_1_num_batches_tracked, b_getattr_l__self___features___16___conv_3_num_batches_tracked, b_getattr_l__self___features___17___conv_0_1_num_batches_tracked, b_getattr_l__self___features___17___conv_1_1_num_batches_tracked, b_getattr_l__self___features___17___conv_3_num_batches_tracked, b_features_18_1_num_batches_tracked, x):
146+
lowered_module_0 = self.lowered_module_0
147+
lowered_module_1 = self.lowered_module_1
148+
executorch_call_delegate_1 = torch.ops.higher_order.executorch_call_delegate(lowered_module_1, x); lowered_module_1 = x = None
149+
getitem_53 = executorch_call_delegate_1[0]; executorch_call_delegate_1 = None
150+
aten_view_copy_default = executorch_exir_dialects_edge__ops_aten_view_copy_default(getitem_53, [1, 1280]); getitem_53 = None
151+
aten_clone_default = executorch_exir_dialects_edge__ops_aten_clone_default(aten_view_copy_default); aten_view_copy_default = None
152+
executorch_call_delegate = torch.ops.higher_order.executorch_call_delegate(lowered_module_0, aten_clone_default); lowered_module_0 = aten_clone_default = None
153+
getitem_52 = executorch_call_delegate[0]; executorch_call_delegate = None
154+
return (getitem_52,)
152155
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
153156
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
154157
-DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
@@ -157,7 +160,7 @@ cmake \
157160
-DPYTHON_EXECUTABLE=python \
158161
-Bcmake-out .
159162
```
160-
Then you can build the runtime componenets with
163+
Then you can build the runtime components with
161164

162165
```bash
163166
cmake --build cmake-out -j9 --target install --config Release
@@ -171,4 +174,19 @@ Now you should be able to find the executable built at `./cmake-out/backends/xnn
171174
```
172175

173176
## Building and Linking with the XNNPACK Backend
174-
You can build the XNNPACK backend [CMake target](https://github.com/pytorch/executorch/blob/main/backends/xnnpack/CMakeLists.txt#L83), and link it with your application binary such as an Android or iOS application. For more information on this you may take a look at this [resource](demo-apps-android.md) next.
177+
aou can build the XNNPACK backend [CMake target](https://github.com/pytorch/executorch/blob/main/backends/xnnpack/CMakeLists.txt#L83), and link it with your application binary such as an Android or iOS application. For more information on this you may take a look at this [resource](demo-apps-android.md) next.
178+
RraphModule(
179+
(lowered_module_0): LoweredBackendModule()
180+
(lowered_module_1): LoweredBackendModule()
181+
)
182+
183+
def forward(self, arg314_1):
184+
lowered_module_0 = self.lowered_module_0
185+
executorch_call_delegate = torch.ops.higher_order.executorch_call_delegate(lowered_module_0, arg314_1); lowered_module_0 = arg314_1 = None
186+
getitem = executorch_call_delegate[0]; executorch_call_delegate = None
187+
aten_view_copy_default = executorch_exir_dialects_edge__ops_aten_view_copy_default(getitem, [1, 1280]); getitem = None
188+
aten_clone_default = executorch_exir_dialects_edge__ops_aten_clone_default(aten_view_copy_default); aten_view_copy_default = None
189+
lowered_module_1 = self.lowered_module_1
190+
executorch_call_delegate_1 = torch.ops.higher_order.executorch_call_delegate(lowered_module_1, aten_clone_default); lowered_module_1 = aten_clone_default = None
191+
getitem_1 = executorch_call_delegate_1[0]; executorch_call_delegate_1 = None
192+
return (getitem_1,)

0 commit comments

Comments
 (0)