Skip to content

Commit 6a2e582

Browse files
kimishpatelmergennachin
authored andcommitted
Update delegate tutorial (#938)
Summary: Pull Request resolved: #938 As title says Reviewed By: mergennachin Differential Revision: D50208348 fbshipit-source-id: 8bd5fbef8bfe4df0c6a0f89b4411fc1f379a43b7
1 parent fc518e7 commit 6a2e582

File tree

1 file changed

+11
-60
lines changed

1 file changed

+11
-60
lines changed

docs/source/compiler-delegate-and-partitioner.md

Lines changed: 11 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -29,13 +29,13 @@
2929

3030
Audience: Vendors, Backend Delegate developers, who are interested in integrating their own compilers and hardware as part of ExecuTorch
3131

32-
## Backend Interfaces: Overview
33-
3432
Backend delegation is an entry point for backends to process and execute PyTorch
3533
programs to leverage performance and efficiency benefits of specialized
3634
backends and hardware, while still providing PyTorch users with an experience
3735
close to that of the PyTorch runtime.
3836

37+
## Backend Interfaces: Overview
38+
3939
At a high level, the entry point for backends is defined by 2 components:
4040

4141
- An IR to represent the program: **Edge Dialect** (which is produced through
@@ -79,10 +79,10 @@ def partition(
7979

8080
During preprocessing, backends are given an edge dialect program,
8181
a list of compile specs specifying the values needed for compilation, and are
82-
expected to return a compiled blob, or binary contains the desired program to be
83-
run in the backend, and profiling information. During serialization, the
82+
expected to return a compiled blob, or binary containing the desired program to be
83+
run in the backend. During serialization, the
8484
compiled blob will be serialized as part of the `.pte` file, and directly loaded to the device. The
85-
API looks something like:
85+
API for this process is:
8686

8787
```python
8888
def preprocess(
@@ -140,12 +140,12 @@ The diagram looks like following
140140
**Figure 3.** The relationship between standard ExecuTorch Runtime and backend entry point.
141141
142142
143-
Once the backend is ready, they can then be registered via the `register_backend` API:
143+
In order to make backend available to ExecuTorch runtime, it must be registered via the `register_backend` API:
144144
```cpp
145145
__ET_NODISCARD Error register_backend(const Backend& backend);
146146
```
147147

148-
`register_backend` can be statically registered like the following
148+
Static registeration, i.e., at libraray init or load time, of a backend can be achieved as follows:
149149
```cpp
150150
namespace {
151151
auto cls = BackendWithCompiler();
@@ -155,62 +155,13 @@ static auto success_with_compiler = register_backend(backend);
155155
```
156156

157157

158-
## SDK: Debug Handle
159-
160-
If there is an error in the backend, for example, if there is any operator that is not
161-
supported by the backend, backend developer can raise an exception with debug handler.
162-
The debug handle can surface back to the Python frontend with the source code information. Below is an
163-
example where the `tan` operator is not supported in `BackendWithCompilerDemo`
164-
backend. Please refer to [SDK delegate integration](./sdk-delegate-integration)
165-
for more details.
166-
167-
A problematic program:
168-
```python
169-
class TanModule(torch.nn.Module):
170-
def __init__(self):
171-
super().__init__()
172-
173-
def forward(self, x):
174-
return torch.tan(x)
175-
176-
tan_module = TanModule()
177-
model_inputs = (torch.ones(1),)
178-
edgeir_m = exir.capture(tan_module, model_inputs).to_edge()
179-
lowered_tan_module = to_backend(
180-
"BackendWithCompilerDemo", edgeir_m, []
181-
)
158+
## Debugging model execution within delegate
182159

183-
class CompositeModelWithTan(torch.nn.Module):
184-
def __init__(self):
185-
super().__init__()
186-
self.lowered_tan = lowered_tan_module
187-
188-
def forward(self, x):
189-
output_from_submodule = self.lowered_tan(x)
190-
return output_from_submodule
191-
192-
composite_model_with_tan = CompositeModelWithTan()
193-
model_inputs = (torch.ones(1),)
194-
195-
composite_model_with_tan(*model_inputs)
196-
197-
exec_prog = (
198-
exir.capture(composite_model_with_tan, model_inputs).to_edge().to_executorch()
199-
)
200-
201-
buff = exec_prog.buffer
202-
model_inputs = torch.ones(1)
203-
204-
# Load and init the program in executor
205-
executorch_module = _load_for_executorch_from_buffer(buff)
206-
207-
# Expect to throw with debug handler here.
208-
model_outputs = executorch_module.forward([model_inputs])
209-
```
160+
Providing consistent debugging experience, be it for runtime failures or performance profiling, is important. ExecuTorch employs native SDK (Software Development Kit) for this purpose, which enables correlating program instructions to original PyTorch code, via debug handles. You can read more about it [here](./sdk-etrecord).
210161

211-
It's expected to throw with the error message mentioning debug handler like `instruction
212-
demo::tan_default<debug_handle>1 is not supported, debug handler is: 1`
162+
Delegated program or subgraphs are opaque to ExecuTorch runtime and appear as a special `call_delegate` instruction, which asks corresponding delegate to handle the execution of the subgraph or program. Due to the opaque nature of delgates, native SDK does not have visibility into delegated program. Thus the debugging, functional or performance, experiences of delegated execution suffers significantly as compared to it's non-delegated counterpart.
213163

164+
In order to provide consistent debugging experience to users, regardless of the use of delegation for a model, SDK provides an interface to correlate delegated (sub)graph to original (sub)graph. The SDK does so via debug handles map which allows delegates to generate internal handles that can be associated with the original (sub)graph consumed by the delegate. Then at runtime, delegates can report error or profiling information using the internal handle, which will be mapped to original (sub)graph using the debug handle map. For more information, please refer to [SDK delegate integration](./sdk-delegate-integration).
214165

215166
## Common Questions
216167

0 commit comments

Comments
 (0)