Skip to content

Commit d1702f0

Browse files
cccclaidbort
authored andcommitted
update delegate doc (#956)
Summary: Pull Request resolved: #956 Reviewed By: mcr229 Differential Revision: D50333307 fbshipit-source-id: 00a8c862e18f6da1fdacac03e2334566381fbb3c
1 parent 2ffe283 commit d1702f0

File tree

5 files changed

+15
-7
lines changed

5 files changed

+15
-7
lines changed
69.8 KB
Loading
174 KB
Loading
103 KB
Loading
-490 Bytes
Loading

docs/source/compiler-delegate-and-partitioner.md

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -55,16 +55,15 @@ A delegate backend implementation is composed of:
5555

5656
The diagram looks like following
5757

58-
<img src="./_static/img/backend_interface.png" alt="drawing" style="width:500px;"/>
58+
<img src="./_static/img/backend_interface.png" alt="drawing" style="width:600px;"/>
5959

6060
**Figure 1.** A high level of entry points for backend interfaces, including both ahead-of-time and runtime.
6161

6262
## Backend Interfaces: Ahead-of-Time Preprocessing
6363

6464
There are mainly two Ahead-of-Time entry point for backend to implement: `partition` and `preprocess`.
6565

66-
`partitioner` is an algorithm implemented by the backend to tag the nodes to be lowered to the backend. `to_backend` API
67-
will apply the partition algorithm and lowered each subgraph, consists of connected tagged nodes, to the backend. Every subgraph
66+
`partitioner` is an algorithm implemented by the backend to tag the nodes to be lowered to the backend. `to_backend` API will apply the partition algorithm and lower each subgraph, which consists of connected tagged nodes, to the targeted backend. Every subgraph
6867
will be sent to the `preprocess` part provided by the backend to compiled as a binary blob.
6968

7069
During partition, the `exported_program` is not allowed to mutate the program, and it's supposed to apply tag to each node. The
@@ -135,7 +134,7 @@ virtual void destroy(__ET_UNUSED DelegateHandle* handle);
135134
136135
The diagram looks like following
137136
138-
<img src="./_static/img/backend_interface_runtime.png" alt="drawing" style="width:500px;"/>
137+
<img src="./_static/img/backend_interface_runtime.png" alt="drawing" style="width:600px;"/>
139138
140139
**Figure 3.** The relationship between standard ExecuTorch Runtime and backend entry point.
141140
@@ -155,13 +154,22 @@ static auto success_with_compiler = register_backend(backend);
155154
```
156155

157156

158-
## Debugging model execution within delegate
157+
## SDK Integration: Debuggability
159158

160159
Providing consistent debugging experience, be it for runtime failures or performance profiling, is important. ExecuTorch employs native SDK (Software Development Kit) for this purpose, which enables correlating program instructions to original PyTorch code, via debug handles. You can read more about it [here](./sdk-etrecord).
161160

162-
Delegated program or subgraphs are opaque to ExecuTorch runtime and appear as a special `call_delegate` instruction, which asks corresponding delegate to handle the execution of the subgraph or program. Due to the opaque nature of delgates, native SDK does not have visibility into delegated program. Thus the debugging, functional or performance, experiences of delegated execution suffers significantly as compared to it's non-delegated counterpart.
161+
Delegated program or subgraphs are opaque to ExecuTorch runtime and appear as a special `call_delegate` instruction, which asks corresponding backend to handle the execution of the subgraph or program. Due to the opaque nature of backend delgates, native SDK does not have visibility into delegated program. Thus the debugging, functional or performance, experiences of delegated execution suffers significantly as compared to it's non-delegated counterpart.
163162

164-
In order to provide consistent debugging experience to users, regardless of the use of delegation for a model, SDK provides an interface to correlate delegated (sub)graph to original (sub)graph. The SDK does so via debug handles map which allows delegates to generate internal handles that can be associated with the original (sub)graph consumed by the delegate. Then at runtime, delegates can report error or profiling information using the internal handle, which will be mapped to original (sub)graph using the debug handle map. For more information, please refer to [SDK delegate integration](./sdk-delegate-integration).
163+
In order to provide consistent debugging experience to users, regardless of the use of delegation for a model, SDK provides an interface to correlate delegated (sub)graph to original (sub)graph. The SDK does so via debug handles map which allows delegates to generate internal handles that can be associated with the original (sub)graph consumed by the delegate. Then at runtime, backend developer can report error or profiling information using the internal handle, which will be mapped to original (sub)graph using the debug handle map. For more information, please refer to [SDK delegate integration](./sdk-delegate-integration).
164+
165+
By leveraging the debug identifier, backend developer can embed the debug as part of the delegated blob
166+
167+
<img src="./_static/img/backend_debug_handle.png" alt="drawing" style="width:600px;"/>
168+
169+
In this way, during execute stage, with the debug identifier, backend developer can associate the failed instruction inside the delegate
170+
back to the exact line of PyThon code.
171+
172+
<img src="./_static/img/backend_debug_handle_example.png" alt="drawing" style="width:700px;"/>
165173

166174
## Common Questions
167175

0 commit comments

Comments
 (0)