You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/kernel-library-selective-build.md
+28-27Lines changed: 28 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -36,17 +36,20 @@ The basic flow looks like this:
36
36
37
37
## APIs
38
38
39
-
We expose a CMake macro `[gen_selected_ops](https://github.com/pytorch/executorch/blob/main/tools/cmake/Codegen.cmake#L12)`, to allow users specifying op info:
39
+
We expose a CMake macro [gen_selected_ops](https://github.com/pytorch/executorch/blob/main/tools/cmake/Codegen.cmake#L12), to allow users specifying op info:
40
40
41
41
```
42
42
gen_selected_ops(
43
-
LIB_NAME # the name of the selective build operator library to be generated
44
-
OPS_SCHEMA_YAML # path to a yaml file containing operators to be selected
45
-
ROOT_OPS # comma separated operator names to be selected
46
-
INCLUDE_ALL_OPS # boolean flag to include all operators
43
+
LIB_NAME # the name of the selective build operator library to be generated
44
+
OPS_SCHEMA_YAML # path to a yaml file containing operators to be selected
45
+
ROOT_OPS # comma separated operator names to be selected
46
+
INCLUDE_ALL_OPS # boolean flag to include all operators
47
+
OPS_FROM_MODEL # path to a pte file of model to select operators from
48
+
DTYPE_SELECTIVE_BUILD # boolean flag to enable dtye selection
47
49
)
48
50
```
49
51
52
+
The macro makes a call to gen_oplist.py, which requires a [distinct selection](https://github.com/BujSet/executorch/blob/main/codegen/tools/gen_oplist.py#L222-L228) of API choice. `OPS_SCHEMA_YAML`, `ROOT_OPS`, `INCLUDE_ALL_OPS`, and `OPS_FROM_MODEL` are mutually exclusive options, and should not be used in conjunction.
50
53
51
54
### Select all ops
52
55
@@ -62,31 +65,29 @@ Context: each kernel library is designed to have a yaml file associated with it.
62
65
63
66
This API lets users pass in a list of operator names. Note that this API can be combined with the API above and we will create a allowlist from the union of both API inputs.
64
67
68
+
### Select ops from model
65
69
66
-
## Example Walkthrough
70
+
This API lets users pass in a pte file of an exported model. When used, the pte file will be parsed to generate a yaml file that enumerates the operators and dtypes used in the model.
Beyond pruning the binary to remove unused operators, the binary size can further reduced by removing unused dtypes. For example, if your model only uses floats for the `add` operator, then including variants of the `add` operators for `doubles` and `ints` is unnecessary. The flag `DTYPE_SELECTIVE_BUILD` can be set to `ON` to support this additional optimization. Currently, dtype selective build is only supported with the model API described above. Once enabled, a header file that specifies only the operators and dtypes used by the model is created and linked against a rebuild of the `portable_kernels` lib. This feature is only supported for the portable kernels library; it's not supported for optimized, quantized or custom kernel libraries.
85
75
86
-
Or
76
+
## Example Walkthrough
87
77
88
-
```
89
-
cmake -D… -DSELECT_OPS_YAML=ON
90
-
```
78
+
In [CMakeLists.txt](https://github.com/BujSet/executorch/blob/main/examples/selective_build/CMakeLists.txt#L48-L72), we have the following cmake config options:
79
+
80
+
1.`EXECUTORCH_SELECT_OPS_YAML`
81
+
2.`EXECUTORCH_SELECT_OPS_LIST`
82
+
3.`EXECUTORCH_SELECT_ALL_OPS`
83
+
4.`EXECUTORCH_SELECT_OPS_FROM_MODEL`
84
+
5.`EXECUTORCH_DTYPE_SELECTIVE_BUILD`
85
+
86
+
These options allow a user to tailor the cmake build process to utilize the different APIs, and results in different invocations on the `gen_selected_ops`[function](https://github.com/BujSet/executorch/blob/main/examples/selective_build/CMakeLists.txt#L110-L123). The following table describes some examples of how the invocation changes when these configs are set:
91
87
92
-
To select from either an operator name list or a schema yaml from kernel library.
88
+
| Example cmake Call | Resultant `gen_selected_ops` Invocation |
Check out `CMakeLists.txt` for demo of 3 selective build APIs:
15
+
Check out `CMakeLists.txt` for demo of selective build APIs:
16
16
1.`SELECT_ALL_OPS`: Select all ops from the dependency kernel libraries, register all of them into ExecuTorch runtime.
17
17
2.`SELECT_OPS_LIST`: Only select operators from a list.
18
18
3.`SELECT_OPS_YAML`: Only select operators from a yaml file.
19
+
4.`SELECT_OPS_FROM_MODEL`: Only select operators from a from an exported model pte.
20
+
5.`DTYPE_SELECTIVE_BUILD`: Enable rebuild of `portable_kernels` to use dtype selection. Currently only supported for `SELECTED_OPS_FROM_MODEL` API and `portable_kernels` lib.
19
21
20
22
Other configs:
21
23
-`MAX_KERNEL_NUM=N`: Only allocate memory for N operators.
22
-
23
-
We have one more API incoming: only select from an exported model file (.pte).
0 commit comments