Skip to content

Commit f673a4b

Browse files
authored
Updating documentation for cmake dtype selective build (#12112)
### Summary Updating documentation for cmake dtype selective build with model API. Table at `selective_build_model_doc/docs/source/kernel-library-selective-build.md` now looks like: ![image](https://github.com/user-attachments/assets/68b7c8fe-dfc5-4ae4-9756-5599680357bf) Fixes #12065 cc @mergennachin @byjlw
1 parent 37ce7de commit f673a4b

File tree

2 files changed

+31
-30
lines changed

2 files changed

+31
-30
lines changed

docs/source/kernel-library-selective-build.md

Lines changed: 28 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -36,17 +36,20 @@ The basic flow looks like this:
3636

3737
## APIs
3838

39-
We expose a CMake macro `[gen_selected_ops](https://github.com/pytorch/executorch/blob/main/tools/cmake/Codegen.cmake#L12)`, to allow users specifying op info:
39+
We expose a CMake macro [gen_selected_ops](https://github.com/pytorch/executorch/blob/main/tools/cmake/Codegen.cmake#L12), to allow users specifying op info:
4040

4141
```
4242
gen_selected_ops(
43-
LIB_NAME # the name of the selective build operator library to be generated
44-
OPS_SCHEMA_YAML # path to a yaml file containing operators to be selected
45-
ROOT_OPS # comma separated operator names to be selected
46-
INCLUDE_ALL_OPS # boolean flag to include all operators
43+
LIB_NAME # the name of the selective build operator library to be generated
44+
OPS_SCHEMA_YAML # path to a yaml file containing operators to be selected
45+
ROOT_OPS # comma separated operator names to be selected
46+
INCLUDE_ALL_OPS # boolean flag to include all operators
47+
OPS_FROM_MODEL # path to a pte file of model to select operators from
48+
DTYPE_SELECTIVE_BUILD # boolean flag to enable dtye selection
4749
)
4850
```
4951

52+
The macro makes a call to gen_oplist.py, which requires a [distinct selection](https://github.com/BujSet/executorch/blob/main/codegen/tools/gen_oplist.py#L222-L228) of API choice. `OPS_SCHEMA_YAML`, `ROOT_OPS`, `INCLUDE_ALL_OPS`, and `OPS_FROM_MODEL` are mutually exclusive options, and should not be used in conjunction.
5053

5154
### Select all ops
5255

@@ -62,31 +65,29 @@ Context: each kernel library is designed to have a yaml file associated with it.
6265

6366
This API lets users pass in a list of operator names. Note that this API can be combined with the API above and we will create a allowlist from the union of both API inputs.
6467

68+
### Select ops from model
6569

66-
## Example Walkthrough
70+
This API lets users pass in a pte file of an exported model. When used, the pte file will be parsed to generate a yaml file that enumerates the operators and dtypes used in the model.
6771

68-
In CMakeLists.txt we have the following logic:
69-
```cmake
70-
set(_kernel_lib)
71-
if(SELECT_ALL_OPS)
72-
gen_selected_ops("" "" "${SELECT_ALL_OPS}")
73-
elseif(SELECT_OPS_LIST)
74-
gen_selected_ops("" "${SELECT_OPS_LIST}" "")
75-
elseif(SELECT_OPS_YAML)
76-
set(_custom_ops_yaml ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops.yaml)
77-
gen_selected_ops("${_custom_ops_yaml}" "" "")
78-
endif()
79-
```
80-
Then when calling CMake, we can do:
72+
### Dtype Selective Build
8173

82-
```
83-
cmake -D… -DSELECT_OPS_LIST="aten::add.out,aten::mm.out”
84-
```
74+
Beyond pruning the binary to remove unused operators, the binary size can further reduced by removing unused dtypes. For example, if your model only uses floats for the `add` operator, then including variants of the `add` operators for `doubles` and `ints` is unnecessary. The flag `DTYPE_SELECTIVE_BUILD` can be set to `ON` to support this additional optimization. Currently, dtype selective build is only supported with the model API described above. Once enabled, a header file that specifies only the operators and dtypes used by the model is created and linked against a rebuild of the `portable_kernels` lib. This feature is only supported for the portable kernels library; it's not supported for optimized, quantized or custom kernel libraries.
8575

86-
Or
76+
## Example Walkthrough
8777

88-
```
89-
cmake -D… -DSELECT_OPS_YAML=ON
90-
```
78+
In [CMakeLists.txt](https://github.com/BujSet/executorch/blob/main/examples/selective_build/CMakeLists.txt#L48-L72), we have the following cmake config options:
79+
80+
1. `EXECUTORCH_SELECT_OPS_YAML`
81+
2. `EXECUTORCH_SELECT_OPS_LIST`
82+
3. `EXECUTORCH_SELECT_ALL_OPS`
83+
4. `EXECUTORCH_SELECT_OPS_FROM_MODEL`
84+
5. `EXECUTORCH_DTYPE_SELECTIVE_BUILD`
85+
86+
These options allow a user to tailor the cmake build process to utilize the different APIs, and results in different invocations on the `gen_selected_ops` [function](https://github.com/BujSet/executorch/blob/main/examples/selective_build/CMakeLists.txt#L110-L123). The following table describes some examples of how the invocation changes when these configs are set:
9187

92-
To select from either an operator name list or a schema yaml from kernel library.
88+
| Example cmake Call | Resultant `gen_selected_ops` Invocation |
89+
| :----: | :---:|
90+
|<code><br> cmake -D… -DSELECT_OPS_LIST="aten::add.out,aten::mm.out" <br></code> | <code><br> gen_selected_ops("" "${SELECT_OPS_LIST}" "" "" "") <br></code> |
91+
|<code><br> cmake -D… -DSELECT_OPS_YAML=ON <br></code> | <code><br> set(_custom_ops_yaml ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops.yaml) <br> gen_selected_ops("${_custom_ops_yaml}" "" "") <br></code> |
92+
|<code><br> cmake -D… -DEXECUTORCH_SELECT_OPS_FROM_MODEL="model.pte.out" <br></code> | <code><br> gen_selected_ops("" "" "" "${_model_path}" "") <br></code> |
93+
|<code><br> cmake -D… -DEXECUTORCH_SELECT_OPS_FROM_MODEL="model.pte.out" -DEXECUTORCH_DTYPE_SELECTIVE_BUILD=ON<br></code> | <code><br> gen_selected_ops("" "" "" "${_model_path}" "ON") <br></code> |

examples/selective_build/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,12 @@ cd executorch
1212
bash examples/selective_build/test_selective_build.sh cmake
1313
```
1414

15-
Check out `CMakeLists.txt` for demo of 3 selective build APIs:
15+
Check out `CMakeLists.txt` for demo of selective build APIs:
1616
1. `SELECT_ALL_OPS`: Select all ops from the dependency kernel libraries, register all of them into ExecuTorch runtime.
1717
2. `SELECT_OPS_LIST`: Only select operators from a list.
1818
3. `SELECT_OPS_YAML`: Only select operators from a yaml file.
19+
4. `SELECT_OPS_FROM_MODEL`: Only select operators from a from an exported model pte.
20+
5. `DTYPE_SELECTIVE_BUILD`: Enable rebuild of `portable_kernels` to use dtype selection. Currently only supported for `SELECTED_OPS_FROM_MODEL` API and `portable_kernels` lib.
1921

2022
Other configs:
2123
- `MAX_KERNEL_NUM=N`: Only allocate memory for N operators.
22-
23-
We have one more API incoming: only select from an exported model file (.pte).

0 commit comments

Comments
 (0)