Skip to content

Commit 5db1bfb

Browse files
[ExecuTorch][Export][1/N] Export API pipeline re-architecture, making it composable
RFC: #12660 This diff introduces composable architecture for the export pipeline. Changes: 1. Introduces notion of `PipelineArtifact` which encompasses stage artifacts and run context necessary to execute a stage and the pipeline. 1. Move all stages to new file to better architect the component 1. Export api will now accept `pipeline_stages` and honors the sequence if provided otherwise fallback to default sequence: `source_transform -> quantize -> torch.export -> to_edge_transform_and_lower -> to_executorch` 1. Validate if pipeline provided is valid, error out if not. 1. Add unittests to test pipeline sequence and stages With this one can execute a partial pipeline, for example, one can just do `TORCH_EXPORT -> TO_EDGE_TRANSFORM_AND_LOWER -> TO_EXECUTORCH` Current limitation: 1. `TORCH_EXPORT` stage is mandatory to avoid someone passing incorrect input such as `ExportedProgram` instead of nn.module, this enforcement will anchor the pipeline to have expected stages to run. - This limitation will be removed if we choose to add the support to take `ExportedProgram` as input. Note: 1. this diff is large because I moved stages to separate file and added bunch of unittests but core functionality that is added can be reviewed in `export.py` file. CC: digantdesai 2. Export component was supposed to mimic stages design in tester and now it is. Fixes: #12928 Differential Revision: [D79120574](https://our.internmc.facebook.com/intern/diff/D79120574/) ghstack-source-id: 299106076 Pull Request resolved: #12936
1 parent 45846c8 commit 5db1bfb

File tree

7 files changed

+946
-916
lines changed

7 files changed

+946
-916
lines changed

export/TARGETS

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ runtime.python_library(
1515
"//caffe2:torch",
1616
"//executorch/exir/backend:backend_api",
1717
"//executorch/exir:pass_manager",
18-
"//executorch/devtools/backend_debug:delegation_info",
1918
"//executorch/extension/export_util:export_util",
2019
]
2120
)
@@ -31,11 +30,33 @@ runtime.python_library(
3130
],
3231
deps = [
3332
":recipe",
33+
":stages",
3434
"//executorch/runtime:runtime",
3535
":recipe_registry"
3636
]
3737
)
3838

39+
40+
runtime.python_library(
41+
name = "stages",
42+
srcs = [
43+
"stages.py",
44+
],
45+
visibility = [
46+
"//executorch/...",
47+
"@EXECUTORCH_CLIENTS",
48+
],
49+
deps = [
50+
":recipe",
51+
"//executorch/devtools/backend_debug:delegation_info",
52+
"//executorch/exir/backend:backend_api",
53+
"//executorch/exir:pass_manager",
54+
"//caffe2:torch",
55+
"//executorch/devtools/backend_debug:delegation_info",
56+
]
57+
)
58+
59+
3960
runtime.python_library(
4061
name = "lib",
4162
srcs = [
@@ -48,6 +69,7 @@ runtime.python_library(
4869
deps = [
4970
":export",
5071
":recipe",
72+
":stages",
5173
":recipe_registry",
5274
":recipe_provider"
5375
],

0 commit comments

Comments
 (0)