1
1
# TRTorch Partitioning
2
2
3
- TRTorch partitioning phase is developed to support automatic fallback feature in TRTorch. This phase won't run by
3
+ TRTorch partitioning phase is developed to support ` automatic fallback ` feature in TRTorch. This phase won't run by
4
4
default until the automatic fallback feature is enabled.
5
5
6
6
On a high level, TRTorch partitioning phase does the following:
@@ -15,6 +15,8 @@ from the user. Shapes can be calculated by running the graphs with JIT.
15
15
it's still a phase in our partitioning process.
16
16
- ` Stitching ` . Stitch all TensorRT engines with PyTorch nodes altogether.
17
17
18
+ Test cases for each of these components could be found [ here] ( https://github.com/NVIDIA/TRTorch/tree/master/tests/core/partitioning ) .
19
+
18
20
Here is the brief description of functionalities of each file:
19
21
- ` PartitionInfo.h/cpp ` : The automatic fallback APIs that is used for partitioning.
20
22
- ` SegmentedBlock.h/cpp ` : The main data structures that is used to maintain information for each segments after segmentation.
@@ -34,8 +36,8 @@ To enable automatic fallback feature, you can set following attributes in Python
34
36
...
35
37
" torch_fallback" : {
36
38
" enabled" : True ,
37
- " min_block_size" : 1 ,
38
- " forced_fallback_ops" : [" aten::foo " ],
39
+ " min_block_size" : 3 ,
40
+ " forced_fallback_ops" : [" aten::add " ],
39
41
}
40
42
})
41
43
```
@@ -58,8 +60,8 @@ auto mod = trtorch::jit::load("trt_ts_module.ts");
58
60
auto input_sizes = std::vector< trtorch::CompileSpec::InputRange > {{in.sizes()}};
59
61
trtorch::CompileSpec cfg(input_sizes);
60
62
cfg.torch_fallback = trtorch::CompileSpec::TorchFallback(true);
61
- cfg.torch_fallback.min_block_size = 1 ;
62
- cfg.torch_fallback.forced_fallback_ops.push_back("aten::foo ");
63
+ cfg.torch_fallback.min_block_size = 2 ;
64
+ cfg.torch_fallback.forced_fallback_ops.push_back("aten::relu ");
63
65
auto trt_mod = trtorch::CompileGraph(mod, cfg);
64
66
auto out = trt_mod.forward({in});
65
67
```
0 commit comments