Skip to content

Commit 5e26564

Browse files
authored
add inference-analysis doc (#11813)
1 parent c2165ff commit 5e26564

File tree

1 file changed

+57
-0
lines changed

1 file changed

+57
-0
lines changed
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Inference Analysis
2+
3+
The `inference/analysis` module is used to analyze and optimize the inference program,
4+
it references some philosophy from `LLVM/analysis`,
5+
and make the various optimization features be pluggable and co-exist in a pipeline.
6+
7+
We borrowed some concepts from LLVM, such as
8+
9+
- [Pass](./pass.h)es to implement optimization that traverse the inference program,
10+
- [DataFlowGraph](./data_flow_graph.h) to represent the data flow graph built from a program,
11+
- [PassManager](./pass_manager.h) to manage a sequence of `Pass`es over a graph.
12+
13+
There are some other basic concepts here
14+
15+
- [Node](./node.h), the node in a `DataFlowGraph`,
16+
- `Function`, the Operator in Fluid,
17+
- `Value`, the Variable in Fluid;
18+
- [Argument](./argument.h), the argument that treat as the input and output of all `Pass`es in the pipeline,
19+
20+
## How it works
21+
22+
The `inference/analysis` module make all the passes in a pipeline, and works in such way:
23+
24+
1. Build a `DataFlowGraph` from a Fluid inference ProgramDesc,
25+
2. Call the middle passes one by one, the same `DataFlowGraph` is passed across all the passes,
26+
3. Transform a new ProgramDesc from the modified `DataFlowGraph`.
27+
28+
The new optimization features can be added as an independent `Pass` and controlled by gflags,
29+
each pass will generate unified debug information or visualization for better debugging.
30+
31+
## Supported Passes
32+
33+
### `FluidToDataFlowGraphPass`
34+
Transform the fluid `ProgramDesc` to a `DataFlowGraph` to give an abstract representation for all the middle passes,
35+
this should be the first pass of the pipeline.
36+
37+
### `DataFlowGraphToFluidPass`
38+
Generate a final `ProgramDesc` from a data flow graph, this should be the last pass of the pipeline.
39+
40+
### `TensorRTSubgraphNodeMarkPass`
41+
Mark the `Node` that are supported by TensorRT,
42+
this pass will generate a visualization file which can be used for debugging.
43+
44+
### `TensorRTSubGraphPass`
45+
Split the sub-graph that are can be accelerated by TensorRT.
46+
47+
### `DFG_GraphvizDrawPass`
48+
This pass is just for debug, it will visualize the `DataFlowGraph` using the [graphviz](http://www.graphviz.org) tool.
49+
50+
It can be used as a helper class that draws the modified graph after each pass.
51+
52+
## Utilities
53+
54+
There is some helper function/class for analysis.
55+
56+
- [dot.h](./dot.h) give a easy to use interface for generating `DOT` codes,
57+
- [graph_traits.h](./graph_traits.h) contains the graph traversal algorithms, it uses `iterator` to make the algorithms easy to share across different passes.

0 commit comments

Comments
 (0)