|
| 1 | +# Inference Analysis |
| 2 | + |
| 3 | +The `inference/analysis` module is used to analyze and optimize the inference program, |
| 4 | +it references some philosophy from `LLVM/analysis`, |
| 5 | +and make the various optimization features be pluggable and co-exist in a pipeline. |
| 6 | + |
| 7 | +We borrowed some concepts from LLVM, such as |
| 8 | + |
| 9 | +- [Pass](./pass.h)es to implement optimization that traverse the inference program, |
| 10 | +- [DataFlowGraph](./data_flow_graph.h) to represent the data flow graph built from a program, |
| 11 | +- [PassManager](./pass_manager.h) to manage a sequence of `Pass`es over a graph. |
| 12 | + |
| 13 | +There are some other basic concepts here |
| 14 | + |
| 15 | +- [Node](./node.h), the node in a `DataFlowGraph`, |
| 16 | + - `Function`, the Operator in Fluid, |
| 17 | + - `Value`, the Variable in Fluid; |
| 18 | +- [Argument](./argument.h), the argument that treat as the input and output of all `Pass`es in the pipeline, |
| 19 | + |
| 20 | +## How it works |
| 21 | + |
| 22 | +The `inference/analysis` module make all the passes in a pipeline, and works in such way: |
| 23 | + |
| 24 | +1. Build a `DataFlowGraph` from a Fluid inference ProgramDesc, |
| 25 | +2. Call the middle passes one by one, the same `DataFlowGraph` is passed across all the passes, |
| 26 | +3. Transform a new ProgramDesc from the modified `DataFlowGraph`. |
| 27 | + |
| 28 | +The new optimization features can be added as an independent `Pass` and controlled by gflags, |
| 29 | +each pass will generate unified debug information or visualization for better debugging. |
| 30 | + |
| 31 | +## Supported Passes |
| 32 | + |
| 33 | +### `FluidToDataFlowGraphPass` |
| 34 | +Transform the fluid `ProgramDesc` to a `DataFlowGraph` to give an abstract representation for all the middle passes, |
| 35 | +this should be the first pass of the pipeline. |
| 36 | + |
| 37 | +### `DataFlowGraphToFluidPass` |
| 38 | +Generate a final `ProgramDesc` from a data flow graph, this should be the last pass of the pipeline. |
| 39 | + |
| 40 | +### `TensorRTSubgraphNodeMarkPass` |
| 41 | +Mark the `Node` that are supported by TensorRT, |
| 42 | +this pass will generate a visualization file which can be used for debugging. |
| 43 | + |
| 44 | +### `TensorRTSubGraphPass` |
| 45 | +Split the sub-graph that are can be accelerated by TensorRT. |
| 46 | + |
| 47 | +### `DFG_GraphvizDrawPass` |
| 48 | +This pass is just for debug, it will visualize the `DataFlowGraph` using the [graphviz](http://www.graphviz.org) tool. |
| 49 | + |
| 50 | +It can be used as a helper class that draws the modified graph after each pass. |
| 51 | + |
| 52 | +## Utilities |
| 53 | + |
| 54 | +There is some helper function/class for analysis. |
| 55 | + |
| 56 | +- [dot.h](./dot.h) give a easy to use interface for generating `DOT` codes, |
| 57 | +- [graph_traits.h](./graph_traits.h) contains the graph traversal algorithms, it uses `iterator` to make the algorithms easy to share across different passes. |
0 commit comments