|
| 1 | +## Motivation |
| 2 | + |
| 3 | +There is a ```gap``` between the ```Program``` defined by |
| 4 | +user and the ```Executable``` that can be scheduled |
| 5 | +efficiently on heterogeneous hardware, either locally |
| 6 | +or distributedly. |
| 7 | + |
| 8 | +Usually, the ```gap``` is bridged by |
| 9 | + |
| 10 | +* A serious transformations with defined order. |
| 11 | + |
| 12 | +* The transformations usually invovle |
| 13 | +```insert, delete, clustering, split, dependency analysis```. |
| 14 | + |
| 15 | +* Has a simple way to verify and debug each transformation. |
| 16 | + |
| 17 | +* Flexible to add, remove or customize transformations to fit |
| 18 | +the requirements of various algorithms (models) and hardware secenarios. |
| 19 | + |
| 20 | +Some other events also push us to a better unified pattern. |
| 21 | + |
| 22 | +* The deep learning framework is built around the concepts of graphs. |
| 23 | +To leverage tools such as compilation (e.g. TVM and nGraph) or |
| 24 | +cross-framework conversion (e.g. ONNX), we also need a intermediate |
| 25 | +representation that can be connected to the rest of the ecosystem. |
| 26 | + |
| 27 | + |
| 28 | +We need a unified pattern to naturally support the requirements |
| 29 | +described above. The pattern should fit both training, inference |
| 30 | +and other offline serielized model transformations. |
| 31 | +Learned from LLVM and other deep learning framework, we draft the |
| 32 | +design below. |
| 33 | + |
| 34 | + |
| 35 | +## Design |
| 36 | + |
| 37 | +### Major Concepts |
| 38 | + |
| 39 | +#### Node |
| 40 | + |
| 41 | +```Node``` represents an operation that performs some computation or |
| 42 | +a variable that is input or output of operation. |
| 43 | + |
| 44 | +```Node```s are connected to other ```Node```s via inputs and outputs. |
| 45 | + |
| 46 | +#### Graph |
| 47 | + |
| 48 | +```Graph``` contains a list of ```Node```s. |
| 49 | + |
| 50 | +TODO: Better definitions for the graph. |
| 51 | + |
| 52 | +```Graph``` can also contain ```Attribute```s. ```Attribute```s |
| 53 | +can be ``any`` thing. For example, it can be a list of "wraper" |
| 54 | +nodes. The ```wrapper``` nodes compose ```Node```s and provide |
| 55 | +helper method for execution. ```Attribute``` can also contain |
| 56 | +other things that describe some properties of the ```Graph```. |
| 57 | + |
| 58 | +#### Pass |
| 59 | + |
| 60 | +```Pass``` represents a transformation of ```Graph```. Its input |
| 61 | +is a ```Graph``` and its output is also a ```Graph```. For example, |
| 62 | +a ```Pass``` can simply print out the ```Graph```. A ```Pass``` |
| 63 | +can also fuse some ```Graph```'s ```Node```s. |
| 64 | + |
| 65 | +#### Optimize |
| 66 | + |
| 67 | +```Optimize``` contains a series of ```Pass``` with defined order. |
| 68 | +```Optimize``` transforms a ```Graph``` that only contains raw |
| 69 | +modeling logic to a ```Graph``` that can be run efficiently while |
| 70 | +maintaining the original modeling logic. |
| 71 | + |
| 72 | + |
| 73 | +### Workflow |
| 74 | + |
| 75 | + |
| 76 | +* Program is first converted to Graph. |
| 77 | +* Graph goes through a series of Pass |
| 78 | +* Graph is transformed from raw model logic to a |
| 79 | +form that is efficient to execute. |
| 80 | + |
| 81 | +Graph->Pass1->Graph->Pass2->Graph->Pass3->Executor |
0 commit comments