|
1 | 1 | # Compiling |
2 | 2 |
|
3 | | -The term "compiling" for ngcsimlib refers to automatic step that happens inside of a context that produces a transformed method for all of its components. This step is the most complicated part of the library and, in general, does not need to be touched or interacted with. Nevertheless, this section will cover all of the steps that the ngcsimlib compilation process does at a high level. This section contains advanced technical/developer-level information: there is an expectation that the reader has an understanding of Python abstract syntax trees (ASTs), globals (global variables), and how to dynamically compile Python code to run/execute. |
| 3 | +The term "compiling" for NGC-Sim-Lib refers to automatic step that happens |
| 4 | +inside of a context that produces a transformed method for all of its |
| 5 | +components. This step is the most complicated part of the library and, in |
| 6 | +general, does not need to be touched or interacted with. Nevertheless, this |
| 7 | +section will cover most of the steps that the NGC-Sim-Lib compilation process |
| 8 | +does at a high level. This section contains advanced technical/developer-level |
| 9 | +information: there is an expectation that the reader has an understanding of |
| 10 | +Python abstract syntax trees (ASTs), Python namespaces, and how to |
| 11 | +dynamically compile Python code and execute it. |
4 | 12 |
|
5 | 13 | ## The Decorator |
6 | 14 |
|
7 | | -In ngcsimlib, there is a decorator marked as `@compilable` which is used to add a flag to methods that the user wants to compile. On its own,this will not do anything; however, this decorator lets the parser distinguish between methods that should be compiled and methods that should be ignored (by the overall back-end compilation process). |
| 15 | +In NGC-Sim-Lib, there is a decorator marked as `@compilable` which is used to |
| 16 | +add a flag to methods that the user wants to compile. On its own, this will not |
| 17 | +do anything; however, this decorator lets the parser distinguish between methods |
| 18 | +that should be compiled and methods that should be ignored. |
8 | 19 |
|
9 | 20 | ## The Step-by-Step NGC-Sim-Lib Parsing Process |
10 | 21 |
|
11 | 22 | The process starts by telling the parser to compile a specific object. |
12 | 23 |
|
13 | 24 | ### Step 1: Compile Children |
14 | 25 |
|
15 | | -The first step to compile any object is to make sure that all of the "compilable" objects of the top level object are essentially compiled. As a result, ngcsimlib will loop through all of the whole objects and will compile each part that it finds that is flagged as compilable (via the decorators mentioned above) and is, furthermore, an instance of a class. |
| 26 | +The first step to compile any object is to make sure that all of the |
| 27 | +"compilable" objects of the top level object are compiled. As a |
| 28 | +result, NGC-Sim-Lib will loop through all of the whole object and will compile |
| 29 | +each part that it finds that is flagged as compilable (via the decorators |
| 30 | +mentioned above) and is, furthermore, an instance of a class. |
16 | 31 |
|
17 | 32 | ### Step 2: Extract Methods to Compile |
18 | 33 |
|
19 | | -While the parser is looping through all of the parts of the top-level object, it is also extracting the methods on/embedded to the object that are flagged as compilable (with the decorator above). ngcsimlib stores them for later; however, this lets the parser know to only loop over the object once. |
| 34 | +While the parser is looping through all of the parts of the top-level object, it |
| 35 | +is also extracting the methods on/embedded to the object that are flagged as |
| 36 | +compilable (with the decorator above). NGC-Sim-Lib stores them for later; |
| 37 | +however, this lets the parser only loop over the object once. |
20 | 38 |
|
21 | 39 | ### Step 3: Parse Each Method |
22 | 40 |
|
23 | | -As each method is its own entry-point into the compiler, this step will run for each method in the top-level object. |
| 41 | +As each method is its own entry-point into the transformer, this step will run |
| 42 | +for each method in the top-level object. |
24 | 43 |
|
25 | 44 | ### Step 3a: Set up a Transformer |
26 | 45 |
|
27 | | -This step sets up a ContextTransformer, which further makes use of a NodeTransformer, and will convert methods from class methods (with the use of `self`), as well as other methods that need to be removed / ignored, into their more context-friendly counterparts. |
| 46 | +This step sets up a `ContextTransformer`, which further makes use of a |
| 47 | +`ast.NodeTransformer`, and will convert methods from class methods (with the use |
| 48 | +of `self`), as well as other methods that need to be removed / ignored, into |
| 49 | +their more context-friendly counterparts. |
28 | 50 |
|
29 | 51 | ### Step 3b: Transform the Function |
30 | 52 |
|
31 | | -There are quite a few pieces of common Python that need to be transformed. This step happens with the overall goal of replacing all object-focused parts with a more global view. This means that a compartment's `.get` and `.set` calls are replaced with direct setting and getting from the global state, based on the compartment's target. This also means that all temporally constant values -- such as `batch_size` -- are moved into the globals space for that specific file and ultimately replaced with the naming convention of `object_path_constant`. One more key step that is performed is to ensure that there is no branching in the code. Specifically, if there is a branch, i.e., an if-statement, ngcsimlib will evaluate it and only keep the branch it will traverse down. This means that there cannot be any branch logic based on inputs or computed values (this is a common restriction for just-in-time compiling). |
| 53 | +There are quite a few pieces of common Python that need to be transformed. This |
| 54 | +step happens with the overall goal of replacing all object-focused parts with a |
| 55 | +more global view. This means that a compartment's `.get` and `.set` calls are |
| 56 | +replaced with direct setting and getting from the global state, based on the |
| 57 | +compartment's target. This also means that all temporally constant values -- |
| 58 | +such as `batch_size` -- are moved into the globals space for that specific file |
| 59 | +and ultimately replaced with the naming convention of `object_path_constant`. |
| 60 | +One more key step that is performed is to ensure that there is no branching in |
| 61 | +the code. Specifically, if there is a branch, i.e., an if-statement, NGC-Sim-Lib |
| 62 | +will evaluate it and only keep the branch it will traverse down. This means that |
| 63 | +there cannot be any branch logic based on inputs or computed values (this is a |
| 64 | +common restriction for just-in-time compiling). |
32 | 65 |
|
33 | 66 | ### Step 3c: Parse Sub-Methods |
34 | 67 |
|
35 | | -Since it is possible to have other class methods that are not marked as entry-points for compilation but still need to be compiled, as step 3b happens, ngcsimlib tracks all of the sub-methods required. Notably, this step goes through and repeats steps 3a and 3b for each of the (sub-)methods with a naming convention similar to the temporally constant values for each method. |
| 68 | +Since it is possible to have other class methods that are not marked as |
| 69 | +entry-points for compilation but still need to be compiled, as step 3b happens, |
| 70 | +NGC-Sim-Lib tracks all of the sub-methods required. Notably, this step goes |
| 71 | +through and repeats steps 3a and 3b for each of the (sub-)methods with a naming |
| 72 | +convention similar to the temporally constant values for each method. |
36 | 73 |
|
37 | 74 | ### Step 3d: Compile the Abstract Syntax Tree (AST) |
38 | 75 |
|
39 | | -Once we have all of the namespace and globals needed to execute the properly-transformed method, the method is compiled with Python and finally executed. |
| 76 | +Once we have all of the namespace and globals needed to execute the |
| 77 | +properly-transformed method, the method is compiled with Python and finally |
| 78 | +executed. |
40 | 79 |
|
41 | 80 | ### Step 3e: Binding |
42 | 81 |
|
43 | | -The final step per method is to bind each to their original method; this replaces each method with an object which, when called, will act like the normal, uncompiled version but has the addition of the `.compiled` attribute. This attribute contains all of the compiled information to be used later (for model / system simulation). This crucially allows for the end user to call `myComponent.myMethod.compiled()` and have it run. The exact type for a `compiled` value can be found in `ngcsimlib._src.parser.utils:CompiledMethod`. |
| 82 | +The final step per method is to bind each to their original method; this |
| 83 | +replaces each method with an object which, when called, will act like the |
| 84 | +normal, uncompiled version but has the addition of the `.compiled` attribute. |
| 85 | +This attribute contains all of the compiled information to be used later (for |
| 86 | +model / system simulation). This crucially allows for the end user to |
| 87 | +call `myComponent.myMethod.compiled()` and have it run. The exact type for |
| 88 | +a `compiled` value can be found |
| 89 | +in `ngcsimlib._src.parser.utils:CompiledMethod`. |
44 | 90 |
|
45 | 91 | ### Step 4: Finishing Up / Final Processing |
46 | 92 |
|
47 | | -Some objects, such as the processes, entail additional steps to modify themselves or their compiled methods in order to align themselves with needed functionality. However, this operation/functionality is found within each class's expanded `compile` method and should be referred to by looking at those methods specifically. |
| 93 | +Some objects, such as the processes, entail additional steps to modify |
| 94 | +themselves or their compiled methods in order to align themselves with needed |
| 95 | +functionality. However, this operation/functionality is found within each |
| 96 | +class's expanded `compile` method and should be referred to by looking at those |
| 97 | +methods specifically. |
48 | 98 |
|
0 commit comments