|
1 | | -# MLC Script Meta |
| 1 | +# Script Meta |
2 | 2 |
|
3 | 3 | This page provides a walkthrough of the `meta.yaml` file. |
4 | 4 |
|
5 | 5 | ## Important Keys and Data Types |
6 | 6 |
|
7 | 7 | - **alias** (`string`) |
| 8 | + |
8 | 9 | - Alias of the script, which can be used instead of tags when running a script. |
9 | 10 |
|
10 | 11 | - **uid** (`string`) |
| 12 | + |
11 | 13 | - Unique identifier for individual scripts. |
| 14 | + |
12 | 15 | - Can be used instead of tags when running a script. |
13 | 16 |
|
14 | 17 | - **automation_alias** (`string`) |
| 18 | + |
15 | 19 | - Alias specific to script automation. |
16 | 20 |
|
17 | 21 | - **automation_uid** (`string`) |
| 22 | + |
18 | 23 | - Unique identifier for script automation. |
19 | 24 |
|
20 | 25 | - **category** (`string`) |
| 26 | + |
21 | 27 | - Defines the script category. |
22 | 28 |
|
23 | 29 | - **tags** (`list of strings`) |
| 30 | + |
24 | 31 | - List of tags users can specify to run the script. |
25 | 32 |
|
26 | 33 | - **default_env** (`dictionary` with `string` values) |
27 | | - - Contains key-value pairs representing environment variables and their default values for a script. |
| 34 | + |
| 35 | + - Contains key-value pairs representing environment variables and their default values for a script. |
| 36 | + |
28 | 37 | - These default values are overridden if the same environment variable is set in script files or inherited from a parent script. |
29 | 38 |
|
30 | 39 | - **env** (`dictionary` with `string` values) |
| 40 | + |
31 | 41 | - Defines environment variables and their corresponding values. |
32 | 42 |
|
33 | 43 | - **input_mapping** (`dictionary` with `string` values) |
| 44 | + |
34 | 45 | - Maps input flags related to a script to corresponding environment variables. |
| 46 | + |
35 | 47 | - Only keys specified under `input_mapping` in `meta.yaml` are mapped to environment variables. |
36 | 48 |
|
37 | 49 | - **env_key_mapping** (`dictionary` with `string` values) |
| 50 | + |
38 | 51 | - Maps one environment key to another. |
39 | 52 |
|
40 | 53 | - **new_env_keys** (`list of strings`) |
| 54 | + |
41 | 55 | - Specifies environment keys that should be passed to a parent script if this script is used as a dependency. |
42 | 56 |
|
43 | 57 | - **new_state_keys** (`list of strings`) |
| 58 | + |
44 | 59 | - Specifies state keys that should be passed to a parent script when used as a dependency. |
45 | 60 |
|
46 | 61 | - **add_deps_recursive** (`dictionary`) |
| 62 | + |
47 | 63 | - Customizes recursive dependencies with nested `tags` and other attributes. |
48 | 64 |
|
49 | 65 | ### Dependencies |
50 | 66 |
|
51 | 67 | Dependencies in a script are specified as a list of dictionaries. |
| 68 | + |
52 | 69 | - Each dictionary can contain: |
| 70 | + |
53 | 71 | - **tags** |
| 72 | + |
54 | 73 | - Comma separated list of tags to identify a dependent script |
| 74 | + |
55 | 75 | - **names** (`list of strings`) |
| 76 | + |
56 | 77 | - Identifiers for dependencies, allowing users to modify the deps executions when needed. |
| 78 | + |
57 | 79 | - **enable_if_env** (`dictionary` with `list of strings` values) |
| 80 | + |
58 | 81 | - Specifies conditions under which a dependency is enabled based on environment variables. The conditions are ANDed. |
| 82 | + |
59 | 83 | - **enable_if_any_env** (`dictionary` with `list of strings` values) |
| 84 | + |
60 | 85 | - Specifies conditions under which a dependency is enabled based on environment variables. The conditions are ORed. |
| 86 | + |
61 | 87 | - **skip_if_env** (`dictionary` with `list of strings` values) |
| 88 | + |
62 | 89 | - Specifies conditions under which a dependency should be skipped. The conditions are ANDed. |
| 90 | + |
63 | 91 | - **skip_if_any_env** (`dictionary` with `list of strings` values) |
| 92 | + |
64 | 93 | - Specifies conditions under which a dependency should be skipped. The conditions are ORed. |
| 94 | + |
65 | 95 | - Any other script meta can also be included which will be passed to the dependent script |
66 | 96 |
|
67 | 97 | In MLC script, there are four types of dependencies. |
| 98 | + |
68 | 99 | - **deps** |
| 100 | + |
69 | 101 | - Executes after processing the script meta, but before the `preprocess` function |
| 102 | + |
70 | 103 | - **prehook_deps** |
| 104 | + |
71 | 105 | - Executes after the `preprocess` function, but before the `run` script |
| 106 | + |
72 | 107 | - **posthook_deps** |
| 108 | + |
73 | 109 | - Executes after the `run` script, but before the `postprocess` function |
| 110 | + |
74 | 111 | - **post_deps** |
| 112 | + |
75 | 113 | - Executes after the `postprocess` function |
76 | | - - More details on script execution flow can be found [here](execution-flow.md). |
| 114 | + |
| 115 | + More details on script execution flow can be found [here](execution-flow.md). |
77 | 116 |
|
78 | 117 |
|
79 | 118 | ### Variation Configuration |
80 | 119 |
|
81 | 120 | - **variations** (`dictionary`) |
| 121 | + |
82 | 122 | - Contains script variations, including attributes like `alias`, `default_variations`, and `group`. |
83 | 123 |
|
84 | 124 | - **group** (`string`) |
| 125 | + |
85 | 126 | - Specifies the script variation group. Maximum one variation from a given group can be selected during a script invocation and if `default:true` is set for a variation in a group, that variation is turned on unless another variation is specified during the script invocation. |
86 | 127 |
|
87 | 128 | - **default_variations** (`dictionary` with `string` values) |
| 129 | + |
88 | 130 | - Specifies default variations for the script. |
89 | 131 |
|
90 | 132 |
|
91 | 133 |
|
92 | 134 | ### Docker Configuration |
93 | 135 |
|
94 | 136 | - **docker** (`dictionary`) |
| 137 | + |
95 | 138 | - Contains Docker-specific configurations: |
| 139 | + |
96 | 140 | - **base_image** (`string`) – Base Docker image |
| 141 | + |
97 | 142 | - **image_name** (`string`) – Docker image name |
| 143 | + |
98 | 144 | - **os** (`string`) – Operating system |
| 145 | + |
99 | 146 | - **os_version** (`string`) – OS version |
| 147 | + |
100 | 148 | - **deps** (`list of dictionaries`) – Specifies dependencies required in Docker |
| 149 | + |
101 | 150 | - **env** (`dictionary` with `string` values) – Defines environment variables inside the container |
| 151 | + |
102 | 152 | - **interactive** (`boolean`) – Indicates if the container should be interactive |
| 153 | + |
103 | 154 | - **extra_run_args** (`string`) – Additional arguments for `docker run` |
| 155 | + |
104 | 156 | - **mounts** (`list of strings`) – Specifies mount paths in the format `"source:destination"` |
| 157 | + |
105 | 158 | - **pre_run_cmds** (`list of strings`) – Commands to execute before starting the container |
| 159 | + |
106 | 160 | - **docker_input_mapping** (`dictionary` with `string` values) – Maps input parameters to Docker environment variables |
| 161 | + |
107 | 162 | - **use_host_user_id** (`boolean`) – Uses the host's user ID inside the container |
| 163 | + |
108 | 164 | - **use_host_group_id** (`boolean`) – Uses the host's group ID inside the container |
| 165 | + |
109 | 166 | - **skip_run_cmd** (`string`) – Command to skip execution |
| 167 | + |
110 | 168 | - **shm_size** (`string`) – Defines shared memory size |
111 | | - - **real_run** (`boolean`) – Indicates if the container run should be real |
112 | | - - **all_gpus** (`string`) – Specifies GPU availability |
| 169 | + |
| 170 | + - **real_run** (`boolean`) – Indicates if the container should do the MLC script run at end, if not only the deps will be run |
| 171 | + |
| 172 | + - **all_gpus** (`string`) – Use all available GPUs |
113 | 173 |
|
114 | 174 | --- |
115 | 175 |
|
0 commit comments