Skip to content

Commit 79180c3

Browse files
authored
Improve documentation. Organise runtest.sh variables. Update and sync configuration files. (#455)
1 parent 4070176 commit 79180c3

15 files changed

+308
-119
lines changed

README.md

Lines changed: 140 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -4,33 +4,59 @@
44

55
## Introduction
66

7-
The main purpose of the Run 3 validation framework is to provide a compact and flexible tool for validation of the
7+
The Run 3 validation framework is a tool for an easy execution, testing and validation of the Run 3 analysis code on large local samples.
8+
9+
Its features include
10+
11+
* simple specification of input datasets,
12+
* simple configuration and activation of analysis tasks,
13+
* easy generation of the O<sup>2</sup> command for complex workflow topologies,
14+
* job parallelisation,
15+
* output merging,
16+
* error checking and reporting,
17+
* specification of postprocessing.
18+
19+
It also provides tools for:
20+
21+
* post mortem debugging of failing jobs,
22+
* comparison of histograms between ROOT files,
23+
* visualisation of workflow dependencies,
24+
* downloading of data samples from the Grid,
25+
* maintenance of Git repositories and installations of aliBuild packages.
26+
27+
The original purpose of the Run 3 validation framework was to provide a compact and flexible tool for validation of the
828
[O<sup>2</sup>(Physics)](https://github.com/AliceO2Group/O2Physics) analysis framework by comparison of its output to its
929
[AliPhysics](https://github.com/alisw/AliPhysics) counterpart.
1030
The general idea is to run the same analysis using AliPhysics and O<sup>2</sup>(Physics) and produce comparison plots.
1131

32+
However, it can be used without AliPhysics as well to run O<sup>2</sup> analyses locally, similar to running trains on AliHyperloop.
33+
This makes it a convenient framework for local development, testing and debugging of O<sup>2</sup>(Physics) code.
34+
1235
## Overview
1336

1437
The validation framework is a general configurable platform that gives user the full control over what is done.
15-
Its flexibility is enabled by strict separation of its specialised components into a system of bash scripts.
38+
Its flexibility is enabled by strict separation of its specialised components into a system of Bash scripts.
1639
Configuration is separate from execution code, input configuration is separate from task configuration, execution steps are separate from the main steering code.
1740

1841
* The steering script [`runtest.sh`](exec/runtest.sh) provides control parameters and interface to the machinery for task execution.
19-
* User provides configuration bash scripts which:
42+
* User provides configuration Bash scripts which:
2043
* modify control parameters,
2144
* produce modified configuration files,
2245
* generate step scripts executed by the framework in the validation steps.
2346

24-
### Execution
47+
## Execution
2548

2649
Execution code can be found in the [`exec`](exec) directory.
2750

51+
**The user should not touch anything in this directory!**
52+
2853
The steering script [`runtest.sh`](exec/runtest.sh) performs the following execution steps:
54+
2955
* Load input specification.
3056
* Load tasks configuration.
3157
* Print out input description.
3258
* Clean before running. (activated by `DOCLEAN=1`)
33-
* Deletes specified files.
59+
* Deletes specified files (produced by previous runs).
3460
* Generate list of input files.
3561
* Modify the JSON file.
3662
* Convert `AliESDs.root` to `AO2D.root`. (activated by `DOCONVERT=1`)
@@ -51,46 +77,107 @@ The steering script [`runtest.sh`](exec/runtest.sh) performs the following execu
5177
* Executes the postprocessing step script.
5278
* This step typically compares AliPhysics and O<sup>2</sup> output and produces plots.
5379
* Clean after running. (activated by `DOCLEAN=1`)
54-
* Deletes specified files.
80+
* Deletes specified (temporary) files.
5581
* Done
5682
* This step is just a visual confirmation that all steps have finished without errors.
5783

5884
All steps are activated by default and some can be disabled individually by setting the respective activation variables to `0` in user's task configuration.
5985

60-
### Configuration
86+
## Configuration
6187

6288
The steering script [`runtest.sh`](exec/runtest.sh) can be executed with the following optional arguments:
6389

6490
```bash
65-
bash [<path>/]runtest.sh [-h] [-i <input config>] [-t <task config>] [-d]
91+
bash [<path>/]runtest.sh [-h] [-i <input-configuration>] [-t <task-configuration>] [-d]
6692
```
6793

68-
`-h` Prints out the usage specification above.
94+
`<input-configuration>` Input specification script. See [Input specification](#input-specification).
6995

70-
`-d` (Debug mode) Prints out more information about settings and execution.
71-
72-
`<input config>` Input specification
73-
* Bash script that modifies input parameters.
74-
* This script defines which data will be processed.
7596
* Defaults to `config_input.sh` (in the current directory).
7697

77-
`<task config>` Task configuration
78-
* Bash script that cleans the directory, deactivates steps, modifies the JSON file, generates step scripts.
79-
* This script defines what the validation steps will do.
98+
`<task-configuration>` Task configuration script. See [Task configuration](#task-configuration).
99+
80100
* Defaults to `config_tasks.sh` (in the current directory).
81-
* Provides these mandatory functions:
82-
* `Clean` Performs cleanup before and after running.
83-
* `AdjustJson` Modifies the JSON file. (e.g. selection cut activation)
84-
* `MakeScriptAli` Generates the AliPhysics step script.
85-
* `MakeScriptO2` Generates the O<sup>2</sup> step script.
86-
* `MakeScriptPostprocess` Generates the postprocessing step script. (e.g. plotting)
87-
* The `Clean` function takes one argument: `$1=1` before running, `$1=2` after running.
88-
* The AliPhysics and O<sup>2</sup> step scripts take two arguments: `$1="<input file>"`, `$2="<JSON file>"`.
89-
* The postprocessing step script takes two arguments: `$1="<O2 output file>"`, `$2="<AliPhysics output file>"`.
90101

91-
Implementation of these configuration scripts is fully up to the user.
102+
`-d` Debug mode. Prints out more information about settings and execution.
103+
104+
`-h` Help. Prints out the usage specification above.
105+
106+
### Input specification
107+
108+
The input specification script is a Bash script that sets input parameters used by the steering script.
109+
110+
**This script defines which data will be processed and how.**
111+
112+
These are the available input parameters and their default values:
113+
114+
* `INPUT_LABEL="nothing"` Input description
115+
* `INPUT_DIR="$PWD"` Input directory
116+
* `INPUT_FILES="AliESDs.root"` Input file pattern
117+
* `INPUT_SYS="pp"` Collision system (`"pp"`, `"PbPb"`)
118+
* `INPUT_RUN=2` LHC Run (2, 3, 5)
119+
* `INPUT_IS_O2=0` Input files are in O<sup>2</sup> format.
120+
* `INPUT_IS_MC=0` Input files are MC data.
121+
* `INPUT_PARENT_MASK=""` Path replacement mask for the input directory of parent files in case of linked derived O<sup>2</sup> input. Set to `";"` if no replacement needed.
122+
* `JSON="dpl-config.json"` O<sup>2</sup> device configuration
123+
124+
This allows you to define several input datasets and switch between them easily by setting the corresponding value of `INPUT_CASE`.
125+
126+
Other available parameters allow you to specify how many input files to process and how to parallelise the job execution.
127+
128+
### Task configuration
92129

93-
Dummy examples can be found in: [`config/config_input_dummy.sh`](config/config_input_dummy.sh), [`config/config_tasks_dummy.sh`](config/config_tasks_dummy.sh).
130+
The task configuration script is a Bash script that modifies the task parameters used by the steering script.
131+
132+
**This script defines which validation steps will run and what they will do.**
133+
134+
* It cleans the directory, deactivates incompatible steps, modifies the JSON file, generates step scripts.
135+
* The body of the script has to provide these mandatory functions:
136+
* `Clean` Performs cleanup before and after running.
137+
* `AdjustJson` Modifies the JSON file (e.g. selection cut activation).
138+
* `MakeScriptAli` Generates the AliPhysics step script `script_ali.sh`.
139+
* `MakeScriptO2` Generates the O<sup>2</sup> step script `script_o2.sh`.
140+
* `MakeScriptPostprocess` Generates the postprocessing step script `script_postprocess.sh` (e.g. plotting).
141+
* The `Clean` function takes one argument: `$1=1` for cleaning before running, `$1=2` for cleaning after running.
142+
* The AliPhysics and O<sup>2</sup> step scripts take two arguments: `$1="<input-file>"`, `$2="<JSON-file>"`.
143+
* The postprocessing step script takes two arguments: `$1="<O2-output-file>"`, `$2="<AliPhysics-output-file>"`.
144+
145+
Configuration that should be defined in the task configuration includes:
146+
147+
* Deactivation of the validation steps (`DOCLEAN`, `DOCONVERT`, `DOALI`, `DOO2`, `DOPOSTPROCESS`)
148+
* Customisation of the commands loading the AliPhysics, O2Physics and postprocessing environments (`ENV_ALI`, `ENV_O2`, `ENV_POST`). By default the latest builds of AliPhysics, O2Physics and ROOT are used, respectively.
149+
* Any other parameters related to "what should run and how", e.g. `SAVETREES`, `MAKE_GRAPH`, `USEALIEVCUTS`
150+
151+
### Workflow specification
152+
153+
The full O<sup>2</sup> command, executed in the O<sup>2</sup> step script to run the activated O<sup>2</sup> workflows, is generated in the `MakeScriptO2` function using a dedicated Python script [`make_command_o2.py`](exec/make_command_o2.py).
154+
This script generates the command using a **YAML database (`workflows.yml`) that specifies workflow options and how workflows depend on each other**.
155+
You can consider a workflow specification in this database to be the equivalent of a wagon definition on AliHyperloop, including the definition of the wagon name, the workflow name, the dependencies and the derived data. The main difference is that the device configuration is stored in the JSON file.
156+
157+
The workflow database has two sections: `options` and `workflows`.
158+
The `options` section defines `global` options, used once at the end of the command, and `local` options, used for every workflow.
159+
The `workflows` section contains the "wagon" definitions.
160+
The available parameters are:
161+
162+
* `executable` Workflow command, if different from the "wagon" name
163+
* This allows you to define multiple wagons for the same workflow.
164+
* `dependencies` **Direct** dependencies (i.e. other wagons **directly** needed to run this wagon)
165+
* Allowed formats: string, list of strings
166+
* Direct dependencies are wagons that produce tables consumed by this wagon. You can figure them out using the [`find_dependencies.py`](https://github.com/AliceO2Group/O2Physics/blob/master/Scripts/find_dependencies.py) script in O2Physics.
167+
* `requires_mc` Boolean parameter to specify whether the workflow can only run on MC
168+
* `options` Command line options. (Currently not supported on AliHyperloop.)
169+
* Allowed formats: string, list of strings, dictionary with keys `default`, `real`, `mc`
170+
* `tables` Descriptions of output tables to be saved as trees
171+
* Allowed formats: string, list of strings, dictionary with keys `default`, `real`, `mc`
172+
173+
The `make_command_o2.py` script allows you to generate a topology graph to visualise the dependencies defined in the database, using [Graphviz](https://graphviz.org/).
174+
Generation of the topology graph can be conveniently enabled with `MAKE_GRAPH=1` in the task configuration.
175+
176+
Dummy examples of the configuration files can be found in:
177+
178+
* [`config/config_input_dummy.sh`](config/config_input_dummy.sh),
179+
* [`config/config_tasks_dummy.sh`](config/config_tasks_dummy.sh),
180+
* [`config/workflows_dummy.yml`](config/workflows_dummy.yml).
94181

95182
## Preparation
96183

@@ -136,7 +223,7 @@ sudo apt install parallel
136223

137224
Now you are ready to run the validation code.
138225

139-
**Make sure that your bash environment is clean!
226+
**Make sure that your Bash environment is clean!
140227
Do not load ROOT, AliPhysics, O<sup>2</sup>, O<sup>2</sup>Physics or any other aliBuild package environment before running the framework!**
141228

142229
Enter any directory and execute the steering script `runtest.sh`.
@@ -156,12 +243,25 @@ If any step fails, the script will display an error message and you should look
156243

157244
If the main log file of a validation step mentions "parallel: This job failed:", inspect the respective log file in the directory of the corresponding job.
158245

246+
## How to add a new workflow
247+
248+
To add a new workflow in the framework configuration, you need to follow these steps.
249+
250+
* Add the workflow in the [task configuration](#task-configuration):
251+
* Add the activation switch: `DOO2_...=0 # name of the workflow (without o2-analysis)`.
252+
* Add the application of the switch in the `MakeScriptO2` function: `[ $DOO2_... -eq 1 ] && WORKFLOWS+=" o2-analysis-..."`.
253+
* If needed, add lines in the `AdjustJson` function to modify the JSON configuration.
254+
* Add the [workflow specification](#workflow-specification) in the workflow database:
255+
* See the dummy example `o2-analysis-workflow` for the full list of options.
256+
* Add the device configuration in the default JSON file.
257+
159258
## Job debugging
160259

161260
If you run many parallelised jobs and some of them don't finish successfully, you can make use of the debugging script [`debug.sh`](exec/debug.sh) in the [`exec`](exec) directory
162261
which can help you figure out what went wrong, where and why.
163262

164263
You can execute the script from the current working directory using the following syntax (options can be combined):
264+
165265
```bash
166266
bash [<path>/]debug.sh [-h] [-t TYPE] [-b [-u]] [-f] [-w] [-e]
167267
```
@@ -180,10 +280,16 @@ bash [<path>/]debug.sh [-h] [-t TYPE] [-b [-u]] [-f] [-w] [-e]
180280
181281
`-e` Show errors (for all jobs).
182282
183-
## Heavy-flavour analyses
283+
## Specific analyses
284+
285+
### Heavy-flavour analyses
184286
185287
Enter the [`codeHF`](codeHF) directory and see the [`README`](codeHF/README.md).
186288
289+
### Jet analyses
290+
291+
Enter the [`codeJE`](codeJE) directory.
292+
187293
## Keep your repositories and installations up to date and clean
188294
189295
With the ongoing fast development, it can easily happen that updating the O<sup>2</sup>Physics part of the validation
@@ -197,8 +303,9 @@ This includes updating alidist, AliPhysics, O<sup>2</sup>(Physics), and this Run
197303
as well as re-building your AliPhysics and O<sup>2</sup>(Physics) installations via aliBuild and deleting obsolete builds.
198304
199305
You can execute the script from any directory on your system using the following syntax:
306+
200307
```bash
201-
python <path to the Run3Analysisvalidation directory>/exec/update_packages.py [-h] [-d] [-l] [-c] database
308+
python [<path>/]exec/update_packages.py [-h] [-d] [-l] [-c] database
202309
```
203310
204311
optional arguments:
@@ -245,7 +352,7 @@ It is possible to check your code locally (before even committing or pushing):
245352
### Space checker
246353
247354
```bash
248-
bash <path to the Run3Analysisvalidation directory>/exec/check_spaces.sh
355+
bash [<path>/]exec/check_spaces.sh
249356
```
250357
251358
### [ClangFormat](https://clang.llvm.org/docs/ClangFormat.html)
@@ -254,7 +361,7 @@ bash <path to the Run3Analysisvalidation directory>/exec/check_spaces.sh
254361
clang-format -style=file -i <file>
255362
```
256363
257-
### [MegaLinter](https://oxsecurity.github.io/megalinter/latest/mega-linter-runner/)
364+
### [MegaLinter](http://megalinter.io/latest/mega-linter-runner/)
258365
259366
```bash
260367
npx mega-linter-runner

codeHF/README.md

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -55,13 +55,3 @@ The postprocessing step produces several plots `comparison_histos_(...).pdf`, `M
5555
To confirm that the output of the default settings looks as expected, compare the produced plots with their reference counterparts `(...)_ref.pdf`.
5656

5757
The complete list of commit hashes used to produce the reference plots can be found in `versions_ref.txt`.
58-
59-
## Add a new workflow
60-
61-
- Add the workflow in the task configuration ([`config_task.sh`](config_tasks.sh)):
62-
- Add the activation switch: `DOO2_...=0 # name of the workflow (without o2-analysis)`.
63-
- Add the application of the switch in the `MakeScriptO2` function: `[ $DOO2_... -eq 1 ] && WORKFLOWS+=" o2-analysis-..."`.
64-
- If needed, add lines in the `AdjustJson` function to modify the JSON configuration.
65-
- Add the workflow specification in the workflow database ([`workflows.yml`](workflows.yml)):
66-
- See the dummy example `o2-analysis-workflow` for the full list of options.
67-
- Add the device configuration in the default JSON file ([`dpl-config_run3.json`](dpl-config_run3.json)).

codeHF/config_input.sh

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -8,29 +8,30 @@ INPUT_CASE=2 # Input case
88

99
NFILESMAX=1 # Maximum number of processed input files. (Set to -0 to process all; to -N to process all but the last N files.)
1010

11-
# Number of input files per job (Automatic optimisation on if < 1.)
11+
# Number of input files per job. (Will be automatically optimised if set to 0.)
1212
NFILESPERJOB_CONVERT=0 # Conversion
1313
NFILESPERJOB_ALI=0 # AliPhysics
1414
NFILESPERJOB_O2=1 # O2
1515

16-
# Maximum number of simultaneously running O2 jobs
16+
# Maximum number of simultaneously running O2 jobs. (Adjust it based on available memory.)
1717
NJOBSPARALLEL_O2=$(python3 -c "print(min(10, round($(nproc) / 2)))")
1818

19-
JSONRUN3="dpl-config_run3.json" # Run 3 tasks parameters
20-
# Run 5 tasks parameters for open HF study
21-
JSONRUN5_HF="dpl-config_run5_hf.json"
22-
# Run 5 tasks parameters for onia studies:
23-
# J/psi and X (higher pt cut on 2-prong decay tracks and no DCA cut on single track)
24-
JSONRUN5_ONIAX="dpl-config_run5_oniaX.json"
25-
JSON="$JSONRUN3"
26-
2719
# Default settings:
28-
# INPUT_FILES="AliESDs.root"
29-
# INPUT_SYS="pp"
30-
# INPUT_RUN=2
31-
# INPUT_IS_O2=0
32-
# INPUT_IS_MC=0
33-
# JSON="$JSONRUN3"
20+
# INPUT_LABEL="nothing" # Input description
21+
# INPUT_DIR="$PWD" # Input directory
22+
# INPUT_FILES="AliESDs.root" # Input file pattern
23+
# INPUT_SYS="pp" # Collision system ("pp", "PbPb")
24+
# INPUT_RUN=2 # LHC Run (2, 3, 5)
25+
# INPUT_IS_O2=0 # Input files are in O2 format.
26+
# INPUT_IS_MC=0 # Input files are MC data.
27+
# INPUT_PARENT_MASK="" # Path replacement mask for the input directory of parent files in case of linked derived O2 input. Set to ";" if no replacement needed.
28+
# JSON="dpl-config.json" # O2 device configuration
29+
30+
# O2 device configuration
31+
JSONRUN3="dpl-config_run3.json" # Run 3
32+
# JSONRUN5_HF="dpl-config_run5_hf.json" # Run 5, open HF
33+
# JSONRUN5_ONIAX="dpl-config_run5_oniaX.json" # Run 5, onia (J/psi and X), (higher pt cut on 2-prong decay tracks and no DCA cut on single track)
34+
JSON="$JSONRUN3"
3435

3536
INPUT_BASE="/data2/data" # alicecerno2
3637

codeHF/config_tasks.sh

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313

1414
####################################################################################################
1515

16-
# Here you can select the AliPhysics and O2Physics branches to load.
16+
# Here you can select the AliPhysics and O2Physics Git branches to load. (You need to have them built with aliBuild.)
1717
# BRANCH_ALI="master"
1818
# ENV_ALI="alienv setenv AliPhysics/latest-${BRANCH_ALI}-o2 -c"
1919
# BRANCH_O2="master"
@@ -29,9 +29,8 @@ DOPOSTPROCESS=1 # Run output postprocessing. (Comparison plots. Requires DOA
2929
# Disable incompatible steps.
3030
[ "$INPUT_IS_O2" -eq 1 ] && { DOCONVERT=0; DOALI=0; }
3131

32-
# O2 database
33-
DATABASE_O2="workflows.yml"
34-
MAKE_GRAPH=0 # Make topology graph.
32+
DATABASE_O2="workflows.yml" # Workflow specification database
33+
MAKE_GRAPH=0 # Make topology graph.
3534

3635
# Activation of O2 workflows
3736
# Trigger selection

codeHF/dpl-config_run3.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
"start-value-enumeration": "0",
77
"end-value-enumeration": "-1",
88
"step-value-enumeration": "1",
9+
"aod-file-private": "@list_o2.txt",
910
"aod-file": "@list_o2.txt",
1011
"aod-parent-base-path-replacement": "PARENT_PATH_MASK",
1112
"aod-parent-access-level": 1

codeHF/workflows.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ workflows:
2020
# default: ""
2121
# real: ""
2222
# mc: "--doMC"
23-
tables: [] # descriptions of tables to be saved in the output tree (format: str, list), see more detailed format below
23+
tables: [] # descriptions of output tables to be saved as trees (format: str, list), see more detailed format below
2424
# tables:
2525
# default: []
2626
# real: []

0 commit comments

Comments
 (0)