Validation

Creating the validation data, histograms and pdf maps

Set appropriate options in default.yml and database_parameters_DNN_fluctuations.yml.

In default.yml:

docreatevaldata: whether to produce a root file with full validation data described on the Validation Data Format page; the result file is needed for other options below to work
docreatepdfmaps: whether to create ND histograms (as *.gzip files) and pdf maps (*.root) for all validation data (currently: mean maps with id 0, 9, 18)
docreatepdfmapforvariable: whether to create ND histograms and pdf maps for the data specified in database_parameters_DNN_fluctuations.yml
domergepdfmaps: whether to merge pdf maps for different mean maps and factors into one file

In database_parameters_DNN_fluctuations.yml:

diroutflattree: where to save validation data and pdf maps
dirouthistograms: where to save validation histograms
pdf_map_var, pdf_map_mean_id: ND histograms and pdf maps will be created for the data with this map factor and mean map id when docreatepdfmapforvariable is set in default.yml

Plotting and browsing pdf maps

The easiest way to examine the result files is to use the interactive Jupyter notebook available here.

Alternatively, one can manually draw plots with ROOT, from *.root pdf maps.

Running the notebook in aliceml via ssh

Enter the notebooks directory:

cd notebooks/

Launch Jupyter without browser (you will later browse on your local machine). It will print an URL with a token, copy and store this for the next step.

python -m notebook --no-browser --port=<aliceml_port_number> # e.g. --port=8887

On your local machine, tunnel the localhost to the notebook port:

ssh -N -L localhost:8888:localhost:<aliceml_port_number> <your_name>@aliceml

You can browse the notebooks at the URL returned to you by Jupyter, just change the port number to 8888.

Examining a pdf map with Jupyter

Once you have Jupyter running according to the instructions above, you should see the notebooks/ directory contents in the browser. Enter model_performance_evaluation.ipynb. You need to adjust the directory variables on the top. Then, follow the rest of code in the notebook.