Skip to content

Commit 1e9f314

Browse files
achiefaachiefa
committed
Upload documentation
1 parent 04156ce commit 1e9f314

File tree

6 files changed

+316
-69
lines changed

6 files changed

+316
-69
lines changed

‎doc/sphinx/source/tutorials/run-qed-fit.rst‎

Lines changed: 217 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -3,32 +3,232 @@
33
==========================
44
How to run a QED fit
55
==========================
6+
This tutorial describes how to run a QED fit using the LuxQED approach, as
7+
described in `arXiv:1607.04266 <https://arxiv.org/abs/1607.04266>`_ and
8+
`arXiv:1708.01256 <https://arxiv.org/abs/1708.01256>`_.
69

7-
It is possible to perform a QED fit adding the key ``fiatlux`` to the runcard. In this way
8-
a photon PDF will be generated using the FiatLux public library that implements the LuxQED
9-
(see :cite:p:`Manohar:2016nzj` and :cite:p:`Manohar:2017eqh`) approach.
10-
The parameters to be added are the following:
10+
Setting up the runcard
11+
----------------------
12+
13+
It is possible to perform a QED fit by adding a ``fiatlux`` block to the
14+
runcard. The following code snippet shows an example of a QED fit configuration:
1115

1216
.. code-block:: yaml
1317
1418
fiatlux:
15-
luxset: NNPDF40_nnlo_as_01180
19+
luxset: 251127-jcm-lh-qed-001
1620
additional_errors: true
1721
luxseed: 1234567890
1822
19-
``luxset`` is the PDF set used to generate the photon PDF with `FiatLux <https://github.com/scarrazza/fiatlux/>`.
20-
The code will generate as many photon replicas as the number of replicas contained in the ``luxset``. Therefore, if the user
21-
tries to generate a replica with ID higher than the maximum ID of the ``luxset``, the code will
22-
raise an error. Moreover, being the LuxQED approach an iterated prcedure, and given that some replicas
23-
do not pass the ``postfit`` selection criteria, the user should make sure that the number of replicas in
24-
the ``luxset`` is high enough such that in the final iteration there will be a number of replicas
25-
higher than the final replicas desired.
26-
``additional_errors`` is a parameter that switches on and off the additional errors of the LuxQED approach,
27-
while ``luxseed`` is the seed used to generate such errors.
28-
This errors should be switched on only in the very last iteration of the procedure.
23+
The parameters contained in the ``fiatlux`` block are:
24+
25+
* ``luxset``
26+
The name of the PDF set used to generate the photon PDF with `FiatLux
27+
<https://github.com/scarrazza/fiatlux/>`_. The code will use as many
28+
photon replicas as the number of replicas contained in the ``luxset``. If
29+
the user tries to generate a replica with ID higher than the maximum ID of
30+
the ``luxset``, the code will start reusing photon replica from the first.
31+
Being the LuxQED approach an iterated procedure, and given that some
32+
replicas do not pass the ``postfit`` selection criteria, the user should
33+
make sure that the number of replicas in the ``luxset`` is high enough
34+
such that in the final iteration there will be a number of replicas higher
35+
than the final replicas desired.
36+
37+
* ``additional_errors``
38+
Boolean flag to switch on and off the additional errors of the LuxQED approach.
39+
40+
.. note::
41+
42+
The ``additional_errors`` flag should be switched on only in the very last
43+
iteration of the procedure.
44+
45+
* ``luxseed``
46+
The seed used to generate the additional errors of the LuxQED as in ``additional_errors``.
47+
48+
The code uses both the ``fiatlux`` block and the ``theoryid`` specified in the
49+
runcard to identify the photon PDF set. As explained below, the code searches
50+
for precomputed photon PDF sets using the pair of ``luxset`` and ``theoryid``
51+
parameters, first locally and then on the NNPDF server (see
52+
:ref:`photon-resources` for details). The photon PDF set will be named
53+
``photon_theoryID_<theoryid>_fit_<luxset>``.
54+
55+
.. _photon-resources:
56+
57+
Running with Photon PDF sets
58+
-----------------------------
59+
60+
The generation of a photon PDF set can add significant overhead to the fitting
61+
procedure. Moreover, the same photon PDF set might be used in different fits. To
62+
minimize the overhead due to the generation of photon PDF sets and avoid
63+
redundant computations, the code looks for precomputed photon resources either
64+
locally or on the NNPDF server. If a desired photon PDF set does not exist in
65+
either of the two locations, it will be computed on the fly and stored locally.
66+
The following sections describe how to use existing photon PDF sets or generate
67+
new ones.
68+
69+
Using available Photon PDF sets
70+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
71+
Before running a QED fit, it is strongly advised to prepare the fit using
72+
``vp-setupfit``, as explained in :ref:`this tutorial <run-n3fit-fit>`. This will
73+
ensure that all the required resources are available, including the photon PDF.
74+
The desired photon PDF is specified by the ``luxset`` parameter in the
75+
``fiatlux`` block and the ``theoryid``, as explained above. The code will first
76+
look for the photon PDF set in the local directory specified in the
77+
:ref:`profile file <nnprofile>`. If the set is not found locally, it will try to
78+
download it from the NNPDF server. The following is an example of running
79+
``vp-setupfit`` using the ``fiatlux`` block shown above:
80+
81+
.. code-block:: bash
82+
83+
$ vp-setupfit qed_example_runcard.yml
84+
[INFO]: Could not find a resource (photonQED): Could not find Photon QED set /user/envs/nnpdf/share/NNPDF/photons_qed/photon_theoryID_702_fit_251127-jcm-lh-qed-001 in theory: 702. Attempting to download it.
85+
[INFO]: Downloading https://data.nnpdf.science/photons/photon_theoryID_702_fit_251127-jcm-lh-qed-001.tar.
86+
[==================================================] (100%)
87+
[INFO]: Extracting archive to /opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/share/NNPDF/photons_qed
88+
[INFO]: Photon QED set found for 702 with luxset 251127-jcm-lh-qed-001.
89+
[WARNING]: Using q2min from runcard
90+
[WARNING]: Using w2min from runcard
91+
Using Keras backend
92+
[INFO]: All requirements processed and checked successfully. Executing actions.
93+
...
94+
95+
This will download and extract the photon PDF set in the local
96+
``photons_qed_path`` specified in the :ref:`profile file <nnprofile>`.
97+
98+
The ``vp-list`` utility tool can be used to list all the available photon PDF
99+
sets locally and on the NNPDF server. To list the available photon PDF sets,
100+
just run:
101+
102+
.. code-block:: bash
103+
104+
$ vp-list photons
105+
[INFO]: The following photons are available locally:
106+
- theoryID_702_fit_251127-jcm-lh-qed-001
107+
[INFO]: The following photons are downloadable:
108+
- theoryID_702_fit_251127-jcm-lh-qed-002
109+
110+
In this example, there is one photon PDF set available locally, and one photon
111+
resource available on the NNPDF server precomputed with theory ID 702 and
112+
``251127-jcm-lh-qed-002`` as input PDF. The user can manually download a photon
113+
PDF set using the ``vp-get`` tool as explained :ref:`here <download>`. For
114+
example:
115+
116+
.. code-block:: bash
117+
118+
$ vp-get photonQED 702 251127-jcm-lh-qed-002
119+
[INFO]: Could not find a resource (photonQED): Could not find Photon QED set /user/envs/nnpdf/share/NNPDF/photons_qed/photon_theoryID_702_fit_251127-jcm-lh-qed-002 in theory: 702. Attempting to download it.
120+
[INFO]: Downloading https://data.nnpdf.science/photons/photon_theoryID_702_fit_251127-jcm-lh-qed-002.tar.
121+
[==================================================] (100%)
122+
[INFO]: Extracting archive to /user/envs/nnpdf/share/NNPDF/photons_qed
123+
PosixPath('/user/envs/nnpdf/share/NNPDF/photons_qed/photon_theoryID_702_fit_251127-jcm-lh-qed-002')
29124
30-
Whenever the photon PDF is generated, it will remain constant during the fit and will be considered in the momentum sum rule.
125+
As in the case of ``vp-setupfit``, this will download and extract the photon PDF
126+
set in the local ``photons_qed_path`` specified in the :ref:`profile file <nnprofile>`.
127+
Once the photon PDF set is available locally, the user can proceed to run the
128+
fit with ``n3fit`` as usual.
31129

32130
.. warning::
33131

34-
At the moment it is not possible to run QED fits in parallel, as the FiatLux library cannot run in parallel.
132+
If ``vp-setupfit`` is not run before ``n3fit``, and the photon PDF set is not
133+
available locally, the code will **not** attempt to download it from the server,
134+
but will directly proceed to compute it on the fly. See the :ref:`next section <generating-photon-sets>` for
135+
more details.
136+
137+
.. _generating-photon-sets:
138+
Generating new Photon PDF sets
139+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
140+
If the desired photon PDF set is not available locally nor on the NNPDF server, the
141+
code will generate the required photon PDF set replicas on the fly using `FiatLux <https://github.com/scarrazza/fiatlux/>`_.
142+
This can be done either during the ``vp-setupfit`` step, which precomputes all photon
143+
replicas before starting the fit, or during the fitting step with ``n3fit``, which
144+
generates the photon replica as needed for each replica being fitted.
145+
146+
.. note::
147+
148+
Generating photon PDF sets on the fly can add significant overhead to the
149+
fitting procedure. It is therefore strongly advised to precompute the photon
150+
PDF set using ``vp-setupfit`` before running the fit with ``n3fit``.
151+
152+
In either case, the generated photon PDF set will be stored locally in the
153+
``photons_qed_path`` specified in the :ref:`profile file <nnprofile>`, so that
154+
it can be reused in future fits. The folder containing the photon replicas will
155+
be named as ``photon_theoryID_<theoryid>_fit_<luxset>`` where ``<theoryid>`` and
156+
``<luxset>`` are the values specified in the runcard. The folder contains a ``npz``
157+
file for each photon replica, named as ``replica_<replica_id>.npz``. Each replica
158+
file contains the photon PDF grid computed with FiatLux at :math:`Q_{\rm init} = 100` GeV,
159+
prior to the evolution through EKO.
160+
161+
.. important::
162+
163+
Automatic upload to the NNPDF server through ``vp-upload`` is **not**
164+
supported at the moment. The user should manually create a ``tar`` archive
165+
file containing the photon replicas and upload it to server. Refer to the
166+
:ref:`profile file <nnprofile>` to find the remote URL where photon PDF sets are stored.
167+
168+
169+
Using ``vp-setupfit`` (preferred)
170+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
171+
172+
In order to trigger the computation of the photon PDF set during ``vp-setupfit``,
173+
the user needs to add the flag ``compute_in_setupfit: true`` in the ``fiatlux`` block
174+
discussed above. The following is an example of running ``vp-setupfit`` with
175+
this flag enabled:
176+
177+
.. code-block:: bash
178+
179+
$ vp-setupfit qed_example_runcard.yml
180+
[INFO]: Could not find a resource (photonQED): Could not find Photon QED set /user/envs/nnpdf/share/NNPDF/photons_qed/photon_theoryID_703_fit_251127-jcm-lh-qed-001 in theory: 703. Attempting to download it.
181+
[ERROR]: Resource not in the remote repository: Photon QED set for TheoryID 703 and luxset 251127-jcm-lh-qed-001 is not available in the remote server.
182+
[WARNING]: Photon QED set for theory 703 with luxset 251127-jcm-lh-qed-001 not found. It will be computed in vp-setupfit.
183+
[INFO]: Forcing photon computation with FiatLux during setupfit.
184+
[WARNING]: Using q2min from runcard
185+
[WARNING]: Using w2min from runcard
186+
Using Keras backend
187+
[INFO]: All requirements processed and checked successfully. Executing actions.
188+
...
189+
190+
In addition, the user can also specify the optional parameter ``eps_base`` to
191+
the ``fiatlux`` block to set the base precision of the FiatLux computation,
192+
which controls the precision on final integration of double integral. By
193+
default, it is set to ``1e-5``. If the ``compute_in_setupfit`` flag is set to
194+
``true`` despite the photon PDF being available locally, the code will
195+
recompute and overwrite the existing photon PDF set
196+
197+
.. tip::
198+
199+
During ``vp-setupfit``, the code tries to distribute the computation of the
200+
photon replicas among the available CPU cores as specified in the
201+
``maxcores`` key of the runcard. This can significantly speed up the
202+
computation of the photon PDF set. Make sure to set ``maxcores`` to a value
203+
compatible with the available hardware. For example, using ``maxcores: 64``
204+
will try to compute up to 64 photon replicas in parallel using 64 GB of RAM.
205+
206+
Once the preparation step is completed, and the photon PDF set is generated and stored
207+
locally, the user can proceed to run the fit with ``n3fit`` as usual.
208+
209+
Using ``n3fit`` (discouraged)
210+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
211+
If the user prefers to compute the photon PDF set during the fitting step with
212+
``n3fit``, no additional flag is needed in the runcard and ``vp-setupfit`` is
213+
not required beforehand (unless other resources are needed such as the
214+
:ref:`theory covariance matrix <vptheorycov-index>`). The code will check for
215+
the presence of the photon PDF set locally before starting the fit for each
216+
replica. If it is not found, it will proceed to compute the photon replica as
217+
needed for each replica being fitted. The following is an example of running
218+
``n3fit`` where the photon PDF set is computed during the fitting step:
219+
220+
.. code-block:: bash
221+
222+
$ n3fit qed_example_runcard.yml 1
223+
[INFO]: Creating replica output folder in /user/runcards/qed_example_runcard/nnfit/replica_1
224+
[WARNING]: Using q2min from runcard
225+
[WARNING]: Using w2min from runcard
226+
Using Keras backend
227+
[WARNING]: No Photon QED set found for Theory 703 with luxset 251127-jcm-lh-qed-001. It will be computed using FiatLux. This may impact performance. It is recommended to precompute the photon set before running the fit. Refer to https://docs.nnpdf.science/tutorials/run-qed-fit.html for more details on precomputing photon PDF sets.
228+
[INFO]: All requirements processed and checked successfully. Executing actions.
229+
230+
.. warning::
231+
232+
Computing the photon PDF set during `n3fit` with multiple replicas or using GPUs
233+
has not been tested and may lead to unexpected behaviour. It is strongly advised to
234+
precompute the photon PDF set using ``vp-setupfit`` before running the fit with ``n3fit``.

‎doc/sphinx/source/vp/download.rst‎

Lines changed: 35 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,16 +6,16 @@ Downloading resources
66
``validphys`` is designed so that, by default, resources stored in known remote
77
locations are downloaded automatically and seamlessly used where necessary.
88
Available resources include PDF sets, completed fits, theories, and results of
9-
past ``validphys`` runs that have been :ref:`uploaded to the server <upload>`.
10-
The ``vp-get`` tool, :ref:`described below <vp-get>`,
9+
past ``validphys`` runs that have been :ref:`uploaded to the server <upload>`.
10+
The ``vp-get`` tool, :ref:`described below <vp-get>`,
1111
can be used to download the same items manually.
1212

1313
Automatic operation
1414
-------------------
1515

1616
By default when some resource such as a PDF is required by ``validphys`` (or
1717
derived tools such as ``vp-setupfit``), the code will first look for it in some
18-
local directory specified in the [profile file](nnprofile). If it is not found
18+
local directory specified in the :ref:`profile file <nnprofile>`. If it is not found
1919
there, it will try to download it from some remote repository (also specified in
2020
the profile).
2121

@@ -70,6 +70,21 @@ Theories
7070
Theories (specified by the ``theoryid`` key) are downloaded and
7171
uncompressed.
7272

73+
Photon PDF sets
74+
Photon PDF sets can be downloaded if they have been previously generated
75+
using `FiatLux <https://github.com/scarrazza/fiatlux/>`_. A photon PDF set
76+
is uniquely identified by the input PDF set used to generate the photon replicas
77+
and the theory ID. See :ref:`this tutorial <run-qed-fit>` for more details on
78+
how to set up a QED fit.
79+
80+
EKOs
81+
Evolution kernels (EKOs) can be downloaded if they have been previously
82+
generated using EKO.
83+
84+
Hyperparameter scans
85+
Results of the hyperparameter scans generated in the hyperoptimization
86+
procedure.
87+
7388
``validphys`` output files
7489
Files produced by ``validphys`` can be used as input to subsequent validphys
7590
analyses (for example χ² tables are used for αs fits). The user needs to
@@ -80,7 +95,7 @@ Theories
8095

8196
.. _vp-get:
8297

83-
The `vp-get` tool
98+
The ``vp-get`` tool
8499
-----------------
85100

86101
The ``vp-get`` tool can be used to download resources manually, in the same way
@@ -99,8 +114,11 @@ They correspond to the resources described :ref:`above <what-can-be-downloaded>`
99114
100115
$ vp-get --list
101116
Available resource types:
117+
- eko
102118
- fit
119+
- hyperscan
103120
- pdf
121+
- photonQED
104122
- theoryID
105123
- vp_output_file
106124
@@ -118,6 +136,15 @@ information on it and bail out:
118136
$ vp-get fit NNPDF31_nlo_as_0118_1000
119137
FitSpec(name='NNPDF31_nlo_as_0118_1000', path=PosixPath('/home/zah/anaconda3/envs/nnpdf-dev/share/NNPDF/results/NNPDF31_nlo_as_0118_1000'))
120138
139+
.. note::
140+
141+
In order to download photon PDF sets, the user needs to specify both the input PDF
142+
set and the theory ID. For example:
143+
144+
.. code:: bash
145+
146+
$ vp-get photonQED 40000000 NNPDF40_nnlo_as_01180
147+
121148
Downloading resources in code (``validphys.loader``)
122149
----------------------------------------------------
123150

@@ -164,18 +191,18 @@ Conversely the ``Loader`` class will only search locally.
164191
----> 1 l.check_dataset('NMC', theoryid=151)
165192
166193
~/nngit/nnpdf/validphys2/src/validphys/loader.py in check_dataset(self, name, rules, sysnum, theoryid, cfac, frac, cuts, use_fitcommondata, fit, weight)
167-
416
194+
416
168195
417 if not isinstance(theoryid, TheoryIDSpec):
169196
--> 418 theoryid = self.check_theoryID(theoryid)
170-
419
197+
419
171198
420 theoryno, _ = theoryid
172199
173200
~/nngit/nnpdf/validphys2/src/validphys/loader.py in check_theoryID(self, theoryID)
174201
288 if not theopath.exists():
175202
289 raise TheoryNotFound(("Could not find theory %s. "
176203
--> 290 "Folder '%s' not found") % (theoryID, theopath) )
177204
291 return TheoryIDSpec(theoryID, theopath)
178-
292
205+
292
179206
180207
TheoryNotFound: Could not find theory 151. Folder '/home/zah/anaconda3/share/NNPDF/data/theory_151' not found
181208
@@ -184,10 +211,9 @@ Output files uploaded to the ``validphys`` can be retrieved specifying their pat
184211
(starting from the report ID). They will be either downloaded (when using
185212
``FallbackLoader``) or retrieved from the cache:
186213

187-
.. code:: python
214+
.. code:: python
188215
189216
from validphys.loader import FallbackLoader as Loader
190217
l = Loader()
191218
l.check_vp_output_file('qTpvLZLwS924oAsmpMzhFw==/figures/f_ns0_fitunderlyinglaw_plot_closure_pdf_histograms_0.pdf')
192219
PosixPath('/home/zah/anaconda3/share/NNPDF/vp-cache/qTpvLZLwS924oAsmpMzhFw==/figures/f_ns0_fitunderlyinglaw_plot_closure_pdf_histograms_0.pdf')
193-

0 commit comments

Comments
 (0)