Skip to content

Commit 10d6c2f

Browse files
authored
Merge pull request #65 from scipp/improve-docs
docs: add description of data reduction procedure
2 parents 7239cfe + 4a76898 commit 10d6c2f

File tree

16 files changed

+298
-75
lines changed

16 files changed

+298
-75
lines changed

bibliography.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Bibliography
2+
============
3+
4+
.. bibliography::

docs/about/bibliography.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Bibliography
2+
============
3+
4+
.. bibliography::

docs/about/index.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,12 @@ Simply download the archive, unzip and view locally in a web browser.
2424
## Source code and development
2525

2626
ESSreflectometry is hosted and developed [on GitHub](https://github.com/scipp/essreflectometry).
27+
28+
29+
```{toctree}
30+
---
31+
hidden:
32+
---
33+
34+
bibliography
35+
```

docs/bibliography.bib

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
@article{STAHN201644,
2+
title = {Focusing neutron reflectometry: Implementation and experience on the TOF-reflectometer Amor},
3+
journal = {Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment},
4+
volume = {821},
5+
pages = {44-54},
6+
year = {2016},
7+
issn = {0168-9002},
8+
doi = {https://doi.org/10.1016/j.nima.2016.03.007},
9+
url = {https://www.sciencedirect.com/science/article/pii/S0168900216300250},
10+
author = {J. Stahn and A. Glavic},
11+
keywords = {Reflectometry, Neutron, Focusing},
12+
abstract = {Neutron reflectometry is a powerful tool to investigate chemical and magnetic depth profiles near surfaces. The advantages of neutrons compared to x-rays are their sensitivity to isotopes, the high penetration capabilities and the high sensitivity to magnetic induction. The biggest disadvantage however is the low flux available, which leads to much longer counting times on much larger samples. In order to boost the performance of neutron reflectometers, a focusing guide system was developed and realised over recent years. Here we report on the application and performance of a down-scaled demonstrator of such a Selene guide, installed as an add-on on the time-of-flight (TOF) reflectometer Amor at the PSI. Due to the limited size of the guide, the flux is concentrated to a footprint of at most 2mm width. It is thus possible to avoid illumination of contacts even on small samples. Despite the fact that typical samples measured on Amor with a size of 10×10mm2 are markedly under illuminated, the presented set-up leads to a reduction in counting time of 80%. The use of the demonstrator thus allows for in-situ or in-operando investigations with a time resolution of a few minutes for a qz range from 0.005Å−1 to 0.08Å−1. Besides a short recapitulation of the concept of focusing reflectometry, a detailed description of the data reduction and its quality is given, followed by an application example.}
13+
}

docs/conf.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
'sphinx_autodoc_typehints',
3333
'sphinx_copybutton',
3434
'sphinx_design',
35+
"sphinxcontrib.bibtex",
3536
'nbsphinx',
3637
'myst_parser',
3738
]
@@ -274,3 +275,8 @@ def do_not_plot(*args, **kwargs):
274275
nbsphinx_execute_arguments = [
275276
"--Session.metadata=scipp_sphinx_build=True",
276277
]
278+
279+
280+
# -- Options for bibtex ----------------------------------------------------
281+
bibtex_bibfiles = ["bibliography.bib"]
282+
bibtex_reference_style = "label"
Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# Focused reflectometry data reduction
2+
3+
The goal of the reflectometry data reduction is to compute the reflectivity $R(Q)$ of the sample as a function of the momentum transfer $Q$.
4+
5+
Based on {cite}`STAHN201644`.
6+
7+
8+
## Target audience
9+
10+
The target audience of this text is anyone who wants to know the reasoning behind the data reduction workflow implemented for the focusing reflectometry instruments ESTIA at ESS and AMOR at PSI.
11+
It is not necessary to read this to be able to use the workflow, for that we recommend the tutorial notebooks as they will give a much more practically useful introduction, but if you are going to develop new features it is probably good to at least skim it.
12+
13+
14+
## Preliminaries
15+
16+
The detector data is a list $EV$ of detected neutron events.
17+
For each event in the list we know its wavelength $\lambda$ and the pixel number $j$ of the detector pixel that it hit.
18+
The detector pixel positions are known and so is the position and the orientation of the sample.
19+
From this information we can compute the reflection angle $\theta$, and the momentum transfer $Q$ caused by the interaction with the sample.
20+
21+
The purpose of this text is not to describe how the event coordinates wavelength, $Q$ and $\theta$ are derived from the raw detector data and the instrument geometry, so for now just take those for given.
22+
For more details see the implementations for the respective instruments [Amor] and [Estia].
23+
24+
To simplify the description it is assumed that the sample- and reference measurements were made over the same length of time, and it is assumed the brightness of the source did not change between the measurements.
25+
26+
27+
## Model of event intensity in the detector
28+
29+
The reflectivity of the sample is related to the intensity of neutron counts in the detector by the model
30+
31+
$$
32+
I_{\text{sam}}(\lambda, j) = F(\theta(\lambda, j, \mu_{\text{sam}}), w_{\text{sam}}) \cdot R(Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}}))) \cdot I_{\text{ideal}}(\lambda, j)
33+
$$ (model)
34+
where $I_{\text{sam}}(\lambda, j)$ represents the expected number of neutrons detected in the $j$ pixel of the detector per unit of wavelength at the wavelength value $\lambda$. $I_{\text{ideal}}$ represents the expected number of neutrons detected if the sample was a perfect reflector and large enough so that the footprint of the focused beam on the sample was small compared to the sample. $F(\theta, w)$ is the fraction of the beam that hits the sample. It depends on the incidence angle $\theta$ and on the size of the sample represented by $w,$ and $\mu_{\text{sam}}$ is the sample rotation.
35+
36+
The model does not hold for any $\lambda,j$. For example, there might be a region in the detector where we can see part of the direct beam. To make this explicit, let $M_{sam}$ represent the region of $\lambda,j$ where the model is expected to hold (the "region of interest").
37+
38+
The ideal intensity $I_{ideal}$ will be estimated from a reference measurement on a neutron supermirror.
39+
How that is done will be described in more detail later, for now assume it is a known quantity.
40+
41+
42+
## Estimating $R(Q)$
43+
Move $F$ to the left-hand-side of equation {eq}`model` and integrate over all $\lambda, j\in M$ contributing to the $Q$-bin $[q_{i}, q_{i+1}]$
44+
45+
$$
46+
\int_{M \cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} \frac{I_{\text{sam}}(\lambda, j)}{F(\theta(\lambda, j, \mu_{\text{sam}}), w_{\text{sam}})} d\lambda \ dj = \\
47+
\int_{M \cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I_{\text{ideal}}(\lambda, j) R(Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}}))) d\lambda \ dj.
48+
$$
49+
Notice that if the $Q$ binning is sufficiently fine then $R(Q)$ is approximately constant in the integration region.
50+
Assuming the binning is fine enough $R(Q)$ can be moved outside the integral and isolated so that
51+
52+
$$
53+
R(Q_{i+\frac{1}{2}}) \approx \frac{\int_{M \cap Q(\lambda, j, \mu_{\text{sam}}) \in [q_{i}, q_{i+1}]} \frac{I_{\text{sam}}(\lambda, j)}{F(\theta(\lambda, j, \mu_{\text{sam}}), w_{\text{sam}})} d\lambda \ dj }{\int_{M \cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I_{\text{ideal}}(\lambda, j) d\lambda \ dj} =: \frac{I_{measured}(Q_{i+\frac{1}{2}})}{I_{\text{ideal}}(Q_{i+\frac{1}{2}})}
54+
$$ (reflectivity)
55+
for $Q_{i+\frac{1}{2}} \in [q_{i}, q_{i+1}]$.
56+
57+
For the integral to make sense the region of interest $M$ has to be contained in the region where {eq}`model` holds, $M\subset M_{sam}$, but there might be other constraints limiting the region of interest $M$ even more, so it is left undefined for now.
58+
59+
60+
## The reference intensity $I_{\text{ideal}}$
61+
$I_{\text{ideal}}$ is estimated from a reference measurement on a neutron supermirror with known reflectivity curve.
62+
The reference measurement intensity $I_{\text{ref}}$ is modeled the same way the sample measurement was
63+
64+
$$
65+
I_{\text{ref}}(\lambda, j) = F(\theta(\lambda, j, \mu_{\text{ref}}), w_{\text{ref}}) \cdot R_{\text{supermirror}}(Q(\lambda, \theta(\lambda, j, \mu_{\text{ref}}))) \cdot I_{\text{ideal}}(\lambda, j)
66+
$$
67+
but in this case $R_{\text{supermirror}}(Q)$ is known.
68+
69+
As before, the model does not hold for any $\lambda,j$. Let $M_{ref}$ represent the region of $\lambda,j$ where the model is expected to hold. $M_{ref}$ is typically not the same as $M_{sam}$. For example, if the supermirror reflectivity is not known for all $Q$ we are not going to have a model for the reference intensity at the $\lambda,j$ corresponding to those $Q$ and that is reflected in $M_{ref}$ but not in $M_{sam}$.
70+
71+
Using the definition in {eq}`reflectivity`
72+
73+
$$
74+
I_{\text{ideal}}(Q_{i+\frac{1}{2}}) = \int_{M \cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} \frac{I_{\text{ref}}(\lambda, j)}{F(\theta(\lambda, j, \mu_{\text{ref}}), w_{\text{ref}}) R_{\text{supermirror}}(Q(\lambda, \theta(\lambda, j, \mu_{\text{ref}})))}
75+
d\lambda \ dj.
76+
$$
77+
For this integral to make sense $M\subset M_{ref}$, so now we have an additional constraint on $M$ to keep in mind.
78+
79+
80+
## Estimating intensities from detector counts
81+
82+
The number of neutron counts in the detector is a Poisson process where the expected number of neutrons per pixel and unit of wavelength are the measurement intensities $I_{sam}$ and $I_{ref}$ defined above.
83+
The expected intensity can be estimated by the measured intensity:
84+
85+
$$
86+
I_{measured}(Q_{i+\frac{1}{2}}) = \int_{M\cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} \frac{I_{\text{sam}}(\lambda, j)}{F(\theta(\lambda, j, \mu_{\text{sam}}), w_{\text{sam}})} d\lambda \ dj \\
87+
\approx \sum_{\substack{k \in EV_{\text{sam}} \\ Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]\\ (\lambda_k, j_k)\in M}} \frac{1}{F(\theta(\lambda_{k}, j_{k}, \mu_{\text{sam}}), w_{\text{sam}})}
88+
$$
89+
where $EV_{\text{sam}}$ refers to the event list from the sample experiment.
90+
91+
The sum is compound Poisson distributed and for such random variables the variance can be estimated by the sum of squared summands
92+
93+
$$
94+
V\bigg[ \sum_{\substack{k \in EV_{\text{sam}} \\ Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]\\ (\lambda_k, j_k)\in M}} \frac{1}{F(\theta(\lambda_{k}, j_{k}, \mu_{\text{sam}}), w_{\text{sam}})} \bigg] \approx
95+
\sum_{\substack{k \in EV_{\text{sam}} \\ Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]\\ (\lambda_k, j_k)\in M}} \frac{1}{F(\theta(\lambda_{k}, j_{k}, \mu_{\text{sam}}), w_{\text{sam}})^2}.
96+
$$
97+
98+
The same estimates are used to approximate the ideal intensity:
99+
100+
$$
101+
I_{\text{ideal}}(Q_{i+\frac{1}{2}}) \approx \sum_{\substack{k \in EV_{\text{ref}} \\ Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]\\ (\lambda_k, j_k)\in M}} \frac{1}{F(\theta(\lambda_{k}, j_{k}, \mu_{\text{ref}}), w_{\text{ref}}) R_{\text{supermirror}}(Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{ref}})))}
102+
$$
103+
The above expressions and {eq}`reflectivity` lets us express $R(Q_{i+{\frac{1}{2}}})$ and its uncertainty in terms of the neutron counts in the detector.
104+
105+
106+
## More efficient evaluation of the reference intensity
107+
The above expression for the reference intensity is cumbersome to compute because it is a sum over the reference measurement event list, and the reference measurement is large compared to the sample measurement.
108+
109+
Therefore we back up a bit. Consider the expression for the reference intensity, replacing the integrand with a generic $I(\lambda, j)$ it looks something like:
110+
111+
$$
112+
I_{\text{ideal}}(Q_{i+\frac{1}{2}}) = \int_{Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I(\lambda, j) \ d\lambda \ dj.
113+
$$
114+
115+
In the previous section we approximated the integral by summing over all events in the reference measurement.
116+
117+
Alternatively, we could define a $\lambda$ grid with edges $\lambda_{k}$ for $k=1\ldots N$ and approximate the integration region as the union of a subset of the grid cells:
118+
119+
$$
120+
\int_{Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I(\lambda, j) \ d\lambda \ dj
121+
\approx \sum_{Q(\bar{\lambda}_{k+\frac{1}{2}},\ \theta(\bar{\lambda}_{k+\frac{1}{2}}, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I_{k+\frac{1}{2},j}
122+
$$
123+
where
124+
125+
$$
126+
I_{k+\frac{1}{2},j} = \int_{\lambda \in [\lambda_{k}, \lambda_{k+1}]} I(\lambda, j) \ d\lambda
127+
$$
128+
and $\bar{\lambda}_{k+\frac{1}{2}} = (\lambda_{k} + \lambda_{k+1}) / 2$.
129+
130+
Why would this be more efficient than the original approach? Note that $I_{k+\frac{1}{2}, j}$ does not depend on $\mu_{\text{sam}}$, and that it can be computed once and reused for all sample measurements.
131+
This allows us to save computing time for each new sample measurement, as long as $|EV_{\text{ref}}| >> NM$ where $M$ is the number of detector pixels and $N$ is the size of the $\lambda$ grid.
132+
133+
However, ideally computing the reference intensity should be quick compared to reducing the sample measurement. And since a reasonable value for $N$ is approximately $500$, and $M\approx 30000$, and a sample measurement is likely less than $10$ million events, the cost of computing the reference measurement is still considerable compared to reducing the sample measurement.
134+
135+
Therefore there's one more approximation that is used to further reduce the cost of computing the reference intensity.
136+
The description of the final approximation is instrument specific.
137+
In the next section it is described specifically for the Amor instrument.
138+
139+
140+
### Evaluating the reference intensity for the Amor instrument
141+
142+
The Amor detector has three logical dimensions, `blade`, `wire` and `strip`. It happens to be the case that $\theta(\lambda, j)$ is almost the same for all $j$ belonging to the same `strip` of the detector.
143+
We can express this as
144+
145+
$$
146+
\theta(\lambda, j, \mu_{\text{sam}}) \approx \bar{\theta}(\lambda, \mathrm{bladewire}(j), \mu_{\text{sam}})
147+
$$
148+
where $\bar{\theta}$ is an approximation for $\theta$ that only depends on the blade and the wire of the pixel where the neutron was detected.
149+
Then the above expression for the reference intensity can be rewritten as
150+
151+
$$
152+
\int_{Q(\lambda, \bar{\theta}(\lambda, z, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} \int_{\mathrm{bladewire}(j) = z} I(\lambda, j) \ dj \ d\lambda \ dz
153+
\approx \sum_{Q(\bar{\lambda}_{k+\frac{1}{2}}, \bar{\theta}(\bar{\lambda}_{k+\frac{1}{2}}, z, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I_{k+\frac{1}{2},z}
154+
$$
155+
where
156+
157+
$$
158+
I_{k+\frac{1}{2},z} = \int_{\lambda \in [\lambda_{k}, \lambda_{k+1}]} \int_{\mathrm{bladewire}(j) = z} I(\lambda, j) \ dj \ d\lambda .
159+
$$
160+
Like before, the benefit of doing this is that
161+
162+
$$
163+
\int_{\lambda \in [\lambda_{k}, \lambda_{k+1}]} \int_{\mathrm{bladewire}(j) = z} I(\lambda, j) \ dj \ d\lambda
164+
$$
165+
can be pre-computed because it doesn't depend on $\mu_{\text{sam}}$.
166+
But unlike before $I_{k+\frac{1}{2},z}$ now has a much more manageable size, about 64x smaller than the first attempt.
167+
This makes it comfortably smaller than the sample measurement.
168+

docs/user-guide/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,5 @@ installation
88
amor/index
99
estia/index
1010
offspec/index
11+
focused-reflectometry-data-reduction
1112
```

requirements/base.txt

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
#
88
annotated-types==0.7.0
99
# via pydantic
10-
click==8.2.0
10+
click==8.2.1
1111
# via dask
1212
cloudpickle==3.1.1
1313
# via dask
@@ -17,21 +17,21 @@ cyclebane==24.10.0
1717
# via sciline
1818
cycler==0.12.1
1919
# via matplotlib
20-
dask==2025.5.0
20+
dask==2025.7.0
2121
# via -r base.in
2222
dnspython==2.7.0
2323
# via email-validator
2424
email-validator==2.2.0
2525
# via scippneutron
26-
essreduce==25.5.1
26+
essreduce==25.7.1
2727
# via -r base.in
28-
fonttools==4.58.0
28+
fonttools==4.59.0
2929
# via matplotlib
30-
fsspec==2025.5.0
30+
fsspec==2025.7.0
3131
# via dask
32-
graphviz==0.20.3
32+
graphviz==0.21
3333
# via -r base.in
34-
h5py==3.13.0
34+
h5py==3.14.0
3535
# via
3636
# scippneutron
3737
# scippnexus
@@ -51,7 +51,7 @@ matplotlib==3.10.3
5151
# via
5252
# mpltoolbox
5353
# plopp
54-
mpltoolbox==25.4.0
54+
mpltoolbox==25.5.0
5555
# via scippneutron
5656
networkx==3.4.2
5757
# via cyclebane
@@ -64,24 +64,24 @@ numpy==2.2.6
6464
# scipp
6565
# scippneutron
6666
# scipy
67-
orsopy==1.2.1
67+
orsopy==1.2.2
6868
# via -r base.in
6969
packaging==25.0
7070
# via
7171
# dask
7272
# lazy-loader
7373
# matplotlib
74-
pandas==2.2.3
74+
pandas==2.3.1
7575
# via -r base.in
7676
partd==1.4.2
7777
# via dask
78-
pillow==11.2.1
78+
pillow==11.3.0
7979
# via matplotlib
80-
plopp==25.5.0
80+
plopp==25.7.0
8181
# via
8282
# -r base.in
8383
# scippneutron
84-
pydantic==2.11.4
84+
pydantic==2.11.7
8585
# via scippneutron
8686
pydantic-core==2.33.2
8787
# via pydantic
@@ -100,21 +100,21 @@ pyyaml==6.0.2
100100
# via
101101
# dask
102102
# orsopy
103-
sciline==25.5.1
103+
sciline==25.5.2
104104
# via
105105
# -r base.in
106106
# essreduce
107-
scipp==25.5.0
107+
scipp==25.5.1
108108
# via
109109
# -r base.in
110110
# essreduce
111111
# scippneutron
112112
# scippnexus
113-
scippneutron==25.5.0
113+
scippneutron==25.7.0
114114
# via
115115
# -r base.in
116116
# essreduce
117-
scippnexus==25.4.0
117+
scippnexus==25.6.0
118118
# via
119119
# -r base.in
120120
# essreduce
@@ -129,15 +129,15 @@ toolz==1.0.0
129129
# via
130130
# dask
131131
# partd
132-
typing-extensions==4.13.2
132+
typing-extensions==4.14.1
133133
# via
134134
# pydantic
135135
# pydantic-core
136136
# sciline
137137
# typing-inspection
138-
typing-inspection==0.4.0
138+
typing-inspection==0.4.1
139139
# via pydantic
140140
tzdata==2025.2
141141
# via pandas
142-
zipp==3.21.0
142+
zipp==3.23.0
143143
# via importlib-metadata

requirements/basetest.txt

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
#
66
# requirements upgrade
77
#
8-
certifi==2025.4.26
8+
certifi==2025.7.14
99
# via requests
1010
charset-normalizer==3.4.2
1111
# via requests
@@ -25,13 +25,15 @@ pluggy==1.6.0
2525
# via pytest
2626
pooch==1.8.2
2727
# via -r basetest.in
28-
pytest==8.3.5
28+
pygments==2.19.2
29+
# via pytest
30+
pytest==8.4.1
2931
# via -r basetest.in
30-
requests==2.32.3
32+
requests==2.32.4
3133
# via pooch
3234
tomli==2.2.1
3335
# via pytest
34-
typing-extensions==4.13.2
36+
typing-extensions==4.14.1
3537
# via exceptiongroup
36-
urllib3==2.4.0
38+
urllib3==2.5.0
3739
# via requests

0 commit comments

Comments
 (0)