Skip to content

Commit d668610

Browse files
committed
Merge branch 'provider-for-qbins' of github.com:scipp/essreflectometry into provider-for-qbins
2 parents 2fe9f5f + 8238a9e commit d668610

23 files changed

+376
-105
lines changed

.copier-answers.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# Changes here will be overwritten by Copier; NEVER EDIT MANUALLY
2-
_commit: 3f79959
2+
_commit: 3561fcd
33
_src_path: gh:scipp/copier_template
44
description: Reflectometry data reduction for the European Spallation Source
55
max_python: '3.13'
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
name: Nightly test using lower bound dependencies
2+
3+
on:
4+
workflow_dispatch:
5+
schedule:
6+
- cron: '30 1 * * 1-5'
7+
8+
jobs:
9+
setup:
10+
name: Setup variables
11+
runs-on: 'ubuntu-24.04'
12+
outputs:
13+
min_python: ${{ steps.vars.outputs.min_python }}
14+
steps:
15+
- uses: actions/checkout@v4
16+
- name: Get Python version for other CI jobs
17+
id: vars
18+
run: echo "min_python=$(< .github/workflows/python-version-ci)" >> "$GITHUB_OUTPUT"
19+
20+
tests:
21+
name: Tests at lower bound
22+
needs: setup
23+
strategy:
24+
matrix:
25+
os: ['ubuntu-24.04']
26+
python:
27+
- version: '${{needs.setup.outputs.min_python}}'
28+
runs-on: ${{ matrix.os }}
29+
env:
30+
ESS_PROTECTED_FILESTORE_USERNAME: ${{ secrets.ESS_PROTECTED_FILESTORE_USERNAME }}
31+
ESS_PROTECTED_FILESTORE_PASSWORD: ${{ secrets.ESS_PROTECTED_FILESTORE_PASSWORD }}
32+
steps:
33+
- uses: actions/checkout@v4
34+
- uses: astral-sh/setup-uv@v6
35+
with:
36+
python-version: ${{ matrix.python.version }}
37+
- run: uv run --extra=test --resolution=lowest-direct pytest

.pre-commit-config.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ repos:
3636
- id: codespell
3737
additional_dependencies:
3838
- tomli
39+
exclude_types:
40+
- svg
3941
- repo: https://github.com/pre-commit/pygrep-hooks
4042
rev: v1.10.0
4143
hooks:

bibliography.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Bibliography
2+
============
3+
4+
.. bibliography::

docs/about/bibliography.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Bibliography
2+
============
3+
4+
.. bibliography::

docs/about/index.md

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
## Development
44

5-
ESSreflectometry is an open source project by the [European Spallation Source ERIC](https://europeanspallationsource.se/) (ESS).
5+
ESSreflectometry is an open source project by the [European Spallation Source ERIC](https://ess.eu/) (ESS).
66

77
## License
88

9-
ESSreflectometry is available as open source under the [BSD-3 license](https://opensource.org/licenses/BSD-3-Clause).
9+
ESSreflectometry is available as open source under the [BSD-3 license](https://opensource.org/license/BSD-3-Clause).
1010

1111
## Citing ESSreflectometry
1212

@@ -24,3 +24,12 @@ Simply download the archive, unzip and view locally in a web browser.
2424
## Source code and development
2525

2626
ESSreflectometry is hosted and developed [on GitHub](https://github.com/scipp/essreflectometry).
27+
28+
29+
```{toctree}
30+
---
31+
hidden:
32+
---
33+
34+
bibliography
35+
```

docs/bibliography.bib

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
@article{STAHN201644,
2+
title = {Focusing neutron reflectometry: Implementation and experience on the TOF-reflectometer Amor},
3+
journal = {Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment},
4+
volume = {821},
5+
pages = {44-54},
6+
year = {2016},
7+
issn = {0168-9002},
8+
doi = {https://doi.org/10.1016/j.nima.2016.03.007},
9+
url = {https://www.sciencedirect.com/science/article/pii/S0168900216300250},
10+
author = {J. Stahn and A. Glavic},
11+
keywords = {Reflectometry, Neutron, Focusing},
12+
abstract = {Neutron reflectometry is a powerful tool to investigate chemical and magnetic depth profiles near surfaces. The advantages of neutrons compared to x-rays are their sensitivity to isotopes, the high penetration capabilities and the high sensitivity to magnetic induction. The biggest disadvantage however is the low flux available, which leads to much longer counting times on much larger samples. In order to boost the performance of neutron reflectometers, a focusing guide system was developed and realised over recent years. Here we report on the application and performance of a down-scaled demonstrator of such a Selene guide, installed as an add-on on the time-of-flight (TOF) reflectometer Amor at the PSI. Due to the limited size of the guide, the flux is concentrated to a footprint of at most 2mm width. It is thus possible to avoid illumination of contacts even on small samples. Despite the fact that typical samples measured on Amor with a size of 10×10mm2 are markedly under illuminated, the presented set-up leads to a reduction in counting time of 80%. The use of the demonstrator thus allows for in-situ or in-operando investigations with a time resolution of a few minutes for a qz range from 0.005Å−1 to 0.08Å−1. Besides a short recapitulation of the concept of focusing reflectometry, a detailed description of the data reduction and its quality is given, followed by an application example.}
13+
}

docs/conf.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@
3232
'sphinx_autodoc_typehints',
3333
'sphinx_copybutton',
3434
'sphinx_design',
35+
'sphinxcontrib.bibtex',
36+
'sphinxcontrib.autodoc_pydantic',
3537
'nbsphinx',
3638
'myst_parser',
3739
]
@@ -274,3 +276,8 @@ def do_not_plot(*args, **kwargs):
274276
nbsphinx_execute_arguments = [
275277
"--Session.metadata=scipp_sphinx_build=True",
276278
]
279+
280+
281+
# -- Options for bibtex ----------------------------------------------------
282+
bibtex_bibfiles = ["bibliography.bib"]
283+
bibtex_reference_style = "label"
Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# Focused reflectometry data reduction
2+
3+
The goal of the reflectometry data reduction is to compute the reflectivity $R(Q)$ of the sample as a function of the momentum transfer $Q$.
4+
5+
Based on {cite}`STAHN201644`.
6+
7+
8+
## Target audience
9+
10+
The target audience of this text is anyone who wants to know the reasoning behind the data reduction workflow implemented for the focusing reflectometry instruments ESTIA at ESS and AMOR at PSI.
11+
It is not necessary to read this to be able to use the workflow, for that we recommend the tutorial notebooks as they will give a much more practically useful introduction, but if you are going to develop new features it is probably good to at least skim it.
12+
13+
14+
## Preliminaries
15+
16+
The detector data is a list $EV$ of detected neutron events.
17+
For each event in the list we know its wavelength $\lambda$ and the pixel number $j$ of the detector pixel that it hit.
18+
The detector pixel positions are known and so is the position and the orientation of the sample.
19+
From this information we can compute the reflection angle $\theta$, and the momentum transfer $Q$ caused by the interaction with the sample.
20+
21+
The purpose of this text is not to describe how the event coordinates wavelength, $Q$ and $\theta$ are derived from the raw detector data and the instrument geometry, so for now just take those for given.
22+
For more details see the implementations for the respective instruments [Amor] and [Estia].
23+
24+
To simplify the description it is assumed that the sample- and reference measurements were made over the same length of time, and it is assumed the brightness of the source did not change between the measurements.
25+
26+
27+
## Model of event intensity in the detector
28+
29+
The reflectivity of the sample is related to the intensity of neutron counts in the detector by the model
30+
31+
$$
32+
I_{\text{sam}}(\lambda, j) = F(\theta(\lambda, j, \mu_{\text{sam}}), w_{\text{sam}}) \cdot R(Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}}))) \cdot I_{\text{ideal}}(\lambda, j)
33+
$$ (model)
34+
where $I_{\text{sam}}(\lambda, j)$ represents the expected number of neutrons detected in the $j$ pixel of the detector per unit of wavelength at the wavelength value $\lambda$. $I_{\text{ideal}}$ represents the expected number of neutrons detected if the sample was a perfect reflector and large enough so that the footprint of the focused beam on the sample was small compared to the sample. $F(\theta, w)$ is the fraction of the beam that hits the sample. It depends on the incidence angle $\theta$ and on the size of the sample represented by $w,$ and $\mu_{\text{sam}}$ is the sample rotation.
35+
36+
The model does not hold for any $\lambda,j$. For example, there might be a region in the detector where we can see part of the direct beam. To make this explicit, let $M_{sam}$ represent the region of $\lambda,j$ where the model is expected to hold (the "region of interest").
37+
38+
The ideal intensity $I_{ideal}$ will be estimated from a reference measurement on a neutron supermirror.
39+
How that is done will be described in more detail later, for now assume it is a known quantity.
40+
41+
42+
## Estimating $R(Q)$
43+
Move $F$ to the left-hand-side of equation {eq}`model` and integrate over all $\lambda, j\in M$ contributing to the $Q$-bin $[q_{i}, q_{i+1}]$
44+
45+
$$
46+
\int_{M \cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} \frac{I_{\text{sam}}(\lambda, j)}{F(\theta(\lambda, j, \mu_{\text{sam}}), w_{\text{sam}})} d\lambda \ dj = \\
47+
\int_{M \cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I_{\text{ideal}}(\lambda, j) R(Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}}))) d\lambda \ dj.
48+
$$
49+
Notice that if the $Q$ binning is sufficiently fine then $R(Q)$ is approximately constant in the integration region.
50+
Assuming the binning is fine enough $R(Q)$ can be moved outside the integral and isolated so that
51+
52+
$$
53+
R(Q_{i+\frac{1}{2}}) \approx \frac{\int_{M \cap Q(\lambda, j, \mu_{\text{sam}}) \in [q_{i}, q_{i+1}]} \frac{I_{\text{sam}}(\lambda, j)}{F(\theta(\lambda, j, \mu_{\text{sam}}), w_{\text{sam}})} d\lambda \ dj }{\int_{M \cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I_{\text{ideal}}(\lambda, j) d\lambda \ dj} =: \frac{I_{measured}(Q_{i+\frac{1}{2}})}{I_{\text{ideal}}(Q_{i+\frac{1}{2}})}
54+
$$ (reflectivity)
55+
for $Q_{i+\frac{1}{2}} \in [q_{i}, q_{i+1}]$.
56+
57+
For the integral to make sense the region of interest $M$ has to be contained in the region where {eq}`model` holds, $M\subset M_{sam}$, but there might be other constraints limiting the region of interest $M$ even more, so it is left undefined for now.
58+
59+
60+
## The reference intensity $I_{\text{ideal}}$
61+
$I_{\text{ideal}}$ is estimated from a reference measurement on a neutron supermirror with known reflectivity curve.
62+
The reference measurement intensity $I_{\text{ref}}$ is modeled the same way the sample measurement was
63+
64+
$$
65+
I_{\text{ref}}(\lambda, j) = F(\theta(\lambda, j, \mu_{\text{ref}}), w_{\text{ref}}) \cdot R_{\text{supermirror}}(Q(\lambda, \theta(\lambda, j, \mu_{\text{ref}}))) \cdot I_{\text{ideal}}(\lambda, j)
66+
$$
67+
but in this case $R_{\text{supermirror}}(Q)$ is known.
68+
69+
As before, the model does not hold for any $\lambda,j$. Let $M_{ref}$ represent the region of $\lambda,j$ where the model is expected to hold. $M_{ref}$ is typically not the same as $M_{sam}$. For example, if the supermirror reflectivity is not known for all $Q$ we are not going to have a model for the reference intensity at the $\lambda,j$ corresponding to those $Q$ and that is reflected in $M_{ref}$ but not in $M_{sam}$.
70+
71+
Using the definition in {eq}`reflectivity`
72+
73+
$$
74+
I_{\text{ideal}}(Q_{i+\frac{1}{2}}) = \int_{M \cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} \frac{I_{\text{ref}}(\lambda, j)}{F(\theta(\lambda, j, \mu_{\text{ref}}), w_{\text{ref}}) R_{\text{supermirror}}(Q(\lambda, \theta(\lambda, j, \mu_{\text{ref}})))}
75+
d\lambda \ dj.
76+
$$
77+
For this integral to make sense $M\subset M_{ref}$, so now we have an additional constraint on $M$ to keep in mind.
78+
79+
80+
## Estimating intensities from detector counts
81+
82+
The number of neutron counts in the detector is a Poisson process where the expected number of neutrons per pixel and unit of wavelength are the measurement intensities $I_{sam}$ and $I_{ref}$ defined above.
83+
The expected intensity can be estimated by the measured intensity:
84+
85+
$$
86+
I_{measured}(Q_{i+\frac{1}{2}}) = \int_{M\cap Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} \frac{I_{\text{sam}}(\lambda, j)}{F(\theta(\lambda, j, \mu_{\text{sam}}), w_{\text{sam}})} d\lambda \ dj \\
87+
\approx \sum_{\substack{k \in EV_{\text{sam}} \\ Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]\\ (\lambda_k, j_k)\in M}} \frac{1}{F(\theta(\lambda_{k}, j_{k}, \mu_{\text{sam}}), w_{\text{sam}})}
88+
$$
89+
where $EV_{\text{sam}}$ refers to the event list from the sample experiment.
90+
91+
The sum is compound Poisson distributed and for such random variables the variance can be estimated by the sum of squared summands
92+
93+
$$
94+
V\bigg[ \sum_{\substack{k \in EV_{\text{sam}} \\ Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]\\ (\lambda_k, j_k)\in M}} \frac{1}{F(\theta(\lambda_{k}, j_{k}, \mu_{\text{sam}}), w_{\text{sam}})} \bigg] \approx
95+
\sum_{\substack{k \in EV_{\text{sam}} \\ Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]\\ (\lambda_k, j_k)\in M}} \frac{1}{F(\theta(\lambda_{k}, j_{k}, \mu_{\text{sam}}), w_{\text{sam}})^2}.
96+
$$
97+
98+
The same estimates are used to approximate the ideal intensity:
99+
100+
$$
101+
I_{\text{ideal}}(Q_{i+\frac{1}{2}}) \approx \sum_{\substack{k \in EV_{\text{ref}} \\ Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]\\ (\lambda_k, j_k)\in M}} \frac{1}{F(\theta(\lambda_{k}, j_{k}, \mu_{\text{ref}}), w_{\text{ref}}) R_{\text{supermirror}}(Q(\lambda_{k}, \theta(\lambda_{k}, j_{k}, \mu_{\text{ref}})))}
102+
$$
103+
The above expressions and {eq}`reflectivity` lets us express $R(Q_{i+{\frac{1}{2}}})$ and its uncertainty in terms of the neutron counts in the detector.
104+
105+
106+
## More efficient evaluation of the reference intensity
107+
The above expression for the reference intensity is cumbersome to compute because it is a sum over the reference measurement event list, and the reference measurement is large compared to the sample measurement.
108+
109+
Therefore we back up a bit. Consider the expression for the reference intensity, replacing the integrand with a generic $I(\lambda, j)$ it looks something like:
110+
111+
$$
112+
I_{\text{ideal}}(Q_{i+\frac{1}{2}}) = \int_{Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I(\lambda, j) \ d\lambda \ dj.
113+
$$
114+
115+
In the previous section we approximated the integral by summing over all events in the reference measurement.
116+
117+
Alternatively, we could define a $\lambda$ grid with edges $\lambda_{k}$ for $k=1\ldots N$ and approximate the integration region as the union of a subset of the grid cells:
118+
119+
$$
120+
\int_{Q(\lambda, \theta(\lambda, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I(\lambda, j) \ d\lambda \ dj
121+
\approx \sum_{Q(\bar{\lambda}_{k+\frac{1}{2}},\ \theta(\bar{\lambda}_{k+\frac{1}{2}}, j, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I_{k+\frac{1}{2},j}
122+
$$
123+
where
124+
125+
$$
126+
I_{k+\frac{1}{2},j} = \int_{\lambda \in [\lambda_{k}, \lambda_{k+1}]} I(\lambda, j) \ d\lambda
127+
$$
128+
and $\bar{\lambda}_{k+\frac{1}{2}} = (\lambda_{k} + \lambda_{k+1}) / 2$.
129+
130+
Why would this be more efficient than the original approach? Note that $I_{k+\frac{1}{2}, j}$ does not depend on $\mu_{\text{sam}}$, and that it can be computed once and reused for all sample measurements.
131+
This allows us to save computing time for each new sample measurement, as long as $|EV_{\text{ref}}| >> NM$ where $M$ is the number of detector pixels and $N$ is the size of the $\lambda$ grid.
132+
133+
However, ideally computing the reference intensity should be quick compared to reducing the sample measurement. And since a reasonable value for $N$ is approximately $500$, and $M\approx 30000$, and a sample measurement is likely less than $10$ million events, the cost of computing the reference measurement is still considerable compared to reducing the sample measurement.
134+
135+
Therefore there's one more approximation that is used to further reduce the cost of computing the reference intensity.
136+
The description of the final approximation is instrument specific.
137+
In the next section it is described specifically for the Amor instrument.
138+
139+
140+
### Evaluating the reference intensity for the Amor instrument
141+
142+
The Amor detector has three logical dimensions, `blade`, `wire` and `strip`. It happens to be the case that $\theta(\lambda, j)$ is almost the same for all $j$ belonging to the same `strip` of the detector.
143+
We can express this as
144+
145+
$$
146+
\theta(\lambda, j, \mu_{\text{sam}}) \approx \bar{\theta}(\lambda, \mathrm{bladewire}(j), \mu_{\text{sam}})
147+
$$
148+
where $\bar{\theta}$ is an approximation for $\theta$ that only depends on the blade and the wire of the pixel where the neutron was detected.
149+
Then the above expression for the reference intensity can be rewritten as
150+
151+
$$
152+
\int_{Q(\lambda, \bar{\theta}(\lambda, z, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} \int_{\mathrm{bladewire}(j) = z} I(\lambda, j) \ dj \ d\lambda \ dz
153+
\approx \sum_{Q(\bar{\lambda}_{k+\frac{1}{2}}, \bar{\theta}(\bar{\lambda}_{k+\frac{1}{2}}, z, \mu_{\text{sam}})) \in [q_{i}, q_{i+1}]} I_{k+\frac{1}{2},z}
154+
$$
155+
where
156+
157+
$$
158+
I_{k+\frac{1}{2},z} = \int_{\lambda \in [\lambda_{k}, \lambda_{k+1}]} \int_{\mathrm{bladewire}(j) = z} I(\lambda, j) \ dj \ d\lambda .
159+
$$
160+
Like before, the benefit of doing this is that
161+
162+
$$
163+
\int_{\lambda \in [\lambda_{k}, \lambda_{k+1}]} \int_{\mathrm{bladewire}(j) = z} I(\lambda, j) \ dj \ d\lambda
164+
$$
165+
can be pre-computed because it doesn't depend on $\mu_{\text{sam}}$.
166+
But unlike before $I_{k+\frac{1}{2},z}$ now has a much more manageable size, about 64x smaller than the first attempt.
167+
This makes it comfortably smaller than the sample measurement.
168+

docs/user-guide/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,5 @@ installation
88
amor/index
99
estia/index
1010
offspec/index
11+
focused-reflectometry-data-reduction
1112
```

0 commit comments

Comments
 (0)