You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: paper/paper.md
+16-16Lines changed: 16 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -84,14 +84,6 @@ authors:
84
84
surname: Márquez Prieto
85
85
orcid: https://orcid.org/0000-0002-8173-2566
86
86
affiliation: 1
87
-
- given-names: Christoph
88
-
surname: Koch
89
-
orcid: https://orcid.org/0000-0002-3984-1523
90
-
affiliation: 1
91
-
- given-names: Heiko B.
92
-
surname: Weber
93
-
orcid: https://orcid.org/0000-0002-6403-9022
94
-
affiliation: 7
95
87
- given-names: Claudia
96
88
surname: Draxl
97
89
orcid: https://orcid.org/0000-0003-3523-6657
@@ -100,6 +92,14 @@ authors:
100
92
surname: Brockhauser
101
93
orcid: https://orcid.org/0000-0002-9700-4803
102
94
affiliation: 1
95
+
- given-names: Christoph
96
+
surname: Koch
97
+
orcid: https://orcid.org/0000-0002-3984-1523
98
+
affiliation: 1
99
+
- given-names: Heiko B.
100
+
surname: Weber
101
+
orcid: https://orcid.org/0000-0002-6403-9022
102
+
affiliation: 7
103
103
104
104
affiliations:
105
105
- name: Physics Department and CSMB, Humboldt-Universität zu Berlin, Zum Großen Windkanal 2, D-12489 Berlin, Germany
@@ -141,27 +141,27 @@ Achieving FAIR (Findable, Accessible, Interoperable, and Reproducible) data prin
141
141
142
142
The `dataconverter`, core module of pynxtools, combines instrument output files and data from electronic lab notebooks into NeXus-compliant HDF5 files. The converter performs three key operations: extracting experimental data through specialized readers, validating against NeXus application definitions to ensure compliance with existence and format constraints, and writing valid NeXus/HDF5 output files.
143
143
144
-
The `dataconverter` provides a CLI to produce NeXus files where users can use one of the built-in readers for generic functionality or technique-specific reader plugins, which are distributed as separate Python packages.
144
+
The `dataconverter` provides a command-line interface (CLI) for generating NeXus files, supporting both built-in readers for general-purpose functionality and technique-specific reader plugins, which are distributed as separate Python packages.
145
145
146
146
For developers, the `dataconverter` provides an abstract `reader` class for building plugins that process experiment-specific formats and populate the NeXus specification. It passes a `Template`, a subclass of Python’s dictionary, to the `reader` as a form to fill. The `Template` ensures structural compliance with the chosen NeXus application definition and organizes data by NeXus's required, recommended, and optional levels.
147
147
148
-
The `dataconverter` validates `reader` output against the selected NeXus application definition, checking for instances of required concepts, complex dependencies (like inheritance and nested group rules), and data integrity (type, shape, constraints). It reports errors for invalid required concepts and emits CLI warnings for unmatched or invalid data, aiding practical NeXus file creation.
148
+
The `dataconverter` validates `reader` output against the selected NeXus application definition, checking for instances of required concepts, complex dependencies (like inheritance and nested group rules), and data integrity (type, shape, constraints). It validates required concepts, reporting errors for any violations, and issues warnings for invalid data, facilitating reliable and practical NeXus file generation.
149
149
150
-
All reader plugins are tested using the pynxtools.testing suite, which runs automatically via GitHub CI to ensure compatibility with the dataconverter, the NeXus specification, and integration across plugins.
150
+
All reader plugins are tested using the `pynxtools.testing` suite, which runs automatically via GitHub CI to ensure compatibility with the dataconverter, the NeXus specification, and integration across plugins.
151
151
152
-
The dataconverter includes an ELN generator that creates either a fillable `YAML` file or a `NOMAD`[@Scheidgen:2023] ELN schema based on a selected NeXus application definition.
152
+
The dataconverter includes `eln_mapper` that creates either a fillable `YAML` file or a `NOMAD`[@Scheidgen:2023] ELN schema based on a selected NeXus application definition.
153
153
154
154
# NeXus reader and annotator
155
155
156
-
`read_nexus` enables semantic access to NeXus files by linking data items to NeXus concepts, allowing applications to locate relevant data without hardcoding file paths. It supports concept-based queries that return all data items associated with a specific NeXus Vocabulary term. Each data item is annotated by traversing its group path and resolving its corresponding NeXus concept, included inherited definitions.
156
+
`read_nexus` enables semantic access to NeXus files by linking data items to NeXus concepts, allowing applications to locate relevant data without hardcoding file paths. It supports concept-based queries that return all data items associated with a specific NeXus vocabulary term. Each data item is annotated by traversing its group path and resolving its corresponding NeXus concept, included inherited definitions.
157
157
158
-
Items not part of the NeXus schema are explicitly marked as such, aiding in validation and debugging. Targeted documentation of individual data items is supported through path-specific annotation. The tool also identifies and summarizes the file’s default plottable data based on the NXdata definition.
158
+
Items not part of the NeXus schema are explicitly marked as such, aiding in validation and debugging. Targeted documentation of individual data items is supported through path-specific annotation. The tool also identifies and summarizes the file’s default plottable data based on the `NXdata` definition.
159
159
160
160
# `NOMAD` integration
161
161
162
-
While `pynxtools` works independently, it can also be integrated directly into Research Data Management Systems (RDMS). Out of the box, the package functions as a plugin within the `NOMAD` platform [@Scheidgen:2023; @Draxl:2019]. This enables data in the NeXus format to be integrated into `NOMAD`'s metadata model, making it searchable and interoperable with other data from theory and experiment. The plugin consists of several key components (so called entry points):
162
+
While `pynxtools` works independently, it can also be integrated directly into any Research Data Management Systems (RDMS). The package works as a plugin within the `NOMAD` platform [@Scheidgen:2023; @Draxl:2019] out of the box. This enables data in the NeXus format to be integrated into `NOMAD`'s metadata model, making it searchable and interoperable with other data from theory and experiment. The plugin consists of several key components (so called entry points):
163
163
164
-
`pynxtools` extends `NOMAD`'s data schema (called `Metainfo`[@Ghiringhelli:2017]) by integrating NeXus definitions as a `NOMAD``Schema Package`, adding NeXus-specific quantities and enabling interoperability through links to other standardized data representations in`NOMAD`. The `dataconverter` is integrated into `NOMAD`, making the conversion of data to NeXus accessible via the `NOMAD` GUI. The `dataconverter` also processes manually entered `NOMAD` ELN data in the conversion.
164
+
`pynxtools` extends `NOMAD`’s data schema, known as `Metainfo`[@Ghiringhelli:2017], by integrating NeXus definitions as a `NOMAD``Schema Package`. This integration introduces NeXus-specific quantities and enables interoperability by linking to other standardized data representations within`NOMAD`. The `dataconverter` is integrated into `NOMAD`, making the conversion of data to NeXus accessible via the `NOMAD` GUI. The `dataconverter` also processes manually entered `NOMAD` ELN data in the conversion.
165
165
166
166
The `NOMAD` Parser module in `pynxtools` (`NexusParser`) extracts structured data from NeXus HDF5 files to populate `NOMAD` with `Metainfo` object instances as defined by the `pynxtools` schema package. This enables ingestion of NeXus data directly into `NOMAD`. Parsed data is post-processed using `NOMAD`'s `Normalization` pipeline. This includes automatic handling of units, linking references (including sample and instrument identifiers defined elsewhere in `NOMAD`), and populating derived quantities needed for advanced search and visualization.
0 commit comments