Skip to content

Commit c74bfdb

Browse files
Major restructuring of the documentation (#662)
Co-authored-by: RubelMozumder <32923026+RubelMozumder@users.noreply.github.com>
1 parent 4e8d6a2 commit c74bfdb

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+1488
-1158
lines changed

.cspell/custom-dictionary.txt

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Custom Dictionary Words
2+
3+
## NeXus related words
4+
AXISNAME
5+
namefitting
6+
NIAC
7+
NXDL
8+
9+
## LateX, XML
10+
11+
# Words related to experimental techniques
12+
13+
# Software
14+
caplog
15+
cnxvalidate
16+
dataconverter
17+
DECTRIS
18+
electronanalyzer
19+
ELECTRONANALYZER
20+
ellipsometry
21+
infty
22+
libiconv
23+
loadgroup
24+
mainfile
25+
MSYS
26+
nexpy
27+
nxdls
28+
PATHCONV
29+
posint
30+
punx
31+
pynxtools
32+
xarrays
33+
xdist
34+
Zenodo
35+
36+
# examples
37+
appdefdir
38+
CSET
39+
ellips
40+
FAIRMAT
41+
filemanager
42+
fxcef
43+
hildebrandt
44+
hixu
45+
infile
46+
kfra
47+
libhdf
48+
mydata
49+
mydatareader
50+
mynxdl
51+
nexusvalidate
52+
nexusvalidation
53+
NULLTERM
54+
otherfile
55+
physik
56+
STRPAD
57+
STRSIZE
58+
xfbaa
59+
60+
# related to experimental techniques
61+
arpes
62+
ARPES
63+
Bluesky
64+
mbar
65+
MPES
66+
Raman
67+
68+
# Names
69+
Forschungsgemeinschaft
70+
José
71+
Kühbach
72+
Lukas
73+
lukaspie
74+
Markus
75+
mkuehbach
76+
Mozumder
77+
NFDI
78+
Pielsticker
79+
Shabih
80+
Sherjeel
81+
sherjeelshabih
82+
83+
# other languages
84+
85+
# en-UK which requires to many changes
86+
87+
# misc
88+
appdefs
89+
hypothes
90+
speciality

.github/workflows/build_docs.yml

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ on:
44
push:
55
branches:
66
- master # Triggers deployment on push to the master branch
7+
- restructure-docs
78

89
env:
910
python-version: 3.12
@@ -27,6 +28,13 @@ jobs:
2728
git config user.name github-actions[bot]
2829
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
2930
31+
- name: Check spelling
32+
uses: streetsidesoftware/cspell-action@v7
33+
with:
34+
files: "docs/**/*.md README.md"
35+
strict: true
36+
incremental_files_only: false
37+
3038
- name: Install uv and set the python version to ${{ env.python-version }}
3139
uses: astral-sh/setup-uv@v5
3240
with:
@@ -42,7 +50,7 @@ jobs:
4250
4351
- name: Install Dependencies
4452
run: |
45-
uv pip install --no-cache-dir ".[docs]"
53+
uv pip install ".[docs]"
4654
4755
- name: Build and Deploy
4856
run: |

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,7 @@ cython_debug/
198198
.pyenv
199199
*.pyc
200200
*.txt
201+
!.cspell/custom-dictionary.txt
201202
!requirements.txt
202203
!dev-requirements.txt
203204
!mkdocs-requirements.txt

MANIFEST.in

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
prune *
22
exclude *
33
recursive-include src/pynxtools *.py
4-
include pyproject.toml LICENSE README.md TROUBLESHOOTING.md dev-requirements.txt
4+
include pyproject.toml LICENSE README.md dev-requirements.txt
55
include src/pynxtools/data/*
66

77
recursive-include src/pynxtools/definitions/base_classes/ *.xml

README.md

Lines changed: 9 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -8,110 +8,19 @@
88
[![Coverage Status](https://coveralls.io/repos/github/FAIRmat-NFDI/pynxtools/badge.svg?branch=master&kill_cache=1)](https://coveralls.io/github/FAIRmat-NFDI/pynxtools?branch=master)
99
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.1323437.svg)](https://doi.org/10.5281/zenodo.13862042)
1010

11-
`pynxtools` is a tool designed for making your experimental data FAIR.
12-
It allows to develop ontologies and to create ontological instances based on the [NeXus format](https://www.nexusformat.org/).
11+
<table align="center">
12+
<tr><td align="center" width="10000">
1313

14-
# Scope
14+
# <strong> pynxtools: FAIR data handling using NeXus </strong>
15+
</td></tr></table>
1516

16-
`pynxtools` is a parser for combining various instrument output formats and electronic lab notebook (ELN) formats into an [HDF5](https://support.hdfgroup.org/HDF5/) file according to NeXus application definitions.
1717

18-
Additionally, the software can be used as a plugin in the research data management system NOMAD for
19-
making experimental data searchable and publishable. NOMAD is developed by the FAIRmat consortium which is a consortium of the German National Research Data Infrastructure (NFDI).
18+
`pynxtools` is an open-source Python package designed for making your experimental data FAIR. `pynxtools` is a parser for combining various instrument output formats and electronic lab notebook (ELN) formats into an HDF5 file according to NeXus application definitions.
2019

21-
# Installation
20+
Additionally, the software can be used as a plugin in the research data management system NOMAD for making experimental data searchable and publishable. NOMAD is developed by the FAIRmat consortium which is a consortium of the German National Research Data Infrastructure (NFDI).
2221

23-
It is recommended to use python 3.12 with a dedicated virtual environment for this package.
24-
Learn how to manage [python versions](https://github.com/pyenv/pyenv) and
25-
[virtual environments](https://realpython.com/python-virtual-environments-a-primer/).
22+
Read more in the [official documentation page](https://fairmat-nfdi.github.io/pynxtools/).
2623

27-
Install the latest stable version of this package from PyPI with
24+
# How to cite this work
2825

29-
```shell
30-
pip install pynxtools
31-
```
32-
33-
You can also install the latest _development_ version with
34-
35-
```shell
36-
pip install git+https://github.com/FAIRmat-NFDI/pynxtools.git
37-
```
38-
39-
# Documentation
40-
Documentation can be found [here](https://fairmat-nfdi.github.io/pynxtools/).
41-
42-
# Repository structure
43-
44-
The software tools are located inside [`src/pynxtools`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/src/pynxtools). They are shipped with unit tests located in [`tests`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/tests).
45-
Some examples from the scientific community are provided in [`examples`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/examples). They guide you through the process of converting instrument data into the NeXus standard and visualising the files' content.
46-
47-
# NOMAD integration
48-
49-
## Does this software require NOMAD or NOMAD OASIS?
50-
51-
No. The data files produced here can be uploaded to NOMAD. Therefore, this tool acts as the framework to design schemas and instances of data within the NeXus universe. It can, however, be used as a NOMAD plugin to parse nexus files, please see the section below for details.
52-
53-
## How to use pynxtools with NOMAD
54-
55-
To use pynxtools with NOMAD, simply install it in the same environment as the `nomad-lab` package.
56-
NOMAD will recognize pynxtools as a plugin automatically and offer automatic parsing of `.nxs` files. In addition, NOMAD will install a schema for NeXus application definitions.
57-
By default, `pynxtools` is already included in the NOMAD [production]https://nomad-lab.eu/prod/v1/gui/ and [staging](https://nomad-lab.eu/prod/v1/staging/gui/) deployments.
58-
59-
# Contributing
60-
61-
## Development install
62-
63-
Install the package with its dependencies:
64-
65-
```shell
66-
git clone https://github.com/FAIRmat-NFDI/pynxtools.git \\
67-
--branch master \\
68-
--recursive pynxtools
69-
cd pynxtools
70-
git submodule sync --recursive
71-
git submodule update --init --recursive --jobs=4
72-
python -m pip install --upgrade pip
73-
python -m pip install -e .
74-
python -m pip install -e ".[dev]"
75-
```
76-
77-
There is also a [pre-commit hook](https://pre-commit.com/#intro) available
78-
which formats the code and checks the linting before actually commiting.
79-
It can be installed with
80-
```shell
81-
pre-commit install
82-
```
83-
from the root of this repository.
84-
85-
## Test this software
86-
87-
Especially relevant for developers, there exists a basic test framework written in
88-
[pytest](https://docs.pytest.org/en/stable/) which can be used as follows:
89-
90-
```shell
91-
python -m pytest -sv tests
92-
```
93-
94-
## Run examples
95-
96-
A number of examples exist which document how the tools can be used. For a standalone
97-
usage convenient jupyter notebooks are available for each tool. To use these notebooks, jupyter
98-
and related tools have to be installed in the development environment as follows:
99-
100-
```shell
101-
python -m pip install jupyter
102-
python -m pip install jupyterlab
103-
python -m pip install jupyterlab_h5web
104-
```
105-
# Troubleshooting
106-
107-
Please check this [guide](https://fairmat-nfdi.github.io/pynxtools/tutorial/troubleshooting.html) for any issues you face with the tool. If you don't find a solution there, please make a new [Github Issue](https://github.com/FAIRmat-NFDI/pynxtools/issues/new?template=bug.yaml).
108-
109-
# Questions, suggestions?
110-
111-
To ask further questions, to make suggestions how we can improve these tools, to get advice
112-
on how to build on this work, or to get your parser included into NOMAD, you can:
113-
114-
- Open an issue on the [pynxtools GitHub](https://github.com/FAIRmat-NFDI/pynxtools/issues)
115-
- Use our forums at [matsci.org](https://matsci.org/c/nomad/32)
116-
- Write to [support@nomad-lab.eu](mailto:support@nomad-lab.eu)
117-
- Contact directly the lead developers of the individual parsers.
26+
Shabih, S., Dobener, F., Albino, A., Brockhauser, S., Brückner, S., Chang, T., Fekete, Á., Ginzburg, L., Haraszti, T., Hildebrandt, R., Kühbach, M., Maier, S., Márquez, J. A., Mozumder, R., Pielsticker, L., Rettig, L., Scheidgen, M., Weber, H. B., Koch, C. T., & Draxl, C. (2025). Pynxtools: A Python Library for NeXus-Compliant Experimental Data Conversion and Integration with NOMAD Platform (v0.10.6). Zenodo. https://doi.org/10.5281/zenodo.15341365

cspell.json

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{
2+
"version": "0.2",
3+
"ignorePaths": [
4+
"./tests/data/*",
5+
"*.toml",
6+
"*.bat",
7+
"*.egg-info"
8+
],
9+
"dictionaryDefinitions": [
10+
{
11+
"name": "custom-dictionary",
12+
"path": "./.cspell/custom-dictionary.txt",
13+
"addWords": true
14+
}
15+
],
16+
"dictionaries": [
17+
"custom-dictionary",
18+
"python"
19+
],
20+
"words": [],
21+
"ignoreWords": [],
22+
"ignoreRegExpList": ["\\bNX\\w*\\b", "https?://[\\S]+"],
23+
"import": [],
24+
"language": "en-US"
25+
}

docs/contact.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Get in contact
2+
3+
NOMAD and `pynxtools` are open source project that warmly welcome community projects, contributions, suggestions, bug fixes, and constructive feedback. `pynxtools` is build mainly within FAIRmat Area B - Experiment.
4+
5+
You can reach us by different channels. You can send as directly an email to the main contributors list:
6+
7+
| Name | E-mail | Github profiles |
8+
|------|------------|-----------------|
9+
| Dr. Lukas Pielsticker | [lukas.pielsticker@physik.hu-berlin.de](mailto:lukas.pielsticker@physik.hu-berlin.de) | [@lukaspie](https://github.com/lukaspie) |
10+
| Dr. Markus Kühbach | [markus.kuehbach@physik.hu-berlin.de](mailto:markus.kuehbach@physik.hu-berlin.de) | [@mkuehbach](https://github.com/mkuehbach) |
11+
| Rubel Mozumder | [rubel.mozumder@physik.hu-berlin.de](mailto:rubel.mozumder@physik.hu-berlin.de) | [@RubelMozumder](https://github.com/RubelMozumder) |
12+
| Sherjeel Shabih | [sherjeel.shabih@physik.hu-berlin.de](mailto:sherjeel.shabih@physik.hu-berlin.de) | [@sherjeelshabih](https://github.com/sherjeelshabih) |
13+
| Dr. José Marquez | [josemarquez@physik.hu-berlin.de](mailto:josemarquez@physik.hu-berlin.de) | [@Pepe-Marquez](https://github.com/Pepe-Marquez) |
14+
15+
Alternatively, you can also:
16+
17+
- Open an issue on the [pynxtools GitHub](https://github.com/FAIRmat-NFDI/pynxtools/issues)
18+
- Join the [NOMAD discord channel](https://discord.gg/Gyzx3ukUw8) and ask us there directly.

docs/getting-started.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Getting started
2+
3+
This is the entry point for anybody that is new to the NeXus data format and to FAIRmat/NOMAD. It serves as a guide to getting started with the `pynxtools` software.
4+
5+
## What should you should know before reading this guide?
6+
7+
Nothing!
8+
9+
However, to get started, it does not hurt to read the following explanations:
10+
11+
- [A primer on NeXus](learn/nexus/nexus-primer.md)
12+
- [What is FAIR data](https://www.nature.com/articles/sdata201618)
13+
14+
## What you will know at the end of this guide?
15+
16+
You will have
17+
18+
- a basic understanding of what this software is about and its capabilities
19+
- how the different software tools are interconnected
20+
21+
## What is NeXus?
22+
23+
NeXus is a data format intended for describing and standardizing experimental data. NeXus provides a specific grammar and syntactic rules via NXDL (NeXus Definition Language) for organizing data within files in addition to a dictionary of well-defined domain-specific field concepts. Since each individual concept is properly documented, it allows communities to agree on terms describing their data. Thus, NeXus acts as a contract on which concepts have to be present and how to name them in a given dataset.
24+
25+
For a more detailed description on the general principles of NeXus we recommend:
26+
27+
- [our learning page for NeXus](learn/nexus/nexus-primer.md)
28+
- [the official NeXus manual](https://manual.nexusformat.org/)
29+
30+
## What is FAIRmat?
31+
32+
FAIRmat is one of the consortia of the German National Research Data Infrastructure (NFDI). It is tasked with building a FAIR data infrastructure for condensed-matter physics and the chemical physics of solids.
33+
34+
## What is NOMAD?
35+
36+
Within FAIRmat, we develop **NOMAD**: an open source research data management system for making materials science data searchable and publishable. NOMAD hosts a wide variety of datasets from different domains of materials science - including, but not limited to, NeXus data.
37+
38+
- [NOMAD Homepage](https://nomad-lab.eu/)
39+
- [NOMAD documentation](https://nomad-lab.eu/prod/v1/staging/docs/)
40+
41+
## What is pynxtools?
42+
43+
`pynxtools` is our main software tool for end-to-end handling of data from experiments using NeXus. It contains a parser for combining various instrument output formats and electronic lab notebook (ELN) formats into an [HDF5](https://support.hdfgroup.org/HDF5/) file according to NeXus application definitions. It provides validation against these NeXus definitions and can be used to annotate existing NeXus files with semantic meaning.
44+
45+
`pynxtools` acts as a Python framework with a built-in API for writing [reader plugins](reference/plugins.md) that provide specialized reading and conversion functionality for different domains of materials science.
46+
47+
`pynxtools` can be used for standalone NeXus conversion, but it can also be used as a _plugin_ to NOMAD, extending NOMAD schemas and parsing capabilities with NeXus-specific capabilities.
48+
49+
## How can I install pynxtools? How can I contribute?
50+
51+
- [Installation tutorial](./tutorial/installation.md)
52+
- [Development tutorial](./tutorial/contributing.md)
53+
54+
## Does `pynxtools` require NOMAD or NOMAD OASIS?
55+
56+
No. With the available [plugins](./reference/plugins.md) or community-developed plugins, you can use `pynxtools` as a standalone tool for converting raw data from experiments to NeXus-compliant files. Therefore, this tool acts as the framework to design instances of data within the NeXus universe. The software _can_, however, be used as a **NOMAD plugin** to parse NeXus files, please see the section below for details.
57+
58+
## How to use `pynxtools` with NOMAD
59+
60+
NeXus is supported by the research data management platform NOMAD (as a [NOMAD plugin](https://nomad-lab.eu/prod/v1/docs/howto/plugins/plugins.html)). Experimental data files that are compatible with a NeXus application definition can easily be uploaded to NOMAD and translated into the [NOMAD Metainfo](https://nomad-lab.eu/prod/v1/gui/analyze/metainfo/pynxtools) data model using `pynxtools`. Therefore, the data file can be recognized by NOMAD's search system. If you want to learn more about uploading NeXus data to NOMAD, please refer to the [NeXus to NOMAD tutorial](./tutorial/nexus-to-nomad.md) of this documentation.
61+
62+
To use the `pynxtools` Python package with NOMAD, simply install it in the same Python environment as the `nomad-lab` package. NOMAD will recognize `pynxtools` as a plugin automatically and offer automatic parsing of `.nxs` files. In addition, NOMAD will automatically transform the NeXus definitions shipped with `pynxtools` into its own datamodel called `Metainfo`. By default, `pynxtools` is already included in the NOMAD [production]https://nomad-lab.eu/prod/v1/gui/ and [staging](https://nomad-lab.eu/prod/v1/staging/gui/) deployments..
63+
64+
!!! info "A note on FAIR data"
65+
66+
FAIRmat's contribution to the existing NeXus standard, together with the tools provided through `pynxtools`, enable scientists and research groups working with data, as well as helping communities implement standardized FAIR research data.
67+
68+
You can think of NeXus fulfilling the interoperability and reproducibility part and a research data management platform like NOMAD the findable and accessible part.
69+
70+
We consider `pynxtools` particularly useful for meeting the following FAIR principle as defined in [FAIR Principles: Interpretations and Implementation Considerations](https://direct.mit.edu/dint/article/2/1-2/10/10017/FAIR-Principles-Interpretations-and-Implementation): F2-4, I2-I3, and R1.
71+
72+
## Where to go next?
73+
74+
We suggest you have a look at one of our tutorials:
75+
76+
- [installing `pynxtools`](tutorial/installation.md)
77+
- [how go convert data to NeXus](tutorial/converting-data-to-nexus.md)

0 commit comments

Comments
 (0)