datalab in situ plugin

A Python plugin for processing and visualizing in situ (NMR, UV-Vis, XRD) data within datalab instances.

The project was originally developed in the Grey Group in the Department of Chemistry at the University of Cambridge, with support and ongoing maintenance from datalab industries ltd..

Features

• Support for NMR, XRD and UV-Vis data, recorded alongside electrochemical experiments, temperature ramps or other time series data (support varies across data types)
• Integration with electrochemical data formats for combined analysis using navani
• Interactive visualization using Bokeh, with heatmaps and spectra selection
• Support for both local files, as well as running as a plugin directly on a datalab instance

Roadmap

There are still several planned features and improvements for this plugin, including:

• Supporting all formats for electrochemistry, NMR, XRD and UV-Vis that are supported by the corresponding non-in situ blocks available in datalab.
• More flexible specification of other time series data across all data types, e.g., temperature or pressure ramps.
• Improved performance and caching of data, especially for large datasets.

Installation

The datalab-app-plugin-insitu package is currently a "core plugin" of datalab (since datalab v0.5.3), i.e., it will be installed on datalab instances by default.

Development installation

We recommend you use uv for managing virtual environments and Python versions.

Once you have uv installed, you can clone this repository and install the package in a fresh virtual environment with:

git clone [email protected]:datalab-org/datalab-app-plugin-insitu
cd datalab-app-plugin-insitu
uv sync --all-extras --dev

You can activate pre-commit in your cloned repository with uv run pre-commit install.

You can run the tests using pytest, optionaly with the marker show_plots to display the plots generated by the tests:

uv run pytest -m show_plots

Usage

Full reference API documentation can be found on ReadTheDocs.

Data Structure Requirements

NMR

Your in situ NMR data should be organized as follows:

data_folder.zip/
├── <nmr_folder>/
│   ├── 1/
│   │   ├── acqus
│   │   └── pdata/
│   │       └── 1/
│   │           └── ascii-spec.txt
│   ├── 2/
│   │   └── ...
│   └── ...
└── <echem_folder>/
    └── *.MPR

with the <nmr_folder> and <echem_folder> names specified at runtime (or via the datalab UI).

UV-Vis

data_folder.zip/
├── <uv-vis folder>/
│   ├── scan_0000.txt
│   ├── scan_0001.txt
│   ├── scan_0002.txt
│   ├── scan_0003.txt
│   ├── scan_0004.txt
│   ├── scan_0005.txt
│   ├── ...
│   └── scan_<N>.txt
├── <uv-vis reference folder>/
│   └── reference.txt
└── <echem_folder>/
    └── <echem_file>

where the <uv-vis folder> contains the UV-Vis scans (named as *_<n>.txt, which will be used for sorting), the <uv-vis reference folder> contains the reference spectrum, and the <echem_folder> contains a electrochemical data file that is loadable by navani.

XRD

data_folder.zip/
├── <xrd folder>/
│   ├── 0000-scan.dat
│   ├── 0001-scan.dat
│   ├── 0002-scan.dat
│   ├── 0003-scan.dat
│   ├── 0004-scan.dat
│   ├── 0005-scan.dat
│   ├── ...
│   └── <N>-scan.dat
└── <log folder>/
    └── time_series_log.csv

where the <xrd folder> contains the XRD scans (named as 0000-scan.dat, 0001-scan.dat, etc.), and the <log folder> contains a time series log file in CSV format, mapping scan number to the time series data, e.g.,

scan_number,temperature
0000, 25.0
0001, 30.0
0002, 35.0
0003, 40.0

License

This project is released under the conditions of the MIT license. Please see LICENSE for the full text of the license.

Contact

For questions and support, please open an issue on the GitHub repository or join the public datalab Slack workspace.