Merge pull request #1 from davistdaniel/feature/workflows

Feature/workflows

Author: davistdaniel

CHANGELOG

@@ -0,0 +1,9 @@
+v0.9.0b1
+
+* Added Workflows for HYSCORE, ESEEM experiments
+* Updated tests in accordance with change of the parameter default for `npts`.
+* Updated docs with Workflow examples
+* Other minor updates to documentation
+* Logo adapted also for dark backgrounds.
+* Added exponential decay as a background correction function.
+* Added workflow method to EprData, detects experiment type from pulse program

README.md

@@ -25,12 +25,10 @@ For EPRpy documentation, see [here](https://davistdaniel.github.io/EPRpy/). Sour
 ## Features
 
 * Read and export EPR data acquired on Bruker EPR spectrometers.
-* Basic processing capabilities such as [interactive baseline correction](https://davistdaniel.github.io/EPRpy/notebooks/examples.html#Baseline-correction), integration etc.
+* Data processing capabilities such as [interactive baseline correction](https://davistdaniel.github.io/EPRpy/notebooks/examples.html#Baseline-correction), integration etc.
 * [Interactive data inspection](https://davistdaniel.github.io/EPRpy/plotting.html#interactive-plots) for 1D and 2D datasets.
 * Generate quick plots of 1D and 2D datasets, compare different datasets.
-
-## Upcoming 
-* Automated workflow templates for specific EPR experiments
+* [Automated processing pipelines](https://davistdaniel.github.io/EPRpy/notebooks/examples.html#Workflows) for specific pulse programs. Read more about [workflows](https://davistdaniel.github.io/EPRpy/workflows.html).
 
 ## Limitations
 * Supports reading of files only in Bruker BES3T format v.1.2 and upto 2D datasets.

docs/source/Examples.rst

@@ -3,7 +3,7 @@ Examples
 
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 3
    :caption: Contents:
 
    notebooks/examples.ipynb

docs/source/conf.py

@@ -9,7 +9,7 @@
 project = 'EPRpy'
 copyright = '2024, Davis Thomas Daniel'
 author = 'Davis Thomas Daniel'
-release = '0.9.0a8'
+release = '0.9.0b1'
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

docs/source/images/eprpy_logo.png

@@ -10,6 +10,7 @@ Welcome to **EPRpy**'s documentation
    :width: 400px
    :align: center
 
+
 **EPRpy** is a Python library designed to streamline the handling, inspection, and processing of Electron Paramagnetic Resonance (EPR) spectroscopic data acquired on Bruker EPR spectrometers. 
 **EPRpy** focusses on ease of use, enabling quick data visualization, data comparisons, and having transperent as well as highly customisable control over data analysis.
 
@@ -49,6 +50,7 @@ Using EPRpy
    loading
    processing
    plotting
+   workflows
    EprData
    Examples
 

docs/source/index.rst

@@ -31,7 +31,7 @@ For complex data, imaginary part can be be visualized by setting ``plot_imag`` t
 .. image:: images/plt_with_imag.png
    :width: 400px
 
-See more `plotting examples <notebooks/examples.html#Plotting>`_ for 1D data.
+See more plotting `examples <notebooks/examples.html#Plotting>`_ for 1D data.
 
 
 2D data can be visualized similarly and has a selection of plot types as shown below. 

docs/source/notebooks/examples.ipynb

@@ -0,0 +1,141 @@
+Workflows
+===============
+
+The ``workflow`` module in **EPRpy** provides high-level processing pipelines for 
+common pulsed-EPR experiments. It is designed to take an ``EprData`` object as input 
+and return a fully processed ``EprData`` object, with all intermediate results and 
+processing steps preserved for reproducibility in the ``data_dict`` method.
+
+Currently, the module supports workflows for:
+
+* **HYSCORE** (Hyperfine Sublevel Correlation) experiments
+* **ESEEM** (Electron Spin Echo Envelope Modulation) experiments
+
+These workflows include baseline correction, windowing, zero-filling, Fourier 
+transformation, and frequency axis generation. Optionally, symmetrisation of HYSCORE 
+spectra and background fitting strategies for ESEEM are provided.
+
+A workflow can be simply used by loading experimental data with a compatible pulse program :
+
+.. code-block:: python
+
+  # import EPRpy
+  import eprpy as epr
+  
+  # Load data by providing path to .DSC or .DTA file
+  hyscore_data = epr.load('hyscore_exp.DSC')
+
+  # process using a workflow, zero fill 1024 points
+  hyscore_processed = hyscore_data.workflow(zf=1024)
+
+
+Have a look at `workflow examples <notebooks/examples.html#Workflows>`_.
+
+Internally, the :ref:`eprworkflowclass` is used when ``EprData.workflow()`` is called. In the example above, ``hyscore_processed`` is an ``EprData`` object, so all methods of ``EprData`` object are also accessible.
+Other parameters which are supported for a workflow is explained in the section :ref:`eprworkflowclass`. In the following, the workflow processing steps are elaborated.
+
+HYSCORE processing
+--------------------
+
+The workflow supports pulse programs:
+    - ``HYSCORE``
+
+
+HYSCORE datasets are procssed by by:
+    - Applying 2D polynomial baseline correction.
+    - Windowing with Hamming functions in both dimensions.
+    - Zero-filling (optional).
+    - Performing a 2D FFT and shifting the result.
+    - Symmetrising the FFT data (optional).
+    - Returning all results (raw, intermediate, final) in a dictionary.
+
+The output contains frequency axes in MHz and processed spectra suitable for 
+plotting and further analysis.
+
+If you have a processed ``EprData`` object from a ``HYSCORE`` data set, the processed arrays can be accessed using ``EprData.data_dict``:
+
+.. code-block:: python
+
+    - 'proc_param': Processing parameters used.
+    - 'raw_data': Original raw data.
+    - 'baseline_dim1', 'baseline_dim2': Baseline correction results for both dimensions.
+    - 'bc_data': Baseline-corrected data.
+    - 'window_dim1', 'window_dim2': Window functions applied.
+    - 'bc_w_data': Baseline-corrected and windowed data.
+    - 'time_axis1', 'time_axis2': Time axes (extended if zero-filled).
+    - 'bc_w_zf_data': Zero-filled, baseline-corrected, windowed data.
+    - 'frequency_axis1', 'frequency_axis2': Frequency axes for FFT.
+    - 'FFT_data': 2D FFT of processed data.
+    - 'FFT_shifted_data': FFT data after shift.
+    - 'data': Absolute value of shifted FFT (final processed data, optionally symmeterised).
+    - 'dims': Frequency axes for plotting.
+    - 'is_complex': Boolean indicating if data is complex (always False).
+    - 'history': List of processing steps.
+
+
+ESEEM processing
+--------------------
+
+The workflow supports pulse programs:
+    - ``2P ESEEM``
+    - ``3P ESEEM``
+    - ``2P ESEEM vs. B0``
+    - ``3P ESEEM vs. B0``
+    - ``3P ESEEM vs tau``
+
+ESEEM datasets are processed by:
+    - Applying background correction (exponential or polynomial).
+    - Windowing with a Hamming function.
+    - Zero-filling (optional).
+    - Performing a 1D FFT (or along the last axis for 2D data).
+    - Returning absolute, shifted FFT spectra along with all intermediate steps.
+
+Results include the processed frequency-domain spectra in MHz, with 
+dimensionality preserved for field-dependent ESEEM maps.
+
+If you have a processed ``EprData`` object from a ``ESEEM`` data set, the processed arrays can be accessed using ``EprData.data_dict``:
+
+.. code-block:: python
+
+    - 'proc_param': Processing parameters used.
+    - 'raw_data': Original raw data.
+    - 'baseline_dim1': Baseline correction results.
+    - 'bc_data': Baseline-corrected data.
+    - 'window_dim1': Window function applied.
+    - 'bc_w_data': Baseline-corrected and windowed data.
+    - 'time_axis1': Time axis (extended if zero-filled).
+    - 'bc_w_zf_data': Zero-filled, baseline-corrected, windowed data.
+    - 'frequency_axis1': Frequency axis for FFT.
+    - 'FFT_data': FFT of processed data.
+    - 'FFT_shifted_data': FFT data after shift.
+    - 'data': Absolute value of shifted FFT (final processed data).
+    - 'dims': Frequency axis (and second axis if 2D data).
+    - 'is_complex': Boolean indicating if data is complex (always False).
+    - 'history': List of processing steps.
+
+
+.. _eprworkflowclass:
+
+EprWorkflow Class
+--------------------
+
+The ``EprWorkflow`` class encapsulates complete workflows for HYSCORE and ESEEM 
+data processing. It manages parameters like zero-filling, polynomial order for 
+baseline correction, and optional symmetrisation. All results, including raw 
+data, corrected data, intermediate steps, and frequency-domain spectra, are 
+stored in a dictionary that is updated step by step.
+
+**Parameters**
+    - **eprdata** (*EprData*) – Instance of the EprData class.
+    - **zf** (*int, optional*) – Number of points to add via zero filling.
+    - **poly_order** (*int, optional, default=3*) – Order of the polynomial for baseline correction in case of 3P ESEEM experiments
+    - **x_max** (*float, optional*) – Upper bound of the Hamming window.
+    - **pick_eseem_points** (*bool, default=False*) – Whether to pick points along the ESEEM decay curve for background calculation. If False, all data points are used.
+    - **symmeterise** (*bool or str, default=False*) – If set, symmetrises HYSCORE FFT results across diagonal or anti-diagonal.
+
+Both ``hyscore()`` and ``eseem()`` return a **dictionary** containing raw data, 
+intermediate results, processing parameters, and final spectra.  
+Some keys are primarily useful for debugging, while others are essential for 
+end-users who wish to analyse or visualise results.
+
+

docs/source/plotting.rst

@@ -7,8 +7,10 @@
 
 # EPRpy
 from eprpy.plotter import eprplot
-from eprpy.processor import _integrate,_scale_between,_baseline_correct
+from eprpy.processor import _integrate,_scale_between,_baseline_correct,_derivative
+from eprpy.workflows import EprWorkflow
 
+warnings.simplefilter("always")
 
 def load(filepath):
 
@@ -38,7 +40,8 @@ def load(filepath):
 
     out_dict = {'dims':None,
                 'data':None,
-                'acq_param':None}
+                'acq_param':None,
+                'workflow_type':None}
 
     dta_filepath,dsc_filepath = check_filepaths(filepath)
     dsc_parameter_dict = read_DSC_file(dsc_filepath)
@@ -411,7 +414,17 @@ def __init__(self,out_dict):
             self.g = ((float(self.acq_param['MWFQ'])/1e+9)/(13.996*(x_g/10000)))
         else:
             self.g = None
-        self.history[0].append(deepcopy(self))
+        self.workflow_type = out_dict['workflow_type']
+        self.history[-1].append(deepcopy(self))
+
+        try:
+            self.pulse_program = self.acq_param["PlsSPELEXPSlct"]
+        except Exception as e:
+            print(
+                "Error ocurred while reading pulse program parameter PlsSPELEXPSlct from DSC file : ",
+                e,
+            )
+            self.pulse_program = "Unknown"
 
 
     def plot(self,g_scale=False,plot_type='stacked', slices='all', spacing=0.5,plot_imag=True,interactive=False):
@@ -429,24 +442,38 @@ def integral(self):
         eprdata_proc = _integrate(self)
         return eprdata_proc
 
-    def baseline_correct(self,interactive=False,
-                      npts=10,method='linear',spline_smooth=1e-5,
-                      order=2):
+    def baseline_correct(self,interactive=False, npts=0, method="linear", spline_smooth=1e-5, order=2,init_vals=None,bounds = (-np.inf, np.inf),fit_eseem_max=False):
 
-        eprdata_proc = _baseline_correct(self,interactive,
-                          npts,method,spline_smooth,order)
+        eprdata_proc = _baseline_correct(self,interactive, npts, method, spline_smooth, order,init_vals,bounds,fit_eseem_max)
         return eprdata_proc
 
 
     def select_region(self,region):
 
-
         assert type(region) in [range,list],'region keyword must be a range object or list.'
         out_dict = deepcopy(self.data_dict)
         out_dict['dims'][-1] =  out_dict['dims'][-1][region]
         out_dict['data'] =  out_dict['data'][...,region]
 
         return EprData(out_dict)
+    
+    def derivative(self,sigma=1,axis=-1):
+
+        epr_data_proc = _derivative(self,sigma,axis)
+        return epr_data_proc
+
+    def workflow(self,zf=0,poly_order=3,x_max=None,pick_eseem_points=False,symmetrise=False,verbose=False):
 
+        if self.pulse_program == "HYSCORE":
+            hyscore_out_dict = EprWorkflow(eprdata=self,zf=zf,poly_order=poly_order,x_max=x_max,pick_eseem_points=pick_eseem_points,symmetrise=symmetrise,verbose=verbose).hyscore()
+            return EprData(hyscore_out_dict)
 
+        elif self.pulse_program in ["2P ESEEM", "3P ESEEM","2P ESEEM vs. B0","3P ESEEM vs. B0","3P ESEEM vs tau"]:
+            eseem_out_dict = EprWorkflow(eprdata=self,zf=zf,poly_order=poly_order,x_max=x_max,pick_eseem_points=pick_eseem_points,verbose=verbose).eseem()
+            return EprData(eseem_out_dict)
 
+        else:
+            raise ValueError(f"No supported workflows found for the pulse program : {self.pulse_program}")
+
+
+