IMDReader Integration (#4923)

* IMDReader integration

* small cleanup

* Chore: Cleanup and small fixes

1. Moved `parse_host_port` to `IMD.py` and deleted `util.py`
2. Cleaned up `test_imd.py` - changes to `assert_*` functions and simplified non-applicable test to pass automatically

* bump CI

* Add minimum docs

* try add mock for import versions

* fix for wrong assert

* fix fixture issues

* add short class description

* add stricter matching to tests

* forbid use of .timeseries() method for streamed trajectories

* Update test_HAS_IMDCLIENT_new_enough to pass

* add imd docs

* add doc for streamreader

* black imd

* add reason for skipped test

* use staticmethod for readers

* black imd test

* try 0.2.0b0 imdclient

* check imd file input is a string

* add some port parsing test

* add test for transformation

* try cut buffer by 10

* add buffer sizes

* fail out on non-v3

* Change to `r` only

* remove parse port

* remove parse port in test

* only import parse port when imdclient exists

* use monkeypatch to avoid mess sys module

* black imd test

* add coverage test

* add tests fix

* disable test on imd version

* imd type hint

* test error on mda not imdclient

* close ref port properly

* test imd import in different file

* add test for stream base

* move _frame to baseclass

* remove onepass

* Update installed optional deps with new imdclient 0.2.2 package

* Update azure-pipelines.yml for 0.2.2 imdclient version

* Update environment.yml

* Update IMD.py version pins for imdclient package 0.2.2

* Update pyproject.toml for imdclient package 0.2.2

* Update version pin to be dynamic in test_imd_import.py

* Update test_imd_import.py

* make changes for new testserver API

* remove n_atoms_mismatch test

* fix str concat

* Fix host port format

* doc tweaks

* typo

* clarify need for gromacs configuration before example command

* buffer docs tweak and typo

* buffer default value change

* Update: AUTHORS

* Update: CHANGELOG

* black `IMD.py`

* black `test_imd.py`

* test_imd.py: import reorder

* placeholder citation removed

* Correction: `_step` called instead of `step`

* Doc: `StreamFrameIteratorSliced`

* Doc: `StreamReaderBase`

* Doc: `StreamReaderBase`'s `__getitem__`

* Edit: `StreamFrameIteratorSliced` docs

* Doc: Update base Reader list

Added to `StreamReaderBase` to list of base Readers.

* Remove: whitespace

* Update: IMD format

* Change: Error Message type

* Update: IMDReader docs

* Remove: comment

* IMDReader docs: Links to protocol

* Units removed

* Chore: small chnage to IMDReader module docs

* Update: sphinx markup - `base.py`

* Added: See also doc

* Version Added: `StreamFrameIteratorSliced`

* seealso to `StreamReaderBase` added in `IMDReader`

* Update: module and class level docs for limitations

* Remove `grompp`

* fix inlink link to imclient

* Update CHANGELOG w `StreamReaderBase`

* Minor clean up

* Typo: precision

* Limitations updated: `IMDReader` and `StreamReaderBase`

* API Doc: `check_slice_indices``

* API Doc: `timeseries`

* API Doc: `copy`

* API Doc: `rewind`

* API Doc: `next`

* API Doc: `_reopen`

* black `IMD.py`

* black `test_imd.py`

* Error raised and tested in `test_imd.py`

* `step`: docstring and test coverage

* `StreamReaderBase` writer doctsring and tests

* `test_wrong_imd_protocol_version`

* black `test_imd.py`

* added test for RuntimeError on IMDReader.OtherWriter

* merged test_imd_import into test_imd

* refactored IMD import test to use a custom state manager

- testing imports of IMD/imdclient module is now isolated from other tests
- testing different versions of imdclient and IMD.HAS_IMDCLIENT is now done in separate tests

* remove __test__

* Modify: `timestep.pyx` to remove usage of `new` keyword

* Docs: Updated
1. API doc on multiple connections
2. Jupyter usage of `close()` for secure closure of connections

* Cleanup: Remove timestep comment

* Bug Fix: Warnign message for incompatible version

* Refactor: simplify IMD tests with pytest monkeypatch

Replace `IMDModuleStateManager` with standard pytest patterns

* Revert "Refactor: simplify IMD tests with pytest monkeypatch"

This reverts commit cd21d6b.

* IMDReader docs: reorganized and reduced multiple connection details

- move details on which engines support multiple connections to imdclient docs (only update one place) and link
- rearranged warning/seealso boxes

* Update package/pyproject.toml

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

* Chore: Minor cleanup

1. Alphabetical order for `imclcient` as optional dependency
2. Changed order in `CHANGELOG`

---------

Co-authored-by: Amruthesh Thirumalaiswamy <amruthesht@users.noreply.github.com>
Co-authored-by: Hugo MacDermott-Opeskin <hugomacdermott@gmail.com>
Co-authored-by: Jennifer A. Clark <jennifer.a.clark13@gmail.com>
Co-authored-by: Yuxuan Zhuang <yuxuan.zhuang@dbb.su.se>
Co-authored-by: Lawson Woods <ljwoods2@asu.edu>
Co-authored-by: Lawson <lwoods@tgen.org>
Co-authored-by: Oliver Beckstein <orbeckst@gmail.com>
Co-authored-by: Yuxuan Zhuang <yuzhuang@stanford.edu>
Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

Author: 7 people

.github/actions/setup-deps/action.yaml

@@ -64,6 +64,8 @@ inputs:
     default: 'h5py>=2.10'
   hole2:
     default: 'hole2'
+  imdclient:
+    default: 'imdclient>=0.2.2'
   joblib:
     default: 'joblib>=0.12'
   netcdf4:
@@ -138,6 +140,7 @@ runs:
           ${{ inputs.gsd }}
           ${{ inputs.h5py }}
           ${{ inputs.hole2 }}
+          ${{ inputs.imdclient }}
           ${{ inputs.joblib }}
           ${{ inputs.netcdf4 }}
           ${{ inputs.networkx }}

azure-pipelines.yml

@@ -113,6 +113,8 @@ jobs:
       pytng>=0.2.3
       rdkit>=2024.03.4
       tidynamics>=1.0.0
+      imdclient>=0.2.2
+
     # remove from azure to avoid test hanging #4707
     # "gsd>3.0.0"
     displayName: 'Install additional dependencies for 64-bit tests'

maintainer/conda/environment.yml

@@ -30,6 +30,7 @@ dependencies:
   - sphinxcontrib-bibtex
   - mdaencore
   - waterdynamics
+  - imdclient>=0.2.2
   - pip:
       - mdahole2
       - pathsimanalysis

package/AUTHORS

@@ -260,6 +260,7 @@ Chronological list of authors
   - Gareth Elliott
   - Marc Schuh
   - Sirsha Ganguly
+  - Amruthesh Thirumalaiswamy
 
 External code
 -------------

package/CHANGELOG

@@ -16,7 +16,7 @@ The rules for this file:
 -------------------------------------------------------------------------------
 ??/??/?? IAlibay, orbeckst, BHM-Bob, TRY-ER, Abdulrahman-PROG, pbuslaev,
          yuxuanzhuang, yuyuan871111, tanishy7777, tulga-rdn, Gareth-elliott,
-         hmacdope, tylerjereddy, cbouy, talagayev, DrDomenicoMarson
+         hmacdope, tylerjereddy, cbouy, talagayev, DrDomenicoMarson, amruthesht
 
 
  * 2.10.0
@@ -42,6 +42,11 @@ Fixes
    directly passing them. (Issue #3520, PR #5006)
 
 Enhancements
+ * Added support for reading and processing streamed data in `coordinates.base`
+   with new `StreamFrameIteratorSliced` and `StreamReaderBase` (Issue #4827, PR #4923)
+ * New coordinate reader: Added `IMDReader` for reading real-time streamed
+   molecular dynamics simulation data using the IMDv3 protocol - requires
+   `imdclient` package (Issue #4827, PR #4923)
  * Added capability to calculate MSD from frames with irregular (non-linear)
    time spacing in analysis.msd.EinsteinMSD with keyword argument
    `non_linear=True` (Issue #5028, PR #5066)
@@ -70,7 +75,7 @@ Enhancements
    so that it gets passed through from the calling functions and classes
    (PR #5038)
  * Moved distopia checking function to common import location in
-   MDAnalysisTest.util (PR #5038)
+   MDAnalysisTest.util (PR #5038)  
  * Enables parallelization for `analysis.polymer.PersistenceLength` (Issue #4671, PR #5074)