update micromed multi segment using the buffer API

Author: samuelgarcia

.github/workflows/core-test.yml

@@ -24,19 +24,21 @@ jobs:
     strategy:
       fail-fast: true
       matrix:
-        os: ["ubuntu-latest", "windows-latest"]
-        # "macos-latest",
-        python-version: ['3.8', '3.9', '3.10', '3.11']
-        numpy-version: ['1.20.3', '1.21.6', '1.22.4', '1.23.5', '1.24.1', '1.25.1']
+        os: ["ubuntu-latest", "windows-latest", "macos-latest"]
+        python-version: ['3.9', '3.10', '3.11', '3.12']
+        numpy-version: ['1.22.4', '1.23.5', '1.24.1', '1.25.1', '1.26.4']
+        # numpy 1.22: 3.10, 1.23: 3.11, 1.24: 3.11, 1.25: 3.11, 1.26: 3.12
         exclude:
-           - python-version: '3.8'
-             numpy-version: '1.25.1'
-           - python-version: '3.10'
-             numpy-version: '1.20.3'
-           - python-version: '3.11'
-             numpy-version: '1.20.3'
            - python-version: '3.11'
-             numpy-version: '1.21.6'
+             numpy-version: '1.22.4'
+           - python-version: '3.12'
+             numpy-version: '1.22.4'
+           - python-version: '3.12'
+             numpy-version: '1.23.5'
+           - python-version: '3.12'
+             numpy-version: '1.24.1'
+           - python-version: '3.12'
+             numpy-version: '1.25.1'
     steps:
         - name: Set up Python ${{ matrix.python-version }}
           uses: actions/setup-python@v4

.github/workflows/io-test.yml

@@ -13,12 +13,12 @@ concurrency:  # Cancel previous workflows on the same pull request
 
 jobs:
   build-and-test:
-    name: Test on (${{ inputs.os }})
+    name: Test on (${{ inputs.os }}) (${{ matrix.python-version}})
     runs-on: ${{ inputs.os }}
     strategy:
       fail-fast: true
       matrix:
-        python-version: ['3.9', ]
+        python-version: ['3.9', '3.11.9']
     defaults:
       # by default run in bash mode (required for conda usage)
       run:
@@ -75,6 +75,7 @@ jobs:
         if: steps.cache-conda-env.outputs.cache-hit != 'true'
         run: |
           conda env update --name neo-test-env --file environment_testing.yml --prune
+          conda install python=${{ matrix.python-version }}
 
       - name: Configure git
         run: |
@@ -107,6 +108,14 @@ jobs:
           sudo apt-get update -qq
           sudo apt-get install -yqq --allow-downgrades libc6:i386 libgcc-s1:i386 libstdc++6:i386 wine
 
+      - name: Pip list
+        run: |
+          pip list
+
+      - name: Conda list
+        run: |
+          conda list
+
       - name: Test with pytest
         env:
           HDF5_PLUGIN_PATH: ${{ github.workspace }}/hdf5_local_plugin_path

.gitignore

@@ -76,4 +76,8 @@ doc/*.plx
 doc/*.nev
 doc/*.ns5
 doc/*.nix
-doc/*.nwb
+doc/*.nwb
+*.plx
+*.smr
+B95.zip
+grouped_ephys

README.rst

@@ -64,7 +64,8 @@ grant agreement FETPI-015879 (FACETS), by the European Union Seventh Framework P
 under grant agreements no. 269921 (BrainScaleS) and no. 604102 (HBP),
 and by the European Union’s Horizon 2020 Framework Programme for
 Research and Innovation under the Specific Grant Agreements No. 720270 (Human Brain Project SGA1),
-No. 785907 (Human Brain Project SGA2) and No. 945539 (Human Brain Project SGA3).
+No. 785907 (Human Brain Project SGA2) and No. 945539 (Human Brain Project SGA3),
+and by the European Union's Research and Innovation Program Horizon Europe Grant Agreement No. 101147319 (EBRAINS 2.0).
 
 .. _OpenElectrophy: https://github.com/OpenElectrophy/OpenElectrophy
 .. _Elephant: http://neuralensemble.org/elephant

codemeta.json

@@ -4,15 +4,15 @@
   "license": "https://spdx.org/licenses/BSD-3-Clause",
   "codeRepository": "https://github.com/NeuralEnsemble/python-neo",
   "contIntegration": "https://github.com/NeuralEnsemble/python-neo/actions",
-  "dateModified": "2024-02-02",
-  "downloadUrl": "https://files.pythonhosted.org/packages/b2/07/b8653dd235919dc13671fc31bd8bcbb57571e72d75e4cb04dc1347416357/neo-0.13.0.tar.gz",
+  "dateModified": "2024-10-14",
+  "downloadUrl": "https://files.pythonhosted.org/packages/e0/0d/e973b7e8464b6f1d88022c46040f203d93c0b080af0e33702bb11873dbbb/neo-0.13.4.tar.gz",
   "issueTracker": "https://github.com/NeuralEnsemble/python-neo/issues",
   "name": "Neo",
-  "version": "0.13.0",
+  "version": "0.13.4",
   "identifier": "RRID:SCR_000634",
   "description": "Neo is a Python package for working with electrophysiology data in Python, together with support for reading a wide range of neurophysiology file formats, including Spike2, NeuroExplorer, AlphaOmega, Axon, Blackrock, Plexon, Tdt, and support for writing to a subset of these formats plus non-proprietary formats including HDF5.\n\nThe goal of Neo is to improve interoperability between Python tools for analyzing, visualizing and generating electrophysiology data by providing a common, shared object model. In order to be as lightweight a dependency as possible, Neo is deliberately limited to represention of data, with no functions for data analysis or visualization.\n\nNeo is used by a number of other software tools, including SpykeViewer (data analysis and visualization), Elephant (data analysis), the G-node suite (databasing), PyNN (simulations), tridesclous_ (spike sorting) and ephyviewer (data visualization).\n\nNeo implements a hierarchical data model well adapted to intracellular and extracellular electrophysiology and EEG data with support for multi-electrodes (for example tetrodes). Neo's data objects build on the quantities package, which in turn builds on NumPy by adding support for physical dimensions. Thus Neo objects behave just like normal NumPy arrays, but with additional metadata, checks for dimensional consistency and automatic unit conversion.",
   "applicationCategory": "neuroscience",
-  "releaseNotes": "https://neo.readthedocs.io/en/stable/releases/0.13.0.html",
+  "releaseNotes": "https://neo.readthedocs.io/en/latest/releases/0.13.4.html",
   "funding": "https://cordis.europa.eu/project/id/945539",
   "developmentStatus": "active",
   "referencePublication": "https://doi.org/10.3389/fninf.2014.00010",
@@ -23,7 +23,7 @@
   "programmingLanguage": ["Python"],
   "operatingSystem": ["Linux", "Windows", "macOS"],
   "softwareRequirements": [
-    "Python (version 3.8+)",
+    "Python (version 3.9+)",
     "see https://github.com/NeuralEnsemble/python-neo/blob/master/pyproject.toml"
   ],
   "relatedLink": ["https://neo.readthedocs.io", "https://neuralensemble.org/community"],
@@ -54,6 +54,7 @@
     { "@type": "Person", "givenName": "Kyler", "familyName": "Brown" },
     { "@type": "Person", "givenName": "Mikkel Elle", "familyName": "Lepperød" },
     { "@type": "Person", "givenName": "C Daniel", "familyName": "Meliza" },
+    { "@type": "Person", "givenName": "Michael", "familyName": "Denker" },
     { "@type": "Person", "givenName": "Julia", "familyName": "Sprenger" },
     { "@type": "Person", "givenName": "Maximilian", "familyName": "Schmidt" },
     { "@type": "Person", "givenName": "Johanna", "familyName": "Senk" },
@@ -103,6 +104,15 @@
     { "@type": "Person", "givenName": "Cody", "familyName": "Baker" },
     { "@type": "Person", "givenName": "Fernando J.", "familyName": "Chaure" },
     { "@type": "Person", "alternateName": "Filipe@github" },
-    { "@type": "Person", "givenName": "Matthias Klumpp" }
+    { "@type": "Person", "givenName": "Matthias", "familyName": "Klumpp" },
+    { "@type": "Person", "givenName": "Greg", "familyName": "Knoll" },
+    { "@type": "Person", "givenName": "Chris", "familyName": "Halcrow" },
+    { "@type": "Person", "givenName": "Szonja", "familyName": "Weigl" },
+    { "@type": "Person", "givenName": "Teo", "familyName": "Lohrer"},
+    { "@type": "Person", "givenName": "Kyu Hun", "familyName": "Lee"},
+    { "@type": "Person", "givenName": "Xin", "familyName": "Niu"},
+    { "@type": "Person", "givenName": "Anthony", "familyName": "Pinto"},
+    { "@type": "Person", "givenName": "Chris", "familyName": "Heydrick"},
+    {"@type": "Person", "givenName": "Nikhil", "familyName": "Chandra"}
   ]
 }

doc/source/authors.rst

@@ -27,6 +27,7 @@ and may not be the current affiliation of a contributor.
 * Kyler Brown
 * Mikkel Elle Lepperød [11]
 * C Daniel Meliza [12]
+* Michael Denker [13, 29]
 * Julia Sprenger [13, 6]
 * Maximilian Schmidt [13]
 * Johanna Senk [13]
@@ -54,7 +55,7 @@ and may not be the current affiliation of a contributor.
 * Aitor Morales-Gregorio [13]
 * Peter N Steinmetz [22]
 * Shashwat Sridhar
-* Alessio Buccino [23]
+* Alessio Buccino [23, 24, 38]
 * Regimantas Jurkus [13]
 * Steffen Buergers [24]
 * Etienne Combrisson [6]
@@ -64,21 +65,30 @@ and may not be the current affiliation of a contributor.
 * Heberto Mayorquin [24]
 * Thomas Perret [25]
 * Kyle Johnsen [26, 27]
-* Zach McKenzie [28]
-* Moritz Alexander Kern
+* Zach McKenzie [28,39]
+* Moritz Alexander Kern [29]
 * Joscha Schmiedt
-* Daniel P. Crepeau
-* Divyansh Gupta
-* Nate Dolensek
-* Philipp Hornauer
+* Daniel P. Crepeau [30]
+* Divyansh Gupta [31]
+* Nate Dolensek [3]
+* Philipp Hornauer [32]
 * Robert Wolff
-* Jules Lebert
+* Jules Lebert [33]
 * Benjamin Heasly
-* Baptiste Grimaud
-* Cody Baker
-* Fernando J. Chaure
+* Baptiste Grimaud [34]
+* Cody Baker [24]
+* Fernando J. Chaure [35]
 * Filipe*github
 * Matthias Klumpp
+* Greg Knoll [36]
+* Chris Halcrow [37]
+* Szonja Weigl [24]
+* Chris Heydrick [40]
+* Kyu Hun Lee
+* Téo Lohrer
+* Anthony Pinto [41]
+* Xin Niu
+* Nikhil Chandra [40]
 
 1. Centre de Recherche en Neuroscience de Lyon, CNRS UMR5292 - INSERM U1028 - Universite Claude Bernard Lyon 1
 2. Unité de Neuroscience, Information et Complexité, CNRS UPR 3293, Gif-sur-Yvette, France
@@ -108,6 +118,21 @@ and may not be the current affiliation of a contributor.
 26. Georgia Institute of Technology
 27. Emory University
 28. Harvard Medical School
+29. Institute for Advanced Simulation (IAS-6), Forschungszentrum Jülich, Germany
+30. Dark Horse Neuro, Inc.
+31. IST Austria
+32. Department of Biosystems Science and Engineering, ETH Zürich, Basel, Switzerland
+33. University College London
+34. École Normale Supérieure Paris-Saclay
+35. Institute of Biomedical Engineering, University of Buenos Aires, Argentina
+36. Bernstein Center for Computational Neuroscience, Berlin, Germany
+37. University of Edinburgh
+38. Allen Institute for Neural Dynamics, Seattle, WA, United States
+39. Massachusetts General Hospital, Department of Molecular Biology
+40. Plexon Inc.
+41. Paris Brain Institute
+
+
 
 If we've somehow missed you off the list we're very sorry - please let us know.
 

doc/source/conf.py

@@ -252,6 +252,7 @@
 )
 
 sphinx_gallery_conf = {
+    # 'only_warn_on_example_error': True, # helps with debugging broken examples
     "examples_dirs": "../../examples",  # path to your example scripts
     "gallery_dirs": "examples",  # path to where to save gallery generated output
 }

doc/source/contributing.rst

@@ -208,6 +208,28 @@ The first time this is run, all of the Neo test files will be downloaded to your
 so the run time can be an hour or more.
 For subsequent runs, the files are already there, so the tests will run much faster.
 
+Because Neo downloads datasets this can lead to issues in the course of offline development or
+for packaging Neo (e.g. for a Linux distribution). In order to not download datasets and to skip
+tests which require downloaded datasets the environment variable :code:`NEO_TESTS_NO_NETWORK` can
+be set to any truthy value (e.g. :code:`'True'``).
+
+For macOS/Linux this can be done by doing:
+
+.. code-block:: bash
+
+    NEO_TESTS_NO_NETWORK='True' pytest .
+
+For Windows this can be done by doing:
+
+.. code-block:: bat
+
+    set NEO_TESTS_NO_NETWORK=true
+
+    pytest . 
+
+This can also be done with a conda environment variable if developing in a conda env. To configure these
+see the docs at `conda env vars documentation`_.
+
 It is often helpful to run only parts of the test suite. To test only the :mod:`neo.core` module,
 which is much quicker than testing :mod:`neo.io`, run::
 
@@ -465,4 +487,5 @@ Making a release
 .. _PyPI: https://pypi.org/project/neo
 .. _`continuous integration server`: https://github.com/NeuralEnsemble/python-neo/actions
 .. _`Read the Docs`: https://neo.readthedocs.io/en/latest/
-.. _`docs configuration page`: https://readthedocs.org/projects/neo/
+.. _`docs configuration page`: https://readthedocs.org/projects/neo/
+.. _`conda env vars documentation`: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#activating-an-environment

doc/source/rawio.rst

@@ -281,6 +281,32 @@ Read event timestamps and times
     In [42]: print(ev_times)
     [ 0.0317]
 
+Signal streams and signal buffers
+---------------------------------
+
+For reading analog signals **neo.rawio** has 2 important concepts:
+
+ 1. The **signal_stream** : it is a group of channels that can be read together using :func:`get_analog_signal_chunk()`.
+    This group of channels is guaranteed to have the same sampling rate, and the same duration per segment.
+    Most of the time, this group of channel is a "logical" group of channels. In short they are from the same headstage
+    or from the same auxiliary board.
+    Optionally, depending on the format, a **signal_stream** can be a slice of or an entire **signal_buffer**.
+
+ 2. The **signal_buffer** : it is group of channels that share the same data layout in a file. The most simple example
+    is channel that can be read by a simple :func:`signals = np.memmap(file, shape=..., dtype=... , offset=...)`.
+    A **signal_buffer** can contain one or several **signal_stream**'s (very often it is only one).
+    There are two kind of formats that handle this concept:
+     
+      * Formats which use :func:`np.memmap()` internally
+      * Formats based on hdf5
+       
+    There are many formats that do not handle this concept:
+
+      * the ones that use an external python package for reading data (edf, ced, plexon2, ...)
+      * the ones with a complicated data layout (e.g. those where the data blocks are split without structure)
+
+    To check if a format makes use of the buffer api you can check the class attribute flag `has_buffer_description_api` of the
+    rawio class.
 
 
 

doc/source/releases.rst

@@ -6,6 +6,10 @@ Release notes
 .. toctree::
    :maxdepth: 1
 
+   releases/0.13.4.rst
+   releases/0.13.3.rst
+   releases/0.13.2.rst
+   releases/0.13.1.rst
    releases/0.13.0.rst
    releases/0.12.0.rst
    releases/0.11.1.rst