Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
940e90e
Add new data fixtures
psomhorst Oct 13, 2025
78f6114
Change config for new fixtures
psomhorst Oct 13, 2025
f7babaa
Adapt loading tests to new data
psomhorst Oct 13, 2025
8387e76
Rename test_loading to test_loading_draeger
psomhorst Oct 13, 2025
7ef1d7f
Reduce precision of estimated sample frequency to 3 decimals
psomhorst Oct 13, 2025
80f6c3b
Migrate sample frequency tests
psomhorst Oct 13, 2025
27dfc8f
Make sample frequency variable for testing different frequencies later
psomhorst Oct 14, 2025
8ecaa81
Make Sequence.concatenate a instance method (can still be used as cla…
psomhorst Oct 14, 2025
03b9fdc
Improve sample frequency estimation
psomhorst Oct 14, 2025
250fee2
Add check for strictly monotonically increasing time
psomhorst Oct 14, 2025
6d9c34a
Migrate part of test_load_partial
psomhorst Oct 14, 2025
93baad7
Migrate part of test_loading_illegal
psomhorst Oct 14, 2025
3136343
Migrate time axis test to separate file
psomhorst Oct 21, 2025
fa1782a
Ensure sample frequency is a float during loading draeger data
psomhorst Oct 21, 2025
130594a
Ensure sample frequency is a float during EITData initialization
psomhorst Oct 21, 2025
7689db4
Migrate to healthy volunteer data
psomhorst Oct 21, 2025
5a24897
Add error or warning when sample frequency could not be estimated.
psomhorst Oct 21, 2025
ea3c4d1
Fix issue where max_frames=0 would act like max_frames=None.
psomhorst Oct 21, 2025
b8697c1
Add error if no frames are to be loaded.
psomhorst Oct 21, 2025
d03ab33
Add pressure pod data fixtures
psomhorst Oct 21, 2025
057ab49
Add pressure pod fixtures to loading tests
psomhorst Oct 21, 2025
31f0040
Update sample frequency tests.
psomhorst Oct 21, 2025
9306287
Add tests for skipping frames.
psomhorst Oct 21, 2025
e67f646
Add todo
psomhorst Oct 21, 2025
25d8db5
Remove outdated tests
psomhorst Oct 21, 2025
06feb55
Migrate events tests, including events on the first frame
psomhorst Oct 21, 2025
010d468
Remove outdated tests
psomhorst Oct 21, 2025
2822205
Migrate rate detection to new test data
psomhorst Oct 22, 2025
3eaf736
Migrate tests to new test data
psomhorst Oct 22, 2025
353d18d
Remove old test data refernce
psomhorst Oct 22, 2025
320bf92
Rename testdata folder to test_data
psomhorst Oct 23, 2025
7a71c47
Add download instructions for test data
psomhorst Oct 23, 2025
28963fd
Fix parametrization
psomhorst Oct 28, 2025
0186bf7
Migrate tests to new data
psomhorst Oct 28, 2025
d78f76f
Fix first_frame not working and simplify code loading Sentec data
psomhorst Oct 30, 2025
073fae9
Add Sentec test data fixtures
psomhorst Oct 30, 2025
cadf05e
Create test files for loading Sentec EIT data
psomhorst Oct 30, 2025
7660292
Fix test name
psomhorst Oct 30, 2025
fd946e3
Remove redundant case
psomhorst Oct 30, 2025
272c2cf
Remove redundant comments
psomhorst Oct 30, 2025
cbb279a
Fix typo in dev README
psomhorst Nov 3, 2025
d8b47e6
Add more healthy volunteer data fixtures
psomhorst Nov 12, 2025
8e933e7
Add time wrapped data to draeger loading test
psomhorst Nov 12, 2025
ca0fc02
Fix assertion error message
psomhorst Nov 12, 2025
69c4c0c
Add timpel data to partial data loading test
psomhorst Nov 12, 2025
cec0ec5
Replace old test data with new test data
psomhorst Nov 12, 2025
dddfc8e
Parametrize single-sequence tests
psomhorst Nov 12, 2025
ac948b3
Add timpel data to breath detection test
psomhorst Nov 12, 2025
56093b9
Remove old tests
psomhorst Nov 12, 2025
eaca692
Add timpel data to pixel breath test
psomhorst Nov 12, 2025
a6ca755
Add timpel data to rate detection test
psomhorst Nov 12, 2025
7b8c161
Add tests for loading timpel data
psomhorst Nov 12, 2025
3ab1ae9
Remove superfluous fixtures from conftest
psomhorst Nov 12, 2025
a26cba6
Add skipped test and todo for multiple Timpel files from one recording
psomhorst Nov 12, 2025
099e4fe
Update dev documentation to reflect finished status of test data migr…
psomhorst Nov 12, 2025
6e0dccd
Fix imports from tests
psomhorst Nov 12, 2025
40e314b
Fix imports from tests
psomhorst Nov 12, 2025
5d8f99a
Fix download instructions
psomhorst Nov 12, 2025
d9c8cfd
Update github action for new test data
psomhorst Nov 12, 2025
642b919
Fix check to skip donwloading test data
psomhorst Nov 12, 2025
2f19ff3
Remove old reference to data directory
psomhorst Nov 12, 2025
0f407ab
Update file download from zenodo
psomhorst Nov 12, 2025
fec69bb
Update Zenodo record id to specific version
psomhorst Nov 12, 2025
65876d7
Add download option using zenodo-get
psomhorst Nov 12, 2025
fc52c3a
Confirm deletion of zenodo-get
psomhorst Nov 12, 2025
ef2e8b5
Fix use of Python 3.12 feature
psomhorst Nov 12, 2025
b52e205
Move data download outside of install action
psomhorst Nov 12, 2025
a2f59a8
Move data download outside of install action
psomhorst Nov 12, 2025
50a94c5
Fix zenodo record ID usage
psomhorst Nov 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 3 additions & 19 deletions .github/actions/install_eitprocessing/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,36 +8,20 @@ inputs:
dependencies:
description: "The optional dependencies of eitprocessing to install"
required: false
extract-data:
description: "Whether to extract testing data"
required: false
default: "false"
token:
description: "GitHub TOKEN"
required: true
data-directory:
description: "Directory where to store eitprocessing data"
zenodo-record-id:
description: "Zenodo record ID for test dataset"
required: false
default: ${{ github.workspace }}/../eitprocessing_data/
default: "17423608"

runs:
using: "composite"
steps:
- uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
- uses: docker/login-action@v3
if: ${{ inputs.extract-data == 'true' }}
with:
registry: ghcr.io
username: psomhorst
password: ${{ inputs.token }}
- uses: shrink/actions-docker-extract@v3
if: ${{ inputs.extract-data == 'true' }}
with:
image: ghcr.io/eit-alive/eittestdata:latest
destination: ${{ inputs.data-directory }}
path: /eitprocessing/.
- uses: actions/cache@v4
id: cache-python-env
with:
Expand Down
34 changes: 29 additions & 5 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,45 @@ jobs:
os: ["ubuntu-latest"]
python-version: ["3.10"]
env:
EIT_PROCESSING_TEST_DATA: ${{ github.workspace }}/../eitprocessing_data/
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
COVERALLS_FLAG_NAME: python-${{ matrix.python-version }}
ZENODO_RECORD_ID: 17423608
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/install_eitprocessing
with:
dependencies: testing
extract-data: true
python-version: ${{ matrix.python-version }}
token: ${{ secrets.GITHUB_TOKEN }}
data-directory: ${{ env.EIT_PROCESSING_TEST_DATA }}
- name: Set data dir
shell: bash
id: set-path
run: echo "data_dir=$GITHUB_WORKSPACE/test_data" >> "$GITHUB_OUTPUT"
- name: Restore dataset cache
id: cache
uses: actions/cache@v4
with:
path: ${{ steps.set-path.outputs.data_dir }}
key: eitprocessing-testdata-zenodo.${{ env.ZENODO_RECORD_ID }}
- name: Install zenodo-get
if: steps.cache.outputs.cache-hit != 'true'
run: python3 -m pip install zenodo-get
shell: bash
- name: Download test dataset
shell: bash
if: steps.cache.outputs['cache-hit'] != 'true'
run: |
mkdir -p ${{ steps.set-path.outputs.data_dir }}
cd ${{ steps.set-path.outputs.data_dir }}
zenodo_get $ZENODO_RECORD_ID
cd -
- name: Uninstall zenodo-get
if: steps.cache.outputs['cache-hit'] != 'true'
run: python3 -m pip uninstall --yes zenodo-get
shell: bash
- name: Run coveralls
run: |
pytest --cov --cov-report xml --cov-report term --cov-report html
git config --global --add safe.directory /ci
pytest --cov --cov-report xml --cov-report term --cov-report html
git config --global --add safe.directory /ci
coveralls --service=github
33 changes: 27 additions & 6 deletions .github/workflows/release_github.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
name: Draft GitHub Release

on:
workflow_dispatch:
inputs:
Expand All @@ -25,6 +24,7 @@ permissions:
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_PAGER: cat
ZENODO_RECORD_ID: 17423608

jobs:
checks:
Expand Down Expand Up @@ -77,8 +77,6 @@ jobs:
os: ["ubuntu-latest"]
python-version: ["3.10", "3.11", "3.12", "3.13"]
name: Build for ${{ matrix.python-version }}, ${{ matrix.os }}
env:
EIT_PROCESSING_TEST_DATA: ${{ github.workspace }}/../eitprocessing_data/
steps:
- name: Checkout repository
uses: actions/checkout@v4
Expand All @@ -87,11 +85,34 @@ jobs:
uses: ./.github/actions/install_eitprocessing
with:
dependencies: testing
extract-data: true
python-version: ${{ matrix.python-version }}
token: ${{ secrets.GITHUB_TOKEN }}
data-directory: ${{ env.EIT_PROCESSING_TEST_DATA }}

- name: Set data dir
shell: bash
id: set-path
run: echo "data_dir=$GITHUB_WORKSPACE/test_data" >> "$GITHUB_OUTPUT"
- name: Restore dataset cache
id: cache
uses: actions/cache@v4
with:
path: ${{ steps.set-path.outputs.data_dir }}
key: eitprocessing-testdata-zenodo.${{ env.ZENODO_RECORD_ID }}
- name: Install zenodo-get
if: steps.cache.outputs.cache-hit != 'true'
run: python3 -m pip install zenodo-get
shell: bash
- name: Download test dataset
shell: bash
if: steps.cache.outputs['cache-hit'] != 'true'
run: |
mkdir -p ${{ steps.set-path.outputs.data_dir }}
cd ${{ steps.set-path.outputs.data_dir }}
zenodo_get $ZENODO_RECORD_ID
cd -
- name: Uninstall zenodo-get
if: steps.cache.outputs['cache-hit'] != 'true'
run: python3 -m pip uninstall --yes zenodo-get
shell: bash
- name: Run pytest
run: pytest -v --runslow

Expand Down
1 change: 0 additions & 1 deletion .github/workflows/release_pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ jobs:
- uses: ./.github/actions/install_eitprocessing
with:
dependencies: publishing
extract-data: false
python-version: "3.10"
- name: Build wheel and source distribution
run: python -m build
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/test_build_documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ jobs:
- uses: ./.github/actions/install_eitprocessing
with:
dependencies: docs
extract-data: false
python-version: "3.10"
- name: Link notebooks
run: ln -s ../../notebooks docs/examples
Expand Down
34 changes: 28 additions & 6 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
name: Build and Test

env:
ZENODO_RECORD_ID: 17423608
on:
push:
branches:
Expand All @@ -14,7 +15,6 @@ on:
branches:
- main
- develop

jobs:
test_and_build:
if: github.event.pull_request.draft == false
Expand All @@ -25,16 +25,38 @@ jobs:
os: ["ubuntu-latest"]
python-version: ["3.10"]
name: Test and build for ${{ matrix.python-version }}, ${{ matrix.os }}
env:
EIT_PROCESSING_TEST_DATA: ${{ github.workspace }}/../eitprocessing_data/
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/install_eitprocessing
with:
dependencies: testing
extract-data: true
python-version: ${{ matrix.python-version }}
token: ${{ secrets.GITHUB_TOKEN }}
data-directory: ${{ env.EIT_PROCESSING_TEST_DATA }}
- name: Set data dir
shell: bash
id: set-path
run: echo "data_dir=$GITHUB_WORKSPACE/test_data" >> "$GITHUB_OUTPUT"
- name: Restore dataset cache
id: cache
uses: actions/cache@v4
with:
path: ${{ steps.set-path.outputs.data_dir }}
key: eitprocessing-testdata-zenodo.${{ env.ZENODO_RECORD_ID }}
- name: Install zenodo-get
if: steps.cache.outputs.cache-hit != 'true'
run: python3 -m pip install zenodo-get
shell: bash
- name: Download test dataset
shell: bash
if: steps.cache.outputs['cache-hit'] != 'true'
run: |
mkdir -p ${{ steps.set-path.outputs.data_dir }}
cd ${{ steps.set-path.outputs.data_dir }}
zenodo_get $ZENODO_RECORD_ID
cd -
- name: Uninstall zenodo-get
if: steps.cache.outputs['cache-hit'] != 'true'
run: python3 -m pip uninstall --yes zenodo-get
shell: bash
- name: Run pytest
run: pytest -v
42 changes: 38 additions & 4 deletions README.dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Please follow these steps:

1. (**important**) announce your plan to the rest of the community _before you start working_. This announcement should be in the form of a (new) issue;
1. (**important**) wait until some kind of consensus is reached about your idea being a good idea;
1. if needed, fork the repository to your own Github profile and create your own feature branch off of the latest master commit. While working on your feature branch, make sure to stay up to date with the master branch by pulling in changes, possibly from the 'upstream' repository (follow the instructions [here](https://help.github.com/articles/configuring-a-remote-for-a-fork/) and [here](https://help.github.com/articles/syncing-a-fork/));
1. if needed, fork the repository to your own GitHub profile and create your own feature branch off of the latest master commit. While working on your feature branch, make sure to stay up to date with the master branch by pulling in changes, possibly from the 'upstream' repository (follow the instructions [here](https://help.github.com/articles/configuring-a-remote-for-a-fork/) and [here](https://help.github.com/articles/syncing-a-fork/));
1. make sure the existing tests still work by running `pytest` (see also [here](#testing-locally));
1. add your own tests (if necessary);
1. update or expand the documentation;
Expand All @@ -31,7 +31,7 @@ readibility or simplicity is more important than absolute correctness.
It is hard to define the precise balance we are looking for, so instead we will refer
to the [Zen of python](https://peps.python.org/pep-0020/).

Note that all contrubtions to this project will be published under our [Apache 2.0 licence]
Note that all contributions to this project will be published under our [Apache 2.0 licence]
(<https://github.com/EIT-ALIVE/eitprocessing/blob/main/LICENSE>).

#### Docstrings
Expand All @@ -49,7 +49,7 @@ place to start. This extension is currently in preview, but seems to work more r
#### Branch naming convention

Please try to adhere to the following branch naming convention:
<github-issue-number>_<brief-description>_<username>.
`<github-issue-number>_<brief-description>_<username>`.
E.g., `042_life_universe_everything_douglasadams`.

This allows, at a single glance, to see in the issue that you're
Expand Down Expand Up @@ -98,6 +98,38 @@ the CI.
Make sure you have developer options installed as described in the [README](README.md)
(otherwise run: `pip install -e .[dev]` on the repository folder in your environment)

##### Downloading test data
Some tests require access to test data. You can download the test data from Zenodo via the button below. Note that for
some reason downloading all files at ones results in a corrupted zip file. Please download the files one by one.

[![](https://zenodo.org/badge/DOI/10.5281/zenodo.17423608.svg)](https://doi.org/10.5281/zenodo.17423608)

Test data should reside in the `test_data/` folder in the root of the repository.

Alternatively, use zenodo-get to download the data directly into the `test_data/` folder:

Using `uv`:

```shell
mkdir -p test_data
cd test_data
uv tool run zenodo_get 17423608
cd -
```

Using `pip`:

```shell
pip install zenodo-get
mkdir -p test_data
cd test_data
zenodo_get 17423608
cd -
```


##### Running tests

For testing all you need to do is run:

```shell
Expand All @@ -120,7 +152,9 @@ coverage report

We use [ruff](https://docs.astral.sh/ruff/) for linting, sorting imports and formatting of python (notebook) files. The configurations of `ruff` are set in [pyproject.toml](pyproject.toml) file.

If you are using VS code, please install and activate the [Ruff extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) to automatically format and check linting.
If you are using VS code, please install and activate the [Ruff
extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) to automatically format and check
linting. Make sure to use the ruff version installed in your environment.

Otherwise, please ensure check both linting (`ruff fix .`) and formatting (`ruff format .`) before requesting a review.

Expand Down
9 changes: 9 additions & 0 deletions eitprocessing/datahandling/eitdata.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,15 @@ def __post_init__(self, suppress_simulated_warning: bool) -> None:
self.path = self.path[0]

self.name = self.name or self.label
old_sample_frequency = self.sample_frequency
self.sample_frequency = float(self.sample_frequency)
if self.sample_frequency != old_sample_frequency:
msg = (
"Sample frequency could not be correctly converted from "
f"{old_sample_frequency} ({type(old_sample_frequency)}) to "
f"{self.sample_frequency:.1f} (float)."
)
raise TypeError(msg)

if (lv := len(self.pixel_impedance)) != (lt := len(self.time)):
msg = f"The number of time points ({lt}) does not match the number of pixel impedance values ({lv})."
Expand Down
Loading