Skip to content

Fix specparam NaN handling and add flexible z-score normalization options#84

Merged
pablomc88 merged 2 commits intomainfrom
feature/zscore-after-epoch
Feb 26, 2026
Merged

Fix specparam NaN handling and add flexible z-score normalization options#84
pablomc88 merged 2 commits intomainfrom
feature/zscore-after-epoch

Conversation

@AlejandroOrozcoValero
Copy link
Contributor

  1. Fix ValueError in specparam peak selection

Problem: When computing power spectra with small nperseg values, some EEG signals produce all-NaN peak power arrays. The np.nanargmax() call on these arrays raises ValueError: All-NaN slice encountered, causing pipeline failures.

Solution: Added validation to check if the peak power array contains all NaN values before calling nanargmax(). If the array is all-NaN, peak selection is safely skipped.

Impact: Prevents pipeline crashes when processing signals with invalid power spectra. Samples with all-NaN peaks have no peak selected, which is the correct behavior. Valid samples are unaffected.

Modified file: ncpi/Features.py (lines 1343-1348)

  1. Add per-epoch z-scoring and last epoch exclusion options

New configuration options:

zscore_after_epoch - Apply z-score normalization after epoching instead of before. This enables per-epoch normalization where each epoch is independently scaled to mean=0 and std=1. Default: False.
exclude_last_epoch - Exclude the last epoch from each time series. Useful for discarding potentially incomplete epochs when signal length is not perfectly divisible by epoch length. Default: False.

Implementation details:

  • Modified processing pipeline to support conditional z-score application based on the zscore_after_epoch flag
  • Added _exclude_last_epoch() method with proper handling of grouped time series
  • Updated pipeline to apply last epoch exclusion after epoching but before optional post-epoch normalization

Modified file: ncpi/EphysDatasetParser.py

  • Config options: lines 183-184
  • Pipeline logic: lines 237-260
  • New method: lines 959-978

…idation to check if peak power array (pws) is all-NaN before calling np.nanargmax(). This prevents ValueError when processing EEG signals with small nperseg that produces invalid power spectra.
Add two new configuration options to ParseConfig:
- zscore_after_epoch: Apply z-score normalization after spoching instead of before (useful for per-epoch normalization)
- exclude_last_epoch: Exclude the last epoch from each time series (useful when signal length is not divisible by epoch length)
@pablomc88 pablomc88 merged commit 67162ea into main Feb 26, 2026
1 check passed
@pablomc88 pablomc88 deleted the feature/zscore-after-epoch branch February 26, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants