Fix specparam NaN handling and add flexible z-score normalization options#84
Merged
Fix specparam NaN handling and add flexible z-score normalization options#84
Conversation
…idation to check if peak power array (pws) is all-NaN before calling np.nanargmax(). This prevents ValueError when processing EEG signals with small nperseg that produces invalid power spectra.
Add two new configuration options to ParseConfig: - zscore_after_epoch: Apply z-score normalization after spoching instead of before (useful for per-epoch normalization) - exclude_last_epoch: Exclude the last epoch from each time series (useful when signal length is not divisible by epoch length)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem: When computing power spectra with small nperseg values, some EEG signals produce all-NaN peak power arrays. The
np.nanargmax()call on these arrays raisesValueError: All-NaN slice encountered, causing pipeline failures.Solution: Added validation to check if the peak power array contains all NaN values before calling
nanargmax(). If the array is all-NaN, peak selection is safely skipped.Impact: Prevents pipeline crashes when processing signals with invalid power spectra. Samples with all-NaN peaks have no peak selected, which is the correct behavior. Valid samples are unaffected.
Modified file:
ncpi/Features.py(lines 1343-1348)New configuration options:
zscore_after_epoch - Apply z-score normalization after epoching instead of before. This enables per-epoch normalization where each epoch is independently scaled to mean=0 and std=1.
Default: False.exclude_last_epoch - Exclude the last epoch from each time series. Useful for discarding potentially incomplete epochs when signal length is not perfectly divisible by epoch length.
Default: False.Implementation details:
zscore_after_epochflag_exclude_last_epoch()method with proper handling of grouped time seriesModified file:
ncpi/EphysDatasetParser.py