Skip to content

fix remove_outlier_replicates to dismiss NaN rows from response colum…#420

Merged
stewarthe6 merged 20 commits into1.8.0from
bug_remove_outlier_replicates
Feb 25, 2026
Merged

fix remove_outlier_replicates to dismiss NaN rows from response colum…#420
stewarthe6 merged 20 commits into1.8.0from
bug_remove_outlier_replicates

Conversation

@paulsonak
Copy link
Collaborator

…n before removing outliers

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the remove_outlier_replicates function to handle NaN values in the response column before performing outlier detection. The fix prevents NaN values from affecting median calculations and ensures that only valid numeric values are considered when identifying outlier replicates.

Changes:

  • Added NaN filtering logic to remove rows with missing response values before outlier detection
  • Added print statement to report the number of rows removed due to missing values

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI commented Jan 29, 2026

@paulsonak I've opened a new pull request, #421, to work on those changes. Once the pull request is ready, I'll request review from you.

paulsonak and others added 8 commits January 29, 2026 12:51
remove log statement in preference of print statements like the rest of the module

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: paulsonak <64031328+paulsonak@users.noreply.github.com>
Added an empty value for the last entry in the CSV file to test that remove_outlier_replicates can handle NaN's
update logic to reflect removal of a NaN row
only print statement if rows were removed

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Document NaN removal in remove_outlier_replicates docstring
@codecov
Copy link

codecov bot commented Jan 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@             Coverage Diff             @@
##            1.8.0     #420       +/-   ##
===========================================
+ Coverage   23.57%   40.51%   +16.94%     
===========================================
  Files          50       50               
  Lines       13514    13518        +4     
===========================================
+ Hits         3186     5477     +2291     
+ Misses      10328     8041     -2287     
Flag Coverage Δ
unittests 40.51% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
atomsci/ddm/utils/curate_data.py 56.22% <100.00%> (+33.02%) ⬆️

... and 27 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@paulsonak
Copy link
Collaborator Author

This solves #419

stewarthe6 and others added 10 commits February 17, 2026 15:34
…emove_outlier_replicates. This is done using the capsys feature of pytest. This means that you should no longer call these test functions without pytest and should only test using pytest.
- Replace individual codecov uploads in each test job with artifact saves
- Add new coverage-merge job that combines all coverage reports
- Upload single merged coverage report to codecov instead of overwriting
- Fixes codecov not capturing integrative test coverage (e.g. test_curation_funcs)
- Ensures complete coverage picture from both unit and integrative tests

Co-authored-by: GitHub Copilot <copilot@github.com>
…lp in the future if coverage reports are missing
@stewarthe6
Copy link
Collaborator

Ok, this passes all tests and code conv is happy.

@stewarthe6 stewarthe6 merged commit 96bc75a into 1.8.0 Feb 25, 2026
8 checks passed
@stewarthe6 stewarthe6 deleted the bug_remove_outlier_replicates branch February 25, 2026 02:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants