Skip to content

Comments

Release 3.1.0#485

Open
martin-springer wants to merge 19 commits intomasterfrom
development
Open

Release 3.1.0#485
martin-springer wants to merge 19 commits intomasterfrom
development

Conversation

@martin-springer
Copy link
Collaborator

@martin-springer martin-springer commented Feb 11, 2026

This pull request implements the v3.1.0 release, focusing on improved compatibility with Python 3.13, pandas 3.0, numpy 2.x, and xgboost 3.x, as well as enhancements to the clipping filter and warning messages. It also removes deprecated normalization functions and updates documentation and requirements. The most significant changes are summarized below.

Enhancements and Feature Changes

  • The clip_filter in TrendAnalysis._filter() now supports using pv_energy when pv_power is unavailable, and enforces a median time step of 60 minutes or less for clipping detection, raising a ValueError otherwise. This improves support for sub-hourly, energy-based analyses and ensures data quality. [1] [2]
  • Added stacklevel to all warnings.warn() calls so warning messages now point to user code rather than internal library code, improving user experience and debugging. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Compatibility and Requirements Updates

  • Updated the test matrix to include Python 3.13 and removed Python 3.9, and excluded clip_filter tests for unsupported environments.
  • Updated requirements to ensure compatibility with Python 3.13, pandas 3.0, numpy 2.x, xgboost 3.x, and other dependencies. Upper version restrictions for pandas and numpy were removed, and many package versions were bumped for compatibility and security.
  • Fixed pandas 3.0 compatibility in normalization.py by using .total_seconds() instead of .view('int64') for datetime calculations, and updated Timedelta strings from 'd' to 'D' to address deprecation warnings. [1] [2] [3]
  • Fixed numpy 2.x compatibility in soiling.py by using .item() and explicit indexing to extract scalars.
  • Fixed xgboost 3.x compatibility in filtering.xgboost_clip_filter() by using xgb.DMatrix with explicit feature names for model prediction. [1] [2] [3]

Deprecations and Removals

  • Removed the deprecated normalization.delta_index and normalization.check_series_frequency functions (deprecated since v2.0.0); private helpers remain for internal use. [1] [2] [3]

Documentation

  • Added the v3.1.0 changelog and updated example notebooks to use the modern temperature coefficient (gamma_pdc). [1] [2]
  • Improved nbval sanitization rules in documentation to handle stacklevel changes in warnings and updated regex indices.

Please refer to the new changelog/v3.1.0.rst for a detailed list of all enhancements, bug fixes, and requirements changes.

  • Code changes are covered by tests
  • Code changes have been evaluated for compatibility/integration with TrendAnalysis
  • New functions added to __init__.py
  • API.rst is up to date, along with other sphinx docs pages
  • Example notebooks are rerun and differences in results scrutinized
  • Updated changelog

martin-springer and others added 9 commits February 4, 2026 16:26
* remove pvlib restriction

* run tests with pvlib 0.13.1

* re-run TA NSRDB notebook

* update changelog

* restrict pandas to <3.0.0

* add nbval workflow filterwarnings

* update changelog

* try limiting numpy version for eager tests

* update python versions in test matrix

* increase pandas requirement to 2.2.3 for python 3.13 compatibility

* increase scipy version for py 3.13 compatibility

* update changelog

* increase h5py requirement for py 3.13 compatiblity

* update changelog

* increase scikit-learn version for py 3.13 compatibility

* update changelog

* update plotly for py 3.13 compatibility

* update setuptools-scm for py 3.13 support

* update six to support py 3.13

* update statsmodels for py 3.13 support

* update threadpoolctl for py 3.13 support

* update tomli for py 3.13 support

* update typing_extensions for py 3.13

* update urllib3 for py 3.13

* update xgboost for py 3.13

* update fonttools for py 3.13

* update idna for py 3.13 support

* update joblib for py 3.13

* updated kiwisolver for py 3.13

* updated matplotlib for py 3.13

* update packaging for py 3.13

* update patsy for py 3.13 support

* update Pillow for py 3.13

* update pyparsing for py 3.13

* update pytz for py 3.13

* update changelog

* try new scikit-learn version

* scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility

* update changelog

* bump urllib3 to satisfy dependabot

* update changelog
* remove pvlib restriction

* run tests with pvlib 0.13.1

* re-run TA NSRDB notebook

* update changelog

* restrict pandas to <3.0.0

* add nbval workflow filterwarnings

* update changelog

* try limiting numpy version for eager tests

* update python versions in test matrix

* increase pandas requirement to 2.2.3 for python 3.13 compatibility

* increase scipy version for py 3.13 compatibility

* update changelog

* increase h5py requirement for py 3.13 compatiblity

* update changelog

* increase scikit-learn version for py 3.13 compatibility

* update changelog

* update plotly for py 3.13 compatibility

* update setuptools-scm for py 3.13 support

* update six to support py 3.13

* update statsmodels for py 3.13 support

* update threadpoolctl for py 3.13 support

* update tomli for py 3.13 support

* update typing_extensions for py 3.13

* update urllib3 for py 3.13

* update xgboost for py 3.13

* update fonttools for py 3.13

* update idna for py 3.13 support

* update joblib for py 3.13

* updated kiwisolver for py 3.13

* updated matplotlib for py 3.13

* update packaging for py 3.13

* update patsy for py 3.13 support

* update Pillow for py 3.13

* update pyparsing for py 3.13

* update pytz for py 3.13

* update changelog

* try new scikit-learn version

* scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility

* update changelog

* bump urllib3 to satisfy dependabot

* update changelog

* set pv power or energy for clip filter

* add sub-hourly check to TA before applying clip_filter

* add test for sub-hourly check in TA clip_filter

* update changelog

* fix linting

* disable clip_filter in existing tests that use daily aggregated data
* remove pvlib restriction

* run tests with pvlib 0.13.1

* re-run TA NSRDB notebook

* update changelog

* restrict pandas to <3.0.0

* add nbval workflow filterwarnings

* update changelog

* try limiting numpy version for eager tests

* update python versions in test matrix

* increase pandas requirement to 2.2.3 for python 3.13 compatibility

* increase scipy version for py 3.13 compatibility

* update changelog

* increase h5py requirement for py 3.13 compatiblity

* update changelog

* increase scikit-learn version for py 3.13 compatibility

* update changelog

* update plotly for py 3.13 compatibility

* update setuptools-scm for py 3.13 support

* update six to support py 3.13

* update statsmodels for py 3.13 support

* update threadpoolctl for py 3.13 support

* update tomli for py 3.13 support

* update typing_extensions for py 3.13

* update urllib3 for py 3.13

* update xgboost for py 3.13

* update fonttools for py 3.13

* update idna for py 3.13 support

* update joblib for py 3.13

* updated kiwisolver for py 3.13

* updated matplotlib for py 3.13

* update packaging for py 3.13

* update patsy for py 3.13 support

* update Pillow for py 3.13

* update pyparsing for py 3.13

* update pytz for py 3.13

* update changelog

* try new scikit-learn version

* scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility

* update changelog

* bump urllib3 to satisfy dependabot

* update changelog

* set pv power or energy for clip filter

* add sub-hourly check to TA before applying clip_filter

* add test for sub-hourly check in TA clip_filter

* update changelog

* fix linting

* disable clip_filter in existing tests that use daily aggregated data

* fix numpy compatibility

* degradation pandas 3.0.0 compatibility

* soiling pandas 3.0.0 compatibility

* update requirements in setup.py

* remove deprecated public functions

* xgboost > 3.x compatibility

* re-work normalization for pandas 3.0 compatibility

* update changelog

* fix scikit-learn requirement

* simplify tz handling in _interpolate_series

* resolve merge conflicts

* fix changelog
Improved logo with transparent background
(115775-Solar Rd Tools Logo Design_horizontal.png is the one with trasparent background but it has crappy resolution)
 same logo and resolution as docs>sphinx>source>images>logo_horizontal_highres.png but with transparent background (I would perhaps place it in that folder instead)
* remove pvlib restriction

* run tests with pvlib 0.13.1

* re-run TA NSRDB notebook

* update changelog

* restrict pandas to <3.0.0

* add nbval workflow filterwarnings

* update changelog

* try limiting numpy version for eager tests

* update python versions in test matrix

* increase pandas requirement to 2.2.3 for python 3.13 compatibility

* increase scipy version for py 3.13 compatibility

* update changelog

* increase h5py requirement for py 3.13 compatiblity

* update changelog

* increase scikit-learn version for py 3.13 compatibility

* update changelog

* update plotly for py 3.13 compatibility

* update setuptools-scm for py 3.13 support

* update six to support py 3.13

* update statsmodels for py 3.13 support

* update threadpoolctl for py 3.13 support

* update tomli for py 3.13 support

* update typing_extensions for py 3.13

* update urllib3 for py 3.13

* update xgboost for py 3.13

* update fonttools for py 3.13

* update idna for py 3.13 support

* update joblib for py 3.13

* updated kiwisolver for py 3.13

* updated matplotlib for py 3.13

* update packaging for py 3.13

* update patsy for py 3.13 support

* update Pillow for py 3.13

* update pyparsing for py 3.13

* update pytz for py 3.13

* update changelog

* try new scikit-learn version

* scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility

* update changelog

* bump urllib3 to satisfy dependabot

* update changelog

* set pv power or energy for clip filter

* add sub-hourly check to TA before applying clip_filter

* add test for sub-hourly check in TA clip_filter

* update changelog

* fix linting

* disable clip_filter in existing tests that use daily aggregated data

* fix numpy compatibility

* degradation pandas 3.0.0 compatibility

* soiling pandas 3.0.0 compatibility

* update requirements in setup.py

* remove deprecated public functions

* xgboost > 3.x compatibility

* re-work normalization for pandas 3.0 compatibility

* update changelog

* fix scikit-learn requirement

* Add stacklevel to warnings.warn() calls

Specify stacklevel parameter in all warnings.warn() calls to ensure
warning messages point to user code rather than rdtools internals:

- stacklevel=2: For public functions/methods called directly by users
- stacklevel=3: For private helper methods called by public methods

Files updated:
- analysis_chains.py: 6 warnings
- filtering.py: 2 warnings
- soiling.py: 4 warnings
- plotting.py: 3 warnings
- normalization.py: 1 warning
- availability.py: 1 warning
- clearsky_temperature.py: 2 warnings

* sanitize line numbers in warning messages

* re-run notebook

* Update nbval sanitization rules to ignore warning traceback code lines

The stacklevel parameter changes which line of code is shown in warning
tracebacks. Add regex rule to sanitize these lines so tests pass regardless
of stacklevel values.

* simplify tz handling in _interpolate_series

* resolve merge conflicts
* remove pvlib restriction

* run tests with pvlib 0.13.1

* re-run TA NSRDB notebook

* update changelog

* restrict pandas to <3.0.0

* add nbval workflow filterwarnings

* update changelog

* try limiting numpy version for eager tests

* update python versions in test matrix

* increase pandas requirement to 2.2.3 for python 3.13 compatibility

* increase scipy version for py 3.13 compatibility

* update changelog

* increase h5py requirement for py 3.13 compatiblity

* update changelog

* increase scikit-learn version for py 3.13 compatibility

* update changelog

* update plotly for py 3.13 compatibility

* update setuptools-scm for py 3.13 support

* update six to support py 3.13

* update statsmodels for py 3.13 support

* update threadpoolctl for py 3.13 support

* update tomli for py 3.13 support

* update typing_extensions for py 3.13

* update urllib3 for py 3.13

* update xgboost for py 3.13

* update fonttools for py 3.13

* update idna for py 3.13 support

* update joblib for py 3.13

* updated kiwisolver for py 3.13

* updated matplotlib for py 3.13

* update packaging for py 3.13

* update patsy for py 3.13 support

* update Pillow for py 3.13

* update pyparsing for py 3.13

* update pytz for py 3.13

* update changelog

* try new scikit-learn version

* scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility

* update changelog

* bump urllib3 to satisfy dependabot

* update changelog

* set pv power or energy for clip filter

* add sub-hourly check to TA before applying clip_filter

* add test for sub-hourly check in TA clip_filter

* update changelog

* fix linting

* disable clip_filter in existing tests that use daily aggregated data

* fix numpy compatibility

* degradation pandas 3.0.0 compatibility

* soiling pandas 3.0.0 compatibility

* update requirements in setup.py

* remove deprecated public functions

* xgboost > 3.x compatibility

* re-work normalization for pandas 3.0 compatibility

* update changelog

* fix scikit-learn requirement

* Initial plan

* Update temperature coefficient from -0.005 to -0.0034 in PVDAQ notebooks

Co-authored-by: martin-springer <97482055+martin-springer@users.noreply.github.com>

* Update pending.rst changelog for temperature coefficient change

Co-authored-by: martin-springer <97482055+martin-springer@users.noreply.github.com>

* re-run notebooks

* re-run notebooks

* re-run notebooks in terminal

---------

Co-authored-by: martin-springer <martinspringer.ms@gmail.com>
Co-authored-by: mspringe <martin.springer@nrel.gov>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: martin-springer <97482055+martin-springer@users.noreply.github.com>
* Bump notebook from 7.2.1 to 7.2.2 in /docs

Bumps [notebook](https://github.com/jupyter/notebook) from 7.2.1 to 7.2.2.
- [Release notes](https://github.com/jupyter/notebook/releases)
- [Changelog](https://github.com/jupyter/notebook/blob/@jupyter-notebook/tree@7.2.2/CHANGELOG.md)
- [Commits](https://github.com/jupyter/notebook/compare/@jupyter-notebook/tree@7.2.1...@jupyter-notebook/tree@7.2.2)

---
updated-dependencies:
- dependency-name: notebook
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump tornado from 6.4.2 to 6.5.1 in /docs

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.4.2 to 6.5.1.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](tornadoweb/tornado@v6.4.2...v6.5.1)

---
updated-dependencies:
- dependency-name: tornado
  dependency-version: 6.5.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* statsmodels 0.14.4 is not able to handle the latest scipy.

* Bump jinja2 from 3.1.5 to 3.1.6 in /docs

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.5...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-version: 3.1.6
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump requests from 2.32.3 to 2.32.4

Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](psf/requests@v2.32.3...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump urllib3 from 2.2.2 to 2.5.0

Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.2 to 2.5.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@2.2.2...2.5.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.5.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* try setup.py now that statsmodels has a new release.

* Update changelog

* Update release date

* Bump pillow from 10.4.0 to 12.1.1

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.4.0 to 12.1.1.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](python-pillow/Pillow@10.4.0...12.1.1)

---
updated-dependencies:
- dependency-name: pillow
  dependency-version: 12.1.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* update changelog

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: cdeline <chris.deline@nrel.gov>
Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov>
Co-authored-by: martin-springer <martinspringer.ms@gmail.com>
@martin-springer martin-springer changed the title Development Release 3.1.0 Feb 11, 2026
@codecov-commenter
Copy link

codecov-commenter commented Feb 11, 2026

Codecov Report

❌ Patch coverage is 98.03922% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 96.18%. Comparing base (076ebff) to head (aac4765).

Files with missing lines Patch % Lines
rdtools/normalization.py 96.66% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master     #485   +/-   ##
=======================================
  Coverage   96.17%   96.18%           
=======================================
  Files          12       12           
  Lines        2276     2280    +4     
=======================================
+ Hits         2189     2193    +4     
  Misses         87       87           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

* Bump nbconvert from 7.16.4 to 7.17.0 in /docs

Bumps [nbconvert](https://github.com/jupyter/nbconvert) from 7.16.4 to 7.17.0.
- [Release notes](https://github.com/jupyter/nbconvert/releases)
- [Changelog](https://github.com/jupyter/nbconvert/blob/main/CHANGELOG.md)
- [Commits](jupyter/nbconvert@v7.16.4...v7.17.0)

---
updated-dependencies:
- dependency-name: nbconvert
  dependency-version: 7.17.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* bump tinycss2

* tinycss2==1.2.1

* fix restrictions in setup.py

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: martin-springer <martinspringer.ms@gmail.com>
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR packages the v3.1.0 release, primarily updating rdtools for compatibility with newer Python and scientific Python ecosystems (Python 3.13, pandas 3.x/numpy 2.x/xgboost 3.x), improving clip_filter behavior in TrendAnalysis, and making warnings point to user code via stacklevel.

Changes:

  • Update dependencies/CI to target newer runtimes and libraries (incl. Python 3.13) and adjust warning filtering/sanitization for notebooks.
  • Modernize pandas/numpy/xgboost compatibility in core time-series utilities (normalization, soiling, filtering) and add clip_filter frequency validation.
  • Update docs/changelog/API docs to reflect removals of deprecated normalization functions and new release notes.

Reviewed changes

Copilot reviewed 19 out of 24 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
setup.py Bumps test requirements and adjusts install requirements/classifiers for newer Python ecosystem.
setup.cfg Adds pytest filterwarnings entry for a new deprecation warning.
requirements.txt Pins updated dependency versions for compatibility/security.
.github/workflows/pytest.yaml Updates CI matrix to test Python 3.10–3.13.
rdtools/analysis_chains.py Enhances clip_filter to use energy if needed and enforces ≤60 min median timestep; adds stacklevel to warnings.
rdtools/filtering.py Updates XGBoost clipping prediction path for xgboost 3.x.
rdtools/normalization.py Refactors datetime arithmetic to use total_seconds() and adds edge handling; adjusts interpolation/resolution behavior.
rdtools/soiling.py Updates warnings stacklevel and adjusts numpy/pandas scalar/axis usage for newer versions.
rdtools/plotting.py Adds stacklevel to experimental-warning emissions.
rdtools/degradation.py Updates Timedelta strings to avoid pandas deprecation warnings.
rdtools/clearsky_temperature.py Adds stacklevel to warnings.
rdtools/availability.py Adds stacklevel to warnings.
rdtools/test/analysis_chains_test.py Adjusts soiling expectations, disables clip filter for daily data tests, adds new clip frequency error test.
rdtools/test/normalization_pvwatts_test.py Updates datetime calculations for pandas 3.x behavior.
rdtools/test/degradation_test.py Updates Timedelta string casing.
docs/sphinx/source/changelog/v3.1.0.rst Adds v3.1.0 changelog entry.
docs/sphinx/source/changelog.rst Includes the new v3.1.0 changelog page.
docs/sphinx/source/api.rst Removes deprecated normalization functions from API docs.
docs/notebook_requirements.txt Updates notebook toolchain pins.
docs/nbval_sanitization_rules.cfg Expands sanitization rules to accommodate stacklevel-related traceback diffs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

martin-springer and others added 9 commits February 12, 2026 09:02
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@martin-springer martin-springer marked this pull request as ready for review February 12, 2026 15:11
@martin-springer
Copy link
Collaborator Author

@mdeceglie - Copilot's review had a couple good suggestions that I added with this PR. It's ready for your review and merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants