Conversation
* remove pvlib restriction * run tests with pvlib 0.13.1 * re-run TA NSRDB notebook * update changelog * restrict pandas to <3.0.0 * add nbval workflow filterwarnings * update changelog * try limiting numpy version for eager tests * update python versions in test matrix * increase pandas requirement to 2.2.3 for python 3.13 compatibility * increase scipy version for py 3.13 compatibility * update changelog * increase h5py requirement for py 3.13 compatiblity * update changelog * increase scikit-learn version for py 3.13 compatibility * update changelog * update plotly for py 3.13 compatibility * update setuptools-scm for py 3.13 support * update six to support py 3.13 * update statsmodels for py 3.13 support * update threadpoolctl for py 3.13 support * update tomli for py 3.13 support * update typing_extensions for py 3.13 * update urllib3 for py 3.13 * update xgboost for py 3.13 * update fonttools for py 3.13 * update idna for py 3.13 support * update joblib for py 3.13 * updated kiwisolver for py 3.13 * updated matplotlib for py 3.13 * update packaging for py 3.13 * update patsy for py 3.13 support * update Pillow for py 3.13 * update pyparsing for py 3.13 * update pytz for py 3.13 * update changelog * try new scikit-learn version * scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility * update changelog * bump urllib3 to satisfy dependabot * update changelog
* remove pvlib restriction * run tests with pvlib 0.13.1 * re-run TA NSRDB notebook * update changelog * restrict pandas to <3.0.0 * add nbval workflow filterwarnings * update changelog * try limiting numpy version for eager tests * update python versions in test matrix * increase pandas requirement to 2.2.3 for python 3.13 compatibility * increase scipy version for py 3.13 compatibility * update changelog * increase h5py requirement for py 3.13 compatiblity * update changelog * increase scikit-learn version for py 3.13 compatibility * update changelog * update plotly for py 3.13 compatibility * update setuptools-scm for py 3.13 support * update six to support py 3.13 * update statsmodels for py 3.13 support * update threadpoolctl for py 3.13 support * update tomli for py 3.13 support * update typing_extensions for py 3.13 * update urllib3 for py 3.13 * update xgboost for py 3.13 * update fonttools for py 3.13 * update idna for py 3.13 support * update joblib for py 3.13 * updated kiwisolver for py 3.13 * updated matplotlib for py 3.13 * update packaging for py 3.13 * update patsy for py 3.13 support * update Pillow for py 3.13 * update pyparsing for py 3.13 * update pytz for py 3.13 * update changelog * try new scikit-learn version * scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility * update changelog * bump urllib3 to satisfy dependabot * update changelog * set pv power or energy for clip filter * add sub-hourly check to TA before applying clip_filter * add test for sub-hourly check in TA clip_filter * update changelog * fix linting * disable clip_filter in existing tests that use daily aggregated data
* remove pvlib restriction * run tests with pvlib 0.13.1 * re-run TA NSRDB notebook * update changelog * restrict pandas to <3.0.0 * add nbval workflow filterwarnings * update changelog * try limiting numpy version for eager tests * update python versions in test matrix * increase pandas requirement to 2.2.3 for python 3.13 compatibility * increase scipy version for py 3.13 compatibility * update changelog * increase h5py requirement for py 3.13 compatiblity * update changelog * increase scikit-learn version for py 3.13 compatibility * update changelog * update plotly for py 3.13 compatibility * update setuptools-scm for py 3.13 support * update six to support py 3.13 * update statsmodels for py 3.13 support * update threadpoolctl for py 3.13 support * update tomli for py 3.13 support * update typing_extensions for py 3.13 * update urllib3 for py 3.13 * update xgboost for py 3.13 * update fonttools for py 3.13 * update idna for py 3.13 support * update joblib for py 3.13 * updated kiwisolver for py 3.13 * updated matplotlib for py 3.13 * update packaging for py 3.13 * update patsy for py 3.13 support * update Pillow for py 3.13 * update pyparsing for py 3.13 * update pytz for py 3.13 * update changelog * try new scikit-learn version * scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility * update changelog * bump urllib3 to satisfy dependabot * update changelog * set pv power or energy for clip filter * add sub-hourly check to TA before applying clip_filter * add test for sub-hourly check in TA clip_filter * update changelog * fix linting * disable clip_filter in existing tests that use daily aggregated data * fix numpy compatibility * degradation pandas 3.0.0 compatibility * soiling pandas 3.0.0 compatibility * update requirements in setup.py * remove deprecated public functions * xgboost > 3.x compatibility * re-work normalization for pandas 3.0 compatibility * update changelog * fix scikit-learn requirement * simplify tz handling in _interpolate_series * resolve merge conflicts * fix changelog
Improved logo with transparent background (115775-Solar Rd Tools Logo Design_horizontal.png is the one with trasparent background but it has crappy resolution) same logo and resolution as docs>sphinx>source>images>logo_horizontal_highres.png but with transparent background (I would perhaps place it in that folder instead)
* remove pvlib restriction * run tests with pvlib 0.13.1 * re-run TA NSRDB notebook * update changelog * restrict pandas to <3.0.0 * add nbval workflow filterwarnings * update changelog * try limiting numpy version for eager tests * update python versions in test matrix * increase pandas requirement to 2.2.3 for python 3.13 compatibility * increase scipy version for py 3.13 compatibility * update changelog * increase h5py requirement for py 3.13 compatiblity * update changelog * increase scikit-learn version for py 3.13 compatibility * update changelog * update plotly for py 3.13 compatibility * update setuptools-scm for py 3.13 support * update six to support py 3.13 * update statsmodels for py 3.13 support * update threadpoolctl for py 3.13 support * update tomli for py 3.13 support * update typing_extensions for py 3.13 * update urllib3 for py 3.13 * update xgboost for py 3.13 * update fonttools for py 3.13 * update idna for py 3.13 support * update joblib for py 3.13 * updated kiwisolver for py 3.13 * updated matplotlib for py 3.13 * update packaging for py 3.13 * update patsy for py 3.13 support * update Pillow for py 3.13 * update pyparsing for py 3.13 * update pytz for py 3.13 * update changelog * try new scikit-learn version * scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility * update changelog * bump urllib3 to satisfy dependabot * update changelog * set pv power or energy for clip filter * add sub-hourly check to TA before applying clip_filter * add test for sub-hourly check in TA clip_filter * update changelog * fix linting * disable clip_filter in existing tests that use daily aggregated data * fix numpy compatibility * degradation pandas 3.0.0 compatibility * soiling pandas 3.0.0 compatibility * update requirements in setup.py * remove deprecated public functions * xgboost > 3.x compatibility * re-work normalization for pandas 3.0 compatibility * update changelog * fix scikit-learn requirement * Add stacklevel to warnings.warn() calls Specify stacklevel parameter in all warnings.warn() calls to ensure warning messages point to user code rather than rdtools internals: - stacklevel=2: For public functions/methods called directly by users - stacklevel=3: For private helper methods called by public methods Files updated: - analysis_chains.py: 6 warnings - filtering.py: 2 warnings - soiling.py: 4 warnings - plotting.py: 3 warnings - normalization.py: 1 warning - availability.py: 1 warning - clearsky_temperature.py: 2 warnings * sanitize line numbers in warning messages * re-run notebook * Update nbval sanitization rules to ignore warning traceback code lines The stacklevel parameter changes which line of code is shown in warning tracebacks. Add regex rule to sanitize these lines so tests pass regardless of stacklevel values. * simplify tz handling in _interpolate_series * resolve merge conflicts
* remove pvlib restriction * run tests with pvlib 0.13.1 * re-run TA NSRDB notebook * update changelog * restrict pandas to <3.0.0 * add nbval workflow filterwarnings * update changelog * try limiting numpy version for eager tests * update python versions in test matrix * increase pandas requirement to 2.2.3 for python 3.13 compatibility * increase scipy version for py 3.13 compatibility * update changelog * increase h5py requirement for py 3.13 compatiblity * update changelog * increase scikit-learn version for py 3.13 compatibility * update changelog * update plotly for py 3.13 compatibility * update setuptools-scm for py 3.13 support * update six to support py 3.13 * update statsmodels for py 3.13 support * update threadpoolctl for py 3.13 support * update tomli for py 3.13 support * update typing_extensions for py 3.13 * update urllib3 for py 3.13 * update xgboost for py 3.13 * update fonttools for py 3.13 * update idna for py 3.13 support * update joblib for py 3.13 * updated kiwisolver for py 3.13 * updated matplotlib for py 3.13 * update packaging for py 3.13 * update patsy for py 3.13 support * update Pillow for py 3.13 * update pyparsing for py 3.13 * update pytz for py 3.13 * update changelog * try new scikit-learn version * scikit-learn version 1.7.2 for py 3.13 and xgboost compatibility * update changelog * bump urllib3 to satisfy dependabot * update changelog * set pv power or energy for clip filter * add sub-hourly check to TA before applying clip_filter * add test for sub-hourly check in TA clip_filter * update changelog * fix linting * disable clip_filter in existing tests that use daily aggregated data * fix numpy compatibility * degradation pandas 3.0.0 compatibility * soiling pandas 3.0.0 compatibility * update requirements in setup.py * remove deprecated public functions * xgboost > 3.x compatibility * re-work normalization for pandas 3.0 compatibility * update changelog * fix scikit-learn requirement * Initial plan * Update temperature coefficient from -0.005 to -0.0034 in PVDAQ notebooks Co-authored-by: martin-springer <97482055+martin-springer@users.noreply.github.com> * Update pending.rst changelog for temperature coefficient change Co-authored-by: martin-springer <97482055+martin-springer@users.noreply.github.com> * re-run notebooks * re-run notebooks * re-run notebooks in terminal --------- Co-authored-by: martin-springer <martinspringer.ms@gmail.com> Co-authored-by: mspringe <martin.springer@nrel.gov> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: martin-springer <97482055+martin-springer@users.noreply.github.com>
* Bump notebook from 7.2.1 to 7.2.2 in /docs Bumps [notebook](https://github.com/jupyter/notebook) from 7.2.1 to 7.2.2. - [Release notes](https://github.com/jupyter/notebook/releases) - [Changelog](https://github.com/jupyter/notebook/blob/@jupyter-notebook/tree@7.2.2/CHANGELOG.md) - [Commits](https://github.com/jupyter/notebook/compare/@jupyter-notebook/tree@7.2.1...@jupyter-notebook/tree@7.2.2) --- updated-dependencies: - dependency-name: notebook dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Bump tornado from 6.4.2 to 6.5.1 in /docs Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.4.2 to 6.5.1. - [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst) - [Commits](tornadoweb/tornado@v6.4.2...v6.5.1) --- updated-dependencies: - dependency-name: tornado dependency-version: 6.5.1 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * statsmodels 0.14.4 is not able to handle the latest scipy. * Bump jinja2 from 3.1.5 to 3.1.6 in /docs Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@3.1.5...3.1.6) --- updated-dependencies: - dependency-name: jinja2 dependency-version: 3.1.6 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Bump requests from 2.32.3 to 2.32.4 Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.32.4. - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.32.3...v2.32.4) --- updated-dependencies: - dependency-name: requests dependency-version: 2.32.4 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Bump urllib3 from 2.2.2 to 2.5.0 Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.2 to 2.5.0. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@2.2.2...2.5.0) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.5.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * try setup.py now that statsmodels has a new release. * Update changelog * Update release date * Bump pillow from 10.4.0 to 12.1.1 Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.4.0 to 12.1.1. - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@10.4.0...12.1.1) --- updated-dependencies: - dependency-name: pillow dependency-version: 12.1.1 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * update changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> Co-authored-by: martin-springer <martinspringer.ms@gmail.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #485 +/- ##
=======================================
Coverage 96.17% 96.18%
=======================================
Files 12 12
Lines 2276 2280 +4
=======================================
+ Hits 2189 2193 +4
Misses 87 87 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
* Bump nbconvert from 7.16.4 to 7.17.0 in /docs Bumps [nbconvert](https://github.com/jupyter/nbconvert) from 7.16.4 to 7.17.0. - [Release notes](https://github.com/jupyter/nbconvert/releases) - [Changelog](https://github.com/jupyter/nbconvert/blob/main/CHANGELOG.md) - [Commits](jupyter/nbconvert@v7.16.4...v7.17.0) --- updated-dependencies: - dependency-name: nbconvert dependency-version: 7.17.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump tinycss2 * tinycss2==1.2.1 * fix restrictions in setup.py --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: martin-springer <martinspringer.ms@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR packages the v3.1.0 release, primarily updating rdtools for compatibility with newer Python and scientific Python ecosystems (Python 3.13, pandas 3.x/numpy 2.x/xgboost 3.x), improving clip_filter behavior in TrendAnalysis, and making warnings point to user code via stacklevel.
Changes:
- Update dependencies/CI to target newer runtimes and libraries (incl. Python 3.13) and adjust warning filtering/sanitization for notebooks.
- Modernize pandas/numpy/xgboost compatibility in core time-series utilities (
normalization,soiling,filtering) and addclip_filterfrequency validation. - Update docs/changelog/API docs to reflect removals of deprecated normalization functions and new release notes.
Reviewed changes
Copilot reviewed 19 out of 24 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| setup.py | Bumps test requirements and adjusts install requirements/classifiers for newer Python ecosystem. |
| setup.cfg | Adds pytest filterwarnings entry for a new deprecation warning. |
| requirements.txt | Pins updated dependency versions for compatibility/security. |
| .github/workflows/pytest.yaml | Updates CI matrix to test Python 3.10–3.13. |
| rdtools/analysis_chains.py | Enhances clip_filter to use energy if needed and enforces ≤60 min median timestep; adds stacklevel to warnings. |
| rdtools/filtering.py | Updates XGBoost clipping prediction path for xgboost 3.x. |
| rdtools/normalization.py | Refactors datetime arithmetic to use total_seconds() and adds edge handling; adjusts interpolation/resolution behavior. |
| rdtools/soiling.py | Updates warnings stacklevel and adjusts numpy/pandas scalar/axis usage for newer versions. |
| rdtools/plotting.py | Adds stacklevel to experimental-warning emissions. |
| rdtools/degradation.py | Updates Timedelta strings to avoid pandas deprecation warnings. |
| rdtools/clearsky_temperature.py | Adds stacklevel to warnings. |
| rdtools/availability.py | Adds stacklevel to warnings. |
| rdtools/test/analysis_chains_test.py | Adjusts soiling expectations, disables clip filter for daily data tests, adds new clip frequency error test. |
| rdtools/test/normalization_pvwatts_test.py | Updates datetime calculations for pandas 3.x behavior. |
| rdtools/test/degradation_test.py | Updates Timedelta string casing. |
| docs/sphinx/source/changelog/v3.1.0.rst | Adds v3.1.0 changelog entry. |
| docs/sphinx/source/changelog.rst | Includes the new v3.1.0 changelog page. |
| docs/sphinx/source/api.rst | Removes deprecated normalization functions from API docs. |
| docs/notebook_requirements.txt | Updates notebook toolchain pins. |
| docs/nbval_sanitization_rules.cfg | Expands sanitization rules to accommodate stacklevel-related traceback diffs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
@mdeceglie - Copilot's review had a couple good suggestions that I added with this PR. It's ready for your review and merge. |
This pull request implements the v3.1.0 release, focusing on improved compatibility with Python 3.13, pandas 3.0, numpy 2.x, and xgboost 3.x, as well as enhancements to the clipping filter and warning messages. It also removes deprecated normalization functions and updates documentation and requirements. The most significant changes are summarized below.
Enhancements and Feature Changes
clip_filterinTrendAnalysis._filter()now supports usingpv_energywhenpv_poweris unavailable, and enforces a median time step of 60 minutes or less for clipping detection, raising aValueErrorotherwise. This improves support for sub-hourly, energy-based analyses and ensures data quality. [1] [2]stacklevelto allwarnings.warn()calls so warning messages now point to user code rather than internal library code, improving user experience and debugging. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]Compatibility and Requirements Updates
clip_filtertests for unsupported environments.normalization.pyby using.total_seconds()instead of.view('int64')for datetime calculations, and updated Timedelta strings from'd'to'D'to address deprecation warnings. [1] [2] [3]soiling.pyby using.item()and explicit indexing to extract scalars.filtering.xgboost_clip_filter()by usingxgb.DMatrixwith explicit feature names for model prediction. [1] [2] [3]Deprecations and Removals
normalization.delta_indexandnormalization.check_series_frequencyfunctions (deprecated since v2.0.0); private helpers remain for internal use. [1] [2] [3]Documentation
gamma_pdc). [1] [2]Please refer to the new
changelog/v3.1.0.rstfor a detailed list of all enhancements, bug fixes, and requirements changes.__init__.py