Multi yoy integration - combines PRs #456, #460 and #464 at once. (#470)

* add keyword 'label' to degradation_timeseries_plot, enabling 'left' and 'center' labeling options.

* Update changelog, add pytests, update sphinx documentation

* fix flake8 grumbles

* update pytests to include axes limits

* fix flake8 grumbles

* add 'label' input option to `degradation_year_on_year`. Fixes #459

* add pytests and update changelog.

* flake8 grumbles

* Minor updates to setup.py (constrain scipy<1.16) and refactor degradation_test

* Custom fix for Pandas < 2.0.0 which can't average two columns of timestamps.

* flake8 grumbles

* keep TZ-aware timestamps.  Update pytests to specifically test _avg_timestamp_old_Pandas

* flake8 grumbles

* try to UTC localize the pytest...

* Add .asfreq() to get pytests to agree

* switch to calendar.timegm to hopefully remove TZ issues..

* regardless of uncertainty_method, return calc_info{'YoY_values')

* update _right dt labels to correct _left labels in degradation_year_on_year

* update _avg_timestamp_old_Pandas to allow for numeric index instead of timestamp

* add left label option to degradation_year_on_year

* update degradation_year_on_year, index set to either left, center or right. Consistent with #394 - multi_yoy

* update return for default = none uncertainty option

* degradation_year_on_year - go back to single return when uncertainty_value = None to avoid breaking pytests.

* add multi-year aggregation of slopes in degradation_year_on_year

* add multi_yoy kwarg in degradation_year_on_year to toggle the multi-YoY function.

* update plotting for detailed=True, allow usage_of_points > 2

* flake8 grumbles

* update plotting detailed=True for  (even) and (odd) number of points coloring

* To allow multi_yoy=True in plotting.degradation_timeseries_plot, resample.mean() the YoY_values.

* flake8 grumbles

* Add warning to degradation_timeseries_plot when multi_YoY=True

* update to warning message in plotting.degradation_timeseries_plot

* fix flake8 grumbles

* nbval fixes from qnguyen345-bare_except_error

* Add pandas 3.0 futurewarning handling

* Try again to solve pandas3.0 futurewarning

* attempt 3 to fix nbval

* Add infer_objects to remove futurewarning

* minor inline comment update

* update plotting tests to be relative value, update ordering of module import in plotting.py, per Copilot review.

* update inline comments and whatsnew docs

* Clean up inline comments per Copilot review

* added multi-YoY pytest - still need to catch warnings

* Add a warnings.catch_warnings to the plotting pytest

* flake8 grumbles

* add multi-YoY=True pytest

* updated changelog

* update changelog

* implement copilot suggestions

* linting

* use s instead of ns for pandas 3 compatibility

* update pandas version comparison

* set matplotlib non-gui backend for tests

* set dtype resolution based on pandas version

* linting import order

* boost degradation.py test coverage

* update changelog

* exclude coverage reports with suffixes

* Update rdtools/degradation.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* remove label=None handling, rely on default 'right' behavior

* refactor dt_center  tz handling for old pandas

* simplify _avg_timestamp_old_Pandas

* degradation_timeseries_plot: change rolling median min_periods to rolling_days / 4.

* remove degradation_timeseries_plot(label=) and just default to center=True

* update sensor_analysis() and clearsky_analysis() docstrings to discuss passing `label=right` kwargs

* flake8 updates

* Initial commit - multi-YoY notebook

* pretty-print notebook dataframes with tabulate.

* update notebook requirements to silence pandas warnings

* add multi-yoy nb to tutorials

* fix nblink path

* Change the yoy_values index to be named 'dt'.  Add new illustrations at the end of the multi-YoY notebook.

---------

Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com>
Co-authored-by: martin-springer <martinspringer.ms@gmail.com>
Co-authored-by: Martin Springer <97482055+martin-springer@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Author: 4 people

.github/workflows/nbval.yaml

@@ -28,8 +28,8 @@ jobs:
         pip install --timeout=300 -r requirements.txt -r docs/notebook_requirements.txt .[test]
     - name: Run notebook and check output
       run: |
-        # --sanitize-with:  pre-process text to remove irrelevant differences (e.g. warning filepaths)
-        pytest --nbval docs/${{ matrix.notebook-file }} --sanitize-with docs/nbval_sanitization_rules.cfg
+        # --nbval-sanitize-with:  pre-process text to remove irrelevant differences (e.g. warning filepaths)
+        pytest --nbval --nbval-sanitize-with docs/nbval_sanitization_rules.cfg docs/${{ matrix.notebook-file }}
     - name: Run notebooks again, save files
       run: |
         pip install nbconvert[webpdf]

.gitignore

@@ -7,9 +7,10 @@
 # ignore byte compiled files
 *.py[co]
 
-# ignore coveralls yaml, coverge dir
+# ignore coveralls yaml, coverage dir
 .coveralls.yml
 .coverage
+.coverage.*
 htmlcov/
 
 # ignore test cache

docs/Multi-year_on_year_example.ipynb

@@ -62377,7 +62377,7 @@
     "# Visualize the results\n",
     "ta_new_filter.plot_degradation_summary('sensor', summary_title='Sensor-based degradation results',\n",
     "                                   scatter_ymin=0.5, scatter_ymax=1.1,\n",
-    "                                   hist_xmin=-30, hist_xmax=45);\n",
+    "                                   hist_xmin=-30, hist_xmax=45)\n",
     "plt.show()"
    ]
   },
@@ -62492,7 +62492,7 @@
     "# Visualize the results\n",
     "ta_stuck_filter.plot_degradation_summary('sensor', summary_title='Sensor-based degradation results',\n",
     "                                         scatter_ymin=0.5, scatter_ymax=1.1,\n",
-    "                                         hist_xmin=-30, hist_xmax=45);\n",
+    "                                         hist_xmin=-30, hist_xmax=45)\n",
     "plt.show()"
    ]
   },

docs/TrendAnalysis_example.ipynb

@@ -200,7 +200,7 @@
     "ax.plot(df.index, df.soiling, 'o', alpha=0.01)\n",
     "#ax.set_ylim(0,1500)\n",
     "fig.autofmt_xdate()\n",
-    "ax.set_ylabel('soiling signal');\n",
+    "ax.set_ylabel('soiling signal')\n",
     "df['power'] = df['power_ac'] * df['soiling']\n",
     "\n",
     "plt.show()"

docs/TrendAnalysis_example_NSRDB.ipynb

@@ -30,7 +30,7 @@ nbconvert==7.17.0
 nbformat==5.10.4
 nest-asyncio==1.6.0
 notebook==7.2.2
-numexpr==2.10.1
+numexpr==2.10.2
 pandocfilters==1.5.1
 parso==0.8.4
 pexpect==4.9.0
@@ -45,6 +45,7 @@ qtconsole==5.5.2
 Send2Trash==1.8.3
 simplegeneric==0.8.1
 soupsieve==2.6
+tabulate==0.9.0
 terminado==0.18.1
 testpath==0.6.0
 tinycss2==1.2.1
@@ -53,3 +54,4 @@ traitlets==5.14.3
 wcwidth==0.2.13
 webencodings==0.5.1
 widgetsnbextension==4.0.11
+

docs/notebook_requirements.txt

@@ -1,5 +1,6 @@
 RdTools Change Log
 ==================
+.. include:: changelog/pending.rst
 .. include:: changelog/v3.1.0.rst
 .. include:: changelog/v3.0.1.rst
 .. include:: changelog/v3.0.0.rst

docs/sphinx/source/changelog.rst

@@ -0,0 +1,61 @@
+*************************
+v3.2.0 (X, X, 2026)
+*************************
+
+Enhancements
+------------
+* :py:func:`~rdtools.degradation.degradation_year_on_year` has new parameter ``label=``
+  to return the calc_info['YoY_values'] as either right labeled (default), left or
+  center labeled. (:issue:`459`)
+* :py:func:`~rdtools.plotting.degradation_timeseries_plot` now defaults to rolling
+  median, centered on the timestamp (pd.rolling(center=True)).
+  (:issue:`455`)
+* :py:func:`~rdtools.degradation.degradation_year_on_year` has new parameter ``multi_yoy``
+  (default False) to trigger multiple YoY degradation calculations similar to Hugo Quest et
+  al 2023. In this mode, instead of a series of 1-year duration slopes, 2-year, 3-year etc
+  slopes are also included.  calc_info['YoY_values'] returns a non-monotonic index
+  in this mode due to multiple overlapping annual slopes.  (:issue:`394`)
+* :py:func:`~rdtools.plotting.degradation_timeseries_plot` now supports ``multi_yoy=True``
+  data by resampling overlapping YoY values to their mean. A warning is issued when this
+  resampling occurs.  (:issue:`394`)
+* :py:func:`~rdtools.plotting.degradation_summary_plots` ``detailed=True`` mode now
+  properly handles points used odd vs even number of times (not just 0, 1, 2).
+  (:issue:`394`)
+* :py:func:`~rdtools.degradation.degradation_year_on_year` now returns
+  ``calc_info['YoY_times']`` DataFrame with ``dt_right``, ``dt_center``, and ``dt_left``
+  columns for each YoY slope.  (:issue:`459`)
+* Added new example notebook ``docs/Multi-year_on_year_example.ipynb`` demonstrating the
+  ``label='center'`` and ``multi_yoy=True`` features of
+  :py:func:`~rdtools.degradation.degradation_year_on_year`.  (:issue:`394`)
+
+Bug Fixes
+---------
+* Fixed ``usage_of_points`` calculation in :py:func:`~rdtools.degradation.degradation_year_on_year`
+  to properly handle ``multi_yoy=True`` mode with overlapping slopes.  (:issue:`394`)
+
+
+Maintenance
+-----------
+* Added ``_avg_timestamp_old_Pandas`` helper function for pandas <2.0 compatibility
+  when calculating center labels.
+* Fixed nbval workflow command syntax (``--sanitize-with`` to ``--nbval-sanitize-with``).
+* Improved pandas 3.0 compatibility with datetime resolution handling.
+* Updated ``docs/notebook_requirements.txt`` to require ``numexpr>=2.10.2`` and
+  ``tabulate>=0.9.0`` to satisfy pandas' optional dependency minimum versions and
+  avoid related warnings/errors.
+
+Testing
+-------
+* Added tests for error handling paths in :py:mod:`~rdtools.degradation`:
+  ``classical_decomposition`` missing/irregular data, ``year_on_year`` circular block
+  validation, no valid pairs error, and ``_mk_test`` edge cases (no trend, ties,
+  decreasing).
+* Added test for ``multi_yoy=True`` parameter in ``degradation_year_on_year``.
+* Set matplotlib backend to ``Agg`` in test ``conftest.py`` to avoid tkinter issues.
+
+
+Contributors
+------------
+* Chris Deline (:ghuser:`cdeline`)
+* Martin Springer (:ghuser:`martin-springer`)
+

docs/sphinx/source/changelog/pending.rst

@@ -26,3 +26,4 @@ This page shows example usage of the RdTools analysis functions.
     examples/TrendAnalysis_example
     examples/TrendAnalysis_example_NSRDB
     examples/system_availability_example
+    examples/Multi-year_on_year_example

docs/sphinx/source/examples.rst

@@ -0,0 +1,3 @@
+{
+    "path": "../../../Multi-year_on_year_example.ipynb"
+}