MESMER-MTP: precipitation by sarasita · Pull Request #837 · MESMER-group/mesmer

sarasita · 2026-01-09T13:35:58Z

Closes #xxx
Tests added
Fully documented, including CHANGELOG.rst

…using cdo remapbil,r20x20 /net/atmos/data/cmip6-ng/pr/mon/g025/pr_mon_IPSL-CM6A-LR_historical_r1i1p1f1_g025.nc /home/scsarah/mesmer/tests/test-data/calibrate-coarse-grid/cmip6-ng/pr/mon/g025/pr_mon_IPSL-CM6A-LR_historical_r1i1p1f1_g025.nc

…ry pretty yet

for more information, see https://pre-commit.ci

…e emulations

for more information, see https://pre-commit.ci

…ation process for emulations

for more information, see https://pre-commit.ci

codecov · 2026-01-13T14:24:11Z

Codecov Report

❌ Patch coverage is 24.42748% with 198 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.78%. Comparing base (0162c58) to head (8468527).

Files with missing lines	Patch %	Lines
mesmer/stats/_regularized_glm.py	19.35%	50 Missing ⚠️
mesmer/stats/_xarray_pipelines.py	21.31%	48 Missing ⚠️
mesmer/stats/_principal_component_decomposition.py	16.98%	44 Missing ⚠️
mesmer/stats/_xarray_kde.py	27.90%	31 Missing ⚠️
mesmer/stats/_xarray_transformers.py	34.21%	25 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #837      +/-   ##
==========================================
- Coverage   99.49%   90.78%   -8.72%     
==========================================
  Files          30       35       +5     
  Lines        1995     2257     +262     
==========================================
+ Hits         1985     2049      +64     
- Misses         10      208     +198

Flag	Coverage Δ
unittests	`90.78% <24.42%> (-8.72%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mathause

You were not in the office therefore I ask here:

What is the difference between the example_mesmer_mtp and example_mesmer_mtp_precipitation notebooks? I see that the first one does not load pr, but doesn't the "p" in "mtp" stand for precipitation?
There seems to be two ways to calculate a PCA:
1. the fit_principal_components function (which applies a StandardScaler and then the PCA); used in the example_mesmer_mtp notebook.
2. the XarrayPipeline, where you explicitely pass a StandardScaler and PCA as steps; used in the example_mesmer_mtp_precipitation notebook
Are these two ways equal? Do we need to keep both? Or which one can we remove?

mathause · 2026-01-15T13:58:11Z

mesmer/stats/_principal_component_decomposition.py

+            "std_scale": (X.dims[1], std.scale_),
+            "std_mean": (X.dims[1], std.mean_),
+            "std_var": (X.dims[1], std.var_),


I find it not very obvious that these belong to the StandardScaler.

mathause · 2026-01-15T14:16:41Z

mesmer/stats/_principal_component_decomposition.py

+    T = _transform_principal_component_decomposition_xr(X=X, params=params)
+
+    return T
+
+
+def _transform_principal_component_decomposition_xr(


(there is not much point to having a wrapper function here)

mathause · 2026-01-15T14:17:23Z

mesmer/stats/_regularized_glm.py

+from mesmer._core.utils import _ignore_warnings
+
+
+# haven't properly commented this yet - WIP


Suggested change

# haven't properly commented this yet - WIP

mathause · 2026-01-15T14:19:06Z

mesmer/stats/_regularized_glm.py

+        n_years = tas.sizes["year"]
+        n_closest = closest_locations.sizes["closest_gridcells"]
+        n_cov = 1 + 2 * n_closest


I have not looked at this in details but this is very use case specific - check if we could move this out of the function somehow.

mathause · 2026-01-15T14:20:15Z

mesmer/stats/_xarray_pipelines.py

+            return param
+
+
+# class XarrayPipeline:


can we delete this commented block?

sarasita · 2026-01-19T09:38:34Z

Related to your initial questions:

example_mesmer_mtp: contains the temperature emulation technique I used in the appendix of the MESMER-M-TP paper; at the time, a working MESMER-M version was not available, but we needed to asses the errors in the precipitation emulations when the framework was forced with emulated (rather than actual) temperature emulations.
example_mesmer_mtp_precipitation: contains the actual precipitation emulation framework.
two ways to compute a pca: that is correct. I find it more intuitive to have a class-based approach, however, upon first implementing in March 25 Victoria suggested to have the structure follow the Yeo-Johnson transformer module which, at the time, consisted of individual functions. Kept both until we decide what we prefer.

for more information, see https://pre-commit.ci

… implementation is still missing tests

for more information, see https://pre-commit.ci

sarasita · 2026-02-26T10:56:58Z

Rewrote parts of the code related to the precipitation emulations:

adjusted the xarray transformer and added tests
rewrote the Gamma GLM (changes from statsmodel implementation to sklearn implementation, exchanged elasticNet against Ridge, added k-fold cross-validation, code is now much faster)

The variability routine has not been changed yet.

for more information, see https://pre-commit.ci

mathause · 2026-03-06T08:44:58Z

mesmer/stats/_regularized_glm.py

+    def __init__(
+        self,
+        design_rule,
+        alpha_grid,


Can alpha_grid be an argument to fit? Then you don't need to "dummy initialize" it on L236.

But it's a argument to __init__ in GammaRegressor so for consistency we could also keep it...

Mh... I suppose either way would be fine.

ok let's keep the current approach the 'ugly' part is only internal

sarasita and others added 10 commits January 13, 2025 10:06

Added functionality for performing a PCA. Very preliminary and not ve…

89e8379

…ry pretty yet

[pre-commit.ci] auto fixes from pre-commit.com hooks

f8666dc

for more information, see https://pre-commit.ci

Preliminary version of a script for generating mesmer-m-tp temperatur…

1a887f0

…e emulations

[pre-commit.ci] auto fixes from pre-commit.com hooks

6aa64b8

for more information, see https://pre-commit.ci

Integrating MESMER-M-TP part-1 (temperature integration): Added gener…

dd107ef

…ation process for emulations

Added style changes

280c2d5

first working draft

a201b92

Merge branch 'main' into mesmermtp_precipitation

d12984d

[pre-commit.ci] auto fixes from pre-commit.com hooks

41516be

for more information, see https://pre-commit.ci

mathause changed the title ~~Mesmermtp precipitation~~ MESMER-MTP: precipitation Jan 12, 2026

remove duplicated notebooks

2ce820b

mathause added 3 commits January 13, 2026 15:42

move pr data to new location

64d74e7

remove output

ae35f10

update notebooks to current mesmer/ xarray

be0f0ad

mathause mentioned this pull request Jan 14, 2026

refactor _ignore_warnings #839

Merged

mathause added 3 commits January 14, 2026 15:54

Merge branch 'main' into mesmermtp_precipitation

2c20be8

apply _ignore_warnings

f164511

small updates

8468527

mathause reviewed Jan 15, 2026

View reviewed changes

mathause mentioned this pull request Jan 18, 2026

add monthly precipitation data for tests #838

Open

3 tasks

mathause and others added 7 commits January 19, 2026 18:58

Merge branch 'main' into mesmermtp_precipitation

4ef816e

speed up data loading and stacking

2c4a4e6

[pre-commit.ci] auto fixes from pre-commit.com hooks

fa77591

for more information, see https://pre-commit.ci

remove commented code block

af0d1c4

Merge branch 'main' into mesmermtp_precipitation

2a6b192

fix forgotten parens

376a66e

small restructuring

227b886

mathause and others added 4 commits February 17, 2026 14:30

small refactoring

b6b6f4c

n_cov -> n_params

820b019

example_mesmer_mtp_precipitation.ipynb: some refactorings and cleanups

6bf8812

[pre-commit.ci] auto fixes from pre-commit.com hooks

86543c2

for more information, see https://pre-commit.ci

mathause mentioned this pull request Feb 19, 2026

on the elastic net regression #841

Open

sarasita and others added 2 commits February 26, 2026 11:33

Changes to the Xarray Transformer and the GLM implementation. The GLM…

2d0f360

… implementation is still missing tests

[pre-commit.ci] auto fixes from pre-commit.com hooks

01b9244

for more information, see https://pre-commit.ci

sarasita closed this Feb 26, 2026

sarasita reopened this Feb 26, 2026

mathause and others added 6 commits March 4, 2026 12:57

Merge branch 'main' into mesmermtp_precipitation

fee6952

use best_alpha instead of best_model internally

aaddfe9

clear output from notebook

8c71c1e

small updates to the notebook

e204f7a

X_var had to be transposed

98e6c6e

[pre-commit.ci] auto fixes from pre-commit.com hooks

11f57da

for more information, see https://pre-commit.ci

mathause reviewed Mar 6, 2026

View reviewed changes

mathause added 3 commits March 9, 2026 17:50

update comment

84e2a12

Merge branch 'main' into mesmermtp_precipitation

9d29750

Merge branch 'main' into mesmermtp_precipitation

8154e3c

		from mesmer._core.utils import _ignore_warnings


		# haven't properly commented this yet - WIP

Conversation

sarasita commented Jan 9, 2026

Uh oh!

codecov bot commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mathause left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sarasita commented Jan 19, 2026

Uh oh!

sarasita commented Feb 26, 2026

Uh oh!

mathause Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Jan 13, 2026 •

edited

Loading

mathause left a comment •

edited

Loading

mathause Mar 6, 2026 •

edited

Loading