Thermosteric sea-level rise patterns #4047

gregrmunday · 2025-05-20T21:20:56Z

Description

This PR calculates and evaluates patterns of thermometric sea-level rise from global thermal expansion and local sea-level changes for a range of CMIP6 models and scenarios.

Systematic calculation and evaluation of these patterns will allow for quick and easy pattern creation for future CMIP generations, and allows the patterns to be used in various sea-level rise emulation models.

Closes Thermosteric sea-level rise patterns #4046
Link to documentation: https://esmvaltool--4047.org.readthedocs.build/en/4047/recipes/recipe_steric_patterns.html

Before you get started

☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

🛠 This pull request has a descriptive title
🛠 Code is written according to the code quality guidelines
🛠 Documentation is available
🛠 Tests run successfully
🛠 The list of authors is up to date
🛠 Any changed dependencies have been added or removed correctly
🛠 All checks below this pull request were successful

New or updated recipe/diagnostic

🧪 Recipe runs successfully
🧪 Recipe is well documented
🧪 Figure(s) and data look as expected from literature
🛠 Provenance information has been added

gregrmunday · 2025-05-20T21:25:39Z

I know that I need a scientific and technical reviewer, I am on the hunt for both :)

gregrmunday · 2025-05-23T20:59:44Z

Hey @valeriupredoi! Was wondering if you could give me a hand with some CircleCI tests failing... it's not clear whether it has anything to do with my code or not. Cheers 😊

ehogan

Many thanks for your contribution @mo-gregmunday! 🥳

ehogan · 2025-07-17T12:06:19Z

esmvaltool/diag_scripts/steric_patterns/sub_funcs.py

@@ -0,0 +1,96 @@
+# (C) Crown Copyright 2022-2025, Met Office.


The code in this module looks very similar to what was included via #2785! Would it be possible to add a single copy of this code to esmvaltool/diag_scripts/shared and use the functions for both recipes, please? 😊

Would this require two additional PR's? One for adding the module to the shared folder and another for updating the other recipe? Or can I just do it in this PR?

esmvaltool/recipes/recipe_steric_patterns.yml

ehogan · 2025-07-17T12:46:44Z

esmvaltool/diag_scripts/steric_patterns/steric_patterns.py

+    fig.savefig(
+        Path(plot_path)
+        / f'detrended_{zostoga.attributes["source_id"]}.png', dpi=150)


Could esmvaltool.diag_scripts.shared.save_figure be used here? 😊

(there are other uses of this in this module!)

For consistency with UKCP18 and AR5, it looks like a quadratic fit is needed for the drift correction

@ehogan The save_figure function definitely could be used here but I think saving provenance for each of these figures is overkill, especially since none of them relate to any previously published work.

@mo-jenniferweeks I think we've discussed this and we agreed to linear drift correction was the correct technique here - let me know if I'm mistaken!

esmvaltool/diag_scripts/steric_patterns/steric_patterns.py

ehogan · 2025-07-17T12:52:37Z

esmvaltool/diag_scripts/steric_patterns/steric_patterns.py

+    np.save(
+        Path(work_path)
+        / f"zos_regression_{scenario}_{model}.npy", slopes)
+    np.save(Path(work_path) / f"zos_mask_{scenario}_{model}.npy", mask)


Could esmvaltool.diag_scripts.shared.save_data be used here? 😊

(there are other uses of this in this module!)

I think the function is designed to save iris cubes instead of NumPy arrays!

esmvaltool/diag_scripts/steric_patterns/steric_patterns.py

ehogan · 2025-07-17T13:21:42Z

Hey @valeriupredoi! Was wondering if you could give me a hand with some CircleCI tests failing... it's not clear whether it has anything to do with my code or not. Cheers 😊

@mo-gregmunday, run the following command in the directory containing your branch and commit the changes:

pre-commit run --files esmvaltool/diag_scripts/steric_patterns/*py

😊

ehogan · 2025-07-17T15:19:22Z

To download (almost all!) the required data for this recipe, I needed to use a specific ESGF node, which I did by creating a ~/.esmvaltool/esgf-pyclient.yml configuration file containing the following:

search_connection:
  urls:
    - 'https://esg-dn1.nsc.liu.se/esg-search'

I did this both at DKRZ and at the MO 👍

ehogan · 2025-07-17T15:20:38Z

Currently missing data:

2025-07-17 15:19:50,639 UTC [1831751] ERROR   Missing data for preprocessor steric_patterns/zostoga_370:
- Missing data for Dataset: zostoga, Omon, CMIP6, GISS-E2-2-G, ScenarioMIP, ssp370, r1i1p3f1, gn
2025-07-17 15:19:50,639 UTC [1831751] ERROR   Missing data for preprocessor steric_patterns/zostoga_585:
- Missing data for Dataset: zostoga, Omon, CMIP6, GISS-E2-2-G, ScenarioMIP, ssp585, r1i1p3f1, gn
2025-07-17 15:19:50,639 UTC [1831751] ERROR   Missing data for preprocessor steric_patterns/zos_370:
- Missing data for Dataset: zos, Omon, CMIP6, GISS-E2-2-G, ScenarioMIP, ssp370, r1i1p3f1, gn
2025-07-17 15:19:50,639 UTC [1831751] ERROR   Missing data for preprocessor steric_patterns/zos_585:
- Missing data for Dataset: zos, Omon, CMIP6, GISS-E2-2-G, ScenarioMIP, ssp585, r1i1p3f1, gn

ehogan · 2025-07-17T15:36:39Z

I have opened #4117 for the data issue 👍

ehogan · 2025-07-23T13:08:28Z

I tried again today to download data and I was able to download the data from LLNL 🥳

The recipe runs at the Met Office (using ESMValTool v2.12.0):

INFO    [2608676] Time for running the recipe was: 0:06:16.962714
INFO    [2608676] Maximum memory used (estimate): 54.6 GB
[...]
INFO    [2608676] Run was successful

However, there are no plots visible via the index.html file. This might be due to the directory structure within the plots directory.

I'm getting a failure on DKRZ:

Traceback (most recent call last):
  File "/home/b/b382148/software/ESMValTool-main/esmvaltool/diag_scripts/steric_patterns/steric_patterns.py", line 531, in <module>
    main(config)
  File "/home/b/b382148/software/ESMValTool-main/esmvaltool/diag_scripts/steric_patterns/steric_patterns.py", line 519, in main
    sf.parallelise(patterns, processes=8)(models, cfg)
  File "/home/b/b382148/software/ESMValTool-main/esmvaltool/diag_scripts/steric_patterns/sub_funcs.py", line 92, in easy_parallise
    result = pool.map_async(config_wrapper, sequence).get()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sw/spack-levante/miniforge3-24.11.3-0-Linux-x86_64-ftdezc/lib/python3.12/multiprocessing/pool.py", line 774, in get
    raise self._value
  File "/sw/spack-levante/miniforge3-24.11.3-0-Linux-x86_64-ftdezc/lib/python3.12/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
^^^^^^^^^^^^^^^
  File "/sw/spack-levante/miniforge3-24.11.3-0-Linux-x86_64-ftdezc/lib/python3.12/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
      ^^^^^^^^^^^^^^^^^
  File "/home/b/b382148/software/ESMValTool-main/esmvaltool/diag_scripts/steric_patterns/steric_patterns.py", line 489, in patterns
    slopes_arr, masks_arr = dyn_steric_regression(
  ^^^^^^^^^^^^^^^^^
  File "/home/b/b382148/software/ESMValTool-main/esmvaltool/diag_scripts/steric_patterns/steric_patterns.py", line 147, in dyn_steric_regression
    zostoga = zostoga[idx]
      ^^^^^^^^^^^^^^^^^
  File "/sw/spack-levante/miniforge3-24.11.3-0-Linux-x86_64-ftdezc/lib/python3.12/site-packages/iris/cube.py", line 3015, in __getitem__
    dimension_mapping, data = iris.util._slice_data_with_keys(cube_data, keys)
  ^^^^^^^^^^^^^^^^^
  File "/sw/spack-levante/miniforge3-24.11.3-0-Linux-x86_64-ftdezc/lib/python3.12/site-packages/iris/util.py", line 854, in _slice_data_with_keys
    data = data[this_slice]
  ^^^^^^^^^^^^^^^^^
  File "/sw/spack-levante/miniforge3-24.11.3-0-Linux-x86_64-ftdezc/lib/python3.12/site-packages/numpy/ma/core.py", line 3289, in __getitem__
    dout = self.data[indx]
  ^^^^^^^^^^^^^^^^^
IndexError: boolean index did not match indexed array along axis 0; size of axis is 3432 but size of corresponding boolean axis is 1032

@mo-gregmunday any thoughts? 🤔

mo-jenniferweeks · 2025-08-15T11:06:41Z

doc/sphinx/source/recipes/recipe_steric_patterns.rst

Should the title be sterodynamic sea level patterns? Or thermosteric and sterodynamic sea level change?

Would a figure like Figure 2 in Palmer et al., (2020) be possible? https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019EF001413

Add something on applying the methods of Palmer et al., (2020) which are rooted in AR5 methods. Could add something about Hermans et al., (2021) also using CMIP6 models with these models to project global mean sea level change and refer to that for the reader to explore the differences? Although, Hermans use a least squares quadratic fit for drift correction - same as Palmer et al. (2020)?

Could add what scenarios are available?

I've updated the title and added the available scenarios.

In theory a figure like Fig. 2 of Palmer et al., (2020) would be possible but I'd need to get historical data from all the models to calculate the 1986-2005 baseline. This would add quite a lot of extra processing and time for this particular PR. However it could definitely be added in the future if that's something you'd like to do :)

I'll get back to you on the references tomorrow :)

@mo-jenniferweeks I've added a note on Palmer et al., (2020) but I'll leave it to you to add in any other references and context you think might be relevant in another PR which should be easy to do! If you'd like to send me anything offline I can add it in here as well of course.

jlenh · 2025-08-20T13:41:16Z

Hej @mo-gregmunday, the freeze for the new release 2.13 is slowly approaching. Do you think you will have time to adress all the remaining comments and issues in the coming 1-2 weeks or should I bump this to the next release? :)

gregrmunday · 2025-08-22T10:05:35Z

Hej @mo-gregmunday, the freeze for the new release 2.13 is slowly approaching. Do you think you will have time to adress all the remaining comments and issues in the coming 1-2 weeks or should I bump this to the next release? :)

Hi @jlenh, I probably won't unfortunately so should probably be bumped to the next release, thanks for checking in!

…nto steric_patterns

gregrmunday · 2025-10-13T16:14:20Z

One more comment!

The logs contain multiple copies of the following message:

WARNING:py.warnings:/home/b/b382148/software/ESMValTool-main/esmvaltool/diag_scripts/steric_patterns/steric_patterns.py:280: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`.
  fig = plt.figure(figsize=(12, 5), layout="constrained")

Would it be possible to implement what is suggested in this message, please? 😊

I've sorted this out :)

gregrmunday · 2025-10-13T16:22:54Z

Hey @valeriupredoi! Was wondering if you could give me a hand with some CircleCI tests failing... it's not clear whether it has anything to do with my code or not. Cheers 😊

@mo-gregmunday, run the following command in the directory containing your branch and commit the changes:
pre-commit run --files esmvaltool/diag_scripts/steric_patterns/*py
😊

I've tried this and pushed the resulting changes up but it's still failing the 'commit' and 'ci/circleci: run_tests'...

gregrmunday · 2025-10-13T16:24:32Z

@ehogan @mo-jenniferweeks

Thank you so much for both of your very thorough reviews! I've addressed each of your comments as best I can. Please let me know if you'd like to see anything else from me :)

valeriupredoi · 2025-10-14T14:01:14Z

@gregrmunday I have just ruff-ed the diag for you (hope you don't mind!). The unofficial way is to run the command pre-commit run --files esmvaltool/diag_scripts/steric_patterns/*py once, figure out what;s it complaining for, then run it again force-fix it if you are OK with the rule; same with the rst docs entry via pre-commit run --all this time around. The correct way to handle Ruff barfs is to actually go fix if you accept the rule (as it should be done in most of the cases), but a quickfix is to run pre-commit twice 😁

gregrmunday · 2025-10-15T11:02:49Z

@gregrmunday I have just ruff-ed the diag for you (hope you don't mind!). The unofficial way is to run the command pre-commit run --files esmvaltool/diag_scripts/steric_patterns/*py once, figure out what;s it complaining for, then run it again force-fix it if you are OK with the rule; same with the rst docs entry via pre-commit run --all this time around. The correct way to handle Ruff barfs is to actually go fix if you accept the rule (as it should be done in most of the cases), but a quickfix is to run pre-commit twice 😁

Thanks @valeriupredoi !! This is useful to know, I'll keep this in mind for next time :)

Added sterodynamic patterns code + recipe

0797796

gregrmunday self-assigned this May 20, 2025

Updated some functions

edf0bdb

gregrmunday requested a review from mo-jenniferweeks May 21, 2025 15:07

gregrmunday and others added 10 commits May 23, 2025 12:54

Add documentation+

ee25f14

Codacy updates

ae2d52e

Codacy updates

9c4b97e

Codacy updates

bb580ca

Codacy updates

b147484

Codacy updates

f44596a

Codacy updates

b70c80b

Codacy update

4ae1e9f

Codacy update

035f329

Merge branch 'main' into steric_patterns

ac0eacd

gregrmunday requested a review from ehogan June 2, 2025 09:51

gregrmunday added 2 commits June 2, 2025 12:16

Merge branch 'main' into steric_patterns

cf3a9be

Merge branch 'main' into steric_patterns

875c296

ehogan added in technical review in scientific review new recipe Use this label if you are adding a new recipe labels Jul 17, 2025

ehogan added this to the v2.13.0 milestone Jul 17, 2025

ehogan requested changes Jul 17, 2025

View reviewed changes

ehogan mentioned this pull request Jul 17, 2025

esmvalcore.esgf._download.DownloadError: Failed to download the following files: Wrong SHA256 checksum for file #4117

Open

mo-jenniferweeks reviewed Aug 15, 2025

View reviewed changes

jlenh modified the milestones: v2.13.0, v2.14.0 Aug 22, 2025

gregrmunday and others added 10 commits October 12, 2025 16:46

PR review fixes

8fef977

Merge branch 'steric_patterns' of github.com:ESMValGroup/ESMValTool i…

1fdc9dc

…nto steric_patterns

Small update

b2f88c3

Updated detrend plot

ecab380

Updated docs

3dbc23c

Updated docs

8c6202b

Updated plots and docs

ef40617

Codacy fixes

4b9cec6

Codacy fixes

37356ac

Merge branch 'main' into steric_patterns

ef64933

gregrmunday added 3 commits October 13, 2025 17:14

Close figures

e7991dd

Pre-commit push

8dee810

After-fix

4b7d013

gregrmunday requested review from ehogan and mo-jenniferweeks October 13, 2025 16:24

gregrmunday and others added 4 commits October 13, 2025 18:08

Small fix

f7ef555

Small fix

62c912a

ruff formatting

dc33254

ruff doc entry

79856af

Merge branch 'main' into steric_patterns

7c57156

		@@ -0,0 +1,96 @@
		# (C) Crown Copyright 2022-2025, Met Office.

Uh oh!

Thermosteric sea-level rise patterns #4047

Are you sure you want to change the base?

Thermosteric sea-level rise patterns #4047

Uh oh!

Conversation

gregrmunday commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Before you get started

Checklist

Uh oh!

gregrmunday commented May 20, 2025

Uh oh!

gregrmunday commented May 23, 2025

Uh oh!

ehogan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mo-jenniferweeks Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ehogan commented Jul 17, 2025

Uh oh!

ehogan commented Jul 17, 2025

Uh oh!

ehogan commented Jul 17, 2025

Uh oh!

ehogan commented Jul 17, 2025

Uh oh!

ehogan commented Jul 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mo-jenniferweeks Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gregrmunday Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jlenh commented Aug 20, 2025

Uh oh!

gregrmunday commented Aug 22, 2025

Uh oh!

gregrmunday commented Oct 13, 2025

Uh oh!

gregrmunday commented May 20, 2025 •

edited

Loading

mo-jenniferweeks Aug 15, 2025 •

edited

Loading

mo-jenniferweeks Aug 15, 2025 •

edited

Loading

gregrmunday Oct 12, 2025 •

edited

Loading