Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
53657ca
lock
cosunae Sep 22, 2025
b8c6440
add cartopy
cosunae Sep 22, 2025
4759a30
generate raw
cosunae Sep 22, 2025
4db180e
add plot rule
cosunae Sep 22, 2025
db9d4db
fix init_time
cosunae Sep 22, 2025
7faa37b
adding plotting files
cosunae Sep 22, 2025
82e0a2c
adding plotting files
cosunae Sep 22, 2025
0b29429
generate colormaps from NCL color table files
clairemerker Sep 26, 2025
bc34ec0
split colormap configs for variables and colormap loader
clairemerker Sep 30, 2025
6cb8bc0
add tests for colormap_loader.py and colormap_defaults.py
clairemerker Sep 30, 2025
9ab55d8
revert change commited accidentally
clairemerker Sep 30, 2025
12ed14c
fix typo
clairemerker Sep 30, 2025
89b3ae8
fix imports for code and tests to fit the project's structure
clairemerker Sep 30, 2025
87596a1
adapt plotting to earthkit-plots
clairemerker Oct 2, 2025
206bc07
improve tests
clairemerker Oct 2, 2025
4c57ffd
Merge branch 'main' into plotting
clairemerker Oct 2, 2025
193cb25
Merge branch 'main' into plotting
dnerini Oct 7, 2025
a6fcea6
Refactor plot rule and adopt marimo (#54)
dnerini Oct 8, 2025
6c5188d
Merge branch 'main' into plotting
dnerini Oct 8, 2025
73544e0
Fix paths
dnerini Oct 8, 2025
aefe77b
Revert using separate folder for notebooks
dnerini Oct 8, 2025
b7642cc
Specify individual showcases dates
dnerini Oct 9, 2025
69f3dc9
Switch to GRIB and fix projection issues (#60)
OpheliaMiralles Oct 14, 2025
b109eaa
Merge branch 'main' into plotting
dnerini Oct 14, 2025
7262e32
Set path to eccodes defintions
dnerini Oct 14, 2025
77b3893
Add Switzerland region to outputs
dnerini Oct 14, 2025
8fa6c67
Batch plot jobs in larger submissions
dnerini Oct 15, 2025
98b9ac9
Merge branch 'main' into plotting
dnerini Oct 15, 2025
4dd752e
Merge branch 'main' into plotting
dnerini Oct 20, 2025
421b478
Merge branch 'main' into plotting
dnerini Oct 20, 2025
61b871e
Merge branch 'main' into plotting
dnerini Oct 21, 2025
d2be5ca
Fix pre-commits
dnerini Oct 21, 2025
9df35a6
Revert change in co1e config
dnerini Oct 21, 2025
257c748
Parse steps
dnerini Oct 21, 2025
ef00ccc
Do not hardcode leadtimes
dnerini Oct 21, 2025
7306694
Plot wind speed
dnerini Oct 21, 2025
4b56b35
Showcase M-2 model
dnerini Oct 21, 2025
e50e6b2
Fix pre-commit
dnerini Oct 21, 2025
bf07cf7
Add global outputs to interpolator showcase
dnerini Oct 22, 2025
a20ae75
Fix wrong inference config...
dnerini Oct 22, 2025
d018591
Plot meteograms
dnerini Oct 22, 2025
d6fd3e3
Merge branch 'main' into plot-time-series
dnerini Nov 7, 2025
b841d19
Fix merge commit
dnerini Nov 7, 2025
dd36d53
Fix path to ICON definitions
dnerini Nov 7, 2025
e8e0f0c
Merge branch 'main' into plot-time-series
dnerini Jan 9, 2026
b7a9ab1
Fix meteograms
dnerini Jan 9, 2026
36f36f2
Fix pre-commit
dnerini Jan 9, 2026
a1e36b7
Reset uv.lock to match main
dnerini Jan 9, 2026
e24707f
Plot baseline
dnerini Jan 9, 2026
f670d25
Add wind
dnerini Jan 9, 2026
d1ed4da
Add station obs from peakweather
dnerini Jan 13, 2026
35ac830
Add missing output
dnerini Jan 13, 2026
bd24df9
Can't plot TOT_PREC since it's not produced by interpolator
dnerini Jan 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions config/showcase.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ runs:

baselines:
- baseline:
baseline_id: COSMO-E
baseline_id: COSMO-E-1h
label: COSMO-E
root: /store_new/mch/msopr/ml/COSMO-E
steps: 0/120/6
root: /store_new/mch/msopr/ml/COSMO-E_hourly
steps: 0/120/1

stratification:
regions:
Expand All @@ -43,7 +43,7 @@ stratification:

analysis:
label: COSMO KENDA
analysis_zarr: /scratch/mch/fzanetta/data/anemoi/datasets/mch-co2-an-archive-0p02-2015-2020-6h-v3-pl13.zarr
analysis_zarr: /scratch/mch/fzanetta/data/anemoi/datasets/mch-co2-an-archive-0p02-2015-2020-1h-v3-pl13.zarr

locations:
output_root: output/
Expand Down
4 changes: 4 additions & 0 deletions pyproject.toml

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To what extent is the model output comparable to station observations? (Effective model resolution vs. point-scale observations, elevation difference, ...). Would a later integration with the MEC make sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess for most (all) parameters the MEC doesn't do anything more sophisticated than just nearest-neighbour / representative grid point + altitude correction. But true, maybe at least the constant lapse-rate correction would be useful.

Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ dependencies = [
"earthkit-plots",
"marimo>=0.16.5",
"geopandas>=0.14.0",
"peakweather",
]

[project.optional-dependencies]
Expand Down Expand Up @@ -58,3 +59,6 @@ packages = [
"src/verification",
"src/data_input"
]

[tool.uv.sources]
peakweather = { git = "https://github.com/MeteoSwiss/PeakWeather.git" }
23 changes: 22 additions & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 8 additions & 1 deletion workflow/Snakefile

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why leave away precipitation?

Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,16 @@ rule showcase_all:
/ "showcases/{run_id}/{init_time}/{init_time}_{param}_{region}.gif",
init_time=[t.strftime("%Y%m%d%H%M") for t in REFTIMES],
run_id=collect_all_candidates(),
param=["TOT_PREC", "T_2M", "SP_10M"],
param=["T_2M", "SP_10M"],
region=["globe", "europe", "switzerland"],
),
expand(
OUT_ROOT / "showcases/{run_id}/{init_time}/{init_time}_{param}_{sta}.png",
init_time=[t.strftime("%Y%m%d%H%M") for t in REFTIMES],
run_id=collect_all_candidates(),
param=["T_2M", "SP_10M"],
sta=["GVE", "KLO", "LUG"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parameters and stations should probably be added to the config.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment still stands. I suggest to make parameters and stations configurable for the meteogram. Could also do this for the other showcase plots (i.e. use the same parameters).

),
expand(
OUT_ROOT / "data/runs/{run_id}/summary.md",
run_id=collect_all_candidates(),
Expand Down
11 changes: 11 additions & 0 deletions workflow/rules/data.smk
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,14 @@ if "extract_cosmo1e" in config.get("include-optional-rules", []):
--steps {params.steps} \
> {log} 2>&1
"""


rule download_obs_from_peakweather:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be more future-proof to download obs from OGD?

localrule: True
output:
peakweather=directory(OUT_ROOT / "data/observations/peakweather"),
run:
from peakweather.dataset import PeakWeatherDataset

# Download the data from Huggingface
ds = PeakWeatherDataset(root=output.peakweather)
46 changes: 46 additions & 0 deletions workflow/rules/plot.smk
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,52 @@ include: "common.smk"
import pandas as pd


def _use_first_baseline_zarr(wc):
"""Get the first available baseline zarr for the given init time."""
for baseline_id in BASELINE_CONFIGS:
root = BASELINE_CONFIGS[baseline_id].get("root")
year = wc.init_time[2:4]
baseline_zarr = f"{root}/FCST{year}.zarr"
if Path(baseline_zarr).exists():
return baseline_zarr
raise ValueError(f"No baseline zarr found for init time {wc.init_time}")


rule plot_meteogram:
input:
script="workflow/scripts/plot_meteogram.mo.py",
inference_okfile=rules.execute_inference.output.okfile,
analysis_zarr=config["analysis"].get("analysis_zarr"),
baseline_zarr=lambda wc: _use_first_baseline_zarr(wc),
peakweather_dir=rules.download_obs_from_peakweather.output.peakweather,
output:
OUT_ROOT / "showcases/{run_id}/{init_time}/{init_time}_{param}_{sta}.png",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a concious decision to produce meteogram plots per run_id? I can see that this ties in nicely with the showcases, but in this case, I would argue we do the meteograms only for an individual run (without a baseline). Otherwise, we should strive to support an arbitrary number of run_ids and baselines in the same (multi-panel) plot.

# localrule: True
resources:
slurm_partition="postproc",
cpus_per_task=1,
runtime="5m",
params:
grib_out_dir=lambda wc: (
Path(OUT_ROOT) / f"data/runs/{wc.run_id}/{wc.init_time}/grib"
).resolve(),
shell:
"""
export ECCODES_DEFINITION_PATH=$(realpath .venv/share/eccodes-cosmo-resources/definitions)
python {input.script} \
--forecast {params.grib_out_dir} --analysis {input.analysis_zarr} \
--baseline {input.baseline_zarr} --peakweather {input.peakweather_dir} \
--date {wildcards.init_time} --outfn {output[0]} \
--param {wildcards.param} --station {wildcards.sta}
# interactive editing (needs to set localrule: True and use only one core)
# marimo edit {input.script} -- \
# --forecast {params.grib_out_dir} --analysis {input.analysis_zarr} \
# --baseline {input.baseline_zarr} --peakweather {input.peakweather_dir} \
# --date {wildcards.init_time} --outfn {output[0]} \
# --param {wildcards.param} --station {wildcards.sta}
"""


rule plot_forecast_frame:
input:
script="workflow/scripts/plot_forecast_frame.mo.py",
Expand Down
Loading
Loading