Handle forecast and ensemble data in CLIMADA

[luseverin]

This ticket discusses ideas on how to better handle hazard data that are forecasts or/and ensembles (i.e. ensembles of simulations from climate models or forecast ensembles) in CLIMADA. This results from the discussion @Evelyn-M, @carmensteinmann, @timschmi95 and I had during the CLIMADA summer day 2025. See this note https://docs.google.com/document/d/14Vd0-w6M4iyRh6Lxzy00cAiOLT5RrFGLH3uvYdw4PfY/edit?tab=t.0#heading=h.jc0sdzqwue8a .

Need
We all thought that a better handling of forecast data (i.e. data that has an additional "leadtime" dimension) and/or ensemble data (i.e. data that has an additional "ensemble member" dimension) would be benificial. In particular, being able to have attributes that mark the leadtime of a hazard and the realization of an ensemble of simulations would allow selection and aggregation along those two dimensions. Those new attributes should also be propagated from the hazard object data to the impact object to allow similar features for the impact object.

Proposed implementention
We thought the easiest implementation would be to add two separate optional attributes to the hazard class:

• lead_time: array the lead time (in hours?) of the different hazard events. Dimension: centroids x events.
• ensemble_id: array integers identifying the realization from an ensemble of simulations. Dimension: centroids x events.

The hazard object and resulting impact object would then be handled differently depending on whether the hazard object is a forecast or an ensemble. For instance, warning discouraging the use of non-sensible methods and metrics (e.g. computing average annual impacts or return periods on a forecast) could be raised. From those two attributes, it can easily be inferred if the hazard data is a forecast or not or if the hazard data stems from an simulation ensemble or not, allowing those checks to be implemented.
In addition, new helper functions or plotting functions could be added to better handle forecast and ensemble data (e.g. group by ensemble realization and aggregate, spaghetti plots for different ensemble members).

Concrete next steps
If the proposed implementation makes sense, one would then need to:

• Add lead_time and ensemble_id attributes to the hazard class
• Replace the date attribute in the hazard class so that it handles timestamps (e.g. subdaily times) instead of just dates
• Make sure the attributes are propagated consistently from a hazard object to an impact object (and solve possible issues that may arise)
• Implement checks to avoid incorrect use of methods when the data is a forecast or an ensemble of simulations
• Add some helper and plotting methods to ease the handling of forecast and ensemble data

@ the core developer team (@emanuel-schmid, @chahank, @peanutfun), please let us know what you think of this approach and what would be the issues/caveats.

[Evelyn-M]

Thanks a lot @luseverin for the summary of our CLIMADA Day discussion!

We have documented 2 use-cases here, which should help reality-check any future feature implementation: https://github.com/Evelyn-M/forecast-usecases/tree/main (@manniepmkam TC displacement forecast and @Evelyn-M's warning use case)

Both demos don't use the Forecast class, but employ workarounds.
The TC forecast usecase (as most so far) simply condenses the lead time component in the Hazard by taking max intensity over the entire forecast time frame.
For a warning context, this information is crucial though; th latter use case hence merges the leadtime and ensemble member dimensions to fit it into a Hazard object, and then unstacks them from the Impact object.

Currently, when merging dimensions, one has to do lots of track-keeping, stacking and unstacking of the ensemble x leadtime dimensions, and one looses nice convenience functions of the impact object, such as the default plotting options, or simple stats (though again, not all of them make sense).

On the upside, keeping track of these dimensions and outputting impact matrices into xarray datasets is pretty simple and gives back a lot of control over stats. Hence it's not super straightforward to me whether one could simply replace the current Forecast Class by convenience wrappers around Hazard and Impact which simply does the track keeping ususally needed.

[peanutfun]

We need a failsafe to avoid plugging HazardForecast into e.g. unsequa, adaptation, because it will work but the results will not make sense

[chahank]

Suggestion by @spjuhel : one could define abstract base hazard and impact classes, and then have derive probabilistic and forecast versions respectively ?

[spjuhel]

Specifically for this problem, the "proper" way to do this, I think, would be to have an abstract class for hazard (HazardBase) which factorizes the code common to both Hazard and HazardForecast, and then have these two be derived from it. That's a big change, it also has several possible cons (which I don't have in mind at the moment).

There might be some middle ground solution.

Edit: @chahank was faster (but less detailled)

[chahank]

Note that you need both a forecast hazard and a forecast impact class.

[ValentinGebhart]

Editable user journey draft