Move TC and StormEurope hazards into Petals

[peanutfun]

Today, the CLIMADA devs decided to move more Hazard-derived classes from the core repository into CLIMADA Petals. The goal is to reduce development overhead in the core. Also, different Hazard types originally served as "demonstrators" in the core. But this purpose has been diminished by more and more hazard data being available from the data API. Therefore, this decision is in line with the original intend of Petals, namely to house hazard (and other) specialization modules.

In a first step, we want to move all TC-related modules:

• https://github.com/CLIMADA-project/climada_python/blob/develop/climada/hazard/tc_clim_change.py
• https://github.com/CLIMADA-project/climada_python/blob/develop/climada/hazard/tc_tracks.py
• https://github.com/CLIMADA-project/climada_python/blob/develop/climada/hazard/tc_tracks_synth.py
• https://github.com/CLIMADA-project/climada_python/blob/develop/climada/hazard/trop_cyclone.py

In a second step, we also want to move the winter storm module:

• https://github.com/CLIMADA-project/climada_python/blob/develop/climada/hazard/storm_europe.py

Users of these modules will have to do the following when they upgrade to a new Climada version:

• Install Petals
• Rename their related imports from climada.hazard.MODULE to climada_petals.hazard.MODULE

At this point, it is still unclear when the move will happen. We want to keep the user experience as nice as possible, with appropriate warnings and/or errors being raised that tell users exactly what to do.

Please note and comment/criticize/suggest..! @ThomasRoosli @mmyrte @tovogt @bguillod @ChrisFairless

[timschmi95]

In particular the TC class is still very useful as demonstrator (in e.g. the CCUR lecture) as it visualizes well how a probabilistic event set is generated, which helps understanding the concept quickly. I think some learning would be lost if we instead just download a hazard set from the API, so for some "Climada introduction"-classes, one would probably also need to install petals in the future.
However, if the TC part in petals remains well-maintained/compatible, I think the additional petals-installation shouldn't be a problem.

[aleeciu]

I support this decision.

[davidnbresch]

We discussed the Pros/Cons in detail. With the API, we do not need TC as a demonstrator to get started with CLIMADA. It was anyway a bit cumbersome that new users had to a create a hazard set first - as many would never do so and rather use existing hazard sets. With the easy petals installtion, we still have a swift way for users to build TC on their own. Same considerations do apply to WS Europe, also a candidate to be moved to petals (one final discussion with Thomas Röösli pending). Fazit: This way, we can provide CLIMADA core plus CLIMADA data API as startting point and do not confuse the first time users with technicalities.

[bguillod]

I think this is a good idea in general.
As for the transition we just need to ensure this is smooth, especially for branches currently on Climada core that target those modules.
Does this mean that any branch will need to be merged before moving those modules to petals? If so, we need to know about the timeline well in advance. If branches can be migrated then it’s less of a problem of course.

[chahank]

This is a good point. I do not know whether one can migrate branches. But, in any case, this is a good occasion to have some efforts to timely either finish the long-standing open pull requests or to close them to be re-opened on petals (manually).

[mmyrte]

Yeah, you could; just use something like git-filter-repo. I used this method the other day to merge multiple repos and branches. As long as the DAG reconverges eventually, all's good.

[chahank]

Excellent, thanks!

[mmyrte]

If you'll indulge me for a bit, let me take this opportunity to bring up a fundamental question once more:

I'm still a little bit unsure about the separation of petals and core in the first place. I appreciate that it makes sense to have the core maintained professionally and to a higher standard, but the divergence between the two repos can ache a little. I know this might be unorthodox, but what if the petals module was moved back to this repo? That would ease the decision to move less-used components into the petals, while making it easier to have core and petals in tune.

For beginners, the setup.py would still make sure that if you install without cloning (e.g. using /usr/bin/pip3 install https://github.com/CLIMADA-project/climada_python/archive/refs/tags/v3.3.3.zip) it's as simple as pie. We'd have to find a solution for installing petals into the system library, but if you're working interactively, you'll be pointing your conda environment/interpreter at the working directory anyways.

I agree that my train of thought isn't fully formed yet, but if this sets someone thinking, that's already something :)

[mmyrte]

Thanks for sharing. Could you make more precise what you mean with divergence between the two repos ?

@chahank if you build a feature in petals that depends on an updated feature in core, then you currently have to manage two repos indepently of one another. If everything's in the same repo but different submodules, a single PR allows for the relevant work on core to be carried out in lockstep/atomically with petals.

[chahank]

I still do not understand, sorry. But in any case, you always have to use compatible versions of petals and core as soon as you are using features of both. So, how would that change now?

[mmyrte]

Exactly: Now you have to know which versions are compatible. If they are provided by the same repo, it's much easier to be sure that they're compatible - a single merge can affect both petals and core. But if I'm not being clear enough, I suppose we'd have to take this discussion offline.

Also, since I'm not involved in climada dev work that much anymore, I don't care overmuch about this, just thought I'd provide an additional option for how to factor code/organize dev work.

[emanuel-schmid]

Compatibility between core and petals is supposed to be sure for

• released versions
  the compatible version of core is explicit in setup.py of petals (this wasn't always the case in the past but we're determined to have it in the future)
• develop branches

If this is not the case, let us know: it's broken and we have to fix it.

Any other compatibility you really have to know about, but I don't see a use case right away.

[peanutfun]

In the future, we should run the testing pipeline for Petals from Core (for each Core PR). This way, we can ensure that changes to the Core will not break Petals. Currently, this is not done automatically