Merge branch 'cross-calibrate-impact-functions' of github.com:CLIMADA-project/climada_python into cross-calibrate-impact-functions

Author: emanuel-schmid

.github/workflows/ci.yml

@@ -47,7 +47,7 @@ jobs:
       -
         name: Install CLIMADA
         run: |
-          python -m pip install ".[test]"
+          python -m pip install "./[dev]"
       -
         name: Run Unit Tests
         run: |

.gitignore

@@ -183,3 +183,5 @@ results/
 !.pylintrc
 !.readthedocs.yml
 !.github
+climada_petals/
+climada_python/

.zenodo.json

@@ -119,6 +119,9 @@
     },
     {
       "name": "Valentin Gebhart"
+    },
+    {
+      "name": "Dahyann Araya"
     }
   ]
 }

AUTHORS.md

@@ -36,3 +36,4 @@
 * Luca Severino
 * Samuel Juhel
 * Valentin Gebhart
+* Dahyann Araya

CHANGELOG.md

@@ -10,11 +10,21 @@ Code freeze date: YYYY-MM-DD
 
 ### Dependency Changes
 
+Added:
+
+- `fiona` >=1.10
+
+Updated:
+
+- `geopandas` >=0.14,<1.0 → >=0.14
+- `pandas` >=2.1,<2.2 → >=2.1
+
 Removed:
 
 - `pandas-datareader`
 
 ### Added
+
 - Added optional parameter to `geo_im_from_array`, `plot_from_gdf`, `plot_rp_imp`, `plot_rp_intensity`,
 `plot_intensity`, `plot_fraction`, `_event_plot` to mask plotting when regions are too far from data points [#1047](https://github.com/CLIMADA-project/climada_python/pull/1047). To recreate previous plots (no masking), the parameter can be set to None.
 - Added instructions to install Climada petals on Euler cluster in `doc.guide.Guide_Euler.ipynb` [#1029](https://github.com/CLIMADA-project/climada_python/pull/1029)
@@ -23,11 +33,14 @@ Removed:
 
 ### Changed
 
+- Changed the default mask_distance in `util.plot.geo_im_from_array` to 0.03 to avoid white gaps in gridded hazard data with comparably low resolution (>80 centroids per axis) [#1073](https://github.com/CLIMADA-project/climada_python/pull/1073)
+- Increased speed of `util.plot.add_shapes` by avoiding for loops, substantially speeding up `Hazard.plot_intensity` and other functions. [#1073](https://github.com/CLIMADA-project/climada_python/pull/1073)
 - `Hazard.local_exceedance_intensity`, `Hazard.local_return_period` and `Impact.local_exceedance_impact`, `Impact.local_return_period`, using the `climada.util.interpolation` module: New default (no binning), binning on decimals, and faster implementation [#1012](https://github.com/CLIMADA-project/climada_python/pull/1012)
 - World Bank indicator data is now downloaded directly from their API via the function `download_world_bank_indicator`, instead of relying on the `pandas-datareader` package [#1033](https://github.com/CLIMADA-project/climada_python/pull/1033)
 - `Exposures.write_hdf5` pickles geometry data in WKB format, which is faster and more sustainable. [#1051](https://github.com/CLIMADA-project/climada_python/pull/1051)
 - The online documentation has been completely overhauled, now uses PyData theme: [#977](https://github.com/CLIMADA-project/climada_python/pull/977)
 - `Input` to impact function calibration tasks now supports adding weights to the data [#1048](https://github.com/CLIMADA-project/climada_python/pull/1048)
+- Add `climada.hazard.xarray` module with helper structures for reading Hazard objects from `xarray` data [#1063](https://github.com/CLIMADA-project/climada_python/pull/1063)
 
 ### Fixed
 
@@ -36,9 +49,13 @@ Removed:
 
 ### Deprecated
 
+- `Hazard.from_xarray_raster_file`. Use `Hazard.from_xarray_raster` and pass the file path as `data` argument [#1063](https://github.com/CLIMADA-project/climada_python/pull/1063)
+
 ### Removed
 
 - `climada.util.interpolation.round_to_sig_digits` [#1012](https://github.com/CLIMADA-project/climada_python/pull/1012)
+- `intensity_thres` of `Hazard`, `StormEurope`, and `TropCyclones` object [#1065](https://github.com/CLIMADA-project/climada_python/pull/1065)
+- Deprecated method `climada.hazard.trop_cyclone.trop_cyclone.TropCyclone.set_from_tracks` [#1065](https://github.com/CLIMADA-project/climada_python/pull/1065)
 
 ## 6.0.1
 
@@ -208,6 +225,7 @@ CLIMADA tutorials. [#872](https://github.com/CLIMADA-project/climada_python/pull
 - `Impact.write_hdf5` now throws an error if `event_name` is does not contain strings exclusively [#894](https://github.com/CLIMADA-project/climada_python/pull/894)
 - Split `climada.hazard.trop_cyclone` module into smaller submodules without affecting module usage [#911](https://github.com/CLIMADA-project/climada_python/pull/911)
 - `yearly_steps` parameter of `TropCyclone.apply_climate_scenario_knu` has been made explicit [#991](https://github.com/CLIMADA-project/climada_python/pull/991)
+- `Hazard.write_hdf5` writes centroids as x,y columns (or as wkb in case of polygons) at a compression level of 9, not as pickled `Shapely` objects anymore, which reduces the size of the files significantly.
 
 ### Fixed
 

climada/engine/cost_benefit.py

@@ -1079,9 +1079,7 @@ def _calc_impact_measures(
         # compute impact for each measure
         for measure in meas_set.get_measure(hazard.haz_type):
             LOGGER.debug("%s impact of measure %s.", when, measure.name)
-            imp_tmp, risk_transf = measure.calc_impact(
-                exposures, imp_fun_set, hazard, assign_centroids=False
-            )
+            imp_tmp, risk_transf = measure.calc_impact(exposures, imp_fun_set, hazard)
             impact_meas[measure.name] = dict()
             impact_meas[measure.name]["cost"] = (
                 measure.cost,

climada/engine/impact.py

@@ -1178,7 +1178,7 @@ def plot_rp_imp(
         return_periods=(25, 50, 100, 250),
         log10_scale=True,
         axis=None,
-        mask_distance=0.01,
+        mask_distance=0.03,
         kwargs_local_exceedance_impact=None,
         **kwargs,
     ):
@@ -1198,8 +1198,8 @@ def plot_rp_imp(
         mask_distance: float, optional
             Only regions are plotted that are closer to any of the data points than this distance,
             relative to overall plot size. For instance, to only plot values
-            at the centroids, use mask_distance=0.01. If None, the plot is not masked.
-            Default is 0.01.
+            at the centroids, use mask_distance=0.03. If None, the plot is not masked.
+            Default is 0.03.
         kwargs_local_exceedance_impact: dict
             Dictionary of keyword arguments for the method impact.local_exceedance_impact.
         kwargs : dict, optional

climada/engine/impact_data.py

@@ -774,7 +774,7 @@ def clean_emdat_df(
         df_data["Start Day"] = np.array(day_list, dtype="int")
         df_data["Start Year"] = np.array(year_list, dtype="int")
         for var in ["Disaster Subtype", "Disaster Type", "Country"]:
-            df_data[VARNAMES_EMDAT[target_version][var]].fillna("None", inplace=True)
+            df_data.fillna({VARNAMES_EMDAT[target_version][var]: "None"}, inplace=True)
 
     # (3) Filter by countries, year range, and disaster type
     # (3.1) Countries:

climada/engine/test/test_cost_benefit.py

@@ -35,6 +35,8 @@
 )
 from climada.entity.disc_rates import DiscRates
 from climada.entity.entity_def import Entity
+from climada.entity.measures import Measure
+from climada.entity.measures.base import LOGGER as ILOG
 from climada.hazard.base import Hazard
 from climada.test import get_test_file
 from climada.util.api_client import Client
@@ -1119,6 +1121,66 @@ def test_remove_measure(self):
         self.assertEqual(len(cost_ben.cost_ben_ratio), 3)
         self.assertEqual(len(cost_ben.benefit), 3)
 
+    def test_measure_exposure_no_centroids(self):
+        """Test centroids assigned if no centroids were assigned in Measure Exposures object"""
+        hazard = Hazard.from_hdf5(HAZ_TEST_TC)
+        entity = Entity.from_excel(ENT_DEMO_TODAY)
+        entity.check()
+        entity.exposures.ref_year = 2018
+
+        # test that warning is not raised when centroid are assigned
+        try:
+            with self.assertLogs(ILOG, level="WARNING") as logs:
+                cost_ben = CostBenefit()
+                cost_ben.calc(
+                    hazard,
+                    entity,
+                    future_year=2040,
+                    risk_func=risk_aai_agg,
+                    imp_time_depen=None,
+                    save_imp=True,
+                )
+                for log in logs.output:
+                    self.assertNotIn(
+                        "No assigned hazard centroids in exposure object after",
+                        log,
+                        "Centroids are already assinged in Measure Exposure object"
+                        "and should not be reassigned.",
+                    )
+        except AssertionError as e:
+            if "no logs" in str(e).lower():
+                pass
+            else:
+                raise
+
+        # add measure with exposure without assigned centroids
+        exp_no_assigned_centroids = entity.exposures.copy()
+        exp_no_assigned_centroids.data = entity.exposures.data.drop(
+            columns=["centr_TC"]
+        )
+        entity.measures.append(
+            Measure(
+                name="no centroids",
+                haz_type="TC",
+                exposures_set=exp_no_assigned_centroids,
+            )
+        )
+
+        # test if warning is raised
+        with self.assertLogs(ILOG, level="WARNING") as logs:
+            cost_ben = CostBenefit()
+            cost_ben.calc(
+                hazard,
+                entity,
+                future_year=2040,
+                risk_func=risk_aai_agg,
+                imp_time_depen=None,
+                save_imp=True,
+            )
+        self.assertIn(
+            "No assigned hazard centroids in exposure object after", logs.output[0]
+        )
+
 
 class TestCalc(unittest.TestCase):
     """Test calc"""

climada/engine/unsequa/calc_cost_benefit.py

@@ -398,7 +398,7 @@ def _map_costben_calc(
             ent_future=ent_fut,
             save_imp=False,
             assign_centroids=False,
-            **cost_benefit_kwargs
+            **cost_benefit_kwargs,
         )
         # Extract from climada.impact the chosen metrics
         uncertainty_values.append(