Merge branch 'develop' into feature/review_yearsets_tutorial

Author: ValentinGebhart

.github/pull_request_template.md

@@ -19,10 +19,10 @@ This PR fixes #
 ### PR Reviewer Checklist
 
 - [ ] Read the [Contribution Guide][contrib]
-- [ ] [CLIMADA Reviewer Checklist](https://climada-python.readthedocs.io/en/latest/guide/Guide_Reviewer_Checklist.html) passed
+- [ ] [CLIMADA Reviewer Checklist](https://climada-python.readthedocs.io/en/latest/development/Guide_Review.html) passed
 - [ ] [Tests][testing] passing
 - [ ] No new [linter issues][linter]
 
 [contrib]: https://github.com/CLIMADA-project/climada_python/blob/main/CONTRIBUTING.md
-[testing]: https://climada-python.readthedocs.io/en/latest/guide/Guide_Continuous_Integration_and_Testing.html
-[linter]: https://climada-python.readthedocs.io/en/stable/guide/Guide_Continuous_Integration_and_Testing.html#3.C.--Static-Code-Analysis
+[testing]: https://climada-python.readthedocs.io/en/latest/development/Guide_continuous_integration_GitHub_actions.html
+[linter]: https://climada-python.readthedocs.io/en/latest/development/Guide_continuous_integration_GitHub_actions.html#static-code-analysis

.github/workflows/ci.yml

@@ -47,7 +47,7 @@ jobs:
       -
         name: Install CLIMADA
         run: |
-          python -m pip install ".[test]"
+          python -m pip install "./[dev]"
       -
         name: Run Unit Tests
         run: |

.zenodo.json

@@ -119,6 +119,9 @@
     },
     {
       "name": "Valentin Gebhart"
+    },
+    {
+      "name": "Dahyann Araya"
     }
   ]
 }

AUTHORS.md

@@ -36,3 +36,4 @@
 * Luca Severino
 * Samuel Juhel
 * Valentin Gebhart
+* Dahyann Araya

CHANGELOG.md

@@ -10,24 +10,36 @@ Code freeze date: YYYY-MM-DD
 
 ### Dependency Changes
 
+Added:
+
+- `fiona` >=1.10
+
+Updated:
+
+- `geopandas` >=0.14,<1.0 → >=0.14
+- `pandas` >=2.1,<2.2 → >=2.1
+
 Removed:
 
 - `pandas-datareader`
 
 ### Added
+
 - Added optional parameter to `geo_im_from_array`, `plot_from_gdf`, `plot_rp_imp`, `plot_rp_intensity`,
 `plot_intensity`, `plot_fraction`, `_event_plot` to mask plotting when regions are too far from data points [#1047](https://github.com/CLIMADA-project/climada_python/pull/1047). To recreate previous plots (no masking), the parameter can be set to None.
 - Added instructions to install Climada petals on Euler cluster in `doc.guide.Guide_Euler.ipynb` [#1029](https://github.com/CLIMADA-project/climada_python/pull/1029)
 
 - `ImpactFunc` and `ImpactFuncSet` now support equality comparisons via `==` [#1027](https://github.com/CLIMADA-project/climada_python/pull/1027)
 
 ### Changed
+
 - Changed the default mask_distance in `util.plot.geo_im_from_array` to 0.03 to avoid white gaps in gridded hazard data with comparably low resolution (>80 centroids per axis) [#1073](https://github.com/CLIMADA-project/climada_python/pull/1073)
 - Increased speed of `util.plot.add_shapes` by avoiding for loops, substantially speeding up `Hazard.plot_intensity` and other functions. [#1073](https://github.com/CLIMADA-project/climada_python/pull/1073)
 - `Hazard.local_exceedance_intensity`, `Hazard.local_return_period` and `Impact.local_exceedance_impact`, `Impact.local_return_period`, using the `climada.util.interpolation` module: New default (no binning), binning on decimals, and faster implementation [#1012](https://github.com/CLIMADA-project/climada_python/pull/1012)
 - World Bank indicator data is now downloaded directly from their API via the function `download_world_bank_indicator`, instead of relying on the `pandas-datareader` package [#1033](https://github.com/CLIMADA-project/climada_python/pull/1033)
 - `Exposures.write_hdf5` pickles geometry data in WKB format, which is faster and more sustainable. [#1051](https://github.com/CLIMADA-project/climada_python/pull/1051)
 - The online documentation has been completely overhauled, now uses PyData theme: [#977](https://github.com/CLIMADA-project/climada_python/pull/977)
+- Add `climada.hazard.xarray` module with helper structures for reading Hazard objects from `xarray` data [#1063](https://github.com/CLIMADA-project/climada_python/pull/1063)
 - The output of the `impact_yearset` was changed to only contain attributes corresponding to the yearly impact set. The default parameters of `impact_yearset`, `sample_events`, and `impact_yearset_from_sampling_vect` functions of the `climada.util.yearsets` module have been changed to sampling with replacement and not applying the correction factor to the impact yearsets. The application of the correction factor and the frequency of the resulting yearly impact object are corrected. [#1075](https://github.com/CLIMADA-project/climada_python/pull/1075)
 
 ### Fixed
@@ -37,9 +49,13 @@ Removed:
 
 ### Deprecated
 
+- `Hazard.from_xarray_raster_file`. Use `Hazard.from_xarray_raster` and pass the file path as `data` argument [#1063](https://github.com/CLIMADA-project/climada_python/pull/1063)
+
 ### Removed
 
 - `climada.util.interpolation.round_to_sig_digits` [#1012](https://github.com/CLIMADA-project/climada_python/pull/1012)
+- `intensity_thres` of `Hazard`, `StormEurope`, and `TropCyclones` object [#1065](https://github.com/CLIMADA-project/climada_python/pull/1065)
+- Deprecated method `climada.hazard.trop_cyclone.trop_cyclone.TropCyclone.set_from_tracks` [#1065](https://github.com/CLIMADA-project/climada_python/pull/1065)
 
 ## 6.0.1
 
@@ -90,6 +106,7 @@ Removed:
 
 ### Added
 
+- `climada.entity.impact_funcs.trop_cyclone.ImpfSetTropCyclone.get_impf_id_regions_per_countries` function [#1034](https://github.com/CLIMADA-project/climada_python/pull/1034)
 - `climada.hazard.tc_tracks.TCTracks.subset_years` function [#1023](https://github.com/CLIMADA-project/climada_python/pull/1023)
 - `climada.hazard.tc_tracks.TCTracks.from_FAST` function, add Australia basin (AU) [#993](https://github.com/CLIMADA-project/climada_python/pull/993)
 - Add `osm-flex` package to CLIMADA core [#981](https://github.com/CLIMADA-project/climada_python/pull/981)
@@ -209,6 +226,7 @@ CLIMADA tutorials. [#872](https://github.com/CLIMADA-project/climada_python/pull
 - `Impact.write_hdf5` now throws an error if `event_name` is does not contain strings exclusively [#894](https://github.com/CLIMADA-project/climada_python/pull/894)
 - Split `climada.hazard.trop_cyclone` module into smaller submodules without affecting module usage [#911](https://github.com/CLIMADA-project/climada_python/pull/911)
 - `yearly_steps` parameter of `TropCyclone.apply_climate_scenario_knu` has been made explicit [#991](https://github.com/CLIMADA-project/climada_python/pull/991)
+- `Hazard.write_hdf5` writes centroids as x,y columns (or as wkb in case of polygons) at a compression level of 9, not as pickled `Shapely` objects anymore, which reduces the size of the files significantly.
 
 ### Fixed
 

climada/engine/impact_data.py

@@ -774,7 +774,7 @@ def clean_emdat_df(
         df_data["Start Day"] = np.array(day_list, dtype="int")
         df_data["Start Year"] = np.array(year_list, dtype="int")
         for var in ["Disaster Subtype", "Disaster Type", "Country"]:
-            df_data[VARNAMES_EMDAT[target_version][var]].fillna("None", inplace=True)
+            df_data.fillna({VARNAMES_EMDAT[target_version][var]: "None"}, inplace=True)
 
     # (3) Filter by countries, year range, and disaster type
     # (3.1) Countries:

climada/engine/unsequa/calc_cost_benefit.py

@@ -398,7 +398,7 @@ def _map_costben_calc(
             ent_future=ent_fut,
             save_imp=False,
             assign_centroids=False,
-            **cost_benefit_kwargs
+            **cost_benefit_kwargs,
         )
         # Extract from climada.impact the chosen metrics
         uncertainty_values.append(

climada/engine/unsequa/test/test_unsequa.py

@@ -650,7 +650,7 @@ def test_calc_sensitivity_all_pass(self):
         }
 
         def test_sensitivity_method(
-            exp_unc, impf_unc, haz_unc, sensitivity_method, param_dict
+            exp_unc, impf_unc, haz_unc, sensitivity_method, param_dict, places
         ):
             """Function to test each seaprate sensitivity method"""
             unc_calc = CalcImpact(exp_unc, impf_unc, haz_unc)
@@ -681,7 +681,7 @@ def test_sensitivity_method(
             self.assertAlmostEqual(
                 param_dict["test_si_value"][0],
                 unc_data.aai_agg_sens_df["aai_agg"][param_dict["test_si_value"][1]],
-                places=5,
+                places=places,
             )
 
             self.assertEqual(unc_data.aai_agg_unc_df.size, unc_data.n_samples)
@@ -695,7 +695,12 @@ def test_sensitivity_method(
         # loop over each method and do test
         for sensitivity_method, method_params in test_dict.items():
             test_sensitivity_method(
-                exp_unc, impf_unc, haz_unc, sensitivity_method, method_params
+                exp_unc,
+                impf_unc,
+                haz_unc,
+                sensitivity_method,
+                method_params,
+                places=2 if sensitivity_method == "rbd_fast" else 5,
             )
 
 

climada/entity/exposures/litpop/litpop.py

@@ -19,6 +19,7 @@
 """
 
 import logging
+import numbers
 from pathlib import Path
 
 import geopandas
@@ -820,12 +821,12 @@ def _from_country(
             total_value = _get_total_value_per_country(iso3a, fin_mode, reference_year)
 
         # disaggregate total value proportional to LitPop values:
-        if isinstance(total_value, (float, int)):
+        if isinstance(total_value, numbers.Number):
             litpop_gdf["value"] = (
                 np.divide(litpop_gdf["value"], litpop_gdf["value"].sum()) * total_value
             )
         elif total_value is not None:
-            raise TypeError("total_value must be int or float.")
+            raise TypeError(f"total_value ({total_value}) must be a number.")
 
         exp = LitPop()
         exp.set_gdf(litpop_gdf)

climada/entity/impact_funcs/test/test_tc.py

@@ -115,7 +115,7 @@ def test_default_values_pass(self):
         self.assertEqual(impfs.size(), 10)
         self.assertEqual(impfs.get_ids()["TC"], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
         self.assertEqual(impf_wp4.intensity_unit, "m/s")
-        self.assertEqual(impf_wp4.name, "North West Pacific (WP4)")
+        self.assertEqual(impf_wp4.name, "North West Pacific")
         self.assertAlmostEqual(v_halfs["WP2"], 188.4, places=7)
         self.assertAlmostEqual(v_halfs["ROW"], 110.1, places=7)
         self.assertListEqual(list(impf_wp4.intensity), list(np.arange(0, 121, 5)))
@@ -135,7 +135,7 @@ def test_RMSF_pass(self):
         self.assertEqual(impfs.size(), 10)
         self.assertEqual(impfs.get_ids()["TC"], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
         self.assertEqual(impf_na1.intensity_unit, "m/s")
-        self.assertEqual(impf_na1.name, "Caribbean and Mexico (NA1)")
+        self.assertEqual(impf_na1.name, "Caribbean and Mexico")
         self.assertAlmostEqual(v_halfs["NA1"], 59.6, places=7)
         self.assertAlmostEqual(v_halfs["ROW"], 73.4, places=7)
         self.assertListEqual(list(impf_na1.intensity), list(np.arange(0, 121, 5)))
@@ -153,7 +153,7 @@ def test_quantile_pass(self):
         self.assertEqual(impfs.size(), 10)
         self.assertEqual(impfs_p10.size(), 10)
         self.assertEqual(impf_si.intensity_unit, "m/s")
-        self.assertEqual(impf_si_p10.name, "South Indian (SI)")
+        self.assertEqual(impf_si_p10.name, "South Indian")
         self.assertAlmostEqual(impf_si_p10.mdd.max(), 0.99999999880, places=5)
         self.assertAlmostEqual(impf_si.calc_mdr(30), 0.01620503041, places=5)
         intensity = np.random.randint(26, impf_si.intensity.max())
@@ -168,6 +168,35 @@ def test_get_countries_per_region(self):
         self.assertListEqual(out[2], [124, 840])
         self.assertListEqual(out[3], ["CAN", "USA"])
 
+    def test_get_imf_id_regions_per_countries(self):
+        """Test get_impf_id_regions_per_countries()"""
+        ifs = ImpfSetTropCyclone()
+        impf_id_reg_id_reg_name = ifs.get_impf_id_regions_per_countries(
+            countries=["CHE"]
+        )
+
+        # the first element of impf_id_reg_id_reg_name [0] is the impact function id,
+        # the second [1] is the region id, the third [2] is the region name.
+        self.assertEqual(impf_id_reg_id_reg_name[0][0], 10)
+        self.assertEqual(impf_id_reg_id_reg_name[1][0], "ROW")
+        self.assertEqual(impf_id_reg_id_reg_name[2][0], "Rest of The World")
+        impf_id_reg_id_reg_name = ifs.get_impf_id_regions_per_countries(countries=[756])
+        self.assertEqual(impf_id_reg_id_reg_name[0][0], 10)
+        self.assertEqual(impf_id_reg_id_reg_name[1][0], "ROW")
+        self.assertEqual(impf_id_reg_id_reg_name[2][0], "Rest of The World")
+
+        impf_id_reg_id_reg_name = ifs.get_impf_id_regions_per_countries(
+            countries=["CHE", 268]
+        )
+        # CHE
+        self.assertEqual(impf_id_reg_id_reg_name[0][0], 10)
+        self.assertEqual(impf_id_reg_id_reg_name[1][0], "ROW")
+        self.assertEqual(impf_id_reg_id_reg_name[2][0], "Rest of The World")
+        # GEO (georgia, 268)
+        self.assertEqual(impf_id_reg_id_reg_name[0][1], 3)
+        self.assertEqual(impf_id_reg_id_reg_name[1][1], "NI")
+        self.assertEqual(impf_id_reg_id_reg_name[2][1], "North Indian")
+
 
 # Execute Tests
 if __name__ == "__main__":