Merge branch 'develop-white' into feature/cyclostrophic-as-parameter

Author: spjuhel

.github/workflows/ci.yml

@@ -12,6 +12,7 @@ jobs:
   build-and-test:
     name: 'Core / Unit Test Pipeline'
     runs-on: ubuntu-latest
+    timeout-minutes: 20
     permissions:
       # For publishing results
       checks: write

.gitignore

@@ -176,3 +176,10 @@ data/ISIMIP_crop/
 
 # climada data results folder:
 results/
+
+# Hidden files we want to track
+!.gitignore
+!.pre-commit-config.yaml
+!.pylintrc
+!.readthedocs.yml
+!.github

.pre-commit-config.yaml

@@ -0,0 +1,23 @@
+# See https://pre-commit.com for more information
+default_language_version:
+  python: python3
+
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v3.2.0
+    hooks:
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+
+  - repo: https://github.com/pycqa/isort
+    rev: '5.13.2'
+    hooks:
+      - id: isort
+        args: ["--profile", "black", "--filter-files"]
+
+  # Using this mirror lets us use mypyc-compiled black, which is about 2x faster
+  - repo: https://github.com/psf/black-pre-commit-mirror
+    rev: '24.4.2'
+    hooks:
+      - id: black-jupyter

CHANGELOG.md

@@ -12,10 +12,19 @@ Code freeze date: YYYY-MM-DD
 
 ### Added
 
+- `climada.util.interpolation` module for inter- and extrapolation util functions used in local exceedance intensity and return period functions [#930](https://github.com/CLIMADA-project/climada_python/pull/930)
+
 ### Changed
 
+- Improved scaling factors implemented in `climada.hazard.trop_cyclone.apply_climate_scenario_knu` to model the impact of climate changes to tropical cyclones [#734](https://github.com/CLIMADA-project/climada_python/pull/734)
+- In `climada.util.plot.geo_im_from_array`, NaNs are plotted in gray while cells with no centroid are not plotted [#929](https://github.com/CLIMADA-project/climada_python/pull/929)
+- Renamed `climada.util.plot.subplots_from_gdf` to `climada.util.plot.plot_from_gdf` [#929](https://github.com/CLIMADA-project/climada_python/pull/929)
+
 ### Fixed
 
+- File handles are being closed after reading netcdf files with `climada.hazard` modules [#953](https://github.com/CLIMADA-project/climada_python/pull/953)
+- Avoids a ValueError in the impact calculation for cases with a single exposure point and MDR values of 0, by explicitly removing zeros in `climada.hazard.Hazard.get_mdr` [#933](https://github.com/CLIMADA-project/climada_python/pull/948)
+
 ### Deprecated
 
 ### Removed
@@ -52,6 +61,19 @@ Updated:
 
 - GitHub actions workflow for CLIMADA Petals compatibility tests [#855](https://github.com/CLIMADA-project/climada_python/pull/855)
 - `climada.util.calibrate` module for calibrating impact functions [#692](https://github.com/CLIMADA-project/climada_python/pull/692)
+- Method `Hazard.check_matrices` for bringing the stored CSR matrices into "canonical format" [#893](https://github.com/CLIMADA-project/climada_python/pull/893)
+- Generic s-shaped impact function via `ImpactFunc.from_poly_s_shape` [#878](https://github.com/CLIMADA-project/climada_python/pull/878)
+- climada.hazard.centroids.centr.Centroids.get_area_pixel
+- climada.hazard.centroids.centr.Centroids.get_dist_coast
+- climada.hazard.centroids.centr.Centroids.get_elevation
+- climada.hazard.centroids.centr.Centroids.get_meta
+- climada.hazard.centroids.centr.Centroids.get_pixel_shapes
+- climada.hazard.centroids.centr.Centroids.to_crs
+- climada.hazard.centroids.centr.Centroids.to_default_crs
+- climada.hazard.centroids.centr.Centroids.write_csv
+- climada.hazard.centroids.centr.Centroids.write_excel
+- climada.hazard.local_return_period [#898](https://github.com/CLIMADA-project/climada_python/pull/898)
+- climada.util.plot.subplots_from_gdf [#898](https://github.com/CLIMADA-project/climada_python/pull/898)
 
 ### Changed
 
@@ -76,20 +98,6 @@ CLIMADA tutorials. [#872](https://github.com/CLIMADA-project/climada_python/pull
 - Fix broken links in `CONTRIBUTING.md` [#900](https://github.com/CLIMADA-project/climada_python/pull/900)
 - When writing `TCTracks` to NetCDF, only apply compression to `float` or `int` data types. This fixes a downstream issue, see [climada_petals#135](https://github.com/CLIMADA-project/climada_petals/issues/135) [#911](https://github.com/CLIMADA-project/climada_python/pull/911)
 
-### Added
-
-- Method `Hazard.check_matrices` for bringing the stored CSR matrices into "canonical format" [#893](https://github.com/CLIMADA-project/climada_python/pull/893)
-- Generic s-shaped impact function via `ImpactFunc.from_poly_s_shape` [#878](https://github.com/CLIMADA-project/climada_python/pull/878)
-- climada.hazard.centroids.centr.Centroids.get_area_pixel
-- climada.hazard.centroids.centr.Centroids.get_dist_coast
-- climada.hazard.centroids.centr.Centroids.get_elevation
-- climada.hazard.centroids.centr.Centroids.get_meta
-- climada.hazard.centroids.centr.Centroids.get_pixel_shapes
-- climada.hazard.centroids.centr.Centroids.to_crs
-- climada.hazard.centroids.centr.Centroids.to_default_crs
-- climada.hazard.centroids.centr.Centroids.write_csv
-- climada.hazard.centroids.centr.Centroids.write_excel
-
 ### Deprecated
 
 - climada.hazard.centroids.centr.Centroids.from_lat_lon
@@ -465,4 +473,3 @@ updated:
 
 - `climada.enginge.impact.Impact.calc()` and `climada.enginge.impact.Impact.calc_impact_yearset()`
 [#436](https://github.com/CLIMADA-project/climada_python/pull/436).
-

climada/engine/cost_benefit.py

@@ -32,7 +32,7 @@
 from tabulate import tabulate
 
 from climada.engine.impact_calc import ImpactCalc
-from climada.engine import Impact, ImpactFreqCurve
+from climada.engine.impact import Impact, ImpactFreqCurve
 
 LOGGER = logging.getLogger(__name__)
 

climada/engine/forecast.py

@@ -186,7 +186,7 @@ def __init__(
         if exposure_name is None:
             try:
                 self.exposure_name = u_coord.country_to_iso(
-                    exposure.gdf.region_id.unique()[0], "name"
+                    exposure.gdf["region_id"].unique()[0], "name"
                 )
             except (KeyError, AttributeError):
                 self.exposure_name = "custom"

climada/engine/impact.py

@@ -243,8 +243,8 @@ def from_eih(cls, exposures, hazard, at_event, eai_exp, aai_agg, imp_mat=None):
             date = hazard.date,
             frequency = hazard.frequency,
             frequency_unit = hazard.frequency_unit,
-            coord_exp = np.stack([exposures.gdf.latitude.values,
-                                  exposures.gdf.longitude.values],
+            coord_exp = np.stack([exposures.gdf['latitude'].values,
+                                  exposures.gdf['longitude'].values],
                                  axis=1),
             crs = exposures.crs,
             unit = exposures.value_unit,
@@ -1081,25 +1081,25 @@ def from_csv(cls, file_name):
         # pylint: disable=no-member
         LOGGER.info('Reading %s', file_name)
         imp_df = pd.read_csv(file_name)
-        imp = cls(haz_type=imp_df.haz_type[0])
-        imp.unit = imp_df.unit[0]
-        imp.tot_value = imp_df.tot_value[0]
-        imp.aai_agg = imp_df.aai_agg[0]
-        imp.event_id = imp_df.event_id[~np.isnan(imp_df.event_id)].values
+        imp = cls(haz_type=imp_df['haz_type'][0])
+        imp.unit = imp_df['unit'][0]
+        imp.tot_value = imp_df['tot_value'][0]
+        imp.aai_agg = imp_df['aai_agg'][0]
+        imp.event_id = imp_df['event_id'][~np.isnan(imp_df['event_id'])].values
         num_ev = imp.event_id.size
-        imp.event_name = imp_df.event_name[:num_ev].values.tolist()
-        imp.date = imp_df.event_date[:num_ev].values
-        imp.at_event = imp_df.at_event[:num_ev].values
-        imp.frequency = imp_df.event_frequency[:num_ev].values
-        imp.frequency_unit = imp_df.frequency_unit[0] if 'frequency_unit' in imp_df \
+        imp.event_name = imp_df['event_name'][:num_ev].values.tolist()
+        imp.date = imp_df['event_date'][:num_ev].values
+        imp.at_event = imp_df['at_event'][:num_ev].values
+        imp.frequency = imp_df['event_frequency'][:num_ev].values
+        imp.frequency_unit = imp_df['frequency_unit'][0] if 'frequency_unit' in imp_df \
                              else DEF_FREQ_UNIT
-        imp.eai_exp = imp_df.eai_exp[~np.isnan(imp_df.eai_exp)].values
+        imp.eai_exp = imp_df['eai_exp'][~np.isnan(imp_df['eai_exp'])].values
         num_exp = imp.eai_exp.size
         imp.coord_exp = np.zeros((num_exp, 2))
-        imp.coord_exp[:, 0] = imp_df.exp_lat[:num_exp]
-        imp.coord_exp[:, 1] = imp_df.exp_lon[:num_exp]
+        imp.coord_exp[:, 0] = imp_df['exp_lat'][:num_exp]
+        imp.coord_exp[:, 1] = imp_df['exp_lon'][:num_exp]
         try:
-            imp.crs = u_coord.to_crs_user_input(imp_df.exp_crs.values[0])
+            imp.crs = u_coord.to_crs_user_input(imp_df['exp_crs'].values[0])
         except AttributeError:
             imp.crs = DEF_CRS
 
@@ -1129,23 +1129,23 @@ def from_excel(cls, file_name):
         dfr = pd.read_excel(file_name)
         imp = cls(haz_type=str(dfr['haz_type'][0]))
 
-        imp.unit = dfr.unit[0]
-        imp.tot_value = dfr.tot_value[0]
-        imp.aai_agg = dfr.aai_agg[0]
+        imp.unit = dfr['unit'][0]
+        imp.tot_value = dfr['tot_value'][0]
+        imp.aai_agg = dfr['aai_agg'][0]
 
-        imp.event_id = dfr.event_id[~np.isnan(dfr.event_id.values)].values
-        imp.event_name = dfr.event_name[:imp.event_id.size].values
-        imp.date = dfr.event_date[:imp.event_id.size].values
-        imp.frequency = dfr.event_frequency[:imp.event_id.size].values
-        imp.frequency_unit = dfr.frequency_unit[0] if 'frequency_unit' in dfr else DEF_FREQ_UNIT
-        imp.at_event = dfr.at_event[:imp.event_id.size].values
+        imp.event_id = dfr['event_id'][~np.isnan(dfr['event_id'].values)].values
+        imp.event_name = dfr['event_name'][:imp.event_id.size].values
+        imp.date = dfr['event_date'][:imp.event_id.size].values
+        imp.frequency = dfr['event_frequency'][:imp.event_id.size].values
+        imp.frequency_unit = dfr['frequency_unit'][0] if 'frequency_unit' in dfr else DEF_FREQ_UNIT
+        imp.at_event = dfr['at_event'][:imp.event_id.size].values
 
-        imp.eai_exp = dfr.eai_exp[~np.isnan(dfr.eai_exp.values)].values
+        imp.eai_exp = dfr['eai_exp'][~np.isnan(dfr['eai_exp'].values)].values
         imp.coord_exp = np.zeros((imp.eai_exp.size, 2))
-        imp.coord_exp[:, 0] = dfr.exp_lat.values[:imp.eai_exp.size]
-        imp.coord_exp[:, 1] = dfr.exp_lon.values[:imp.eai_exp.size]
+        imp.coord_exp[:, 0] = dfr['exp_lat'].values[:imp.eai_exp.size]
+        imp.coord_exp[:, 1] = dfr['exp_lon'].values[:imp.eai_exp.size]
         try:
-            imp.crs = u_coord.to_csr_user_input(dfr.exp_crs.values[0])
+            imp.crs = u_coord.to_csr_user_input(dfr['exp_crs'].values[0])
         except AttributeError:
             imp.crs = DEF_CRS
 
@@ -1324,14 +1324,14 @@ def video_direct_impact(exp, impf_set, haz_list, file_name='',
                  np.array([haz.intensity.max() for haz in haz_list]).max()]
 
         if 'vmin' not in args_exp:
-            args_exp['vmin'] = exp.gdf.value.values.min()
+            args_exp['vmin'] = exp.gdf['value'].values.min()
 
         if 'vmin' not in args_imp:
             args_imp['vmin'] = np.array([imp.eai_exp.min() for imp in imp_list
                                          if imp.eai_exp.size]).min()
 
         if 'vmax' not in args_exp:
-            args_exp['vmax'] = exp.gdf.value.values.max()
+            args_exp['vmax'] = exp.gdf['value'].values.max()
 
         if 'vmax' not in args_imp:
             args_imp['vmax'] = np.array([imp.eai_exp.max() for imp in imp_list

climada/engine/impact_calc.py

@@ -27,7 +27,7 @@
 import geopandas as gpd
 
 from climada import CONFIG
-from climada.engine import Impact
+from climada.engine.impact import Impact
 
 LOGGER = logging.getLogger(__name__)
 
@@ -154,8 +154,8 @@ def impact(self, save_mat=True, assign_centroids=True,
                     exp_gdf.size, self.n_events)
         imp_mat_gen = self.imp_mat_gen(exp_gdf, impf_col)
 
-        insured = ('cover' in exp_gdf and exp_gdf.cover.max() >= 0) \
-               or ('deductible' in exp_gdf and exp_gdf.deductible.max() > 0)
+        insured = ('cover' in exp_gdf and exp_gdf['cover'].max() >= 0) \
+               or ('deductible' in exp_gdf and exp_gdf['deductible'].max() > 0)
         if insured:
             LOGGER.info("cover and/or deductible columns detected,"
                         " going to calculate insured impact")
@@ -253,8 +253,8 @@ def minimal_exp_gdf(self, impf_col, assign_centroids, ignore_cover, ignore_deduc
                              " Run 'exposures.assign_centroids()' beforehand or set"
                              " 'assign_centroids' to 'True'")
         mask = (
-            (self.exposures.gdf.value.values == self.exposures.gdf.value.values)  # value != NaN
-            & (self.exposures.gdf.value.values != 0)                              # value != 0
+            (self.exposures.gdf['value'].values == self.exposures.gdf['value'].values)# value != NaN
+            & (self.exposures.gdf['value'].values != 0)                              # value != 0
             & (self.exposures.gdf[self.hazard.centr_exp_col].values >= 0)    # centroid assigned
         )
 
@@ -320,7 +320,7 @@ def _chunk_exp_idx(haz_size, idx_exp_impf):
                 )
             idx_exp_impf = (exp_gdf[impf_col].values == impf_id).nonzero()[0]
             for exp_idx in _chunk_exp_idx(self.hazard.size, idx_exp_impf):
-                exp_values = exp_gdf.value.values[exp_idx]
+                exp_values = exp_gdf['value'].values[exp_idx]
                 cent_idx = exp_gdf[self.hazard.centr_exp_col].values[exp_idx]
                 yield (
                     self.impact_matrix(exp_values, cent_idx, impf),
@@ -363,10 +363,10 @@ def insured_mat_gen(self, imp_mat_gen, exp_gdf, impf_col):
                 haz_type=self.hazard.haz_type,
                 fun_id=impf_id)
             if 'deductible' in exp_gdf:
-                deductible = exp_gdf.deductible.values[exp_idx]
+                deductible = exp_gdf['deductible'].values[exp_idx]
                 mat = self.apply_deductible_to_mat(mat, deductible, self.hazard, cent_idx, impf)
             if 'cover' in exp_gdf:
-                cover = exp_gdf.cover.values[exp_idx]
+                cover = exp_gdf['cover'].values[exp_idx]
                 mat = self.apply_cover_to_mat(mat, cover)
             yield (mat, exp_idx)