Merge branch 'develop' into feature/get-region-per-country

Author: NicolasColombi

.github/CODEOWNERS

@@ -0,0 +1,7 @@
+# This is a comment.
+# Each line is a file pattern followed by one or more owners.
+
+# These owners will be the default owners for everything in
+# the repo. Unless a later match takes precedence, they will
+# be requested for review when someone opens a pull request.
+*       @emanuel-schmid @chahank @peanutfun

.github/workflows/pull-request.yml

@@ -16,7 +16,7 @@ jobs:
           ref: ${{ github.event.pull_request.head.sha }}
       -
         name: Checkout target commit
-        run: git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --depth=1 origin ${{ github.event.pull_request.base.ref }}
+        run: git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --depth=50 origin ${{ github.event.pull_request.base.ref }}
       -
         name: Set up Python 3.11
         uses: actions/setup-python@v5

.gitignore

@@ -183,3 +183,5 @@ results/
 !.pylintrc
 !.readthedocs.yml
 !.github
+climada_petals/
+climada_python/

.readthedocs.yml

@@ -25,3 +25,6 @@ python:
 
 formats:
   - pdf
+
+sphinx:
+  configuration: doc/conf.py

CHANGELOG.md

@@ -10,16 +10,36 @@ Code freeze date: YYYY-MM-DD
 
 ### Dependency Changes
 
+Removed:
+
+- `pandas-datareader`
+- `intensity_thres` of `Hazard`, `StormEurope`, and `TropCyclones` object [#1065](https://github.com/CLIMADA-project/climada_python/pull/1065)
+- Deprecated method `climada.hazard.trop_cyclone.trop_cyclone.TropCyclone.set_from_tracks` [#1065](https://github.com/CLIMADA-project/climada_python/pull/1065)
+
 ### Added
+- Added optional parameter to `geo_im_from_array`, `plot_from_gdf`, `plot_rp_imp`, `plot_rp_intensity`,
+`plot_intensity`, `plot_fraction`, `_event_plot` to mask plotting when regions are too far from data points [#1047](https://github.com/CLIMADA-project/climada_python/pull/1047). To recreate previous plots (no masking), the parameter can be set to None.
 - Added instructions to install Climada petals on Euler cluster in `doc.guide.Guide_Euler.ipynb` [#1029](https://github.com/CLIMADA-project/climada_python/pull/1029)
 
+- `ImpactFunc` and `ImpactFuncSet` now support equality comparisons via `==` [#1027](https://github.com/CLIMADA-project/climada_python/pull/1027)
+
 ### Changed
+- Changed the default mask_distance in `util.plot.geo_im_from_array` to 0.03 to avoid white gaps in gridded hazard data with comparably low resolution (>80 centroids per axis) [#1073](https://github.com/CLIMADA-project/climada_python/pull/1073)
+- Increased speed of `util.plot.add_shapes` by avoiding for loops, substantially speeding up `Hazard.plot_intensity` and other functions. [#1073](https://github.com/CLIMADA-project/climada_python/pull/1073)
 - `Hazard.local_exceedance_intensity`, `Hazard.local_return_period` and `Impact.local_exceedance_impact`, `Impact.local_return_period`, using the `climada.util.interpolation` module: New default (no binning), binning on decimals, and faster implementation [#1012](https://github.com/CLIMADA-project/climada_python/pull/1012)
+- World Bank indicator data is now downloaded directly from their API via the function `download_world_bank_indicator`, instead of relying on the `pandas-datareader` package [#1033](https://github.com/CLIMADA-project/climada_python/pull/1033)
+- `Exposures.write_hdf5` pickles geometry data in WKB format, which is faster and more sustainable. [#1051](https://github.com/CLIMADA-project/climada_python/pull/1051)
+- The online documentation has been completely overhauled, now uses PyData theme: [#977](https://github.com/CLIMADA-project/climada_python/pull/977)
+
 ### Fixed
 
+- NaN plotting issues in `geo_im_from_array`[#1038](https://github.com/CLIMADA-project/climada_python/pull/1038)
+- Broken ECMWF links in pydoc of `climada.hazard.storm_europe` relocated. [#944](https://github.com/CLIMADA-project/climada_python/pull/944)
+
 ### Deprecated
 
 ### Removed
+
 - `climada.util.interpolation.round_to_sig_digits` [#1012](https://github.com/CLIMADA-project/climada_python/pull/1012)
 
 ## 6.0.1
@@ -43,27 +63,27 @@ Added:
 
 Updated:
 
-- `cartopy` >=0.23 → >=0.24
-- `cfgrib` >=0.9.9,<0.9.10 &rarr; >=0.9
-- `dask` >=2024.2,<2024.3 &rarr; >=2025.2
-- `eccodes` >=2.27,<2.28 &rarr; >=2.40
-- `gdal` >=3.6 &rarr; >=3.10
-- `geopandas` >=0.14 &rarr; >=0.14,<1.0
-- `h5py` >=3.8 &rarr; >=3.12
-- `haversine` >=2.8 &rarr; >=2.9
-- `matplotlib-base` >=3.9 &rarr; >=3.10
-- `netcdf4` >=1.6 &rarr; >=1.7
-- `numba` >=0.60 &rarr; >=0.61
-- `pillow` =9.4 &rarr; =11.1
-- `pyproj` >=3.5 &rarr; >=3.7
-- `pytables` >=3.7 &rarr; >=3.10
-- `python` =3.9 &rarr; =3.11
-- `rasterio` >=1.3 &rarr; >=1.4
-- `scikit-learn` >=1.5 &rarr; >=1.6
-- `scipy` >=1.13 &rarr; >=1.14,<1.15
-- `tqdm` >=4.66 &rarr; >=4.67
-- `xarray` >=2024.6 &rarr; >=2025.1
-- `xlsxwriter` >=3.1 &rarr; >=3.2
+- `cartopy` >=0.23 → >=0.24
+- `cfgrib` >=0.9.9,<0.9.10 → >=0.9
+- `dask` >=2024.2,<2024.3 → >=2025.2
+- `eccodes` >=2.27,<2.28 → >=2.40
+- `gdal` >=3.6 → >=3.10
+- `geopandas` >=0.14 → >=0.14,<1.0
+- `h5py` >=3.8 → >=3.12
+- `haversine` >=2.8 → >=2.9
+- `matplotlib-base` >=3.9 → >=3.10
+- `netcdf4` >=1.6 → >=1.7
+- `numba` >=0.60 → >=0.61
+- `pillow` =9.4 → =11.1
+- `pyproj` >=3.5 → >=3.7
+- `pytables` >=3.7 → >=3.10
+- `python` =3.9 → =3.11
+- `rasterio` >=1.3 → >=1.4
+- `scikit-learn` >=1.5 → >=1.6
+- `scipy` >=1.13 → >=1.14,<1.15
+- `tqdm` >=4.66 → >=4.67
+- `xarray` >=2024.6 → >=2025.1
+- `xlsxwriter` >=3.1 → >=3.2
 
 Removed:
 
@@ -75,7 +95,7 @@ Removed:
 - `climada.hazard.tc_tracks.TCTracks.subset_years` function [#1023](https://github.com/CLIMADA-project/climada_python/pull/1023)
 - `climada.hazard.tc_tracks.TCTracks.from_FAST` function, add Australia basin (AU) [#993](https://github.com/CLIMADA-project/climada_python/pull/993)
 - Add `osm-flex` package to CLIMADA core [#981](https://github.com/CLIMADA-project/climada_python/pull/981)
-- `doc.tutorial.climada_entity_Exposures_osm.ipynb` tutorial explaining how to use `osm-flex`with CLIMADA
+- `doc.tutorial.climada_entity_Exposures_osm.ipynb` tutorial explaining how to use `osm-flex` with CLIMADA
 - `climada.util.coordinates.bounding_box_global` function [#980](https://github.com/CLIMADA-project/climada_python/pull/980)
 - `climada.util.coordinates.bounding_box_from_countries` function [#980](https://github.com/CLIMADA-project/climada_python/pull/980)
 - `climada.util.coordinates.bounding_box_from_cardinal_bounds` function [#980](https://github.com/CLIMADA-project/climada_python/pull/980)
@@ -107,8 +127,8 @@ Removed:
 - the _geometry_ column of the inherent `GeoDataFrame` is set up at initialization
 - latitude and longitude column are no longer present there (the according arrays can be retrieved as properties of the Exposures object: `exp.latitude` instead of `exp.gdf.latitude.values`).
 - `Exposures.gdf` has been renamed to `Exposures.data` (it still works though, as it is a property now pointing to the latter)
-- the `check` method does not add a default "IMPF_" column to the GeoDataFrame anymore
-- Updated IBTrACS version from v4.0 to v4.1 ([#976](https://github.com/CLIMADA-project/climada_python/pull/976)
+- the `check` method does not add a default `'IMPF_'` column to the GeoDataFrame anymore
+- Updated IBTrACS version from v4.0 to v4.1 [#976](https://github.com/CLIMADA-project/climada_python/pull/976)
 - Fix xarray future warning in TCTracks for .dims to .sizes
 - Fix hazard.concatenate type test for pathos pools
 
@@ -142,20 +162,20 @@ Added:
 
 Updated:
 
-- `bottleneck` >=1.3 &rarr; >=1.4
-- `cartopy` >=0.22 &rarr; >=0.23
-- `contextily` >=1.5 &rarr; >=1.6
-- `dask` >=2024.1,<2024.3 &rarr; >=2024.2,<2024.3
-- `matplotlib-base` >=3.8 &rarr; >=3.9
-- `numba` >=0.59 &rarr; >=0.60
-- `numexpr` >=2.9 &rarr; >=2.10
-- `pint` >=0.23 &rarr; >=0.24
-- `pycountry` >=22.3 &rarr; >=24.6
-- `requests` >=2.31 &rarr; >=2.32
-- `salib` >=1.4 &rarr; >=1.5
-- `scikit-learn` >=1.4 &rarr; >=1.5
-- `scipy` >=1.12 &rarr; >=1.13
-- `xarray` >=2024.2 &rarr; >=2024.6
+- `bottleneck` >=1.3 → >=1.4
+- `cartopy` >=0.22 → >=0.23
+- `contextily` >=1.5 → >=1.6
+- `dask` >=2024.1,<2024.3 → >=2024.2,<2024.3
+- `matplotlib-base` >=3.8 → >=3.9
+- `numba` >=0.59 → >=0.60
+- `numexpr` >=2.9 → >=2.10
+- `pint` >=0.23 → >=0.24
+- `pycountry` >=22.3 → >=24.6
+- `requests` >=2.31 → >=2.32
+- `salib` >=1.4 → >=1.5
+- `scikit-learn` >=1.4 → >=1.5
+- `scipy` >=1.12 → >=1.13
+- `xarray` >=2024.2 → >=2024.6
 
 ### Added
 
@@ -190,6 +210,7 @@ CLIMADA tutorials. [#872](https://github.com/CLIMADA-project/climada_python/pull
 - `Impact.from_hdf5` now calls `str` on `event_name` data that is not strings, and issue a warning then [#894](https://github.com/CLIMADA-project/climada_python/pull/894)
 - `Impact.write_hdf5` now throws an error if `event_name` is does not contain strings exclusively [#894](https://github.com/CLIMADA-project/climada_python/pull/894)
 - Split `climada.hazard.trop_cyclone` module into smaller submodules without affecting module usage [#911](https://github.com/CLIMADA-project/climada_python/pull/911)
+- `yearly_steps` parameter of `TropCyclone.apply_climate_scenario_knu` has been made explicit [#991](https://github.com/CLIMADA-project/climada_python/pull/991)
 
 ### Fixed
 
@@ -260,17 +281,17 @@ Added:
 
 Updated:
 
-- `contextily` >=1.3 &rarr; >=1.5
-- `dask` >=2023 &rarr; >=2024
-- `numba` >=0.57 &rarr; >=0.59
-- `pandas` >=2.1 &rarr; >=2.1,<2.2
-- `pint` >=0.22 &rarr; >=0.23
-- `scikit-learn` >=1.3 &rarr; >=1.4
-- `scipy` >=1.11 &rarr; >=1.12
-- `sparse` >=0.14 &rarr; >=0.15
-- `xarray` >=2023.8 &rarr; >=2024.1
-- `overpy` =0.6 &rarr; =0.7
-- `peewee` =3.16.3 &rarr; =3.17.1
+- `contextily` >=1.3 → >=1.5
+- `dask` >=2023 → >=2024
+- `numba` >=0.57 → >=0.59
+- `pandas` >=2.1 → >=2.1,<2.2
+- `pint` >=0.22 → >=0.23
+- `scikit-learn` >=1.3 → >=1.4
+- `scipy` >=1.11 → >=1.12
+- `sparse` >=0.14 → >=0.15
+- `xarray` >=2023.8 → >=2024.1
+- `overpy` =0.6 → =0.7
+- `peewee` =3.16.3 → =3.17.1
 
 Removed:
 
@@ -279,7 +300,7 @@ Removed:
 ### Added
 
 - Convenience method `api_client.Client.get_dataset_file`, combining `get_dataset_info` and `download_dataset`, returning a single file objet. [#821](https://github.com/CLIMADA-project/climada_python/pull/821)
-- Read and Write methods to and from csv files for the `DiscRates` class. [#818](ttps://github.com/CLIMADA-project/climada_python/pull/818)
+- Read and Write methods to and from csv files for the `DiscRates` class. [#818](https://github.com/CLIMADA-project/climada_python/pull/818)
 - Add `CalcDeltaClimate` to unsequa module to allow uncertainty and sensitivity analysis of impact change calculations [#844](https://github.com/CLIMADA-project/climada_python/pull/844)
 - Add function `safe_divide` in util which handles division by zero and NaN values in the numerator or denominator [#844](https://github.com/CLIMADA-project/climada_python/pull/844)
 - Add reset_frequency option for the impact.select() function. [#847](https://github.com/CLIMADA-project/climada_python/pull/847)
@@ -313,13 +334,13 @@ Release date: 2023-09-27
 
 Added:
 
-- `matplotlib-base` None &rarr; >=3.8
+- `matplotlib-base` None → >=3.8
 
 Changed:
 
-- `geopandas` >=0.13 &rarr; >=0.14
-- `pandas` >=1.5,<2.0 &rarr; >=2.1
-- `salib` >=1.3.0 &rarr; >=1.4.7
+- `geopandas` >=0.13 → >=0.14
+- `pandas` >=1.5,<2.0 → >=2.1
+- `salib` >=1.3.0 → >=1.4.7
 
 Removed:
 
@@ -348,37 +369,37 @@ Added:
 
 Changed:
 
-- `cartopy` >=0.20.0,<0.20.3 &rarr; >=0.21
-- `cfgrib` >=0.9.7,<0.9.10 &rarr; =0.9.9
-- `contextily` >=1.0 &rarr; >=1.3
-- `dask` >=2.25 &rarr; >=2023
-- `eccodes` [auto] &rarr; =2.27
-- `gdal` !=3.4.1 &rarr; >=3.6
-- `geopandas` >=0.8 &rarr; >=0.13
-- `h5py` >=2.10 &rarr; >=3.8
-- `haversine` >=2.3 &rarr; >=2.8
-- `matplotlib` >=3.2,< 3.6 &rarr; >=3.7
-- `netcdf4` >=1.5 &rarr; >=1.6
-- `numba` >=0.51,!=0.55.0 &rarr; >=0.57
-- `openpyxl` >=3.0 &rarr; >=3.1
-- `pandas-datareader` >=0.9 &rarr; >=0.10
-- `pathos` >=0.2 &rarr; >=0.3
-- `pint` >=0.15 &rarr; >=0.22
-- `proj` !=9.0.0 &rarr; >=9.1
-- `pycountry` >=20.7 &rarr; >=22.3
-- `pytables` >=3.6 &rarr; >=3.7
-- `rasterio` >=1.2.7,<1.3 &rarr; >=1.3
-- `requests` >=2.24 &rarr; >=2.31
-- `salib` >=1.3.0 &rarr; >=1.4
-- `scikit-learn` >=1.0 &rarr; >=1.2
-- `scipy` >=1.6 &rarr; >=1.10
-- `sparse` >=0.13 &rarr; >=0.14
-- `statsmodels` >=0.11 &rarr; >=0.14
-- `tabulate` >=0.8 &rarr; >=0.9
-- `tqdm` >=4.48 &rarr; >=4.65
-- `xarray` >=0.13 &rarr; >=2023.5
-- `xlrd` >=1.2 &rarr; >=2.0
-- `xlsxwriter` >=1.3 &rarr; >=3.1
+- `cartopy` >=0.20.0,<0.20.3 → >=0.21
+- `cfgrib` >=0.9.7,<0.9.10 → =0.9.9
+- `contextily` >=1.0 → >=1.3
+- `dask` >=2.25 → >=2023
+- `eccodes` [auto] → =2.27
+- `gdal` !=3.4.1 → >=3.6
+- `geopandas` >=0.8 → >=0.13
+- `h5py` >=2.10 → >=3.8
+- `haversine` >=2.3 → >=2.8
+- `matplotlib` >=3.2,< 3.6 → >=3.7
+- `netcdf4` >=1.5 → >=1.6
+- `numba` >=0.51,!=0.55.0 → >=0.57
+- `openpyxl` >=3.0 → >=3.1
+- `pandas-datareader` >=0.9 → >=0.10
+- `pathos` >=0.2 → >=0.3
+- `pint` >=0.15 → >=0.22
+- `proj` !=9.0.0 → >=9.1
+- `pycountry` >=20.7 → >=22.3
+- `pytables` >=3.6 → >=3.7
+- `rasterio` >=1.2.7,<1.3 → >=1.3
+- `requests` >=2.24 → >=2.31
+- `salib` >=1.3.0 → >=1.4
+- `scikit-learn` >=1.0 → >=1.2
+- `scipy` >=1.6 → >=1.10
+- `sparse` >=0.13 → >=0.14
+- `statsmodels` >=0.11 → >=0.14
+- `tabulate` >=0.8 → >=0.9
+- `tqdm` >=4.48 → >=4.65
+- `xarray` >=0.13 → >=2023.5
+- `xlrd` >=1.2 → >=2.0
+- `xlsxwriter` >=1.3 → >=3.1
 
 Removed:
 

CONTRIBUTING.md

@@ -12,11 +12,11 @@ For orientation, these are some categories of possible contributions we can thin
 * **New Modules and Utility Functions:** Did you create a function or an entire module you find useful for your work? Maybe you are not the only one! Feel free to simply raise a pull request for functions that improve, e.g., plotting or data handling. As an entire module has to be carefully integrated into the framework, it might help if you talk to us first so we can design the module and plan the next steps. You can do that by raising an issue or starting a [discussion](https://github.com/CLIMADA-project/climada_python/discussions) on GitHub.
 
 A good place to start a personal discussion is our monthly CLIMADA developers call.
-Please contact the [lead developers](https://wcr.ethz.ch/research/climada.html) if you want to join.
+Please contact the [lead developers](https://climada.ethz.ch/team/) if you want to join.
 
 ## Why Should You Contribute?
 
-* You will be listed as author of the CLIMADA repository in the [AUTHORS](AUTHORS.md) file.
+* You will be listed as author of the CLIMADA repository in the [AUTHORS][authors] file.
 * You will improve the quality of the CLIMADA software for you and for everybody else using it.
 * You will gain insights into scientific software development.
 
@@ -40,7 +40,7 @@ To contribute follow these steps:
     ```bash
     pylint
     ```
-6. Add your name to the [AUTHORS](AUTHORS.md) file.
+6. Add your name to the [AUTHORS][authors] file.
 7. Push your updates to the remote repository:
 
     ```bash
@@ -83,4 +83,5 @@ It also contains a checklist for both pull request authors and reviewers to guid
 
 [docs]: https://climada-python.readthedocs.io/en/latest/
 [devguide]: https://climada-python.readthedocs.io/en/latest/#developer-guide
-[testing]: https://climada-python.readthedocs.io/en/latest/guide/Guide_Testing.html
+[testing]: https://climada-python.readthedocs.io/en/latest/development/Guide_Testing.html
+[authors]: https://github.com/CLIMADA-project/climada_python/blob/main/AUTHORS.md

climada/engine/cost_benefit.py

@@ -1079,9 +1079,7 @@ def _calc_impact_measures(
         # compute impact for each measure
         for measure in meas_set.get_measure(hazard.haz_type):
             LOGGER.debug("%s impact of measure %s.", when, measure.name)
-            imp_tmp, risk_transf = measure.calc_impact(
-                exposures, imp_fun_set, hazard, assign_centroids=False
-            )
+            imp_tmp, risk_transf = measure.calc_impact(exposures, imp_fun_set, hazard)
             impact_meas[measure.name] = dict()
             impact_meas[measure.name]["cost"] = (
                 measure.cost,

climada/engine/impact.py

@@ -1178,6 +1178,7 @@ def plot_rp_imp(
         return_periods=(25, 50, 100, 250),
         log10_scale=True,
         axis=None,
+        mask_distance=0.03,
         kwargs_local_exceedance_impact=None,
         **kwargs,
     ):
@@ -1194,6 +1195,11 @@ def plot_rp_imp(
             plot impact as log10(impact). Default: True
         smooth : bool, optional
             smooth plot to plot.RESOLUTIONxplot.RESOLUTION. Default: True
+        mask_distance: float, optional
+            Only regions are plotted that are closer to any of the data points than this distance,
+            relative to overall plot size. For instance, to only plot values
+            at the centroids, use mask_distance=0.03. If None, the plot is not masked.
+            Default is 0.03.
         kwargs_local_exceedance_impact: dict
             Dictionary of keyword arguments for the method impact.local_exceedance_impact.
         kwargs : dict, optional
@@ -1242,7 +1248,12 @@ def plot_rp_imp(
             )
 
         axis = u_plot.plot_from_gdf(
-            impacts_stats, title, column_labels, axis=axis, **kwargs
+            impacts_stats,
+            title,
+            column_labels,
+            axis=axis,
+            mask_distance=mask_distance,
+            **kwargs,
         )
         return axis, impacts_stats_vals