Merge branch 'develop' into feature/from_netcdf_fast

Author: NicolasColombi

.readthedocs.yml

@@ -1,5 +1,8 @@
 version: 2
 
+sphinx:
+  configuration: doc/conf.py
+
 build:
   os: "ubuntu-22.04"
   tools:

CHANGELOG.md

@@ -12,6 +12,11 @@ Code freeze date: YYYY-MM-DD
 
 ### Added
 
+- Add `osm-flex` package to CLIMADA core [#981](https://github.com/CLIMADA-project/climada_python/pull/981)
+- `doc.tutorial.climada_entity_Exposures_osm.ipynb` tutorial explaining how to use `osm-flex`with CLIMADA
+- `climada.util.coordinates.bounding_box_global` function [#980](https://github.com/CLIMADA-project/climada_python/pull/980)
+- `climada.util.coordinates.bounding_box_from_countries` function [#980](https://github.com/CLIMADA-project/climada_python/pull/980)
+- `climada.util.coordinates.bounding_box_from_cardinal_bounds` function [#980](https://github.com/CLIMADA-project/climada_python/pull/980)
 - `climada.engine.impact.Impact.local_return_period` method [#971](https://github.com/CLIMADA-project/climada_python/pull/971)
 - `doc.tutorial.climada_util_local_exceedance_values.ipynb` tutorial explaining `Hazard.local_exceedance_intensity`, `Hazard.local_return_period`, `Impact.local_exceedance_impact`, and `Impact.local_return_period` methods [#971](https://github.com/CLIMADA-project/climada_python/pull/971)
 - `Hazard.local_exceedance_intensity`, `Hazard.local_return_period` and `Impact.local_exceedance_impact`, that all use the `climada.util.interpolation` module [#918](https://github.com/CLIMADA-project/climada_python/pull/918)
@@ -28,6 +33,8 @@ Code freeze date: YYYY-MM-DD
 
 ### Changed
 
+- `Centroids.append` now takes multiple arguments and provides a performance boost when doing so [#989](https://github.com/CLIMADA-project/climada_python/pull/989)
+- `climada.util.coordinates.get_country_geometries` function: Now throwing a ValueError if unregognized ISO country code is given (before, the invalid ISO code was ignored) [#980](https://github.com/CLIMADA-project/climada_python/pull/980)
 - Improved scaling factors implemented in `climada.hazard.trop_cyclone.apply_climate_scenario_knu` to model the impact of climate changes to tropical cyclones [#734](https://github.com/CLIMADA-project/climada_python/pull/734)
 - In `climada.util.plot.geo_im_from_array`, NaNs are plotted in gray while cells with no centroid are not plotted [#929](https://github.com/CLIMADA-project/climada_python/pull/929)
 - Renamed `climada.util.plot.subplots_from_gdf` to `climada.util.plot.plot_from_gdf` [#929](https://github.com/CLIMADA-project/climada_python/pull/929)
@@ -43,6 +50,7 @@ Code freeze date: YYYY-MM-DD
 
 ### Fixed
 
+- Resolved an issue where windspeed computation was much slower than in Climada v3 [#989](https://github.com/CLIMADA-project/climada_python/pull/989)
 - File handles are being closed after reading netcdf files with `climada.hazard` modules [#953](https://github.com/CLIMADA-project/climada_python/pull/953)
 - Avoids a ValueError in the impact calculation for cases with a single exposure point and MDR values of 0, by explicitly removing zeros in `climada.hazard.Hazard.get_mdr` [#933](https://github.com/CLIMADA-project/climada_python/pull/948)
 

Makefile

@@ -29,11 +29,11 @@ install_test : ## Test installation was successful
 
 .PHONY : data_test
 data_test : ## Test data APIs
-	python script/jenkins/test_data_api.py
+	pytest $(PYTEST_JUNIT_ARGS) script/jenkins/test_data_api.py
 
 .PHONY : notebook_test
 notebook_test : ## Test notebooks in doc/tutorial
-	python script/jenkins/test_notebooks.py report
+	pytest $(PYTEST_JUNIT_ARGS) script/jenkins/test_notebooks.py
 
 .PHONY : integ_test
 integ_test : ## Integration tests execution with xml reports

climada/entity/exposures/base.py

@@ -132,12 +132,40 @@ def gdf(self):
     @property
     def latitude(self):
         """Latitude array of exposures"""
-        return self.data.geometry.y.values
+        try:
+            return self.data.geometry.y.values
+        except ValueError as valerr:
+            nonpoints = list(
+                self.data[
+                    self.data.geometry.type != "Point"
+                ].geometry.type.drop_duplicates()
+            )
+            if nonpoints:
+                raise ValueError(
+                    "Can only calculate latitude from Points."
+                    f" GeoDataFrame contains {', '.join(nonpoints)}."
+                    " Please see the lines_polygons module tutorial."
+                ) from valerr
+            raise
 
     @property
     def longitude(self):
         """Longitude array of exposures"""
-        return self.data.geometry.x.values
+        try:
+            return self.data.geometry.x.values
+        except ValueError as valerr:
+            nonpoints = list(
+                self.data[
+                    self.data.geometry.type != "Point"
+                ].geometry.type.drop_duplicates()
+            )
+            if nonpoints:
+                raise ValueError(
+                    "Can only calculate longitude from Points."
+                    f" GeoDataFrame contains {', '.join(nonpoints)}."
+                    " Please see the lines_polygons module tutorial."
+                ) from valerr
+            raise
 
     @property
     def geometry(self):

climada/entity/exposures/test/test_base.py

@@ -27,7 +27,7 @@
 import rasterio
 import scipy as sp
 from rasterio.windows import Window
-from shapely.geometry import Point
+from shapely.geometry import MultiPolygon, Point, Polygon
 from sklearn.metrics import DistanceMetric
 
 import climada.util.coordinates as u_coord
@@ -652,6 +652,39 @@ def test_to_crs_epsg_crs(self):
             Exposures.to_crs(self, crs="GCS", epsg=26915)
         self.assertEqual("one of crs or epsg must be None", str(cm.exception))
 
+    def test_latlon_with_polygons(self):
+        """Check for proper error message if the data frame contains non-Point shapes"""
+        poly = Polygon(
+            [(10.0, 0.0), (10.0, 1.0), (11.0, 1.0), (11.0, 0.0), (10.0, 0.0)]
+        )
+        point = Point((1, -1))
+        multi = MultiPolygon(
+            [
+                (
+                    ((0.0, 0.0), (0.0, 1.0), (1.0, 1.0), (1.0, 0.0)),
+                    [((0.1, 1.1), (0.1, 1.2), (0.2, 1.2), (0.2, 1.1))],
+                )
+            ]
+        )
+        poly = Polygon()
+        exp = Exposures(geometry=[poly, point, multi, poly])
+        with self.assertRaises(ValueError) as valer:
+            exp.latitude
+        self.assertEqual(
+            "Can only calculate latitude from Points."
+            " GeoDataFrame contains Polygon, MultiPolygon."
+            " Please see the lines_polygons module tutorial.",
+            str(valer.exception),
+        )
+        with self.assertRaises(ValueError) as valer:
+            exp.longitude
+        self.assertEqual(
+            "Can only calculate longitude from Points."
+            " GeoDataFrame contains Polygon, MultiPolygon."
+            " Please see the lines_polygons module tutorial.",
+            str(valer.exception),
+        )
+
 
 class TestImpactFunctions(unittest.TestCase):
     """Test impact function handling"""

climada/hazard/centroids/centr.py

@@ -331,11 +331,16 @@ def from_pnt_bounds(cls, points_bounds, res, crs=DEF_CRS):
             }
         )
 
-    def append(self, centr):
-        """Append Centroids
+    def append(self, *centr):
+        """Append Centroids to the current centroid object for concatenation.
+
+        This method checks that all centroids use the same CRS, appends the list of centroids to
+        the initial Centroid object and eventually concatenates them to create a single centroid
+        object with the union of all centroids.
 
         Note that the result might contain duplicate points if the object to append has an overlap
-        with the current object.
+        with the current object. Remove duplicates by either using :py:meth:`union`
+        or calling :py:meth:`remove_duplicate_points` after appending.
 
         Parameters
         ----------
@@ -351,22 +356,25 @@ def append(self, centr):
         union : Union of Centroid objects.
         remove_duplicate_points : Remove duplicate points in a Centroids object.
         """
-        if not u_coord.equal_crs(self.crs, centr.crs):
-            raise ValueError(
-                f"The given centroids use different CRS: {self.crs}, {centr.crs}. "
-                "The centroids are incompatible and cannot be concatenated."
-            )
-        self.gdf = pd.concat([self.gdf, centr.gdf])
+        for other in centr:
+            if not u_coord.equal_crs(self.crs, other.crs):
+                raise ValueError(
+                    f"The given centroids use different CRS: {self.crs}, {other.crs}. "
+                    "The centroids are incompatible and cannot be concatenated."
+                )
+        self.gdf = pd.concat([self.gdf] + [other.gdf for other in centr])
 
     def union(self, *others):
-        """Create the union of Centroids objects
+        """Create the union of the current Centroids object with one or more other centroids
+        objects by passing the list of centroids to :py:meth:`append` for concatenation and then
+        removes duplicates.
 
         All centroids must have the same CRS. Points that are contained in more than one of the
         Centroids objects will only be contained once (i.e. duplicates are removed).
 
         Parameters
         ----------
-        others : list of Centroids
+        others : Centroids
             Centroids contributing to the union.
 
         Returns
@@ -375,8 +383,8 @@ def union(self, *others):
             Centroids object containing the union of all Centroids.
         """
         centroids = copy.deepcopy(self)
-        for cent in others:
-            centroids.append(cent)
+        centroids.append(*others)
+
         return centroids.remove_duplicate_points()
 
     def remove_duplicate_points(self):

climada/hazard/centroids/test/test_centr.py

@@ -816,6 +816,20 @@ def test_append_dif_crs(self):
         with self.assertRaises(ValueError):
             self.centr.append(centr2)
 
+    def test_append_multiple_arguments(self):
+        """Test passing append() multiple arguments in the form of a list of Centroids."""
+        # create a single centroid
+        lat, lon = np.array([1, 2]), np.array([1, 2])
+        centr = Centroids(lat=lat, lon=lon)
+        # create a list of centroids
+        coords = [(np.array([3, 4]), np.array([3, 4]))]
+        centroids_list = [Centroids(lat=lat, lon=lon) for lat, lon in coords]
+
+        centr.append(*centroids_list)
+
+        np.testing.assert_array_equal(centr.lat, [1, 2, 3, 4])
+        np.testing.assert_array_equal(centr.lon, [1, 2, 3, 4])
+
     def test_remove_duplicate_pass(self):
         """Test remove_duplicate_points"""
         centr = Centroids(

climada/util/coordinates.py

@@ -783,6 +783,12 @@ def get_country_geometries(
     if country_names:
         if isinstance(country_names, str):
             country_names = [country_names]
+
+        # raise error if a country name is not recognized
+        for country_name in country_names:
+            if not country_name in nat_earth[["ISO_A3", "WB_A3", "ADM0_A3"]].values:
+                raise ValueError(f"ISO code {country_name} not recognized.")
+
         country_mask = np.isin(
             nat_earth[["ISO_A3", "WB_A3", "ADM0_A3"]].values,
             country_names,
@@ -1687,6 +1693,89 @@ def _ensure_utf8(val):
     return admin1_info, admin1_shapes
 
 
+def bounding_box_global():
+    """
+    Return global bounds in EPSG 4326
+
+    Returns
+    -------
+    tuple:
+        The global bounding box as (min_lon, min_lat, max_lon, max_lat)
+    """
+    return (-180, -90, 180, 90)
+
+
+def bounding_box_from_countries(country_names, buffer=1.0):
+    """
+    Return bounding box in EPSG 4326 containing given countries.
+
+    Parameters
+    ----------
+    country_names : list or str
+        list with ISO 3166 alpha-3 codes of countries, e.g ['ZWE', 'GBR', 'VNM', 'UZB']
+    buffer : float, optional
+        Buffer to add to both sides of the bounding box. Default: 1.0.
+
+    Returns
+    -------
+    tuple
+        The bounding box containing all given coutries as (min_lon, min_lat, max_lon, max_lat)
+    """
+
+    country_geometry = get_country_geometries(country_names).geometry
+    longitudes, latitudes = [], []
+    for multipolygon in country_geometry:
+        if isinstance(multipolygon, Polygon):  # if entry is polygon
+            for coord in polygon.exterior.coords:  # Extract exterior coordinates
+                longitudes.append(coord[0])
+                latitudes.append(coord[1])
+        else:  # if entry is multipolygon
+            for polygon in multipolygon.geoms:
+                for coord in polygon.exterior.coords:  # Extract exterior coordinates
+                    longitudes.append(coord[0])
+                    latitudes.append(coord[1])
+
+    return latlon_bounds(np.array(latitudes), np.array(longitudes), buffer=buffer)
+
+
+def bounding_box_from_cardinal_bounds(*, northern, eastern, western, southern):
+    """
+    Return and normalize bounding box in EPSG 4326 from given cardinal bounds.
+
+    Parameters
+    ----------
+    northern : (int, float)
+        Northern boundary of bounding box
+    eastern : (int, float)
+        Eastern boundary of bounding box
+    western : (int, float)
+        Western boundary of bounding box
+    southern : (int, float)
+        Southern boundary of bounding box
+
+    Returns
+    -------
+    tuple
+        The resulting normalized bounding box (min_lon, min_lat, max_lon, max_lat) with -180 <= min_lon < max_lon < 540
+
+    """
+
+    # latitude bounds check
+    if not ((90 >= northern > southern >= -90)):
+        raise ValueError(
+            "Given northern bound is below given southern bound or out of bounds"
+        )
+
+    eastern = (eastern + 180) % 360 - 180
+    western = (western + 180) % 360 - 180
+
+    # Ensure eastern > western
+    if western > eastern:
+        eastern += 360
+
+    return (western, southern, eastern, northern)
+
+
 def get_admin1_geometries(countries):
     """
     return geometries, names and codes of admin 1 regions in given countries