Merge branch 'master' into issue693-sar_backscatter-get-coefficients-from-schema

Author: ElienVandermaesenVITO

CHANGELOG.md

@@ -9,14 +9,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
-- Added `show_error_logs` argument to `cube.execute_batch()`/`job.start_and_wait()`/... to toggle the automatic printing of error logs on failure ([#505](https://github.com/Open-EO/openeo-python-client/issues/505))
+- Add initial support for accessing [Federation Extension](https://github.com/Open-EO/openeo-api/tree/master/extensions/federation) related metadata ([#668](https://github.com/Open-EO/openeo-python-client/issues/668))
 
 ### Changed
 
+- Improved tracking of metadata changes with `resample_spatial` and `resample_cube_spatial` ([#690](https://github.com/Open-EO/openeo-python-client/issues/690))
+- Move `ComparableVersion` to `openeo.utils.version` (related to [#611](https://github.com/Open-EO/openeo-python-client/issues/611))
+- Deprecate `openeo.rest.rest_capabilities.RESTCapabilities` and introduce replacement `openeo.rest.capabilities.OpenEoCapabilities` ([#611](https://github.com/Open-EO/openeo-python-client/issues/611), [#610](https://github.com/Open-EO/openeo-python-client/issues/610))
+- `MultiBackendJobManager`: start new jobs before downloading the results of finished jobs to use time more efficiently ([#633](https://github.com/Open-EO/openeo-python-client/issues/633))
+
 ### Removed
 
+- Remove unnecessary base class `openeo.capabilities.Capabilities` [#611](https://github.com/Open-EO/openeo-python-client/issues/611)
+
+### Fixed
+
+- `CsvJobDatabase`: workaround GeoPandas issue (on Python>3.9) when there is a column named "crs" ([#714](https://github.com/Open-EO/openeo-python-client/issues/714))
+
+
+## [0.37.0] - 2025-01-21 - "SAP10" release
+
+### Added
+
+- Added `show_error_logs` argument to `cube.execute_batch()`/`job.start_and_wait()`/... to toggle the automatic printing of error logs on failure ([#505](https://github.com/Open-EO/openeo-python-client/issues/505))
+- Added `Connection.web_editor()` to build link to the openEO backend in the openEO Web Editor
+- Add support for `log_level` in `create_job()` and `execute_job()` ([#704](https://github.com/Open-EO/openeo-python-client/issues/704))
+- Add initial support for "geometry" dimension type in `CubeMetadata` ([#705](https://github.com/Open-EO/openeo-python-client/issues/705))
+- Add support for parameterized `bands` argument in `load_stac()`
+- Argument `spatial_extent` in `load_collection()`/`load_stac()`: add support for Shapely objects, loading GeoJSON from a local path and loading geometry from GeoJSON/GeoParquet URL. ([#678](https://github.com/Open-EO/openeo-python-client/issues/678))
+
+### Changed
+
+- Raise exception when providing empty bands array to `load_collection`/`load_stac` ([#424](https://github.com/Open-EO/openeo-python-client/issues/424), [Open-EO/openeo-processes#372](https://github.com/Open-EO/openeo-processes/issues/372))
+- Start showing deprecation warnings on usage of GeoJSON "GeometryCollection" (in `filter_spatial`, `aggregate_spatial`, `chunk_polygon`, `mask_polygon`). Use a GeoJSON FeatureCollection instead. ([#706](https://github.com/Open-EO/openeo-python-client/issues/706), [Open-EO/openeo-processes#389](https://github.com/Open-EO/openeo-processes/issues/389))
+- The `context` parameter is now used in `execute_local_udf` ([#556](https://github.com/Open-EO/openeo-python-client/issues/556)
+
 ### Fixed
 
+- Clear capabilities cache on login ([#254](https://github.com/Open-EO/openeo-python-client/issues/254))
+
 
 ## [0.36.0] - 2024-12-10
 

docs/api.rst

@@ -63,19 +63,30 @@ openeo.metadata
    :members: CollectionMetadata, BandDimension, SpatialDimension, TemporalDimension
 
 
+openeo.rest.capabilities
+------------------------
+
+.. automodule:: openeo.rest.capabilities
+   :members: OpenEoCapabilities
+
+
+openeo.rest.models
+-------------------
+
+.. automodule:: openeo.rest.models.general
+    :members:
+
+.. automodule:: openeo.rest.models.logs
+    :members: LogEntry, normalize_log_level
+
+
 openeo.api.process
 --------------------
 
 .. automodule:: openeo.api.process
     :members: Parameter
 
 
-openeo.api.logs
------------------
-
-.. automodule:: openeo.api.logs
-    :members: LogEntry, normalize_log_level
-
 
 openeo.rest.connection
 ----------------------

docs/auth.rst

@@ -22,29 +22,6 @@ we start form a back-end connection::
 
     connection = openeo.connect("https://openeo.example.com")
 
-Basic HTTP Auth
-===============
-
-Let's start with the easiest authentication method,
-based on the Basic HTTP authentication scheme.
-It is however *not recommended* for various reasons,
-such as its limited *security* measures.
-For example, if you are connecting to a back-end with a ``http://`` URL
-instead of a ``https://`` one, you should certainly not use basic HTTP auth.
-
-With these security related caveats out of the way, you authenticate
-using your username and password like this::
-
-    connection.authenticate_basic("john", "j0hn123")
-
-Subsequent usage of the connection object ``connection`` will
-use authenticated calls.
-For example, show information about the authenticated user::
-
-    >>> connection.describe_account()
-    {'user_id': 'john'}
-
-
 
 OpenID Connect Based Authentication
 ===================================
@@ -90,15 +67,15 @@ in the context of working with openEO:
     * :ref:`authenticate_oidc_refresh_token`
 
 
-OpenID Connect is clearly more complex than Basic HTTP Auth.
+
 In the sections below we will discuss the practical details of each flow.
 
 General options
 ---------------
 
 *   A back-end might support **multiple OpenID Connect providers**.
     The openEO Python Client Library will pick the first one by default,
-    but another another provider can specified explicity with the ``provider_id`` argument, e.g.:
+    but another another provider can specified explicitly with the ``provider_id`` argument, e.g.:
 
     .. code-block:: python
 
@@ -329,7 +306,6 @@ For example:
 
 
 
-
 .. _auth_configuration_files:
 
 Auth config files and ``openeo-auth`` helper tool
@@ -403,27 +379,6 @@ the current configuration and stored refresh tokens, e.g.::
 The sensitive information (like passwords) are redacted by default.
 
 
-
-Basic HTTP Auth config
------------------------
-
-With the ``add-basic`` subcommand you can add Basic HTTP Auth credentials
-for a given back-end to the config.
-It will interactively ask for username and password and
-try if these credentials work::
-
-    $ openeo-auth add-basic https://openeo.example.com/
-    Enter username and press enter: john
-    Enter password and press enter:
-    Trying to authenticate with 'https://openeo.example.com'
-    Successfully authenticated 'john'
-    Saved credentials to '/home/john/.config/openeo-python-client/auth-config.json'
-
-Now you can authenticate in your application without having to
-specify username and password explicitly::
-
-    connection.authenticate_basic()
-
 OpenID Connect configs
 -----------------------
 
@@ -472,6 +427,54 @@ For example::
 
 
 
+
+Basic HTTP Auth
+===============
+
+The easiest authentication method is
+based on the Basic HTTP authentication scheme.
+It is however *not recommended* for various reasons,
+such as its limited *security* measures.
+For example, if you are connecting to a back-end with a ``http://`` URL
+instead of a ``https://`` one, you should certainly not use basic HTTP auth.
+
+With these security related caveats out of the way, you authenticate
+using your username and password like this::
+
+    connection.authenticate_basic("john", "j0hn123")
+
+Subsequent usage of the connection object ``connection`` will
+use authenticated calls.
+For example, show information about the authenticated user::
+
+    >>> connection.describe_account()
+    {'user_id': 'john'}
+
+
+
+
+Basic HTTP Auth config
+-----------------------
+
+With the ``add-basic`` subcommand you can add Basic HTTP Auth credentials
+for a given back-end to the config.
+It will interactively ask for username and password and
+try if these credentials work::
+
+    $ openeo-auth add-basic https://openeo.example.com/
+    Enter username and press enter: john
+    Enter password and press enter:
+    Trying to authenticate with 'https://openeo.example.com'
+    Successfully authenticated 'john'
+    Saved credentials to '/home/john/.config/openeo-python-client/auth-config.json'
+
+Now you can authenticate in your application without having to
+specify username and password explicitly::
+
+    connection.authenticate_basic()
+
+
+
 .. _default_url_and_auto_auth:
 
 Default openEO back-end URL and auto-authentication

docs/basics.rst

@@ -317,6 +317,7 @@ and observations are filtered out:
 
 .. image:: _static/images/basics/evi-masked-composite.png
 
+.. _aggregate-spatial-evi
 
 Aggregated EVI timeseries
 ===========================

docs/cookbook/job_manager.rst

@@ -1,3 +1,5 @@
+.. _job-manager
+
 ====================================
 Multi Backend Job Manager
 ====================================

docs/cookbook/sampling.md

@@ -1,21 +1,37 @@
 
 # Dataset sampling
 
-
 A number of use cases do not require a full datacube to be computed,
 but rather want to extract a result at specific locations.
 Examples include extracting training data for model calibration, or computing the result for
 areas where validation data is available.
 
 An important constraint is that most implementations assume that sampling is an operation 
 on relatively small areas, of for instance up to 512x512 pixels (but often much smaller). 
-When extracting larger areas, it is recommended to look into running a separate job per 'sample'.
+When extracting polygons with larger areas, it is recommended to look into running a separate job per 'sample'.
+Some more important performance notices are mentioned later in the chapter, please read them carefully 
+to get best results.
 
 Sampling can be done for points or polygons:
 
 - point extractions basically result in a 'vector cube', so can be exported into tabular formats.
 - polygon extractions  can be stored to an individual netCDF per polygon so in this case the output is a sparse raster cube.
 
+Note that sampling many points or polygons may require to send a large amount of geometry, which sometimes makes the size
+of the requests too large when it is included inline as GeoJson. Therefore, we recommend to upload your vector data to a
+public url, and to load it in openEO using {py:meth}`openeo.rest.connection.Connection.load_url`.
+
+## Sampling at point locations
+
+To sample point locations, the `openeo.rest.datacube.DataCube.aggregate_spatial` method can be used. The reducer can be a 
+commonly supported reducer like `min`, `max` or `mean` and will receive only one value as input in most cases. Note that
+in edge cases, a point can intersect with up to 4 pixels. If this is not desirable, it might be worth trying to align 
+points with pixel centers, which does require more advanced knowledge of the pixel grid of your data cube.
+
+More information on `aggregate_spatial` is available [here](_aggregate-spatial-evi).
+
+## Sampling polygons as rasters
+
 To indicate to openEO that we only want to compute the datacube for certain polygon features, we use the
 `openeo.rest.datacube.DataCube.filter_spatial` method.
 
@@ -59,3 +75,5 @@ When doing large scale (e.g. continental) sampling, it is usually not possible o
 batch job. The recommendation here is to apply a spatial grouping to your sampling locations, with a single group covering
 an area of around 100x100km. The optimal size of a group may be backend dependant. Also remember that when working with
 data in the UTM projection, you may want to avoid covering multiple UTM zones in a single group.
+
+See also how to manage [multiple jobs](_job-manager).

docs/federation-extension.rst

@@ -0,0 +1,70 @@
+
+.. _federation-extension:
+
+===========================
+openEO Federation Extension
+===========================
+
+
+The `openEO Federation extension <https://github.com/Open-EO/openeo-api/tree/master/extensions/federation>`_
+is a set of additional specifications,
+on top of the standard openEO API specification,
+to address the need for extra metadata in the context
+of federated openEO processing,
+where multiple (separately operated) openEO services are bundled together
+behind a single API endpoint.
+
+
+Accessing federation extension metadata
+========================================
+
+The openEO Python client library provides access to this additional metadata
+in a couple of resources.
+
+.. versionadded:: 0.38.0
+    initial support to access federation extension related metadata.
+
+.. warning:: this API is experimental and subject to change.
+
+
+Backend details
+---------------
+
+Participating backends in a federation are listed under the ``federation`` field
+of the capabilities document (``GET /``) and can be inspected
+using :py:meth:`OpenEoCapabilities.ext_federation_backend_details() <openeo.rest.capabilities.OpenEoCapabilities.ext_federation_backend_details>`:
+
+.. code-block:: python
+
+    import openeo
+    connection = openeo.connect(url=...)
+    capabilities = connection.capabilities()
+    print("Federated backends:", capabilities.ext_federation_backend_details())
+
+
+Unavailable backends (``federation:missing``)
+----------------------------------------------
+
+When listing resources like
+collections (with :py:meth:`Connection.list_collections() <openeo.rest.connection.Connection.list_collections>`),
+processes (with :py:meth:`Connection.list_processes() <openeo.rest.connection.Connection.list_processes>`),
+jobs (with :py:meth:`Connection.list_jobs() <openeo.rest.connection.Connection.list_jobs>`),
+etc.,
+there might be items missing due to federation participants being temporarily unavailable.
+These missing federation components are listed in the response under the ``federation:missing`` field
+and can be inspected as follows:
+
+.. code-block:: python
+
+    import openeo
+    connection = openeo.connect(url=...)
+    collections = connection.list_collections()
+    print("Number of collections:", len(collections))
+    print("Missing federation components:", collections.ext_federation_missing())
+
+
+Note that the ``collections`` object in this example, returned by
+:py:meth:`Connection.list_collections() <openeo.rest.connection.Connection.list_collections>`,
+acts at the surface as a simple list of dictionaries with collection metadata,
+but also provides additional properties/methods like
+:py:attr:`ext_federation_missing() <openeo.rest.models.general.CollectionListingResponse.ext_federation_missing>`.

docs/index.rst

@@ -64,6 +64,7 @@ Table of contents
    process_mapping
    development
    best_practices
+   Federation extension <federation-extension>
    changelog
 
 

openeo/_version.py

@@ -1 +1 @@
-__version__ = "0.37.0a1"
+__version__ = "0.38.0a3"