Merge pull request #1174 from xcube-dev/toniof-xxx-support_jpeg2000

Add support for JPEG 2000 to file system data stores

Author: TonioF

CHANGES.md

@@ -5,6 +5,12 @@
   `from xcube.core.store import get_filename_extensions`.
   It allows for retrieving mappings of recognized filename extensions to 
   respective data openers and writers.  (#726)
+* The file data stores (e.g., `store = new_data_store("file")`) now support 
+  reading JPEG2000 files as datasets or multi-level datasets from file
+  systems. Openers and writers are available as `dataset:jpeg2000:<protocol>` or
+  `mldataset:jpeg2000:<protocol>`, where protocol is the respective file system. 
+  These openers and writers are available to any file system data store, 
+  e.g., `"file"`, `"s3"`.
 
 ### Other changes
 

docs/source/dataaccess.md

@@ -299,8 +299,8 @@ The following `storage_options` can be used for the `ftp` data store:
 * `password` - User's password, if using
 
 All filesystem data stores can open datasets from various data formats. 
-Datasets in Zarr, GeoTIFF / COG, or NetCDF format will be provided either by
-[xarray.Dataset] or xcube [MultiLevelDataset] instances.
+Datasets in Zarr, GeoTIFF / COG, JPEG 2000, or NetCDF format will be provided 
+either by [xarray.Dataset] or xcube [MultiLevelDataset] instances.
 Datasets stored in GeoJSON or ESRI Shapefile will yield 
 [geopandas.GeoDataFrame] instances.
 

docs/source/mldatasets.md

@@ -25,7 +25,7 @@ In xcube, multi-resolution datasets are represented by the abstract class
 `xcube.core.mldataset.MultiLevelDataset`. The xcube data store framework
 refers to this datatype using the alias `mldataset`. The corresponding
 default data format is the xcube _Levels_ format, named `levels`.
-xcube also supports the Cloud Optimized GeoTIFF (COG) format 
+xcube also supports the Cloud Optimized GeoTIFF (COG) and the JPEG 2000 format 
 for reading multi-resolution datasets.
 
 The xcube Levels Format

environment.yml

@@ -20,6 +20,7 @@ dependencies:
   - geopandas >=0.8
   - jdcal >=1.4
   - jsonschema >=3.2
+  - libgdal-jp2openjpeg
   - mashumaro
   - matplotlib-base >=3.8.3
   - netcdf4 >=1.5

examples/serve/demo/sample-sb.jp2

@@ -2,6 +2,7 @@
 # Permissions are hereby granted under the terms of the MIT License:
 # https://opensource.org/licenses/MIT.
 
+import dask
 import json
 import os.path
 from test.cli.helpers import CliTest
@@ -71,7 +72,9 @@ def test_data(self):
                 "                           cube.nc  <no title>\n"
                 "                    sample-cog.tif  <no title>\n"
                 "                sample-geotiff.tif  <no title>\n"
-                "6 data resources found.\n"
+                "                     sample-sb.jp2  <no title>\n"
+                "                        sample.jp2  <no title>\n"
+                "8 data resources found.\n"
             ),
             result.stdout,
         )
@@ -143,11 +146,14 @@ def setUpClass(cls) -> None:
             yaml.dump(cls.STORE_CONF, fp)
         with open("store-conf.json", "w") as fp:
             json.dump(cls.STORE_CONF, fp)
+        cls._dask_scheduler = dask.config.get("scheduler", None)
+        dask.config.set(scheduler="single-threaded")
 
     @classmethod
     def tearDownClass(cls) -> None:
         rimraf("store-conf.json")
         rimraf("store-conf.yml")
+        dask.config.set(scheduler=cls._dask_scheduler)
 
     def tearDown(self) -> None:
         rimraf("store-dump.json")