STAC SENTINEL2_L2A SCL band alignment issue after resample_spatial + aggregate_spatial

[JeroenVerstraelen]

Minimal code to reproduce:

import openeo

connection = openeo.connect("openeo.dev.warsaw.openeo.dataspace.copernicus.eu").authenticate_oidc()

spatial_extent = {
    "west": 3649940,   # center 3652500 - 256*20/2
    "south": 2094940,  # center 2097500 - 256*20/2
    "east": 3655060,   # center 3652500 + 256*20/2
    "north": 2100060,  # center 2097500 + 256*20/2
    "crs": "EPSG:3035",
}

datacube = connection.load_collection(
    "SENTINEL2_L2A",
    spatial_extent=spatial_extent,
    temporal_extent=["2024-01-01", "2024-01-10"],
    bands=["SCL"],
    max_cloud_cover=90,
)

datacube = datacube.resample_spatial(
    resolution=20,
    projection="EPSG:3035",
    method="near",
)

feature_collections = {
        "geometries": {
            "features": [
                {
                    "geometry": {
                        "coordinates": [
                            [
                                [
                                    2.087900765675578,
                                    41.59887309605375
                                ],
                                [
                                    2.0701235855616362,
                                    41.73630244825719
                                ],
                                [
                                    1.8876825985159764,
                                    41.72163837980604
                                ],
                                [
                                    1.9058605900418466,
                                    41.584247051069255
                                ],
                                [
                                    2.087900765675578,
                                    41.59887309605375
                                ]
                            ]
                        ],
                        "type": "Polygon"
                    },
                    "properties": {},
                    "type": "Feature"
                }
            ],
            "type": "FeatureCollection"
        },
}

datacube = datacube.aggregate_spatial(geometries = feature_collections["geometries"], reducer="mean")

datacube.download(
    outputfile="output_agg_spatial.json",
)

Batch job: j-260224151703475b9f9544194372d97f

Full error:

Traceback (most recent call last):
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 2004, in apply_process
    return process_function(args=ProcessArgs(args, process_id=process_id), env=env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1513, in aggregate_spatial
    return cube.aggregate_spatial(geometries=geoms, reducer=reduce_pg, target_dimension=target_dimension)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/geopysparkdatacube.py", line 97](vscode://file//opt/venv/lib64/python3.11/site-packages/openeogeotrellis/geopysparkdatacube.py?line=97&workspace=/home/jeroen/Projects/openEO/openeo-geopyspark-driver), in run
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  [File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/geopysparkdatacube.py", line 1759](vscode://file//opt/venv/lib64/python3.11/site-packages/openeogeotrellis/geopysparkdatacube.py?line=1759&workspace=/home/jeroen/Projects/openEO/openeo-geopyspark-driver), in aggregate_spatial
    return self.zonal_statistics(geometries, visitor.builder)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/geopysparkdatacube.py", line 1839](vscode://file//opt/venv/lib64/python3.11/site-packages/openeogeotrellis/geopysparkdatacube.py?line=1839&workspace=/home/jeroen/Projects/openEO/openeo-geopyspark-driver), in zonal_statistics
    self._compute_stats_geotrellis().compute_generic_timeseries_from_datacube(
  File "/usr/local/spark/python/lib/py4j-0.10.9.9-src.zip/py4j/java_gateway.py", line 1362, in __call__
    return_value = get_return_value(
                   ^^^^^^^^^^^^^^^^^
  File "/usr/local/spark/python/lib/py4j-0.10.9.9-src.zip/py4j/protocol.py", line 327, in get_return_value
    raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling o2138.compute_generic_timeseries_from_datacube.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 6.1 failed 4 times, most recent failure: Lost task 0.3 in stage 6.1 (TID 82) (10.42.233.83 executor 4): java.lang.IllegalArgumentException: requirement failed: x-aligned: offset by CellSize(20.0,20.0) 0.5000035031232983
	at scala.Predef$.require(Predef.scala:337)
	at geotrellis.layer.LayoutTileSource$.geotrellis$layer$LayoutTileSource$$requireGridAligned(LayoutTileSource.scala:228)
	at geotrellis.layer.LayoutTileSource.(LayoutTileSource.scala:34)
	at org.openeo.geotrellis.layers.LayoutTileSourceFixed.(FileLayerProvider.scala:55)
	at org.openeo.geotrellis.layers.FileLayerProvider.$anonfun$convertToRasterRegions$4(FileLayerProvider.scala:1256)
	at scala.collection.Iterator$$anon$10.nextCur(Iterator.scala:594)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:608)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:601)
	at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:143)
	at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:57)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:111)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:54)
	at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:171)
	at org.apache.spark.scheduler.Task.run(Task.scala:147)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$5(Executor.scala:647)
	at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:80)
	at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:77)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:650)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Driver stacktrace:
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$3(DAGScheduler.scala:2935)
	at scala.Option.getOrElse(Option.scala:201)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2935)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2927)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2927)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1295)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1295)
	at scala.Option.foreach(Option.scala:437)
	at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1295)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:3207)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:3141)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:3130)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:50)
	at org.apache.spark.util.Utils$.getTryWithCallerStacktrace(Utils.scala:1439)
	at org.apache.spark.util.LazyTry.get(LazyTry.scala:58)
	at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:131)
	at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:192)
	at org.apache.spark.sql.classic.DataFrameWriter.runCommand(DataFrameWriter.scala:622)
	at org.apache.spark.sql.classic.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:273)
	at org.apache.spark.sql.classic.DataFrameWriter.saveInternal(DataFrameWriter.scala:241)
	at org.apache.spark.sql.classic.DataFrameWriter.save(DataFrameWriter.scala:118)
	at org.apache.spark.sql.DataFrameWriter.csv(DataFrameWriter.scala:426)
	at org.openeo.geotrellis.aggregate_polygon.AggregatePolygonProcess.aggregateByDateAndPolygon(AggregatePolygonProcess.scala:373)
	at org.openeo.geotrellis.aggregate_polygon.AggregatePolygonProcess.aggregateSpatialGeneric(AggregatePolygonProcess.scala:322)
	at org.openeo.geotrellis.ComputeStatsGeotrellisAdapter.compute_generic_timeseries_from_datacube(ComputeStatsGeotrellisAdapter.scala:107)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:75)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:52)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)
	at py4j.Gateway.invoke(Gateway.java:282)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:184)
	at py4j.ClientServerConnection.run(ClientServerConnection.java:108)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: org.apache.spark.util.Utils$OriginalTryStackTraceException: Full stacktrace of original doTryWithCallerStacktrace caller
		at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$3(DAGScheduler.scala:2935)
		at scala.Option.getOrElse(Option.scala:201)
		at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2935)
		at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2927)
		at scala.collection.immutable.List.foreach(List.scala:334)
		at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2927)
		at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1295)
		at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1295)
		at scala.Option.foreach(Option.scala:437)
		at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1295)
		at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:3207)
		at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:3141)
		at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:3130)
		at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:50)
Caused by: java.lang.IllegalArgumentException: requirement failed: x-aligned: offset by CellSize(20.0,20.0) 0.5000035031232983
	at scala.Predef$.require(Predef.scala:337)
	at geotrellis.layer.LayoutTileSource$.geotrellis$layer$LayoutTileSource$$requireGridAligned(LayoutTileSource.scala:228)
	at geotrellis.layer.LayoutTileSource.(LayoutTileSource.scala:34)
	at org.openeo.geotrellis.layers.LayoutTileSourceFixed.(FileLayerProvider.scala:55)
	at org.openeo.geotrellis.layers.FileLayerProvider.$anonfun$convertToRasterRegions$4(FileLayerProvider.scala:1256)
	at scala.collection.Iterator$$anon$10.nextCur(Iterator.scala:594)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:608)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:601)
	at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:143)
	at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:57)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:111)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:54)
	at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:171)
	at org.apache.spark.scheduler.Task.run(Task.scala:147)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$5(Executor.scala:647)
	at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:80)
	at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:77)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:650)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	... 1 more


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  [File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py", line 1177](vscode://file//opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py?line=1177&workspace=/home/jeroen/Projects/openEO/openeo-geopyspark-driver), in start_main
    main(sys.argv)
  [File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py", line 260](vscode://file//opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py?line=260&workspace=/home/jeroen/Projects/openEO/openeo-geopyspark-driver), in main
    run_driver()
  [File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py", line 221](vscode://file//opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py?line=221&workspace=/home/jeroen/Projects/openEO/openeo-geopyspark-driver), in run_driver
    run_job(
  [File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/utils.py", line 66](vscode://file//opt/venv/lib64/python3.11/site-packages/openeogeotrellis/utils.py?line=66&workspace=/home/jeroen/Projects/openEO/openeo-geopyspark-driver), in memory_logging_wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  [File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py", line 351](vscode://file//opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py?line=351&workspace=/home/jeroen/Projects/openEO/openeo-geopyspark-driver), in run_job
    result = ProcessGraphDeserializer.evaluate(process_graph, env=env, do_dry_run=tracer)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 496, in evaluate
    result = convert_node(top_level_node, env=env)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 580, in convert_node
    process_result = apply_process(
                     ^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1984, in apply_process
    args = {name: convert_node(expr, env=env) for (name, expr) in sorted(args.items())}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1984, in 
    args = {name: convert_node(expr, env=env) for (name, expr) in sorted(args.items())}
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 621, in convert_node
    return [convert_node(x, env=env) for x in processGraph]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 621, in 
    return [convert_node(x, env=env) for x in processGraph]
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 580, in convert_node
    process_result = apply_process(
                     ^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1984, in apply_process
    args = {name: convert_node(expr, env=env) for (name, expr) in sorted(args.items())}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1984, in 
    args = {name: convert_node(expr, env=env) for (name, expr) in sorted(args.items())}
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 607, in convert_node
    return convert_node(processGraph['node'], env=env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 580, in convert_node
    process_result = apply_process(
                     ^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 2014, in apply_process
    raise OpenEOApiException(f"Unexpected error during {process_id!r}: {detail}. The process had these arguments: {args!r} ") from e
openeo_driver.errors.OpenEOApiException: Unexpected error during 'aggregate_spatial': Exception during Spark execution: java.lang.IllegalArgumentException: requirement failed: x-aligned: offset by CellSize(20.0,20.0) 0.5000035031232983. The process had these arguments: {'data': GeopysparkDataCube(metadata=GeopysparkCubeMetadata(dimension_names=['x', 'y', 't', 'bands'], band_names=['SCL'])), 'geometries': {'features': [{'geometry': {'coordinates': [[[2.087900765675578, 41.59887309605375], [2.0701235855616362, 41.73630244825719], [1.8876825985159764, 41.72163837980604], [1.9058605900418466, 41.584247051069255], [2.087900765675578, 41.59887309605375]]], 'type': 'Polygon'}, 'properties': {}, 'type': 'Feature'}], 'type': 'FeatureCollection'}, 'reducer': {'process_graph': {'mean1': {'process_id': 'mean', 'arguments': {'data': {'from_parameter': 'data'}}, 'result': True}}}}

[JeroenVerstraelen]

Might be related to #1546

[JeroenVerstraelen]

The one feature passed via the FixedOpenSearchClient:
https://gist.githubusercontent.com/JeroenVerstraelen/a78444ddd1f40e137633ad6555f70863/raw/b79529c9f8540f53a21b6d3efaa4fe380dcd5cea/curl_response.json

[JeroenVerstraelen]

This should be the FixedFeaturesOpenSearchClient we send to Scala:

result = {FixedFeaturesOpenSearchClient} org.openeo.geotrellis.file.FixedFeaturesOpenSearchClient@ac499858
 features = {list: 1} [Feature(S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849,Extent(1.7835166850224629, 41.45751520585338, 3.1186941736230867, 42.45262974528233),2024-01-07T10:44:31.024Z,[Lorg.openeo.opensearch.OpenSearchResponses$Link;@1330b008,Some(10.0),None,Some(POLYGON ((1.783516685 42.4462357021, 3.1186941736 42.4526297457, 3.1168731032 41.463692517, 1.8021752047 41.4575152063, 1.783516685 42.4462357021))),Some(EPSG:32631),GeneralProperties(None,None,None,None,None),Some(Extent(399960.0, 4590240.0, 509760.0, 4700040.0)),None,0.0,Some(https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l2a/items/S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849))]
  0 = {JVMFeature} Feature(S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849,Extent(1.7835166850224629, 41.45751520585338, 3.1186941736230867, 42.45262974528233),2024-01-07T10:44:31.024Z,[Lorg.openeo.opensearch.OpenSearchResponses$Link;@1330b008,Some(10.0),None,Some(POLYGON ((1.783516685 42.4462357021, 3.1186941736 42.4526297457, 3.1168731032 41.463692517, 1.8021752047 41.4575152063, 1.783516685 42.4462357021))),Some(EPSG:32631),GeneralProperties(None,None,None,None,None),Some(Extent(399960.0, 4590240.0, 509760.0, 4700040.0)),None,0.0,Some(https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l2a/items/S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849))
   bbox = {dict: 4} {'xmax': 3.1186941736230867, 'xmin': 1.7835166850224629, 'ymax': 42.45262974528233, 'ymin': 41.45751520585338}
   cloudCover = {float} 0.0
   crs = {JVMCRS} EPSG:32631
   deduplicationOrderValue = {NoneType} None
   generalProperties = {JVMGeneralProperties} GeneralProperties(None,None,None,None,None)
   geometry = {JVMPolygon} POLYGON ((1.783516685 42.4462357021, 3.1186941736 42.4526297457, 3.1168731032 41.463692517, 1.8021752047 41.4575152063, 1.783516685 42.4462357021))
   id = {str} 'S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849'
   links = {list: 1} [Link(href=/eodata/Sentinel-2/MSI/L2A/2024/01/07/S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849.SAFE/GRANULE/L2A_T31TDG_A044621_20240107T104432/IMG_DATA/R20m/T31TDG_20240107T104431_SCL_20m.jp2, title=SCL_20m, Some(pixelValueOffset= 1.0,), Some(pixelValueOffset= 0.0,) bandNames=[SCL], rel=None)]
   nominalDate = {str} '2024-01-07T10:44:31.024Z'
   rasterExtent = {dict: 4} {'xmax': 509760.0, 'xmin': 399960.0, 'ymax': 4700040.0, 'ymin': 4590240.0}
   resolution = {float} 10.0
   selfUrl = {str} 'https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l2a/items/S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849'
   tileID = {NoneType} None

[jdries]

The loadparameters are also relevant:
load_params={'temporal_extent': ('2024-01-01', '2024-01-10'), 'spatial_extent': {'west': 3649940, 'south': 2094940, 'east': 3655060, 'north': 2100060, 'crs': 'EPSG:3035'}, 'global_extent': {'west': 3649940, 'south': 2094940, 'east': 3655060, 'north': 2100060, 'crs': 'EPSG:3035'}, 'bands': ['SCL'], 'properties': {'eo:cloud_cover': {'process_graph': {'lte1': {'process_id': 'lte', 'arguments': {'x': {'from_parameter': 'value'}, 'y': 90}, 'result': True}}}},

They do not line up with the cube generated in scala:
Loading https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l2a with params DataCubeParameters(128, {}, FloatingLayoutScheme, ByDay, None, None, NearestNeighbor, 0.0, 0.0) and bands SCL initial layout: LayoutDefinition(Extent(3644849.9999299375, 2089789.9993396755, 3660209.9999299375, 2105149.9993396755),CellSize(20.0,20.0),6x6 tiles,768x768 pixels)

Which is also reported:

Trying to construct a datacube with a bounds Extent(3644849.9999299375, 2089849.9993317127, 3660149.999938692, 2105149.9993396755) that is not entirely inside the global bounds: Extent(3649940.0, 2094940.0, 3655060.0, 2100060.0).

Weak spatial extent reported by python:
'weak_spatial_extent': {'west': 3644849.9999330803, 'south': 2089849.9993279488, 'east': 3660149.9999350472, 'north': 2105149.999335291, 'crs': 'EPSG:3035'},

So it looks like global bounds is not correct?

[JeroenVerstraelen]

LoadParameters on CDSE prod are identical:

load_collection: Creating raster datacube for SENTINEL2_L2A with arguments {'temporal_extent': ('2024-01-01', '2024-01-10'), 'spatial_extent': {'west': 3649940, 'south': 2094940, 'east': 3655060, 'north': 2100060, 'crs': 'EPSG:3035'}, 'global_extent': {'west': 3649940, 'south': 2094940, 'east': 3655060, 'north': 2100060, 'crs': 'EPSG:3035'}, 'bands': ['SCL'], 'properties': {'eo:cloud_cover': {'process_graph': {'lte1': {'process_id': 'lte', 'arguments': {'x': {'from_parameter': 'value'}, 'y': 90}, 'result': True}}}}, 'aggregate_spatial_geometries': DriverVectorCube( geometries: geometry 0 POLYGON ((2.0879 41.59887, 2.07012 41.7363, 1.... cube: Size: 8B array([nan]) Coordinates: * geometry (geometry) int64 8B 0 Attributes: vector_cube_dummy: True crs: 'EPSG:4326' bbox: (np.float64(1.8876825985159764), np.float64(41.584247051069255), np.float64(2.087900765675578), np.float64(41.73630244825719))), 'sar_backscatter': None, 'process_types': set(), 'custom_mask': {}, 'data_mask': None, 'target_crs': 'EPSG:3035', 'target_resolution': (20, 20), 'resample_method': 'near', 'pixel_buffer': None}, environment: {'vault_token': None, 'sentinel_hub_client_alias': None, 'max_soft_errors_ratio': 0.1, 'dependencies': [], 'pyramid_levels': 'highest', 'require_bounds': True, 'correlation_id': 'j-2602250723424e9ea749ecff1c918bfc', 'user': User('e3e8dcf3-ac7c-46c3-8bdd-034bf22dc342', None), 'openeo_api_version': '1.2', 'global_extent': BoundingBox(west=3649940, south=2094940, east=3655060, north=2100060, crs='EPSG:3035')}

Logs PROD

Feb 25, 2026 @ 08:24:24.545	Resampling datacube not necessary, resolution already at 20	INFO	geopysparkdatacube.py

Feb 25, 2026 @ 08:24:24.539	Reprojecting datacube with crs EPSG:3035 and layout LayoutDefinition(extent=Extent(xmin=3644849.9999299375, ymin=2089789.9993396755, xmax=3660209.9999299375, ymax=2105149.9993396755), tileLayout=TileLayout(layoutCols=6, layoutRows=6, tileCols=128, tileRows=128)) to EPSG:3035 and 20	INFO	geopysparkdatacube.py	

Feb 25, 2026 @ 08:24:24.458	Created cube for Sentinel2 with metadata TileLayerMetadata(uint8ud0,LayoutDefinition(Extent(3644849.9999299375, 2089789.9993396755, 3660209.9999299375, 2105149.9993396755),CellSize(20.0,20.0),6x6 tiles,768x768 pixels),Extent(3644849.9999299375, 2089849.9993317127, 3660149.999938692, 2105149.9993396755),EPSG:3035,KeyBounds(SpaceTimeKey(0,0,1704412800000),SpaceTimeKey(5,5,1704585600000))) and partitioner org.openeo.geotrelliscommon.package$ConfigurableSpaceTimePartitioner@445f115d	INFO	FileLayerProvider.scala	

Feb 25, 2026 @ 08:24:24.393	Cube partitioner index: org.openeo.geotrelliscommon.package$ConfigurableSpaceTimePartitioner@445f115d	INFO	FileLayerProvider.scala	

Feb 25, 2026 @ 08:24:24.342	Datacube is sparse: false, requiring 37 keys out of 36. 	INFO	DatacubeSupport.scala	

Feb 25, 2026 @ 08:24:21.298	Datacube requires approximately 36 spatial keys.	INFO	FileLayerProvider.scala	

Feb 25, 2026 @ 08:24:20.335	https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?box=1.887682598467632%2C41.58406743928556%2C2.088161943486884%2C41.73632139931961&page=1&maxRecords=1000&status=ONLINE&dataset=ESA-DATASET&productType=L2A&cloudCover=%5B0%2C90%5D&startDate=2024-01-01T00%3A00%3A00Z&completionDate=2024-01-10T00%3A00%3A00.000000001Z returned 200	INFO	OpenSearchClient.scala	

Feb 25, 2026 @ 08:24:19.091	Loading Sentinel2 with params DataCubeParameters(128, {}, FloatingLayoutScheme, ByDay, None, None, NearestNeighbor, 0.0, 0.0) and bands IMG_DATA_Band_SCL_20m_Tile1_Data initial layout: LayoutDefinition(Extent(3644849.9999299375, 2089789.9993396755, 3660209.9999299375, 2105149.9993396755),CellSize(20.0,20.0),6x6 tiles,768x768 pixels)	INFO	FileLayerProvider.scala	

Feb 25, 2026 @ 08:24:19.078	Trying to construct a datacube with a bounds Extent(3644849.9999299375, 2089849.9993317127, 3660149.999938692, 2105149.9993396755) that is not entirely inside the global bounds: Extent(3649940.0, 2094940.0, 3655060.0, 2100060.0). 	WARNING	DatacubeSupport.scala

Logs DEV

Feb 24, 2026 @ 16:17:41.554	Resampling datacube not necessary, resolution already at 20	INFO	geopysparkdatacube.py	

Feb 24, 2026 @ 16:17:41.549	Reprojecting datacube with crs EPSG:3035 and layout LayoutDefinition(extent=Extent(xmin=3644849.9999299375, ymin=2089789.9993396755, xmax=3660209.9999299375, ymax=2105149.9993396755), tileLayout=TileLayout(layoutCols=6, layoutRows=6, tileCols=128, tileRows=128)) to EPSG:3035 and 20	INFO	geopysparkdatacube.py	

Feb 24, 2026 @ 16:17:41.495	Created cube for https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l2a with metadata TileLayerMetadata(uint8ud0,LayoutDefinition(Extent(3644849.9999299375, 2089789.9993396755, 3660209.9999299375, 2105149.9993396755),CellSize(20.0,20.0),6x6 tiles,768x768 pixels),Extent(3644849.9999299375, 2089849.9993317127, 3660149.999938692, 2105149.9993396755),EPSG:3035,KeyBounds(SpaceTimeKey(0,0,1704585600000),SpaceTimeKey(5,5,1704585600000))) and partitioner org.openeo.geotrelliscommon.package$ConfigurableSpaceTimePartitioner@1bd76583	INFO	FileLayerProvider.scala	

Feb 24, 2026 @ 16:17:41.450	Cube partitioner index: org.openeo.geotrelliscommon.package$ConfigurableSpaceTimePartitioner@1bd76583	INFO	FileLayerProvider.scala	

Feb 24, 2026 @ 16:17:41.397	Datacube is sparse: false, requiring 37 keys out of 36. 	INFO	DatacubeSupport.scala	

Feb 24, 2026 @ 16:17:38.510	Datacube requires approximately 36 spatial keys.	INFO	FileLayerProvider.scala	

Feb 24, 2026 @ 16:17:38.197	Loading https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l2a with params DataCubeParameters(128, {}, FloatingLayoutScheme, ByDay, None, None, NearestNeighbor, 0.0, 0.0) and bands SCL initial layout: LayoutDefinition(Extent(3644849.9999299375, 2089789.9993396755, 3660209.9999299375, 2105149.9993396755),CellSize(20.0,20.0),6x6 tiles,768x768 pixels)	INFO	FileLayerProvider.scala	

Feb 24, 2026 @ 16:17:38.188	Trying to construct a datacube with a bounds Extent(3644849.9999299375, 2089849.9993317127, 3660149.999938692, 2105149.9993396755) that is not entirely inside the global bounds: Extent(3649940.0, 2094940.0, 3655060.0, 2100060.0). 	WARNING	DatacubeSupport.scala

Differences

Prod: KeyBounds(SpaceTimeKey(0,0,1704412800000),SpaceTimeKey(5,5,1704585600000)
Dev: KeyBounds(SpaceTimeKey(0,0,1704585600000),SpaceTimeKey(5,5,1704585600000))

So OpenSearch found multiple dates, while STAC only found one.

[JeroenVerstraelen]

Dry run extracted these source constraints also match up.

[JeroenVerstraelen]

Changing focus to the OpenSearchClient for a moment.

Prod
https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?box=1.887682598467632%2C41.58406743928556%2C2.088161943486884%2C41.73632139931961&page=1&maxRecords=1000&status=ONLINE&dataset=ESA-DATASET&productType=L2A&cloudCover=%5B0%2C90%5D&startDate=2024-01-01T00%3A00%3A00Z&completionDate=2024-01-10T00%3A00%3A00.000000001Z

• 3 features
  • /eodata/Sentinel-2/MSI/L2A/2024/01/07/S2A_MSIL2A_20240107T104431_N0510_R008_T31TCG_20240107T145849.SAFE
  • /eodata/Sentinel-2/MSI/L2A/2024/01/07/S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849.SAFE
  • /eodata/Sentinel-2/MSI/L2A/2024/01/05/S2B_MSIL2A_20240105T105339_N0510_R051_T31TCG_20240105T122825.SAFE
• search bbox: [1.887682598467632, 41.58406743928556, 2.088161943486884, 41.73632139931961]
• search temporal interval: [2024-01-01T00:00:00Z, 024-01-10T00:00:00Z] (completionDate)

Dev
https://gist.githubusercontent.com/JeroenVerstraelen/a78444ddd1f40e137633ad6555f70863/raw/b79529c9f8540f53a21b6d3efaa4fe380dcd5cea/curl_response.json (curl request)

• 1 feature
  • SCL_20m: /eodata/Sentinel-2/MSI/L2A/2024/01/07/S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849.SAFE/GRANULE/L2A_T31TDG_A044621_20240107T104432/IMG_DATA/R20m/T31TDG_20240107T104431_SCL_20m.jp2
  • SCL_60m: /eodata/Sentinel-2/MSI/L2A/2024/01/07/S2A_MSIL2A_20240107T104431_N0510_R008_T31TDG_20240107T145849.SAFE/GRANULE/L2A_T31TDG_A044621_20240107T104432/IMG_DATA/R60m/T31TDG_20240107T104431_SCL_60m.jp2
• search bbox: [1.8876825985159764, 41.584247051069255, 2.087900765675578, 41.73630244825719] (differs)
• search temporal interval: [2024-01-01T00:00:00Z, 2024-01-10T23:59:59.999000Z] (datetime) (differs)

[JeroenVerstraelen]

We do not include SCL_60m in our OpenSearch version (layercatalog.json):

        {
          "name": "SCL",
          "aliases": ["IMG_DATA_Band_SCL_20m_Tile1_Data"],
          "gsd": 20
        },

So I wonder what the nuances are that we include it in a feature flag in the STAC version (layercatalog.json):

          "asset_id_to_bands_map": {
            "AOT_10m": ["AOT"],
            "AOT_20m": ["AOT"],
            "AOT_60m": ["AOT"],
            "SCL_20m": ["SCL"],
            "SCL_60m": ["SCL"],
            "WVP_10m": ["WVP"],
            "WVP_20m": ["WVP"],
            "WVP_60m": ["WVP"],
            "CLD_20m": ["CLD"],
            "CLD_60m": ["CLD"],
            "SNW_20m": ["SNW"],
            "SNW_60m": ["SNW"]
          },