Scenario Failure: fusets_mogpr

[github-actions]

Benchmark scenario ID: fusets_mogpr
Benchmark scenario definition: https://github.com/ESA-APEx/apex_algorithms/blob/44da560a781bdf18146b5b3856e74ba403009287/algorithm_catalog/vito/fusets_mogpr/benchmark_scenarios/fusets_mogpr.json
openEO backend: openeofed.dataspace.copernicus.eu

GitHub Actions workflow run: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19451636188
Workflow artifacts: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19451636188#artifacts

Test start: 2025-11-18 02:20:19.234728+00:00
Test duration: 0:20:09.385184
Test outcome: ❌ failed

Last successful test phase: download-reference
Failure in test phase: compare

Contact Information

Name	Organization	Contact
Bram Janssen	VITO	Contact via VITO (VITO Website, GitHub)

Process Graph

{
  "fusetsmogpr": {
    "process_id": "fusets_mogpr",
    "namespace": "https://raw.githubusercontent.com/ESA-APEx/apex_algorithms/6a70b010e7af41a19ce6d728972f1561295dd8c5/algorithm_catalog/vito/fusets_mogpr/openeo_udp/fusets_mogpr.json",
    "arguments": {
      "s1_collection": "RVI",
      "s2_collection": "NDVI",
      "spatial_extent": {
        "coordinates": [
          [
            [
              5.178303838475193,
              51.252856237848164
            ],
            [
              5.178003609252369,
              51.25109194151486
            ],
            [
              5.179280940922463,
              51.25103833409551
            ],
            [
              5.179565949577788,
              51.25278555186941
            ],
            [
              5.178303838475193,
              51.252856237848164
            ]
          ]
        ],
        "type": "Polygon"
      },
      "temporal_extent": [
        "2021-05-01",
        "2021-08-15"
      ]
    }
  },
  "save1": {
    "process_id": "save_result",
    "arguments": {
      "data": {
        "from_node": "fusetsmogpr"
      },
      "format": "NETCDF"
    },
    "result": true
  }
}

Error Logs

scenario = BenchmarkScenario(id='fusets_mogpr', description='Multi output gaussian process regression example on NDVI timeseries'...unner/work/apex_algorithms/apex_algorithms/algorithm_catalog/vito/fusets_mogpr/benchmark_scenarios/fusets_mogpr.json'))
connection_factory = <function connection_factory.<locals>.get_connection at 0x7f4c5f357920>
tmp_path = PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0')
track_metric = <function track_metric.<locals>.track at 0x7f4c5f357a60>
track_phase = <function track_phase.<locals>.track at 0x7f4c5f357ba0>
upload_assets_on_fail = <function upload_assets_on_fail.<locals>.collect at 0x7f4c5f357c40>
request = <FixtureRequest for <Function test_run_benchmark[fusets_mogpr]>>

    @pytest.mark.parametrize(
        "scenario",
        [
            # Use scenario id as parameterization id to give nicer test names.
            pytest.param(uc, id=uc.id)
            for uc in get_benchmark_scenarios()
        ],
    )
    def test_run_benchmark(
        scenario: BenchmarkScenario,
        connection_factory,
        tmp_path: Path,
        track_metric,
        track_phase,
        upload_assets_on_fail,
        request,
    ):
        track_metric("scenario_id", scenario.id)

        with track_phase(phase="connect"):
            # Check if a backend override has been provided via cli options.
            override_backend = request.config.getoption("--override-backend")
            backend_filter = request.config.getoption("--backend-filter")
            if backend_filter and not re.match(backend_filter, scenario.backend):
                # TODO apply filter during scenario retrieval, but seems to be hard to retrieve cli param
                pytest.skip(
                    f"skipping scenario {scenario.id} because backend {scenario.backend} does not match filter {backend_filter!r}"
                )
            backend = scenario.backend
            if override_backend:
                _log.info(f"Overriding backend URL with {override_backend!r}")
                backend = override_backend

            connection: openeo.Connection = connection_factory(url=backend)

        with track_phase(phase="create-job"):
            # TODO #14 scenario option to use synchronous instead of batch job mode?
            job = connection.create_job(
                process_graph=scenario.process_graph,
                title=f"APEx benchmark {scenario.id}",
                additional=scenario.job_options,
            )
            track_metric("job_id", job.job_id)

        with track_phase(phase="run-job"):
            # TODO: monitor timing and progress
            # TODO: abort excessively long batch jobs? https://github.com/Open-EO/openeo-python-client/issues/589
            job.start_and_wait()
            # TODO: separate "job started" and run phases?

        with track_phase(phase="collect-metadata"):
            collect_metrics_from_job_metadata(job, track_metric=track_metric)

            results = job.get_results()
            collect_metrics_from_results_metadata(results, track_metric=track_metric)

        with track_phase(phase="download-actual"):
            # Download actual results
            actual_dir = tmp_path / "actual"
            paths = results.download_files(target=actual_dir, include_stac_metadata=True)

            # Upload assets on failure
            upload_assets_on_fail(*paths)

        with track_phase(phase="download-reference"):
            reference_dir = download_reference_data(
                scenario=scenario, reference_dir=tmp_path / "reference"
            )

        with track_phase(
            phase="compare", describe_exception=analyse_results_comparison_exception
        ):
            # Compare actual results with reference data
>           assert_job_results_allclose(
                actual=actual_dir,
                expected=reference_dir,
                tmp_path=tmp_path,
                rtol=scenario.reference_options.get("rtol", 1e-6),
                atol=scenario.reference_options.get("atol", 1e-6),
                pixel_tolerance=scenario.reference_options.get("pixel_tolerance", 0.0),
            )

tests/test_benchmarks.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

actual = PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/actual')
expected = PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/reference')

    def assert_job_results_allclose(
        actual: Union[BatchJob, JobResults, str, Path],
        expected: Union[BatchJob, JobResults, str, Path],
        *,
        rtol: float = _DEFAULT_RTOL,
        atol: float = _DEFAULT_ATOL,
        pixel_tolerance: float = _DEFAULT_PIXELTOL,
        tmp_path: Optional[Path] = None,
    ):
        """
        Assert that two job results sets are equal (with tolerance).

        :param actual: actual job results, provided as :py:class:`~openeo.rest.job.BatchJob` object,
            :py:meth:`~openeo.rest.job.JobResults` object or path to directory with downloaded assets.
        :param expected: expected job results, provided as :py:class:`~openeo.rest.job.BatchJob` object,
            :py:meth:`~openeo.rest.job.JobResults` object or path to directory with downloaded assets.
        :param rtol: relative tolerance
        :param atol: absolute tolerance
        :param pixel_tolerance: maximum fraction of pixels (in percent)
            that is allowed to be significantly different (considering ``atol`` and ``rtol``)
        :param tmp_path: root temp path to download results if needed.
            It's recommended to pass pytest's `tmp_path` fixture here
        :raises AssertionError: if not equal within the given tolerance

        .. versionadded:: 0.31.0

        .. warning::
            This function is experimental and subject to change.
        """
        issues = _compare_job_results(
            actual, expected, rtol=rtol, atol=atol, pixel_tolerance=pixel_tolerance, tmp_path=tmp_path
        )
        if issues:
>           raise AssertionError("\n".join(issues))
E           AssertionError: Issues for file 'openEO.nc':
E           Issues for variable 'NDVI':
E           Coordinates mismatch for dimension 't': ['2021-05-01T00:00:00.000000000' '2021-05-03T00:00:00.000000000'
E            '2021-05-04T00:00:00.000000000' '2021-05-06T00:00:00.000000000'
E            '2021-05-09T00:00:00.000000000' '2021-05-10T00:00:00.000000000'
E            '2021-05-12T00:00:00.000000000' '2021-05-13T00:00:00.000000000'
E            '2021-05-15T00:00:00.000000000' '2021-05-16T00:00:00.000000000'
E            '2021-05-21T00:00:00.000000000' '2021-05-22T00:00:00.000000000'
E            '2021-05-25T00:00:00.000000000' '2021-05-28T00:00:00.000000000'
E            '2021-05-30T00:00:00.000000000' '2021-05-31T00:00:00.000000000'
E            '2021-06-01T00:00:00.000000000' '2021-06-08T00:00:00.000000000'
E            '2021-06-09T00:00:00.000000000' '2021-06-11T00:00:00.000000000'
E            '2021-06-12T00:00:00.000000000' '2021-06-14T00:00:00.000000000'
E            '2021-06-15T00:00:00.000000000' '2021-06-16T00:00:00.000000000'
E            '2021-06-21T00:00:00.000000000' '2021-06-23T00:00:00.000000000'
E            '2021-06-24T00:00:00.000000000' '2021-06-26T00:00:00.000000000'
E            '2021-06-27T00:00:00.000000000' '2021-07-03T00:00:00.000000000'
E            '2021-07-05T00:00:00.000000000' '2021-07-06T00:00:00.000000000'
E            '2021-07-09T00:00:00.000000000' '2021-07-11T00:00:00.000000000'
E            '2021-07-12T00:00:00.000000000' '2021-07-14T00:00:00.000000000'
E            '2021-07-17T00:00:00.000000000' '2021-07-20T00:00:00.000000000'
E            '2021-07-21T00:00:00.000000000' '2021-07-26T00:00:00.000000000'
E            '2021-07-29T00:00:00.000000000' '2021-07-30T00:00:00.000000000'
E            '2021-08-02T00:00:00.000000000' '2021-08-04T00:00:00.000000000'
E            '2021-08-05T00:00:00.000000000' '2021-08-07T00:00:00.000000000'
E            '2021-08-10T00:00:00.000000000' '2021-08-13T00:00:00.000000000'] != ['2021-05-01T00:00:00.000000000' '2021-05-03T00:00:00.000000000'
E            '2021-05-04T00:00:00.000000000' '2021-05-06T00:00:00.000000000'
E            '2021-05-07T00:00:00.000000000' '2021-05-09T00:00:00.000000000'
E            '2021-05-10T00:00:00.000000000' '2021-05-12T00:00:00.000000000'
E            '2021-05-13T00:00:00.000000000' '2021-05-15T00:00:00.000000000'
E            '2021-05-16T00:00:00.000000000' '2021-05-18T00:00:00.000000000'
E            '2021-05-19T00:00:00.000000000' '2021-05-21T00:00:00.000000000'
E            '2021-05-22T00:00:00.000000000' '2021-05-25T00:00:00.000000000'
E            '2021-05-27T00:00:00.000000000' '2021-05-28T00:00:00.000000000'
E            '2021-05-30T00:00:00.000000000' '2021-05-31T00:00:00.000000000'
E            '2021-06-01T00:00:00.000000000' '2021-06-02T00:00:00.000000000'
E            '2021-06-05T00:00:00.000000000' '2021-06-06T00:00:00.000000000'
E            '2021-06-08T00:00:00.000000000' '2021-06-09T00:00:00.000000000'
E            '2021-06-11T00:00:00.000000000' '2021-06-12T00:00:00.000000000'
E            '2021-06-14T00:00:00.000000000' '2021-06-15T00:00:00.000000000'
E            '2021-06-16T00:00:00.000000000' '2021-06-17T00:00:00.000000000'
E            '2021-06-18T00:00:00.000000000' '2021-06-20T00:00:00.000000000'
E            '2021-06-21T00:00:00.000000000' '2021-06-23T00:00:00.000000000'
E            '2021-06-24T00:00:00.000000000' '2021-06-26T00:00:00.000000000'
E            '2021-06-27T00:00:00.000000000' '2021-06-29T00:00:00.000000000'
E            '2021-07-02T00:00:00.000000000' '2021-07-03T00:00:00.000000000'
E            '2021-07-05T00:00:00.000000000' '2021-07-06T00:00:00.000000000'
E            '2021-07-08T00:00:00.000000000' '2021-07-09T00:00:00.000000000'
E            '2021-07-11T00:00:00.000000000' '2021-07-12T00:00:00.000000000'
E            '2021-07-14T00:00:00.000000000' '2021-07-15T00:00:00.000000000'
E            '2021-07-17T00:00:00.000000000' '2021-07-18T00:00:00.000000000'
E            '2021-07-20T00:00:00.000000000' '2021-07-21T00:00:00.000000000'
E            '2021-07-23T00:00:00.000000000' '2021-07-24T00:00:00.000000000'
E            '2021-07-26T00:00:00.000000000' '2021-07-27T00:00:00.000000000'
E            '2021-07-29T00:00:00.000000000' '2021-07-30T00:00:00.000000000'
E            '2021-08-01T00:00:00.000000000' '2021-08-02T00:00:00.000000000'
E            '2021-08-04T00:00:00.000000000' '2021-08-05T00:00:00.000000000'
E            '2021-08-07T00:00:00.000000000' '2021-08-08T00:00:00.000000000'
E            '2021-08-10T00:00:00.000000000' '2021-08-11T00:00:00.000000000'
E            '2021-08-13T00:00:00.000000000' '2021-08-14T00:00:00.000000000']
E           Shape mismatch: (48, 22, 13) != (70, 22, 13)
E           Left and right DataArray objects are not close
E           Differing dimensions:
E               (t: 48, y: 22, x: 13) != (t: 70, y: 22, x: 13)
E           Differing values:
E           L
E               array([[[0.771753, 0.883915, ..., 0.844254, 0.835507],
E                       [0.773455, 0.859335, ..., 0.800632, 0.805507],
E                       ...,
E                       [0.856733, 0.837003, ..., 0.795507, 0.710954],
E                       [0.85888 , 0.832066, ..., 0.75917 , 0.822665]],
E
E                      [[0.689931, 0.871564, ..., 0.84595 , 0.838386],
E                       [0.762784, 0.85209 , ..., 0.798707, 0.806461],
E                       ...,
E                       [0.84868 , 0.838156, ..., 0.814907, 0.69955 ],
E                       [0.852071, 0.819354, ..., 0.778458, 0.845907]],
E
E                      ...,
E
E                      [[0.779451, 0.881323, ..., 0.868071, 0.855663],
E                       [0.849817, 0.860136, ..., 0.797537, 0.806353],
E                       ...,
E                       [0.852319, 0.860358, ..., 0.823804, 0.791739],
E                       [0.855211, 0.877841, ..., 0.863737, 0.796131]],
E
E                      [[0.789354, 0.876496, ..., 0.845201, 0.811117],
E                       [0.851216, 0.861061, ..., 0.799507, 0.807587],
E                       ...,
E                       [0.850817, 0.85234 , ..., 0.827194, 0.819885],
E                       [0.851584, 0.853758, ..., 0.839596, 0.794814]]], shape=(48, 22, 13))
E           R
E               array([[[0.779571, 0.888816, ..., 0.843326, 0.840004],
E                       [0.774682, 0.77043 , ..., 0.738716, 0.678624],
E                       ...,
E                       [0.866368, 0.855249, ..., 0.802093, 0.674559],
E                       [0.866331, 0.826874, ..., 0.793126, 0.787232]],
E
E                      [[0.731947, 0.888852, ..., 0.845942, 0.829075],
E                       [0.763833, 0.755404, ..., 0.730626, 0.663462],
E                       ...,
E                       [0.845666, 0.843128, ..., 0.808993, 0.657086],
E                       [0.846949, 0.80912 , ..., 0.800771, 0.766645]],
E
E                      ...,
E
E                      [[0.792853, 0.885432, ..., 0.844786, 0.831449],
E                       [0.853772, 0.861507, ..., 0.782855, 0.778632],
E                       ...,
E                       [0.850382, 0.850807, ..., 0.813363, 0.857644],
E                       [0.848869, 0.852967, ..., 0.825004, 0.811901]],
E
E                      [[0.791539, 0.885514, ..., 0.818256, 0.825988],
E                       [0.853873, 0.862271, ..., 0.766543, 0.769633],
E                       ...,
E                       [0.855736, 0.870099, ..., 0.805276, 0.810298],
E                       [0.847268, 0.855199, ..., 0.818124, 0.807418]]], shape=(70, 22, 13))
E           Differing coordinates:
E           L * t        (t) datetime64[ns] 384B 2021-05-01 2021-05-03 ... 2021-08-13
E           R * t        (t) datetime64[ns] 560B 2021-05-01 2021-05-03 ... 2021-08-14
E           Issues for variable 'RVI':
E           Coordinates mismatch for dimension 't': ['2021-05-01T00:00:00.000000000' '2021-05-03T00:00:00.000000000'
E            '2021-05-04T00:00:00.000000000' '2021-05-06T00:00:00.000000000'
E            '2021-05-09T00:00:00.000000000' '2021-05-10T00:00:00.000000000'
E            '2021-05-12T00:00:00.000000000' '2021-05-13T00:00:00.000000000'
E            '2021-05-15T00:00:00.000000000' '2021-05-16T00:00:00.000000000'
E            '2021-05-21T00:00:00.000000000' '2021-05-22T00:00:00.000000000'
E            '2021-05-25T00:00:00.000000000' '2021-05-28T00:00:00.000000000'
E            '2021-05-30T00:00:00.000000000' '2021-05-31T00:00:00.000000000'
E            '2021-06-01T00:00:00.000000000' '2021-06-08T00:00:00.000000000'
E            '2021-06-09T00:00:00.000000000' '2021-06-11T00:00:00.000000000'
E            '2021-06-12T00:00:00.000000000' '2021-06-14T00:00:00.000000000'
E            '2021-06-15T00:00:00.000000000' '2021-06-16T00:00:00.000000000'
E            '2021-06-21T00:00:00.000000000' '2021-06-23T00:00:00.000000000'
E            '2021-06-24T00:00:00.000000000' '2021-06-26T00:00:00.000000000'
E            '2021-06-27T00:00:00.000000000' '2021-07-03T00:00:00.000000000'
E            '2021-07-05T00:00:00.000000000' '2021-07-06T00:00:00.000000000'
E            '2021-07-09T00:00:00.000000000' '2021-07-11T00:00:00.000000000'
E            '2021-07-12T00:00:00.000000000' '2021-07-14T00:00:00.000000000'
E            '2021-07-17T00:00:00.000000000' '2021-07-20T00:00:00.000000000'
E            '2021-07-21T00:00:00.000000000' '2021-07-26T00:00:00.000000000'
E            '2021-07-29T00:00:00.000000000' '2021-07-30T00:00:00.000000000'
E            '2021-08-02T00:00:00.000000000' '2021-08-04T00:00:00.000000000'
E            '2021-08-05T00:00:00.000000000' '2021-08-07T00:00:00.000000000'
E            '2021-08-10T00:00:00.000000000' '2021-08-13T00:00:00.000000000'] != ['2021-05-01T00:00:00.000000000' '2021-05-03T00:00:00.000000000'
E            '2021-05-04T00:00:00.000000000' '2021-05-06T00:00:00.000000000'
E            '2021-05-07T00:00:00.000000000' '2021-05-09T00:00:00.000000000'
E            '2021-05-10T00:00:00.000000000' '2021-05-12T00:00:00.000000000'
E            '2021-05-13T00:00:00.000000000' '2021-05-15T00:00:00.000000000'
E            '2021-05-16T00:00:00.000000000' '2021-05-18T00:00:00.000000000'
E            '2021-05-19T00:00:00.000000000' '2021-05-21T00:00:00.000000000'
E            '2021-05-22T00:00:00.000000000' '2021-05-25T00:00:00.000000000'
E            '2021-05-27T00:00:00.000000000' '2021-05-28T00:00:00.000000000'
E            '2021-05-30T00:00:00.000000000' '2021-05-31T00:00:00.000000000'
E            '2021-06-01T00:00:00.000000000' '2021-06-02T00:00:00.000000000'
E            '2021-06-05T00:00:00.000000000' '2021-06-06T00:00:00.000000000'
E            '2021-06-08T00:00:00.000000000' '2021-06-09T00:00:00.000000000'
E            '2021-06-11T00:00:00.000000000' '2021-06-12T00:00:00.000000000'
E            '2021-06-14T00:00:00.000000000' '2021-06-15T00:00:00.000000000'
E            '2021-06-16T00:00:00.000000000' '2021-06-17T00:00:00.000000000'
E            '2021-06-18T00:00:00.000000000' '2021-06-20T00:00:00.000000000'
E            '2021-06-21T00:00:00.000000000' '2021-06-23T00:00:00.000000000'
E            '2021-06-24T00:00:00.000000000' '2021-06-26T00:00:00.000000000'
E            '2021-06-27T00:00:00.000000000' '2021-06-29T00:00:00.000000000'
E            '2021-07-02T00:00:00.000000000' '2021-07-03T00:00:00.000000000'
E            '2021-07-05T00:00:00.000000000' '2021-07-06T00:00:00.000000000'
E            '2021-07-08T00:00:00.000000000' '2021-07-09T00:00:00.000000000'
E            '2021-07-11T00:00:00.000000000' '2021-07-12T00:00:00.000000000'
E            '2021-07-14T00:00:00.000000000' '2021-07-15T00:00:00.000000000'
E            '2021-07-17T00:00:00.000000000' '2021-07-18T00:00:00.000000000'
E            '2021-07-20T00:00:00.000000000' '2021-07-21T00:00:00.000000000'
E            '2021-07-23T00:00:00.000000000' '2021-07-24T00:00:00.000000000'
E            '2021-07-26T00:00:00.000000000' '2021-07-27T00:00:00.000000000'
E            '2021-07-29T00:00:00.000000000' '2021-07-30T00:00:00.000000000'
E            '2021-08-01T00:00:00.000000000' '2021-08-02T00:00:00.000000000'
E            '2021-08-04T00:00:00.000000000' '2021-08-05T00:00:00.000000000'
E            '2021-08-07T00:00:00.000000000' '2021-08-08T00:00:00.000000000'
E            '2021-08-10T00:00:00.000000000' '2021-08-11T00:00:00.000000000'
E            '2021-08-13T00:00:00.000000000' '2021-08-14T00:00:00.000000000']
E           Shape mismatch: (48, 22, 13) != (70, 22, 13)
E           Left and right DataArray objects are not close
E           Differing dimensions:
E               (t: 48, y: 22, x: 13) != (t: 70, y: 22, x: 13)
E           Differing values:
E           L
E               array([[[0.276105, 0.354538, ..., 0.373056, 0.355261],
E                       [0.342372, 0.373904, ..., 0.406917, 0.41499 ],
E                       ...,
E                       [0.434685, 0.396209, ..., 0.431287, 0.497189],
E                       [0.753757, 0.281527, ..., 0.611761, 0.493749]],
E
E                      [[0.114468, 0.254632, ..., 0.375129, 0.35859 ],
E                       [0.340446, 0.354645, ..., 0.415952, 0.417551],
E                       ...,
E                       [0.313243, 0.173899, ..., 0.38943 , 0.509212],
E                       [0.052928, 0.254551, ..., 0.556757, 0.573127]],
E
E                      ...,
E
E                      [[0.291314, 0.333571, ..., 0.402173, 0.378566],
E                       [0.356151, 0.376033, ..., 0.421449, 0.41726 ],
E                       ...,
E                       [0.360139, 0.357076, ..., 0.370232, 0.412014],
E                       [0.255322, 0.378666, ..., 0.313571, 0.403127]],
E
E                      [[0.310877, 0.294525, ..., 0.374214, 0.327063],
E                       [0.356404, 0.378491, ..., 0.412196, 0.420573],
E                       ...,
E                       [0.33579 , 0.317588, ..., 0.36292 , 0.382339],
E                       [0.152729, 0.327559, ..., 0.382414, 0.398628]]], shape=(48, 22, 13))
E           R
E               array([[[0.296749, 0.378629, ..., 0.369668, 0.291813],
E                       [0.348964, 0.341047, ..., 0.342291, 0.348734],
E                       ...,
E                       [0.750469, 0.392641, ..., 0.425863, 0.559072],
E                       [0.752764, 0.206424, ..., 0.499764, 0.465316]],
E
E                      [[0.201865, 0.388018, ..., 0.373842, 0.308881],
E                       [0.348026, 0.352344, ..., 0.424704, 0.35227 ],
E                       ...,
E                       [0.144218, 0.33813 , ..., 0.394043, 0.585623],
E                       [0.05364 , 0.078125, ..., 0.479193, 0.521752]],
E
E                      ...,
E
E                      [[0.323211, 0.343485, ..., 0.371998, 0.14725 ],
E                       [0.355796, 0.365563, ..., 0.390447, 0.39802 ],
E                       ...,
E                       [0.24989 , 0.372667, ..., 0.373892, 0.280868],
E                       [0.153194, 0.207836, ..., 0.413988, 0.397689]],
E
E                      [[0.320593, 0.346943, ..., 0.329675, 0.124693],
E                       [0.355802, 0.361382, ..., 0.245393, 0.373247],
E                       ...,
E                       [0.473405, 0.459425, ..., 0.411178, 0.352813],
E                       [0.245825, 0.297638, ..., 0.432501, 0.409979]]], shape=(70, 22, 13))
E           Differing coordinates:
E           L * t        (t) datetime64[ns] 384B 2021-05-01 2021-05-03 ... 2021-08-13
E           R * t        (t) datetime64[ns] 560B 2021-05-01 2021-05-03 ... 2021-08-14

/opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/site-packages/openeo/testing/results.py:515: AssertionError
----------------------------- Captured stdout call -----------------------------
0:00:00 Job 'cdse-j-2511180220234e8ba513abe919ab8011': send 'start'
0:00:15 Job 'cdse-j-2511180220234e8ba513abe919ab8011': created (progress 0%)
0:00:20 Job 'cdse-j-2511180220234e8ba513abe919ab8011': created (progress 0%)
0:00:27 Job 'cdse-j-2511180220234e8ba513abe919ab8011': created (progress 0%)
0:00:35 Job 'cdse-j-2511180220234e8ba513abe919ab8011': created (progress 0%)
0:00:45 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:00:58 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:01:14 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:01:33 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:01:57 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:02:27 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:03:05 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:03:52 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:04:50 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:05:51 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:06:51 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:07:52 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:08:53 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:09:53 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:10:54 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:11:54 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:12:55 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:13:55 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:14:55 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:15:56 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:16:56 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:17:57 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:18:57 Job 'cdse-j-2511180220234e8ba513abe919ab8011': running (progress N/A)
0:19:58 Job 'cdse-j-2511180220234e8ba513abe919ab8011': finished (progress 100%)
------------------------------ Captured log call -------------------------------
INFO     conftest:conftest.py:131 Connecting to 'openeofed.dataspace.copernicus.eu'
INFO     openeo.config:config.py:193 Loaded openEO client config from sources: []
INFO     conftest:conftest.py:144 Checking for auth_env_var='OPENEO_AUTH_CLIENT_CREDENTIALS_CDSEFED' to drive auth against url='openeofed.dataspace.copernicus.eu'.
INFO     conftest:conftest.py:148 Extracted provider_id='CDSE' client_id='openeo-apex-benchmarks-service-account' from auth_env_var='OPENEO_AUTH_CLIENT_CREDENTIALS_CDSEFED'
INFO     openeo.rest.connection:connection.py:255 Found OIDC providers: ['CDSE']
INFO     openeo.rest.auth.oidc:oidc.py:404 Doing 'client_credentials' token request 'https://identity.dataspace.copernicus.eu/auth/realms/CDSE/protocol/openid-connect/token' with post data fields ['grant_type', 'client_id', 'client_secret', 'scope'] (client_id 'openeo-apex-benchmarks-service-account')
INFO     openeo.rest.connection:connection.py:354 Obtained tokens: ['access_token', 'id_token']
INFO     openeo.rest.job:job.py:436 Downloading Job result asset 'openEO.nc' from https://s3.waw3-1.openeo.v1.dataspace.copernicus.eu/openeo-data-prod-waw4-1/batch_jobs/j-2511180220234e8ba513abe919ab8011/openEO.nc?X-Proxy-Head-As-Get=true&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ab24deb1a8104912bbda28950b4ba036%2F20251118%2Fwaw4-1%2Fs3%2Faws4_request&X-Amz-Date=20251118T024023Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Security-Token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlX2FybiI6ImFybjpvcGVuZW93czppYW06Ojpyb2xlL29wZW5lby1kYXRhLXByb2Qtd2F3NC0xLXdvcmtzcGFjZSIsImluaXRpYWxfaXNzdWVyIjoib3BlbmVvLnByb2Qud2F3My0xLm9wZW5lby1pbnQudjEuZGF0YXNwYWNlLmNvcGVybmljdXMuZXUiLCJodHRwczovL2F3cy5hbWF6b24uY29tL3RhZ3MiOnsicHJpbmNpcGFsX3RhZ3MiOnsiam9iX2lkIjpbImotMjUxMTE4MDIyMDIzNGU4YmE1MTNhYmU5MTlhYjgwMTEiXSwidXNlcl9pZCI6WyI2YTc3ZmNkMS05YzA4LTQ2ZTktYjg3NS01NGZiOTk5YWIyMDAiXX0sInRyYW5zaXRpdmVfdGFnX2tleXMiOlsidXNlcl9pZCIsImpvYl9pZCJdfSwiaXNzIjoic3RzLndhdzMtMS5vcGVuZW8udjEuZGF0YXNwYWNlLmNvcGVybmljdXMuZXUiLCJzdWIiOiJvcGVuZW8tZHJpdmVyIiwiZXhwIjoxNzYzNDM3MjIzLCJuYmYiOjE3NjM0MzM2MjMsImlhdCI6MTc2MzQzMzYyMywianRpIjoiZWI2NDdhMjItZDEyZC00MzEzLWE3MjAtMTI2NTYzNzliNGEyIiwiYWNjZXNzX2tleV9pZCI6ImFiMjRkZWIxYTgxMDQ5MTJiYmRhMjg5NTBiNGJhMDM2In0.wGNeGxZ1_ixYSbmPhYuuUAiV9A1fMoQOd0cQEWqm2a16VbmsD1yewsslyuQF7vu-pcZl3R_A2_Ms3VqUfaDssiyduQxhu_7qqq8AtEfHF6xRqoSRStpLEXosMaJONTZMWOReZJXkCNoccFl37vhWD4GR2G0O4bIDXyXNgvgYL-vZY_c_JGKhgnNqOD2vZh608mi8FxOVe8Y0_y7VFGsZxxoibjAovOnpnfVO-lFChF2Nw04_jfmDcLSyQCAVYv02w2tzmo34IuAI-FJL3MPtzmn5VokpimIIWOpMD0wKArFdyE-dR2MSmQ7DxUJzZ5ti-ZftlDDONKMIlh4gMj8Cog&X-Amz-Signature=529048dd1c49b3c8758b1a6931d992fb0f2499d0059c1f0f00697fc7b0f34177 to /home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/actual/openEO.nc
INFO     apex_algorithm_qa_tools.scenarios:util.py:345 Downloading reference data for scenario.id='fusets_mogpr' to reference_dir=PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/reference'): start 2025-11-18 02:40:26.171235
INFO     apex_algorithm_qa_tools.scenarios:util.py:345 Downloading source='https://s3.waw3-1.cloudferro.com/apex-benchmarks/gh-16156495398!tests_test_benchmarks.py__test_run_benchmark_fusets_mogpr_!actual/openEO.nc' to path=PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/reference/openEO.nc'): start 2025-11-18 02:40:26.171545
INFO     apex_algorithm_qa_tools.scenarios:util.py:351 Downloading source='https://s3.waw3-1.cloudferro.com/apex-benchmarks/gh-16156495398!tests_test_benchmarks.py__test_run_benchmark_fusets_mogpr_!actual/openEO.nc' to path=PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/reference/openEO.nc'): end 2025-11-18 02:40:27.489463, elapsed 0:00:01.317918
INFO     apex_algorithm_qa_tools.scenarios:util.py:345 Downloading source='https://s3.waw3-1.cloudferro.com/apex-benchmarks/gh-16156495398!tests_test_benchmarks.py__test_run_benchmark_fusets_mogpr_!actual/job-results.json' to path=PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/reference/job-results.json'): start 2025-11-18 02:40:27.489733
INFO     apex_algorithm_qa_tools.scenarios:util.py:351 Downloading source='https://s3.waw3-1.cloudferro.com/apex-benchmarks/gh-16156495398!tests_test_benchmarks.py__test_run_benchmark_fusets_mogpr_!actual/job-results.json' to path=PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/reference/job-results.json'): end 2025-11-18 02:40:28.514237, elapsed 0:00:01.024504
INFO     apex_algorithm_qa_tools.scenarios:util.py:351 Downloading reference data for scenario.id='fusets_mogpr' to reference_dir=PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/reference'): end 2025-11-18 02:40:28.514449, elapsed 0:00:02.343214
INFO     openeo.testing.results:results.py:418 Comparing job results: PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/actual') vs PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_fusets_mogp0/reference')

[github-actions]

Report of latest run:

Benchmark scenario ID: fusets_mogpr
Benchmark scenario definition: https://github.com/ESA-APEx/apex_algorithms/blob/60326e39bc5b02387b690b4f47f983f4427c990d/algorithm_catalog/vito/fusets_mogpr/benchmark_scenarios/fusets_mogpr.json
openEO backend: openeofed.dataspace.copernicus.eu

GitHub Actions workflow run: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19523161317
Workflow artifacts: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19523161317#artifacts

Test start: 2025-11-20 02:18:21.961371+00:00
Test duration: 0:22:19.160518
Test outcome: ❌ failed

Last successful test phase: download-reference
Failure in test phase: compare

[github-actions]

Report of latest run:

Benchmark scenario ID: fusets_mogpr
Benchmark scenario definition: https://github.com/ESA-APEx/apex_algorithms/blob/ac5ce8e08459a563ac3aef912b6c38595a71f690/algorithm_catalog/vito/fusets_mogpr/benchmark_scenarios/fusets_mogpr.json
openEO backend: openeofed.dataspace.copernicus.eu

GitHub Actions workflow run: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19611544487
Workflow artifacts: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19611544487#artifacts

Test start: 2025-11-23 13:01:11.709634+00:00
Test duration: 0:22:20.420766
Test outcome: ❌ failed

Last successful test phase: download-reference
Failure in test phase: compare

[github-actions]

Report of latest run:

Benchmark scenario ID: fusets_mogpr
Benchmark scenario definition: https://github.com/ESA-APEx/apex_algorithms/blob/f455f453657d851be4f3f6426cc797b233e689f8/algorithm_catalog/vito/fusets_mogpr/benchmark_scenarios/fusets_mogpr.json
openEO backend: openeofed.dataspace.copernicus.eu

GitHub Actions workflow run: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19824064972
Workflow artifacts: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19824064972#artifacts

Test start: 2025-12-01 13:20:30.955216+00:00
Test duration: 0:20:21.278756
Test outcome: ❌ failed

Last successful test phase: download-reference
Failure in test phase: compare

[github-actions]

Report of latest run:

Benchmark scenario ID: fusets_mogpr
Benchmark scenario definition: https://github.com/ESA-APEx/apex_algorithms/blob/44c2d7db930923caefd0accd98ebc4276b992b75/algorithm_catalog/vito/fusets_mogpr/benchmark_scenarios/fusets_mogpr.json
openEO backend: openeofed.dataspace.copernicus.eu

GitHub Actions workflow run: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19963955973
Workflow artifacts: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19963955973#artifacts

Test start: 2025-12-05 13:10:25.835987+00:00
Test duration: 0:18:05.845439
Test outcome: ❌ failed

Last successful test phase: create-job
Failure in test phase: run-job

[github-actions]

Report of latest run:

Benchmark scenario ID: fusets_mogpr
Benchmark scenario definition: https://github.com/ESA-APEx/apex_algorithms/blob/44c2d7db930923caefd0accd98ebc4276b992b75/algorithm_catalog/vito/fusets_mogpr/benchmark_scenarios/fusets_mogpr.json
openEO backend: openeofed.dataspace.copernicus.eu

GitHub Actions workflow run: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19997742274
Workflow artifacts: https://github.com/ESA-APEx/apex_algorithms/actions/runs/19997742274#artifacts

Test start: 2025-12-07 02:34:38.851060+00:00
Test duration: 0:31:31.436589
Test outcome: ❌ failed

Last successful test phase: download-reference
Failure in test phase: compare

[github-actions]

Report of latest run:

Benchmark scenario ID: fusets_mogpr
Benchmark scenario definition: https://github.com/ESA-APEx/apex_algorithms/blob/9410664effa078591315129b82b74d3c3f6927ff/algorithm_catalog/vito/fusets_mogpr/benchmark_scenarios/fusets_mogpr.json
openEO backend: openeofed.dataspace.copernicus.eu

GitHub Actions workflow run: https://github.com/ESA-APEx/apex_algorithms/actions/runs/20064906988
Workflow artifacts: https://github.com/ESA-APEx/apex_algorithms/actions/runs/20064906988#artifacts

Test start: 2025-12-09 13:20:30.061718+00:00
Test duration: 0:11:52.821350
Test outcome: ❌ failed

Last successful test phase: create-job
Failure in test phase: run-job

[Pratichhya]

Traceback (most recent call last):
  File "/usr/local/spark/python/lib/pyspark.zip/pyspark/worker.py", line 1227, in main
    func, profiler, deserializer, serializer = read_command(pickleSer, infile)
  File "/usr/local/spark/python/lib/pyspark.zip/pyspark/worker.py", line 90, in read_command
    command = serializer._read_with_length(file)
  File "/usr/local/spark/python/lib/pyspark.zip/pyspark/serializers.py", line 174, in _read_with_length
    return self.loads(obj)
  File "/usr/local/spark/python/lib/pyspark.zip/pyspark/serializers.py", line 472, in loads
    return cloudpickle.loads(obj, encoding=encoding)
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/geopysparkcubemetadata.py", line 13, in <module>
    from openeo_driver.util.geometry import BoundingBox
  File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/util/geometry.py", line 17, in <module>
    from openeo_driver.errors import OpenEOApiException
  File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/errors.py", line 33, in <module>
    from openeo_driver.specs import SPECS_ROOT
  File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/specs/__init__.py", line 4, in <module>
    from openeo_driver.utils import read_json
  File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/utils.py", line 16, in <module>
    from openeo.utils.version import ComparableVersion
ModuleNotFoundError: No module named 'openeo.utils'

`

[Pratichhya]

The service is removed and is replaced by a new one: https://github.com/ESA-APEx/apex_algorithms/tree/main/algorithm_catalog/vito/mogpr_s1s2 The new version simply migrated the actual implementation to Terrascope openEO cataloge repository.

So maybe this can be closed as service is no longer available under this name?