Replace vcr-py for tests (#1198)

* Adapt test.sh to use proxay.

* wip replace vcr-py with proxy.

* working on formatting and cleaning responses.

* Add type hint.

* Update uv lock.

* Change API requests in tests to use local webknossos.

* Experiment with failing tests.

* Remove download from pytest fixture.

* Update test-cassettes and fix some tests.

* Run linter.

* Remove time strings from tests.

* Adapt tests with zarr streaming.

* Working on tests that use annotations.

* Add debugging test mode that starts replay proxy.

* Add fixture for aiohttp use env variables.

* Set l4_sample public for running the remote_mags tests.

* Add cassette files for tests.

* Move access to remote MagViews out of fixture to capture requests in proxay.

* Run linter.

* Update cassette for annotation_upload_download_roundtrip.

* Update readme and contribution guideline.

* Update ci.

* Remove groups from CI.

* Update cassettes.

* Revert changes of ci and add timeout to test.sh.

* Update timeout in test.sh.

* Update test_duration file.

* Adapt test examples.

* Remove unused import.

* Skip flaky tests for now.

* Update tests to use l4_sample dataset.

* Update cassettes.

* Fix test for apply_merger_mode.

* Fix remote_datasets test.

* Fix tests for adding remote mag and layer.

* Run linter and typechecker.

* Fix test learned_segmenter.

Author: markbader

.github/workflows/ci.yml

@@ -166,7 +166,10 @@ jobs:
         with:
           # Install a specific version of uv.
           version: "0.4.22"
-      
+
+      - name: Install proxay
+        run: npm install -g proxay
+              
       - name: Set up Python ${{ matrix.python-version }}
         run: uv python install ${{ matrix.python-version }}
 

CONTRIBUTING.md

@@ -133,16 +133,25 @@ Internal workflows for scalable minds:
 
 The `webknossos` folder contains examples, which are not part of the package, but are tested via `tests/test_examples.py` and added to the documentation (see `docs/src/webknossos-py/examples`).
 
+To run the `./test.sh` script it is necessary to install `proxay`. This is either done with [NPM](https://www.npmjs.com) or [yarn](https://yarnpkg.com/getting-started/install):
+```bash
+npm install --global proxay
+
+# or if you're using yarn
+yarn global add proxay
+```
+
 The tests also contain functionality for the WEBKNOSSOS client. There a two modes to run the tests:
 
 1. `./test.sh --refresh-snapshots`, sending network requests to a WEBKNOSSOS instance:
-  This expects a local WEBKNOSSOS setup with specific test data, which is shipped with WEBKNOSSOS. If you're starting and running WEBKNOSSOS manually, please use port 9000 (the default) and run the `tools/postgres/dbtool.js prepare-test-db` script in the WEBKNOSSOS repository (⚠️ this overwrites your local WEBKNOSSOS database). Alternatively, a `docker compose` setup is started automatically for the tests, see `./test.sh` and `tests/docker-compose.yml` for details. The network requests & response are recorded as "cassettes" by [vcr.py](https://vcrpy.readthedocs.io), see next point:
-2. `./test.sh` replays responses from previous network snapshots using [vcr.py](https://vcrpy.readthedocs.io) via [pytest-recording](https://github.com/kiwicom/pytest-recording). No additional network requests are allowed in this mode.
+  This expects a local WEBKNOSSOS setup with specific test data, which is shipped with WEBKNOSSOS. If you're starting and running WEBKNOSSOS manually, please use port 9000 (the default) and run the `tools/postgres/dbtool.js prepare-test-db` script in the WEBKNOSSOS repository (⚠️ this overwrites your local WEBKNOSSOS database). Alternatively, a `docker compose` setup is started automatically for the tests, see `./test.sh` and `tests/docker-compose.yml` for details. The network requests & response are recorded as "cassettes" by [proxay](https://github.com/airtasker/proxay), see next point:
+2. `./test.sh` replays responses from previous network snapshots using [proxay](https://github.com/airtasker/proxay).
 
 `./test.sh --store-durations` updates the durations for
 [`pytest-split`](https://jerry-git.github.io/pytest-split),
 which is used in the CI to split the tests for different runners.
 
+
 #### `cluster_tools` package
 
 For testing the `slurm` setup a `docker compose` setup is available. Please see the [respective Readme](https://github.com/scalableminds/webknossos-libs/blob/master/cluster_tools/README.md) for details.

cluster_tools/dockered-slurm/docker-compose.yml

@@ -1,5 +1,3 @@
-version: "2.2"
-
 services:
   mysql:
     image: mysql:5.7
@@ -16,7 +14,7 @@ services:
 
   slurmdbd:
     image: scalableminds/slurm-docker-cluster:master__11274637426
-    command: ["slurmdbd"]
+    command: [ "slurmdbd" ]
     container_name: slurmdbd
     hostname: slurmdbd
     volumes:
@@ -32,7 +30,7 @@ services:
 
   slurmctld:
     image: scalableminds/slurm-docker-cluster:master__11274637426
-    command: ["slurmctld"]
+    command: [ "slurmctld" ]
     container_name: slurmctld
     environment:
       USER: "root"
@@ -53,7 +51,7 @@ services:
 
   c1:
     image: scalableminds/slurm-docker-cluster:master__11274637426
-    command: ["slurmd"]
+    command: [ "slurmd" ]
     hostname: c1
     container_name: c1
     volumes:
@@ -71,7 +69,7 @@ services:
 
   c2:
     image: scalableminds/slurm-docker-cluster:master__11274637426
-    command: ["slurmd"]
+    command: [ "slurmd" ]
     hostname: c2
     container_name: c2
     volumes: