Actually use snapshots in nightly step that says so (#1403)

* actually use snapshots in nightly step that says so

* correctly terminate proxay and its subprocesses

* format

* avoid terminating python along with proxay

* fix logging

* attempt to make splitting order deterministic

* change splitting algorithm

* store all test durations

* no test groups in nightly

* cleanup

* no new session on windows

Author: fm3

.github/workflows/nightly.yml

@@ -12,7 +12,6 @@ jobs:
       max-parallel: 4
       matrix:
         python-version: ["3.10", "3.11", "3.12", "3.13"]
-        group: [1, 2, 3]
       fail-fast: false
     defaults:
       run:
@@ -38,12 +37,12 @@ jobs:
       - name: Python tests, refreshing the network snapshots
         env:
           WK_TOKEN: ${{ secrets.WK_TOKEN }}
-        run: uv run test.py --refresh-snapshots --splits 3 --group ${{ matrix.group }}
+        run: uv run test.py --refresh-snapshots
 
       - name: Python tests, using the new snapshots
         env:
           WK_TOKEN: ${{ secrets.WK_TOKEN }}
-        run: uv run test.py --refresh-snapshots --splits 3 --group ${{ matrix.group }}
+        run: uv run test.py
 
       - uses: slackapi/[email protected]
         if: failure() || cancelled()

webknossos/test.py

@@ -199,17 +199,30 @@ def proxay(mode: Literal["record", "replay"], quiet: bool) -> Iterator[None]:
             )
         else:
             proxay_process = subprocess.Popen(
-                cmd,
-                shell=IS_WINDOWS,
+                cmd, shell=IS_WINDOWS, start_new_session=(not IS_WINDOWS)
             )
         sleep(1)
         yield
     finally:
         if proxay_process is not None:
-            print("Terminating proxay", flush=True)
-            proxay_process.terminate()
-            proxay_process.wait(5)
-            proxay_process.kill()
+            if IS_WINDOWS:
+                print("Terminating proxay...")
+                proxay_process.terminate()
+            else:
+                print(
+                    "Terminating proxay and its subprocesses with killpg (SIGTERM)...",
+                    flush=True,
+                )
+                pgid = os.getpgid(proxay_process.pid)
+                try:
+                    os.killpg(pgid, signal.SIGTERM)
+                    proxay_process.wait(timeout=5)
+                except subprocess.TimeoutExpired:
+                    print(
+                        "Terminating proxay and its subprocesses with killpg (SIGKILL)...",
+                        flush=True,
+                    )
+                    os.killpg(pgid, signal.SIGKILL)
 
 
 def run_pytest(args: list[str]) -> None: