Moving to uv + hatch (#1029)

* try to move to uv+hatch

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update test.yml

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* problems of vibe coding :(

* retry

* try out this setting

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix2

* remove scanpy specific options

* remove junit

* remove juntxml uplaod

* dont publish debug data and upload test data

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* upload figures

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* just slack the tests for less flaky tests

* update assertion

* update matrix

* update matrix

* update configs

* ofc you can't set os in hatch

* fix hatch

* set notebook to main again

* simplify test.yml

* clean up

* fix the yaml names

* no need for build in test.yml

* undo test change and remove tox.ini

* fix macos dir error

* fix archive filepath problem

* add tolerance for tests/graph/test_ppatterns.py::test_spatial_autocorr_reproducibility[1-moran]

* fix attempt for coverage

* fix the path

* don't specify .xml

* match the versions so they are the same

* undo the tolerance

* update the python versions in hatch

* point to old commit

* replace bash script in CI to avoid .sh'es

* use matrix.os instead of runner.os

* redo scheduled job

* remove unnecessary lines

* remove scripts instead will use uv defaults

* remove set -u

* use hatch matrix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix the bash code

* fix names and where matrix is

* update matrix os

* incorperate the cache again as in old code

* reduce number of versions and conditionally upload coverage

* update docs

* use v5 instead

* the results aren't from version 6

* check if the floating point dep. is related to matrix list

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update yml

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add 3.12

* fix the issue finally

* change the versions

* 3.11, 3.12, 3.13 ubuntu and 3.12 macos. bc macos is slow

* check if this is the issue

* pin versions

* no pytest-cov

* undo the version pin testing

* Python source: 'from __future__ imports must occur at the beginning of the file' at line 10

* add tolerance and explain why

* refer to issue instead of commit

* Allow download fail on darwin

* give it as condition expression

* simplify the matrix and get rid of python 3.10 ci test

* mark internet required

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Philipp A. <[email protected]>

Author: 3 people

.github/workflows/test.yml

@@ -1,52 +1,58 @@
-name: Test
+name: CI
 
 on:
   schedule:
     - cron: 00 00 * * 1 # every Monday at 00:00
   push:
-    branches: [main]
+    branches:
+      - main
+      - "[0-9]+.[0-9]+.x"
   pull_request:
-    branches: [main]
-  workflow_dispatch:
-    inputs:
-      reason:
-        description: Reason for the workflow dispatch. Only "release" is valid.
-        required: true
-        default: release
+
+env:
+  PYTEST_ADDOPTS: "-v --color=yes -n auto"
+  FORCE_COLOR: "1"
+  MPLBACKEND: agg
+  # It's impossible to ignore SyntaxWarnings for a single module,
+  # so because leidenalg 0.10.0 has them, we pre-compile things: https://github.com/vtraag/leidenalg/issues/173
+  UV_COMPILE_BYTECODE: "1"
 
 jobs:
   test:
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
       matrix:
-        python: ["3.10", "3.11", "3.12"]
-        os: [ubuntu-latest]
         include:
-          - python: "3.12"
+          - name: hatch-test.py3.11-stable
+            os: ubuntu-latest
+            python: "3.11"
+          - name: hatch-test.py3.12-stable
+            os: ubuntu-latest
+            python: "3.12"
+          - name: hatch-test.py3.13-stable
+            os: ubuntu-latest
+            python: "3.13"
+            test-type: "coverage"
+          - name: hatch-test.py3.13-pre
             os: macos-latest
-            pre-release: "allow"
-            name: "Python 3.12 (pre-release)"
-
-    env:
-      OS: ${{ matrix.os }}
-      PYTHON: ${{ matrix.python }}
-      UV_PRERELEASE: ${{ matrix.pre-release || 'disallow' }}
-
+            python: "3.13"
+    env: # environment variable for use in codecov's env_vars tagging
+      ENV_NAME: ${{ matrix.name }}
     steps:
-      - uses: actions/checkout@v5
-      - name: Set up Python ${{ matrix.python }}
-        uses: astral-sh/setup-uv@v6
+      - uses: actions/checkout@v4
         with:
-          python-version: ${{ matrix.python }}
-          enable-cache: true
+          fetch-depth: 0
+          filter: blob:none
 
-      - name: Install dependencies
-        run: |
-          ./.scripts/ci/install_dependencies.sh
+      - uses: astral-sh/setup-uv@v6
+        with:
+          enable-cache: true
+          python-version: ${{ matrix.python }}
+          cache-dependency-glob: pyproject.toml
 
-      - name: Install pip dependencies
-        run: uv tool install tox --with=tox-uv --with=tox-gh-actions
+      - name: Ensure figure directory exists
+        run: mkdir -p "$GITHUB_WORKSPACE/tests/figures"
 
       - name: Restore data cache
         id: data-cache
@@ -59,46 +65,54 @@ jobs:
       - name: Download datasets
         if: steps.data-cache.outputs.cache-hit != 'true'
         run: |
-          tox -e download-data
+          uvx hatch run ${{ matrix.name }}:download
 
-      # caching .tox is not encouraged, but since we're private and this shaves off ~1min from the step
-      # if any problems occur and/or once the package is public, this can be removed
-      - name: Restore tox cache
-        uses: actions/cache@v4
-        with:
-          path: .tox
-          key: tox-${{ runner.os }}-${{ matrix.python }}-${{ hashFiles('**/requirements.txt', '**/tox.ini') }}
+      - name: System dependencies (Linux)
+        if: matrix.os == 'ubuntu-latest'
+        run: |
+          sudo apt-get update -y
+          sudo apt-get install automake -y
+
+          # PyQt5 related
+          sudo apt install libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcb-xfixes0 -y
+          sudo Xvfb :42 -screen 0 1920x1080x24 -ac +extension GLX </dev/null &
+
+      - name: System dependencies (macOS)
+        if: matrix.os == 'macos-latest'
+        run: brew install automake
+
+      - name: Install dependencies
+        run: uvx hatch -v env create ${{ matrix.name }}
 
-      - name: Test
-        timeout-minutes: 60
-        env:
-          MPLBACKEND: agg
-          PLATFORM: ${{ matrix.os }}
-          DISPLAY: :42
-          PYTEST_ADDOPTS: "-n auto"
+      - name: Run tests
+        if: matrix.test-type == null
+        run: uvx hatch run ${{ matrix.name }}:run
+      - name: Run tests (coverage)
+        if: matrix.test-type == 'coverage'
         run: |
-          tox -vv
-      # check if this can be deprecated
-      #- name: List figures for potential debugging
-      #      ls -alh /home/runner/work/squidpy/squidpy/tests/figures
+          uvx hatch run ${{ matrix.name }}:run-cov
+          uvx hatch run ${{ matrix.name }}:coverage combine
+          uvx hatch run ${{ matrix.name }}:coverage xml
 
       - name: Archive figures generated during testing
         if: always()
         uses: actions/upload-artifact@v4
         with:
-          name: visual_test_results_${{ matrix.os }}-python${{ matrix.python }}
-          path: /home/runner/work/squidpy/squidpy/tests/figures/*
+          name: visual_test_results_${{ runner.os }}_py${{ matrix.name }}
+          path: ${{ github.workspace }}/tests/figures/*
 
-      - name: Upload coverage to Codecov
+      - name: Upload coverage data
+        if: matrix.test-type == 'coverage'
         uses: codecov/codecov-action@v5
         with:
-          name: coverage
-          verbose: true
-          token: ${{ secrets.CODECOV_TOKEN }} # required
+          token: ${{ secrets.CODECOV_TOKEN }}
+          env_vars: ENV_NAME
+          fail_ci_if_error: true
 
   check:
     if: always()
-    needs: [test]
+    needs:
+      - test
     runs-on: ubuntu-latest
     steps:
       - uses: re-actors/alls-green@release/v1

.scripts/ci/install_dependencies.sh

@@ -61,9 +61,6 @@ Tests structure:
 Code style guide
 ----------------
 We rely on ``black`` and ``isort`` to do the most of the formatting - both of them are integrated as pre-commit hooks.
-You can use ``tox`` to check the changes::
-
-    tox -e lint
 
 Furthermore, we also require that:
 
@@ -75,26 +72,28 @@ Furthermore, we also require that:
 
 Testing
 -------
-We use ``tox`` to automate our testing, as well as linting and documentation creation. To run the tests, run::
 
-    tox -e py{38,39,310}-{linux,macos}
+
+We use [pytest][] to test squidpy.
+To run the tests, simply run `hatch test`.
+
+It can take a while to run the whole test suite. There are a few ways to cut down on this while working on a PR:
+
+1. Only run a subset of the tests.
+   This can be done by specifying paths or test name patterns using the `-k` argument (e.g. `hatch test tests/graph/test_nhood.py` or `hatch test -k "test_umap*"`)
+2. Run the tests in parallel using the `-n` argument (e.g. `hatch test -n 8`).
+
 
 depending on the Python version(s) in your ``PATH`` and your operating system. We use ``flake8`` and ``mypy`` to further
 analyze the code. Use ``# noqa: <error1>,<error2>`` to ignore certain ``flake8`` errors and
 ``# type: ignore[error1,error2]`` to ignore specific ``mypy`` errors.
 
-To run only a subset of tests, run::
 
-    tox -e <environment> -- <name>
 
-where ``<name>`` can be a path to a test file/directory or a name of a test function/class.
-For example, to run only the tests in the ``nhood`` module, use::
 
-    tox -e py39-linux -- tests/graph/test_nhood.py
 
-If needed, a specific ``tox`` environment can be recreated as::
+[pytest]: https://docs.pytest.org/en/stable/
 
-    tox -e <environment> --recreate
 
 Writing documentation
 ---------------------
@@ -106,20 +105,25 @@ We use ``numpy``-style docstrings for the documentation with the following addit
 - prefer putting references in the ``references.bib`` instead under the ``References`` sections of the docstring.
 - use ``docrep`` for repeating documentation.
 
-In order to build the documentation, run::
 
-    tox -e docs
+To build the docs, run run::
+    hatch run docs:build
+
+Afterwards, you can run run::
+    hatch run docs:open 
+
+to open {file}`docs/_build/html/index.html`.
 
 Since the tutorials are hosted on a separate repository (see `Writing tutorials/examples`_), we download the newest
 tutorials/examples from there and build the documentation here.
 
 To validate the links inside the documentation, run::
 
-    tox -e check-docs
+    hatch run docs:check
 
 If you need to clean the artifacts from previous documentation builds, run::
 
-    tox -e clean-docs
+    hatch run docs:clean
 
 Writing tutorials/examples
 --------------------------

CONTRIBUTING.rst

@@ -1,7 +1,7 @@
 include requirements.txt
 include LICENSE
 include README.rst
-include tox.ini
+include hatch.toml
 
 prune tests/_data
 prune tests/_images

MANIFEST.in

@@ -0,0 +1,59 @@
+[envs.default]
+installer = "uv"
+features = ["dev"]
+
+[envs.coverage]
+extra-dependencies = [
+    "coverage[toml]",
+    "diff_cover",
+]
+
+[envs.coverage.scripts]
+clean = "coverage erase"
+report = "coverage report --omit='tox/*'"
+xml = "coverage xml --omit='tox/*' -o coverage.xml"
+diff = "diff-cover --compare-branch origin/main coverage.xml"
+
+[envs.docs]
+features = ["docs"]
+extra-dependencies = [
+    "setuptools",
+]
+
+[envs.docs.scripts]
+build = "make -C docs html {args}"
+clean = "make -C docs clean"
+check = "make -C docs linkcheck {args}"
+
+[envs.data]
+[envs.data.scripts]
+download = "python ./.scripts/ci/download_data.py {args}"
+
+
+[envs.hatch-test]
+features = ["test"]
+extra-dependencies = [
+    "pytest",
+    "pytest-xdist", 
+    "pytest-cov",
+    "pytest-mock",
+    "pytest-timeout",
+]
+
+
+
+[[envs.hatch-test.matrix]]
+deps = ["stable"]
+python = ["3.11", "3.12", "3.13"]
+
+# Test the newest supported Python version also with pre-release deps
+[[envs.hatch-test.matrix]]
+deps = [ "pre" ]
+python = [ "3.13" ]
+
+[envs.hatch-test.overrides]
+# If the matrix variable `deps` is set to "pre",
+# set the environment variable `UV_PRERELEASE` to "allow".
+matrix.deps.env-vars = [
+  { key = "UV_PRERELEASE", value = "allow", if = [ "pre" ] },
+]