CI and development environment overhaul (#2298)

* CI and development environment overhaul

This removes the docker image logic from the CI setup and instead
replaces it with a setup based on pixi. As part of that it moves the
dependency specification from localbuild/meta.yaml to pixi.toml and
pixi.lock. This turns the MSS repository into a single source of truth
for both the application code as well as the development environment
(whereas the latter was previously only specified in the docker images,
and not reproducible in any way).

Setting up a development environment is as simple as installing pixi and
running `pixi shell` (or `pixi run <cmd>`, or `pixi install` to just
create the environment, etc.). This environment will, by construction,
be the same that is used in the CI as well (modulo platform
differences).

There is a new workflow that periodically (once a week on Monday)
recreates the pixi lockfile and opens a PR for that update. The checks
in that PR essentially serve as a replacement for the previous scheduled
runs to ensure that no dependency update breaks MSS. Merging that PR is
a manual step that can be done just as with any other PR and would then
update the environment on the given target branch. This is essentially
what was previously the triggering of a docker image creation.

Including new dependencies can be done with `pixi add`, which will also
automatically add the dependency to the (pre-existing) lockfile. This
means dependency additions can be part of the PR that necessitate them
and they won't affect the entire environment (as they previously did,
where they would trigger a full image rebuild) but instead just add that
new package to the existing specification.

* fixup! CI and development environment overhaul

* Update commands in development docs

* fixup! CI and development environment overhaul

---------

Co-authored-by: ReimarBauer <[email protected]>

Author: matrss

.gitattributes

@@ -1 +1,3 @@
 * text=auto
+# GitHub syntax highlighting
+pixi.lock linguist-language=YAML

.github/workflows/build_docs_gallery.yml

@@ -1,20 +1,20 @@
 name: Build Gallery
 
 on:
-   pull_request:
+  pull_request:
 
 jobs:
   Test-MSS-docs:
     runs-on: ubuntu-latest
-
-    container:
-      image: openmss/testing-develop
-
     steps:
     - uses: actions/checkout@v4
-
+    - uses: prefix-dev/[email protected]
+      with:
+        pixi-version: latest
+        cache: true
+        environments: dev
     - name: Create gallery
       timeout-minutes: 5
       run: |
         cd docs
-        mamba run --no-capture-output -n mssenv python conf.py
+        pixi run -e dev python conf.py

.github/workflows/lint.yml

@@ -18,30 +18,26 @@ jobs:
     timeout-minutes: 10
     steps:
     - uses: actions/checkout@v4
-    - name: Set up Python 3
-      uses: actions/setup-python@v5
+    - uses: prefix-dev/[email protected]
       with:
-        python-version: "3.x"
+        pixi-version: latest
+        cache: true
+        environments: dev
     - name: Lint with codespell
-      run: |
-        python -m pip install --upgrade pip
-        pip install codespell
-        git ls-files -z | xargs -0 codespell --check-filenames
+      run: git ls-files -z | xargs -0 pixi run -e dev codespell --check-filenames
 
   flake8:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
     - uses: actions/checkout@v4
-    - name: Set up Python 3.10
-      uses: actions/setup-python@v5
+    - uses: prefix-dev/[email protected]
       with:
-        python-version: "3.10"
+        pixi-version: latest
+        cache: true
+        environments: dev
     - name: Lint with flake8
-      run: |
-        python -m pip install --upgrade pip
-        pip install flake8 flake8-builtins
-        git ls-files -z '**.py' | xargs -0 flake8
+      run: git ls-files -z '**.py' | xargs -0 pixi run -e dev flake8
 
   no-crlf-in-git:
     runs-on: ubuntu-latest

.github/workflows/testing-all-oses.yml

@@ -21,27 +21,11 @@ jobs:
         os: ["macos-13", "macos-14", "ubuntu-latest"]
     steps:
     - uses: actions/checkout@v4
-    - name: Build requirements.txt file
-      run: |
-        sed -n '/^requirements:/,/^test:/p' localbuild/meta.yaml |
-          sed -e "s/.*- //" |
-          sed -e "s/menuinst.*//" |
-          sed -e "s/.*://" > requirements.tmp.txt
-        cat requirements.d/development.txt >> requirements.tmp.txt
-        echo "pytest-randomly" >> requirements.tmp.txt
-        sed -e '/^$/d' -e '/^#.*$/d' -e 's/\s*# \[not win\]$//' requirements.tmp.txt > requirements.txt
-        rm requirements.tmp.txt
-        cat requirements.txt
-    - name: Get current year and calendar week
-      id: year-and-week
-      run: echo "year-and-week=$(date +%Y-%V)" >> "$GITHUB_OUTPUT"
-    - uses: mamba-org/setup-micromamba@v2
+    - uses: prefix-dev/[email protected]
       with:
-        environment-file: requirements.txt
-        environment-name: ci
-        cache-environment: true
-        # Set the cache key in a way that the cache is invalidated every week on monday
-        cache-environment-key: environment-${{ steps.year-and-week.outputs.year-and-week }}
+        pixi-version: latest
+        cache: true
+        environments: dev
     - name: Run tests
       timeout-minutes: 40
-      run: micromamba run -n ci env QT_QPA_PLATFORM=offscreen pytest -v -n logical --durations=20 --cov=mslib tests
+      run: pixi run -e dev env QT_QPA_PLATFORM=offscreen pytest -v -n logical --durations=20 --cov=mslib tests

.github/workflows/testing-develop.yml

@@ -0,0 +1,39 @@
+name: Update pixi lockfile
+
+on:
+  schedule:
+  # At 04:00 on Monday
+    - cron: "0 4 * * 1"
+  # ...or manually
+  workflow_dispatch:
+
+jobs:
+  pixi-update:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        base_branch: ["develop", "stable"]
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ matrix.base_branch }}
+      - name: Generate new lockfile
+        uses: prefix-dev/[email protected]
+        with:
+          pixi-version: latest
+          run-install: false
+      - name: Update pixi lockfile
+        run: |
+          set -o pipefail
+          pixi update --json | pixi exec pixi-diff-to-markdown >> diff.md
+      - name: Create or update pull request
+        uses: peter-evans/create-pull-request@v7
+        with:
+          token: ${{ secrets.PAT }}
+          branch: automation/update-pixi-lockfile
+          delete-branch: true
+          add-paths: pixi.lock
+          commit-message: Update pixi lockfile
+          title: Update pixi lockfile
+          body-path: diff.md