Move listener to uwsgi daemon and add lag checker GH action

.github/workflows/toolforge-check-lag.yml

@@ -0,0 +1,38 @@
+name: Toolforge Check Lag
+on:
+  schedule:
+    - cron: "42 * * * *"
+  workflow_dispatch:
+
+jobs:
+  check_lag:
+    name: Check Lag
+    runs-on: ubuntu-latest
+
+    strategy:
+      max-parallel: 4
+      matrix:
+        tool: ["editgroups"]
+
+    steps:
+      - name: Configure SSH key
+        uses: shimataro/ssh-key-action@v2
+        with:
+          key: ${{ secrets.SSH_PRIVATE_KEY }}
+          known_hosts: ${{ secrets.SSH_KNOWN_HOSTS }}
+
+      - name: Run deployment commands
+        uses: appleboy/[email protected]
+        with:
+          host: ${{ vars.SSH_HOST }}
+          username: ${{ secrets.SSH_USER }}
+          key: ${{ secrets.SSH_PRIVATE_KEY }}
+          port: ${{ vars.SSH_PORT }}
+          request_pty: true
+          script: |
+            set -xe
+            become ${{ matrix.tool }} webservice python3.11 shell -- webservice-python-bootstrap
+            become ${{ matrix.tool }} webservice python3.11 shell -- ./www/python/venv/bin/python ./www/python/src/manage.py check_lag
+            if (( $? != 0 )); then
+              become ${{ matrix.tool }} webservice python3.11 restart
+            fi

.github/workflows/toolforge-deploy.yml

@@ -36,5 +36,4 @@ jobs:
             become ${{ matrix.tool }} webservice python3.11 shell -- '$HOME/www/python/venv/bin/python' '$HOME/www/python/src/manage.py' collectstatic --noinput
             become ${{ matrix.tool }} webservice python3.11 restart
             become ${{ matrix.tool }} webservice python3.11 status
-            become ${{ matrix.tool }} ./www/python/src/restart_listener.sh
             become ${{ matrix.tool }} ./www/python/src/restart_celery.sh

deployment/celery.yaml

@@ -1,4 +1,4 @@
-# Run the EventStream listener on kubernetes
+# Run Celery on kubernetes
 apiVersion: apps/v1
 kind: Deployment
 metadata:

docs/architecture.rst

@@ -37,7 +37,7 @@ This process can be invoked directly as a script::
 
    python listener.py
 
-Or it can be run as a Kubernetes pod using the ``deployment/listener.yaml`` configuration provided.
+It can be run as an `attached daemon to uwsgi <https://uwsgi-docs.readthedocs.io/en/latest/AttachingDaemons.html>`_.
 
 In the interest of running EditGroups on other Wikibase instances which do not have an associated EventStream
 endpoint, it would be great to add the possibility on ingesting edits by polling the recent changes too.

docs/install.rst

@@ -129,7 +129,10 @@ Put the following content in ``~/www/python/uwsgi.ini``::
   [uwsgi]
   static-map = /static=/data/project/editgroups/www/python/src/static
 
-and run ``./manage.py collectstatic`` in the ``~/www/python/src`` directory.
+  master = true
+  attach-daemon = /data/project/editgroups/www/python/venv/bin/python3 /data/project/editgroups/www/python/src/listener.py
+
+and run ``./manage.py collectstatic`` in the ``~/www/python/src`` directory. The listener will be an attached dameon, restarting with webservice restart.
 
 
 Configure OAuth login:
@@ -156,9 +159,8 @@ Go to the webservice, login with OAuth to the application. This will create a ``
    user.is_staff = True
    user.save()
 
-Launch the listener and Celery in Kubernetes. These deployment files may need to be adapted if you are not deploying the tool as the ``editgroups`` toolforge tool but another tool id:
+Launch Celery in Kubernetes. These deployment files may need to be adapted if you are not deploying the tool as the ``editgroups`` toolforge tool but another tool id:
 
-- ``kubectl create -f deployment/listener.yaml``
 - ``kubectl create -f deployment/celery.yaml``
 
 Backup the database regularly with:

store/management/__init__.py

@@ -0,0 +1 @@
+

store/management/commands/__init__.py

@@ -0,0 +1 @@
+

store/management/commands/check_lag.py

@@ -0,0 +1,15 @@
+from django.core.management.base import BaseCommand
+from django.core.management.base import CommandError
+
+from store.models import Edit
+
+
+class Command(BaseCommand):
+    MAX_SECONDS = 3000
+    help = f"Exits with 1 if current lag is bigger than {MAX_SECONDS} seconds"
+
+    def handle(self, *args, **options):
+        lag = Edit.current_lag().total_seconds()
+        print(f"Current lag: {lag:.2f} seconds")
+        if lag > 3000:
+            raise CommandError(f"Current lag is bigger than {self.MAX_SECONDS}")