CU-8699nbgbh Test against changes (#26)

* CU-8699nbgbh: Run tests on tutorials using changes in the PR branch

* CU-8699nbgbh: Add method to change install targets of notebooks

* CU-8699nbgbh: Update notebook install targets during CI to refer to PR

* CU-8699nbgbh: Check out changes for preflight at CI

* CU-8699nbgbh: Update notebook install patcher

* CU-8699nbgbh: Patch requirements.txt alongside notebooks

* CU-8699nbgbh: Rename patch script

* CU-8699nbgbh: Update patch file in workflow

* CU-8699nbgbh: Move install patching before installation

* CU-8699nbgbh: Move CI helper script to root

* CU-8699nbgbh: Update patch method to be more robust / flexible

* CU-8699nbgbh: Update demo web app dependencies to PR during CI

* CU-8699nbgbh: Run demo app build upon MedCAT changes

* CU-8699nbgbh: Fix usage of branch in demo app workflow

* CU-8699nbgbh: Fix usage of branch name in demo app workflow (try 2)

* Revert "CU-8699nbgbh: Fix usage of branch name in demo app workflow (try 2)"

This reverts commit f1fc582.

* CU-8699nbgbh: Move install patcher

* CU-8699nbgbh: Move the notebook install patcher to correct folder

* CU-8699nbgbh: Only do notebooks for install patcher

* CU-8699nbgbh: Remove branch from install patcher (not relevant)

* CU-8699nbgbh: Fix notebook install fixer

* CU-8699nbgbh: Update workflow to use correct notebook patch functionality and avoid preflight

* CU-8699nbgbh: Update workflow to install branch-specific changes after requiremnets (v2 tutorials workflow)

* CU-8699nbgbh: Remove usage of notebook install fixes from demo web app workflow

* CU-8699nbgbh: Install medcat based on local / PR changes during demo web app workflow

* CU-8699nbgbh: Fix basic and meta workflow for tutorials

* CU-8699nbgbh: Fix install targets for v2 tutorials

* CU-8699nbgbh: Fix demo web app workflow

* CU-8699nbgbh: Update patch info / output

* CU-8699nbgbh: Use absolute path for notebook install fixes

* CU-8699nbgbh: Fix rel-cat tutorial requirements

* CU-8699nbgbh: Fix order or installation for demo web app (reinstall after main install)

* CU-8699nbgbh: [TEMP] Add debug output for rel-cat notebook tests

* CU-8699nbgbh: Add MetaCAT during RelCAT tutorials

* CU-8699nbgbh: Make sure to import RelCAT after package install

* Revert "CU-8699nbgbh: Make sure to import RelCAT after package install"

This reverts commit 2fc7758.

* CU-8699nbgbh: just install pip at notebook run time during tests

* CU-8699nbgbh: Install all necessary optional dependencies at once at test time

* CU-8699nbgbh: Install all necessary optional dependencies at once at test time (nr2)

* CU-8699nbgbh: Install PR-based MedCAT for medcat-service tests

* CU-8699nbgbh: Run medcat-service tests upon core libe update

* CU-8699nbgbh: Test docker image against current medcat state / changes in medcat-service

* CU-8699nbgbh: Fix medcat-v2 install target when installing from local

* CU-8699nbgbh: Make sure the core library is available for install during docker image creation

* CU-8699nbgbh: Fix medcat lib path in Dockerfile(s)

* CU-8699nbgbh: Fix medcat lib path in Dockerfiles - make sure its a relative path

* CU-8699nbgbh: Copy medcat-v2 before building images

* CU-8699nbgbh: Update docker build workflow to include checking out repo and using local context for build

* CU-8699nbgbh: Update docker context path for build workflow

* CU-8699nbgbh: Update Dockerfile path for GPU-based service

* CU-8699nbgbh: Make sure to run gpu-based service with PR changes

Author: mart-r

.github/workflows/medcat-demo-app_build.yml

@@ -6,6 +6,7 @@ on:
   pull_request:
     paths:
       - 'medcat-demo-app/**'
+      - 'medcat-v2/**'
       - '.github/workflows/medcat-demo-app**'
 defaults:
   run:
@@ -14,13 +15,18 @@ defaults:
 jobs:
   integration-test:
     runs-on: ubuntu-latest
+    env:
+      MEDCAT_BRANCH: ${{ github.head_ref || github.ref_name }}
 
     steps:
     - uses: actions/checkout@v3
 
     - name: Set up Docker Compose
       run: sudo apt-get update && sudo apt-get install -y docker-compose
 
+    - name: Make medact-v2 available within webapp
+      run: cp -r ../medcat-v2 webapp/medcat-v2
+
     - name: Build and start containers
       run: docker-compose -f docker-compose-test.yml up -d --build
 

.github/workflows/medcat-service_docker.yml

@@ -7,6 +7,7 @@ on:
       - 'medcat-service/v*.*.*'  # e.g., medcat-serice/v0.1.1
   pull_request:
     paths:
+      - 'medcat-v2/**'
       - 'medcat-service/**'
       - '.github/workflows/medcat-service**'
   release:
@@ -24,7 +25,10 @@ jobs:
     runs-on: ubuntu-latest
     outputs:
       image_version: ${{ steps.meta.outputs.version }}
-    steps:        
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
       - name: Log in to Docker Hub
         uses: docker/login-action@v1
         with:
@@ -51,17 +55,22 @@ jobs:
             # Create version tag based on tag prefix
             type=match,pattern=medcat-service/v(\d+\.\d+\.\d+),group=1
 
+      - name: Make medact-v2 available within build
+        run: cp -r ../medcat-v2 medcat-v2
+
       - name: Build and push Docker MedCATservice image
         id: docker_build
         uses: docker/build-push-action@v6
         with:
-          context: "{{defaultContext}}:medcat-service" # Set the working directory as the action doesnt use the working-directory default
+          context: ./medcat-service/
           push: true
           allow: network.host
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
           cache-from: type=registry,ref=cogstacksystems/medcat-service:buildcache
           cache-to: type=registry,ref=cogstacksystems/medcat-service:buildcache,mode=max
+          build-args: |
+            REINSTALL_CORE_FROM_LOCAL=true
       
       - name: Extract metadata (tags, labels) for Docker MedCATservice-gpu
         id: meta-gpu
@@ -79,19 +88,23 @@ jobs:
             type=sha
             # Create version tag based on tag prefix
             type=match,pattern=medcat-service/v(\d+\.\d+\.\d+),group=1
+          build-args: |
+            REINSTALL_CORE_FROM_LOCAL=true
 
       - name: Build and push Docker Jupyter singleuser image with GPU support
         id: docker_build_gpu
         uses: docker/build-push-action@v6
         with:
-          context: "{{defaultContext}}:medcat-service" # Set the working directory as the action doesnt use the working-directory default
-          file : "Dockerfile_gpu"
+          context: ./medcat-service/
+          file : "./medcat-service/Dockerfile_gpu"
           allow: network.host
           tags: ${{ steps.meta-gpu.outputs.tags }}
           labels: ${{ steps.meta-gpu.outputs.labels }}
           push: true
           cache-from: type=registry,ref=cogstacksystems/medcat-service-gpu:buildcache
           cache-to: type=registry,ref=cogstacksystems/medcat-service-gpu:buildcache,mode=max
+          build-args: |
+            REINSTALL_CORE_FROM_LOCAL=true
           
       - name: Image digest
         run: echo ${{ steps.docker_build.outputs.digest }}

.github/workflows/medcat-service_run-tests.yml

@@ -5,6 +5,7 @@ on:
     branches: [ main ]
   pull_request:
     paths:
+      - 'medcat-v2/**'
       - 'medcat-service/**'
       - '.github/workflows/medcat-service**'
 
@@ -39,6 +40,7 @@ jobs:
           python -m pip install virtualenv setuptools
           python -m pip install isort flake8 mypy stubs types-Flask types-simplejson types-setuptools types-requests
           python -m pip install -r ./requirements.txt --extra-index-url https://download.pytorch.org/whl/cpu/;
+          python -m pip install "../medcat-v2[meta-cat,spacy,deid]"
 
       - name: Check linting and types
         run: |

.github/workflows/medcat-v2-tutorials_main.yml

@@ -23,10 +23,17 @@ jobs:
         with:
           python-version: "3.11"
           cache: 'pip'
+
+      - name: Update install targets in notebooks
+        run: |
+          python .ci/patch_notebook_installs.py .
+
       - name: Install dependencies
         run: |
           pip install -U pip
           pip install -r requirements.txt
+          pip install "../medcat-v2[spacy,deid,rel-cat,meta-cat]"
+
       - name: Install IPython kernel
         run: |
           python -m ipykernel install --name smoketests --user
@@ -44,7 +51,6 @@ jobs:
           pytest --nbmake -n=auto --nbmake-kernel=smoketests --nbmake-timeout=1800 ./notebooks/introductory/custom/*.ipynb
 
   migrate-and-advanved:
-
     runs-on: ubuntu-latest
     strategy:
       matrix:
@@ -60,14 +66,21 @@ jobs:
         uses: actions/setup-python@v5
         with:
           python-version: "3.11"
+
+      - name: Update install targets in notebooks
+        run: |
+          python .ci/patch_notebook_installs.py .
+
       - name: Install dependencies
         run: |
           pip install -U pip
           pip install -r requirements.txt
+          pip install "../medcat-v2[spacy,deid,rel-cat,meta-cat]"
+
       - name: Install IPython kernel
         run: |
           python -m ipykernel install --name smoketests --user
       - name: Smoke test tutorial
         run: |
-          pytest --collect-only --nbmake ${{ matrix.part }}
-          pytest --nbmake -n=auto --nbmake-kernel=smoketests --nbmake-timeout=1800 ${{ matrix.part }}
+          pytest --capture=no --collect-only --nbmake ${{ matrix.part }}
+          pytest --capture=no --nbmake -n=auto --nbmake-kernel=smoketests --nbmake-timeout=1800 ${{ matrix.part }}

medcat-demo-app/docker-compose-test.yml

@@ -4,6 +4,8 @@ services:
   medcatweb:
     build:
       context: ./webapp
+      args:
+        REINSTALL_CORE_FROM_LOCAL: "true"
     command: >
       bash -c "/etc/init.d/cron start &&
                python /webapp/manage.py runserver 0.0.0.0:8000"

medcat-demo-app/webapp/Dockerfile

@@ -16,6 +16,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     sqlite3 \
     libsqlite3-dev
 
+
 # Install Python dependencies
 ARG USE_CPU_TORCH=false
 # NOTE: Allow building without GPU so as to lower image size (disabled by default)
@@ -25,6 +26,13 @@ RUN pip install -U pip && if [ "$USE_CPU_TORCH" = "true" ]; then \
                               pip install -r /webapp/requirements.txt; \
                           fi
 
+# NOTE: for test/CI time, this will install medcat based on local path
+ARG REINSTALL_CORE_FROM_LOCAL=false
+RUN if [ "$REINSTALL_CORE_FROM_LOCAL" = "true" ]; then \
+        echo "Reinstalling medcat based on relative path"; \
+        pip install -e "/webapp/medcat-v2[meta-cat,spacy]"; \
+    fi
+
 # Get the spacy model (for later copy)
 RUN python -m spacy download en_core_web_md
 

medcat-service/Dockerfile

@@ -28,5 +28,12 @@ RUN for spacy_model in $SPACY_MODELS; do python -m spacy download $spacy_model;
 # Copy the remaining files
 COPY . /cat
 
+# NOTE: for test/CI time, this will install medcat based on local path
+ARG REINSTALL_CORE_FROM_LOCAL=false
+RUN if [ "$REINSTALL_CORE_FROM_LOCAL" = "true" ]; then \
+        echo "Reinstalling medcat based on relative path"; \
+        pip install -e "./medcat-v2[meta-cat,spacy]"; \
+    fi
+
 # Now run the simple api
 CMD ["/bin/bash", "start_service_production.sh"]

medcat-service/Dockerfile_gpu

@@ -46,5 +46,12 @@ RUN for spacy_model in $SPACY_MODELS; do python -m spacy download $spacy_model;
 # Copy the remaining files
 COPY . /cat
 
+# NOTE: for test/CI time, this will install medcat based on local path
+ARG REINSTALL_CORE_FROM_LOCAL=false
+RUN if [ "$REINSTALL_CORE_FROM_LOCAL" = "true" ]; then \
+        echo "Reinstalling medcat based on relative path"; \
+        pip install -e "./medcat-v2[meta-cat,spacy]"; \
+    fi
+
 # Now run the simple api
 CMD ["/bin/bash", "start_service_production.sh"]

medcat-v2-tutorials/.ci/patch_notebook_installs.py

@@ -0,0 +1,61 @@
+import sys
+import pathlib
+import re
+from functools import partial
+
+
+# rel_install_path = "../medcat-v2/"
+# abs_install_path = str(pathlib.Path(rel_install_path).resolve())
+
+# Matches either:
+# 1. `! pip install medcat[extras]`
+# 2. `! pip install medcat[extras] @ git+...`
+shell_pattern = re.compile(
+    r'(!\s*pip\s+install\s+)(\\["\']?)medcat(\[.*?\])'
+    r'(\s*@\s*git\+[^"\'\s]+)?\2'
+)
+req_txt_pattern = re.compile(
+    r'^(medcat(\[.*?\])?)\s*@\s*git\+\S+', flags=re.MULTILINE
+)
+
+
+def repl_nb(m, file_path: pathlib.Path):
+    # extras = m[3]
+    old_url = m[4]
+    if old_url and "medcat/v" in old_url:
+        print(f"[WARN] {file_path} refers to alpha/tagged release: "
+              f"{old_url.strip()}")
+    # to_write = f'{m[1]}\\"{abs_install_path}{extras}\\"'
+    to_write = '! pip install \\"pip\\"'
+    print(f"[PATCHED] {file_path}\n with: '{to_write}'")
+    return to_write
+
+
+def do_patch(nb_path: pathlib.Path,
+             regex: re.Pattern = shell_pattern, repl_method=repl_nb):
+    nb_text = nb_path.read_text(encoding="utf-8")
+
+    repl = partial(repl_method, file_path=nb_path)
+    new_text = regex.sub(repl, nb_text)
+
+    if nb_text != new_text:
+        nb_path.write_text(new_text, encoding="utf-8")
+
+
+def main(path: str):
+    for nb_path in pathlib.Path(path).rglob("**/*.ipynb"):
+        do_patch(nb_path)
+
+
+if __name__ == "__main__":
+    if len(sys.argv) != 2:
+        print("Usage: python patch_notebook_installs.py <path>")
+        sys.exit(1)
+
+    path = sys.argv[1]
+
+    if not pathlib.Path(path).exists():
+        print(f"Path {path} does not exist.")
+        sys.exit(1)
+
+    main(path)

medcat-v2-tutorials/notebooks/introductory/relcat/1._Supervised_Training_Relation_Extraction.ipynb

@@ -11,7 +11,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "b52c34cc",
    "metadata": {},
    "outputs": [
@@ -101,7 +101,7 @@
    ],
    "source": [
     "# Install medcat\n",
-    "! pip install \"medcat[spacy,meta-cat] @ git+https://github.com/CogStack/cogstack-nlp@medcat/v0.11.2#subdirectory=medcat-v2\" # NOTE: VERSION-STRING"
+    "! pip install \"medcat[spacy,rel-cat,meta-cat] @ git+https://github.com/CogStack/cogstack-nlp@medcat/v0.11.2#subdirectory=medcat-v2\" # NOTE: VERSION-STRING"
    ]
   },
   {