Add debugpy to Dockerfile for service containers

.pre-commit-config.yaml

@@ -4,6 +4,12 @@ repos:
     hooks:
       - id: check-added-large-files
       - id: check-yaml
+        # These yaml files output valid yaml only after templating
+        ignore:
+          - "template/{% if helm %}helm{% endif %}/{{ chart_name }}/templates/"
+          - "template/{% if helm %}helm{% endif %}/{{ chart_name }}/Chart.yaml"
+          - "{% if helm %}helm{% endif %}/{{ chart_name }}/templates/"
+          - "{% if helm %}helm{% endif %}/{{ chart_name }}/Chart.yaml"
       - id: check-merge-conflict
       - id: end-of-file-fixer
 

copier.yml

@@ -103,6 +103,26 @@ docker:
         Would you like to publish your project in a Docker container?
         You should select this if you are making a service.
 
+helm:
+    type: bool
+    when: "{{ github_org == 'DiamondLightSource'}}  and docker"
+    help: |
+        Would you like to publish a Helm chart?
+        You should select this if you are making a long-running service.
+        This requires that releases use Semantic Versioning
+
+chart_name:
+    type: str
+    when: helm
+    default: "{{ package_name | replace('_', '') | lower() }}"
+    help: |
+        Chart names must be lower case letters and numbers. Words may be separated with dashes (-)
+        But dashes may cause issues with templating.
+    validator: >-
+        {% if not (package_name | regex_search('^[a-z0-9]+(?:-?[a-z0-9])+$')) %}
+        {{chart_name}} is not a valid Helm chart name
+        {% endif %}
+
 docs_type:
     type: str
     help: |

docs/explanations/decisions/0020-debugpy.md

@@ -0,0 +1,19 @@
+# 20. Add debugpy to service Dockerfile
+
+Date: 2025-03-18
+
+## Status
+
+Accepted
+
+## Context
+
+Developers require a way to debug containerized services in e.g. Kubernetes.
+
+## Decision
+
+Add debugpy to the container image that is produced for services.
+
+## Consequences
+
+Debugging containerized services has a clear and documented path.

docs/how-to/coverage.md

@@ -1,4 +1,3 @@
-
 # How to check code coverage
 
 Code coverage is reported to the command line and to a `cov.xml` file by the command `tox -e tests`. The file is uploaded to the Codecov service in CI.

docs/how-to/deploy-cluster.md

@@ -0,0 +1,116 @@
+# Deploy with Helm
+
+If your project is a service, you may wish to deploy it into a Kubernetes cluster using Helm. 
+If enabled, a `helm/` directory is created, which bundles Kubernetes resources into a top level resource, `Chart`, and templates resources to inject specified `values`.
+
+```
+    service/
+    ├── Chart.yaml    # Definition of the resource
+    ├── values.yaml   # Defaults for templating
+    ├── charts/       # [Optionally] other charts to deploy with
+    └── templates/    # Templated Kubernetes resources
+```
+
+`templates/` includes among others:
+- `deployment.yaml`: creates a pod including your container image
+- `service.yaml`: manages Kubernetes networking, potentially exposing your service
+- `ingress.yaml`:  optionally maps a DNS entry to the Kubernetes networking
+
+Assuming your container is published to the GitHub container registry, `values.yaml` will be pre-configured to use your built container and enable debugging.
+
+```yaml
+image:
+  repository: ghcr.io/{{ organisation }}/{{ repo_name }}
+  pullPolicy: Always
+  # Overrides the image tag whose default is the chart appVersion.
+  tag: ""
+
+# Use `kubectl port forward` to access from your machine
+debug:
+  # Whether the container should start in debug mode
+  enabled: false
+  # Whether to suspend the process until a debugger connects
+  suspend: false
+  # Port to listen for the debugger on
+  port: 5678
+```
+
+To enable debugging, the CMD arguments of the Dockerfile have been overwritten by the analogous `args` from Kubernetes.
+The `ENTRYPOINT` and `CMD` concepts in the Dockerfile are analogous to Kubernetes' `command` and `args`.
+If `command` is set, it overrides `ENTRYPOINT` and uses `args` if set, ignoring `CMD`.
+If `args` is set, `ENTRYPOINT` remains and `CMD` is replaced.
+
+
+```yaml
+  args:
+  {{- if .Values.debug.enabled}}
+  - "-Xfrozen_modules=off"
+  - "-m"
+  - "debugpy"
+  {{- if .Values.debug.suspend }}
+  - "--wait-for-client"
+  {{- end }}
+  - "--listen"
+  - "0.0.0.0:{{ .Values.debug.port }}"
+  {{- end }}
+  - "-m"
+  - "service"
+  - "--version"
+```
+
+It is recommended to preserve all of the templates within `templates/`: resources you do not need can be disabled from `values.yaml` while maintaining the ability to deploy or extend the chart.
+
+## Connecting to a container in debug mode
+
+`kubectl port forward` forwards your development machine's port 5678 to the container's:
+
+```sh
+$ kubectl get pods
+NAME      READY   STATUS    RESTARTS     AGE
+service   1/1     Running   0 (1h ago)   1h
+$ kubectl port forward pod/service 5678:5678
+Forwarding from 127.0.0.1:5678 -> 5678
+```
+
+Check out the version of your service that was built into the container and configure your IDE to attach to a remote debugpy process:
+
+The following is a launch configuration from VSCode `launch.json`.
+`"remoteRoot"` should match for the version of Python your `Dockerfile` is built from and use your service's name.
+`"justMyCode": False` was found to be required for breakpoints to be active.
+`"autoReload"` Configured hot swapping of code from your developer machine to the deployed instance.
+
+> ⚠️ **Changes made by autoReload are not preserved.** Code changes made while debugging or resolving an issue should be committed, pushed and built into a new container as soon as possible.
+
+
+```json
+{
+  "name": "Python Debugger: Remote Attach",
+  "type": "debugpy",
+  "request": "attach",
+  "connect": {
+    "host": "localhost",
+    "port": 5678
+  },
+  "pathMappings": [
+    {
+      "localRoot": "${workspaceFolder}/src",
+      "remoteRoot": "/venv/lib/<Python version>/site-packages/<service>"
+    }
+  ],
+  "justMyCode": false,
+  "autoReload": {
+    "enable": true,
+    "exclude": [
+      "**/.git/**",
+      "**/__pycache__/**",
+      "**/node_modules/**",
+      "**/.metadata/**",
+      "**/site-packages/**"
+    ],
+    "include": [
+      "**/*.py",
+      "**/*.pyw"
+    ]
+  }
+}
+```

example-answers.yml

@@ -6,6 +6,7 @@ component_lifecycle: experimental
 description: An expanded https://github.com/DiamondLightSource/python-copier-template to illustrate how it looks with all the options enabled.
 distribution_name: dls-python-copier-template-example
 docker: true
+helm: false
 docs_type: sphinx
 git_platform: github.com
 github_org: DiamondLightSource

template/Dockerfile.jinja

@@ -17,13 +17,16 @@ FROM developer AS build
 COPY . /context
 WORKDIR /context
 RUN touch dev-requirements.txt && pip install -c dev-requirements.txt .
+RUN pip install debugpy
 
 # The runtime stage copies the built venv into a slim runtime container
 FROM python:${PYTHON_VERSION}-slim AS runtime
-# Add apt-get system dependecies for runtime here if needed
+# Add apt-get system dependencies for runtime here if needed
 COPY --from=build /venv/ /venv/
 ENV PATH=/venv/bin:$PATH
 
-# change this entrypoint if it is not the same as the repo
-ENTRYPOINT ["{{ repo_name }}"]
-CMD ["--version"]{% endif %}
+# change CMD if it is not the same as the repo
+ENTRYPOINT ["python"]
+# Allows for modifying the ENTRYPOINT for debugging, e.g.
+#ENTRYPOINT ["python", "-m", "debugpy"]
+CMD ["-m", "{{ repo_name }}", "--version"]{% endif %}

template/{% if git_platform=="github.com" %}.github{% endif %}/workflows/ci.yml.jinja

@@ -41,6 +41,13 @@ jobs:
     permissions:
       contents: read
       packages: write
+{% endif %}{% if helm %}
+  helm:
+    needs: container
+    uses: ./.github/workflows/_helm.yml
+    permissions:
+      contents: read
+      packages: write
 {% endif %}{% if sphinx %}
   docs:
     needs: check

template/{% if git_platform=="github.com" %}.github{% endif %}/workflows/{% if helm %}_helm.yml{% endif %}

@@ -0,0 +1,45 @@
+{% raw -%}on:
+  workflow_call:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Assign environment variables
+        # This is required because oci requires the repository is lowercase
+        run: |
+          echo "CHART_REPO=ghcr.io/${OWNER@L}" >>${GITHUB_ENV}
+          echo "CHART_NAME=${CHART@L}" >>${GITHUB_ENV}
+        env:
+          OWNER: "${{ github.repository_owner }}"
+          CHART: {% endraw %}{{ chart_name }}{% raw %}
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Create tag for publishing chart
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ghcr.io/${{ github.repository }}
+          tags: |
+            type=semver,pattern={{version}}
+
+      - name: Install Helm
+        uses: Azure/setup-helm@v4
+        id: install
+
+      - name: Lint Helm chart
+        run: helm lint helm/${{ env.CHART_NAME }}
+
+      - name: Log in to GitHub Container Registry
+        if: github.ref_type == 'tag'
+        run: |
+          echo ${{ secrets.GITHUB_TOKEN }} | helm registry login ${{ env.CHART_REPO }} --username ${{ github.repository_owner }} --password-stdin
+
+      - name: Publish Helm chart to container registry
+        if: github.ref_type == 'tag'
+        run: |
+          helm dependencies update helm/${{ env.CHART_NAME }}
+          helm package helm/${{ env.CHART_NAME }} --version ${{ steps.meta.outputs.version }} --app-version ${{ steps.meta.outputs.version }} -d /tmp/
+          helm push /tmp/${{ env.CHART_NAME }}-${{ steps.meta.outputs.version }}.tgz oci://${{ env.CHART_REPO }}/charts{% endraw %}

template/{% if helm %}helm{% endif %}/{{ chart_name }}/.helmignore

@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/

template/{% if helm %}helm{% endif %}/{{ chart_name }}/Chart.yaml

@@ -0,0 +1,24 @@
+apiVersion: v2
+name: {{ chart_name }}
+description: A Helm chart for Kubernetes
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+appVersion: "1.16.0"

template/{% if helm %}helm{% endif %}/{{ chart_name }}/templates/NOTES.txt

@@ -0,0 +1,22 @@
+{% raw -%}1. Get the application URL by running these commands:
+{{- if .Values.ingress.enabled }}
+{{- range $host := .Values.ingress.hosts }}
+  {{- range .paths }}
+  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
+  {{- end }}
+{{- end }}
+{{- else if contains "NodePort" .Values.service.type }}
+  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "{% endraw %}{{ chart_name }}{% raw %}.fullname" . }})
+  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
+  echo http://$NODE_IP:$NODE_PORT
+{{- else if contains "LoadBalancer" .Values.service.type }}
+     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
+           You can watch its status by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "{% endraw %}{{ chart_name }}{% raw %}.fullname" . }}'
+  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "{% endraw %}{{ chart_name }}{% raw %}.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
+  echo http://$SERVICE_IP:{{ .Values.service.port }}
+{{- else if contains "ClusterIP" .Values.service.type }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "{% endraw %}{{ chart_name }}{% raw %}.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
+  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
+  echo "Visit http://127.0.0.1:8080 to use your application"
+  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
+{{- end }}{% endraw %}