Merge pull request #33 from jordanbean-msft/main

Updates to Kubernetes deployment

Author: satomic

.gitignore

@@ -32,4 +32,5 @@ debug_*
 
 # .env files
 *.env
-elasticsearch_env
+elasticsearch_env
+aks-deployment.my.yml

deploy/aks-deployment.yml

@@ -148,16 +148,19 @@ spec:
           livenessProbe:
             httpGet:
               path: /api/health
-              port: 80
+              port: 8080
             initialDelaySeconds: 90
             periodSeconds: 30
             failureThreshold: 5
           readinessProbe:
             httpGet:
               path: /api/health
-              port: 80
+              port: 8080
             initialDelaySeconds: 30
             periodSeconds: 10
+          securityContext:
+            runAsUser: 472
+            runAsGroup: 472
       volumes:
         - name: grafana-data
           persistentVolumeClaim:
@@ -173,7 +176,7 @@ spec:
   ports:
     - protocol: TCP
       port: 80
-      targetPort: 80
+      targetPort: 8080
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress

deploy/kubernetes.md

@@ -20,13 +20,15 @@ Dependent technology stack:
 Available [here](aks-deployment.yml).
 
 ## Deploy in Kubernetes
+
 This document describes how to deploy the application in Kubernetes.
 
 Variables used in the manifest:
- - `__GITHUB_PAT__`
- - `__GRAFANA_USERNAME__`
- - `__GRAFANA_PASSWORD__`
- - `__ORGANIZATION_SLUGS__`
+
+- `__GITHUB_PAT__`
+- `__GRAFANA_USERNAME__`
+- `__GRAFANA_PASSWORD__`
+- `__ORGANIZATION_SLUGS__`
 
 Replacement can be done using `envsubst` tool:
 
@@ -42,6 +44,7 @@ envsubst < deploy/aks-deployment.yml | kubectl apply -f -
 ### Deployment in Azure Kubernetes Service - AKS
 
 If you're using Azure Kubernetes Service - AKS:
+
 1. Enable [App Routing](https://learn.microsoft.com/en-us/azure/aks/app-routing)
 2. Update variables.
 3. Apply the manifest in your cluster.
@@ -51,4 +54,49 @@ If you're using Azure Kubernetes Service - AKS:
 1. Update PVC (Persitent Volume Claim) to one supported in your cluster
 2. Update Ingress
 3. Update variables.
-4. Apply the manifest in your cluster.
+4. Apply the manifest in your cluster.
+
+### Troubleshooting Grafana "database is locked" Error
+
+Grafana database can lock which can cause container to fail, see [Grafana, SQLite, and database is locked](https://opsverse.io/2022/12/15/grafana-sqlite-and-database-is-locked/)
+
+Use following Job to unlock the database.
+
+```yaml
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: fix-grafana-once
+spec:
+  template:
+    spec:
+      containers:
+        - name: fix-grafana
+          image: keinos/sqlite3
+          imagePullPolicy: IfNotPresent
+          command:
+          - "/bin/sh"
+          - "-c"
+          - "/usr/bin/sqlite3 /var/lib/grafana/grafana.db '.clone /var/lib/grafana/grafana.db.clone'; mv /var/lib/grafana/grafana.db.clone /var/lib/grafana/grafana.db; chmod a+w /var/lib/grafana/grafana.db"
+          volumeMounts:
+            - name: grafana-data
+              mountPath: /var/lib/grafana
+              subPath: grafana
+          securityContext:
+            runAsUser: 0
+          resources:
+            requests:
+              cpu: "0.25"
+              memory: "256Mi"
+            limits:
+              cpu: "0.5"
+              memory: "512Mi"
+      restartPolicy: Never
+      volumes:
+        - name: grafana-data
+          persistentVolumeClaim:
+            claimName: shared-pvc
+```
+
+Force restart grafana container after to re-create.