Add kustomize pr overlay and update README

Author: iperezx

kubernetes/README.md

@@ -33,6 +33,11 @@ Copy the template and fill in your HuggingFace token (base64-encoded):
 cp base/huggingface-secret.template.yaml base/._huggingface-secret.yaml
 ```
 
+To generateb base64 encoded Hugging Face token:
+```
+echo -n "your_hf_token_here" | base64
+```
+
 ### 2. Sage User Secret
 
 Copy the Sage user secret template and add your Sage account name and password:
@@ -41,7 +46,12 @@ Copy the Sage user secret template and add your Sage account name and password:
 cp base/sage-user-secret.template.yaml base/._sage-user-secret.yaml
 ```
 
-- Encode username and password values as above.
+Base64 encoded SAGE_USER and SAGE_PASS to generate:
+``
+echo -n "your_username_here" | base64
+echo -n "your_password_here" | base64
+```
+
 - Update the `SAGE_USER` and `SAGE_PASS` fields.
 
 > **Important:** 
@@ -66,6 +76,59 @@ Or, using kubectl (if it supports native kustomize):
 kubectl apply -k base/
 ```
 
+Deploy all services:
+```
+kubectl kustomize nrp-dev | kubectl apply -f -
+kubectl kustomize nrp-prod | kubectl apply -f -
+```
+
+Delete all services:
+```
+kubectl kustomize nrp-dev | kubectl delete -f -
+kubectl kustomize nrp-prod | kubectl delete -f -
+```
+
+Debugging - output to yaml:
+```
+kubectl kustomize nrp-dev -o hybrid-search-dev.yaml
+kubectl kustomize nrp-prod -o hybrid-search-dev.yaml
+```
+
+## Testing a Pull Request
+For testing a Pull Request (PR), the overlay [prs](/kubernetes/prs/) is provided. Github Actions is setup to create an image for each PR so that we can manually test or in the future automatically test an instance of the image search deployed on k8s.
+
+The following manual steps are required for now:
+- [kubernetes/prs/kustomization.yaml](/kubernetes/prs/kustomization.yaml)
+    - change the `namePrefix` to the name of the PR
+    - change `commonLabels.env` to the name of the PR
+    - change the `newTag` to the name of the PR for each service that needs it
+- port-forwarding for any of the services to test out (update `pr`):
+    - `kubectl port-forward svc/pr-triton 8001:8001`: triton endpoint to call the LLM models locally
+    - `kubectl port-forward svc/pr-gradio-ui 7860:7860`: Search UI
+    - `kubectl port-forward svc/pr-weaviate 8080:8080`: Weaviate REST endpoint
+    - `kubectl port-forward svc/pr-weaviate 50051:50051`: Weaviate GRPC endpoint
+    - `kubectl port-forward svc/pr-weavloader-metrics 5555:5555`: Weavloader Flower endpoint
+    - `kubectl port-forward svc/pr-weavloader-metrics 8081:8080`: Weavloader Prometheus endpoint
+
+Deploy:
+```
+kubectl kustomize prs | kubectl apply -f -
+```
+
+Delete all services:
+```
+kubectl kustomize prs | kubectl delete -f -
+```
+
+Debugging - output to yaml:
+```
+kubectl kustomize prs -o hybrid-search-pr.yaml
+```
+
+Notes:
+- Make sure that your PR is up-to-date with `main` so that the services that were not modified are reflected for the `latest` tag. This can be also be checked with the [docker-compose](/docker-compose.yml) local deployment (after the PR is up-to-date with `main`) to see if the changes in the PR are working with the rest of the services that were not modified.
+- Users can utilized this overlay to combine it with their local docker compose instance to use a triton instance that has an NVIDIA GPU. This involves commenting out the ports from the docker compose manifest file for triton and doing the kubectl port-forwarding described above.
+
 ## Managing and Customizing
 
 You can extend or patch this `base/` deployment using kustomize overlays for different environments, resource limits, or development setups. See included overlays (such as those in benchmark subfolders) for example usage.

kubernetes/prs/gpus.yaml

@@ -0,0 +1,21 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: PLACEHOLDER
+spec:
+  template:
+    spec:
+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+            - matchExpressions:
+              - key: nvidia.com/gpu.product
+                operator: In
+                values:
+                - NVIDIA-A10
+      tolerations:
+      - key: nautilus.io/reservation
+        operator: Equal
+        value: sage
+        effect: NoSchedule

kubernetes/prs/gradio-env.yaml

@@ -0,0 +1,12 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: gradio-ui
+spec:
+  template:
+    spec:
+      containers:
+        - name: gradio-ui
+          env:
+            - name: UNALLOWED_NODES
+              value: "W042,N001,V012,W015,W01C,W01E,W024,W026,W02C,W02D,W02E,W02F,W031,W040,W046,W047,W048,W049,W04A,W051,W055,W059,W05A,W05B,W05C,W05D,W05E,W05F,W060,W061,W062,W063,W064,W065,W066,W06E,W072,W073,W074,W075,W076,W077,W078,W079,W07A,W07B,W07D,W07E,W07F,W080,W081,W086,W088,W089,W08A,W08B,W08D,W08E,W08F,W090,W091,W092,W094,W096,W099,W09B,W09E,W0A0,W0A1,W0BB,W0BC"

kubernetes/prs/kustomization.yaml

@@ -0,0 +1,42 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: sage
+
+namePrefix: pr2-
+commonLabels:
+  env: pr2
+
+resources:
+- ../base
+
+patches:
+  - path: gpus.yaml
+    target:
+      kind: Deployment
+      labelSelector: "app in (reranker-transformers, triton)"
+  - path: taint.yaml
+    target:
+      kind: Deployment
+      labelSelector: "app in (gradio-ui, weaviate, weavloader)"
+  - path: gradio-env.yaml
+    target:
+      kind: Deployment
+      labelSelector: "app=gradio-ui"
+  - path: weavloader-env.yaml
+    target:
+      kind: Deployment
+      labelSelector: "app=weavloader"
+
+images:
+  - name: semitechnologies/reranker-transformers
+    newTag: cross-encoder-ms-marco-MiniLM-L-6-v2-latest
+  - name: semitechnologies/weaviate
+    newTag: 1.32.0
+  - name:  gitlab-registry.nrp-nautilus.io/ndp/sage/nrp-image-search/triton
+    newTag: pr-2
+  - name:  gitlab-registry.nrp-nautilus.io/ndp/sage/nrp-image-search/gradio-ui
+    newTag: latest
+  - name:  gitlab-registry.nrp-nautilus.io/ndp/sage/nrp-image-search/weavmanage
+    newTag: latest
+  - name:  gitlab-registry.nrp-nautilus.io/ndp/sage/nrp-image-search/weavloader
+    newTag: pr-2

kubernetes/prs/taint.yaml

@@ -0,0 +1,21 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: PLACEHOLDER
+spec:
+  template:
+    spec:
+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+            - matchExpressions:
+              - key: nautilus.io/reservation
+                operator: In
+                values:
+                - sage
+      tolerations:
+      - key: nautilus.io/reservation
+        operator: Equal
+        value: sage
+        effect: NoSchedule

kubernetes/prs/weavloader-env.yaml

@@ -0,0 +1,12 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: weavloader
+spec:
+  template:
+    spec:
+      containers:
+        - name: weavloader
+          env:
+            - name: UNALLOWED_NODES
+              value: "W042,N001,V012,W015,W01C,W01E,W024,W026,W02C,W02D,W02E,W02F,W031,W040,W046,W047,W048,W049,W04A,W051,W055,W059,W05A,W05B,W05C,W05D,W05E,W05F,W060,W061,W062,W063,W064,W065,W066,W06E,W072,W073,W074,W075,W076,W077,W078,W079,W07A,W07B,W07D,W07E,W07F,W080,W081,W086,W088,W089,W08A,W08B,W08D,W08E,W08F,W090,W091,W092,W094,W096,W099,W09B,W09E,W0A0,W0A1,W0BB,W0BC"