Clean up image, build ollama image with pre-pulled model, update compose

Author: cupofpython

Dockerfile

@@ -12,20 +12,18 @@ FROM node:${NODE_VERSION}-alpine
 
 WORKDIR /usr/src/app
 
-# Download dependencies as a separate step to take advantage of Docker's caching.
-# Leverage a cache mount to /root/.npm to speed up subsequent builds.
-# Leverage a bind mounts to package.json and package-lock.json to avoid having to copy them into
-# into this layer.
-RUN --mount=type=bind,source=package.json,target=package.json \
-    --mount=type=bind,source=package-lock.json,target=package-lock.json \
-    --mount=type=cache,target=/root/.npm \
-    npm ci --omit=dev
-
 # Run the application as a non-root user.
 USER node
 
 # Copy the rest of the source files into the image.
-COPY . .
+
+COPY package.json package-lock.json /usr/src/app/
+
+COPY server.js .
+
+COPY src/ .
+
+COPY public/ .
 
 # Expose the port that the application listens on.
 EXPOSE 3000

README.md

@@ -25,6 +25,10 @@ I used compose to develop this locally.
 - `docker compose up --build`
 - When done, `docker compose down`
 
+#### Build the Ollama image with model pre pulled
+- Build: `docker build -f ollama/Dockerfile -t ollama_model --platform=linux/amd64 .`
+- Run: `docker run -d --platform=linux/amd64 ollama_model`
+
 ### On EKS or MiniKube
 - Set up an EKS cluster. I followed this [tutorial](https://medium.com/@tamerbenhassan/deploying-a-simple-application-using-eks-step-by-step-guide-512b1559a7bd).
 - `kubectl apply -k out/overlays/desktop`
@@ -33,5 +37,8 @@ I used compose to develop this locally.
 - I had to rebuild for AMD when the image was not able to be pulled by the pod. It should know to pull the AMD image build.
 - The default image size for your nodes is m5.large. Increase your resource requests as needed.
 
+- Create a node group for the model containers with the taint and labels set correctly (to model=true)
+- Set up your Route 53 by pointing the alias to the K8s cluster public domain, and request a certificate and put the CNAME name and values into Route 53. Additionally, add your pre-generated nameservers to the domain service DNS tab.
+
 #### General Notes:
 - Switch kube contexts when working with MiniKube vs. EKS. Get contexts by running `kubectl config get-contexts` and swtich by running: `kubectl config use-context {NAME}`

compose.yaml

@@ -17,16 +17,15 @@ services:
       context: .
     volumes:
       - ./:/usr/src/app
-      - /usr/src/app/node_modules
+      #- /usr/src/app/node_modules
     env_file:
       - .env.compose
     ports:
       - 3000:3000
       - 5002:5002
   model:
     container_name: model
-    image: ollama/ollama:0.6.2
+    build:
+      context: ./ollama
     ports:
       - 11434:11434
-    post_start:
-      - command: ollama pull llama3.2

ollama/Dockerfile

@@ -0,0 +1,7 @@
+FROM ollama/ollama:0.6.2
+
+# Pre-pull the model during build
+RUN ollama serve & \
+    sleep 3 && \
+    ollama pull llama3.2 && \
+    pkill ollama

out/base/model-deployment.yaml

@@ -32,7 +32,7 @@ spec:
                   effect: "NoSchedule"
             containers:
                 - name: ollama
-                  image: ollama/ollama:0.6.2
+                  image: samanthamorris684/ollama:latest
                   imagePullPolicy: IfNotPresent
                   ports:
                     - name: model-11434
@@ -44,7 +44,4 @@ spec:
                     limits:
                       cpu: "7000m"
                       memory: "30Gi"
-                  lifecycle:
-                    postStart:
-                      exec:
-                        command: ["/bin/sh", "-c", "ollama pull llama3.2"]
+