adding load balanced replicas' docker compose configuration

updating README with the replicas compose docs

Author: moda20

README.md

@@ -21,15 +21,38 @@ The service uses the following environment variables:
 
 ### Running the Service
 
+#### Running a single instance
+
+1.  **Build and Run:** Execute the following command in the root directory:
+
+    ```bash
+    docker compose -f docker/compose.yml up --build -d
+    ```
+
+    The first run will download the `clip-ViT-B-32` model by default (or the model you put in the env variables) (approx. 600MB) and store it in the persistent `model_cache` volume.
+
+2.  **Access the API:** The service will be available at `http://localhost:8000`.
+
+#### Running a load-balanced n number of instances
+
+The load balancing is handled by nginx and a set of replica containers are created. calling the nginx server would automatically choose the target instance.
+
+**Note :** In order to have a single downloading instance of the default model, it's better to run the single instance compose file to populate the volume with the model files.
+After that all instances will just read and load their copy of the model into VRAM. 
+
+**Note :** Ensure that you have enough VRAM to hold the model copies, otherwise some instances might crash or spill into system RAM.
+
 1.  **Build and Run:** Execute the following command in the root directory:
 
     ```bash
-    docker compose up --build -d
+    docker compose -f docker/compose_load_balanced.yml up --build -d
     ```
+    This will also download the default model, 
+
+2.  **Access the API:** The service will be available at `http://localhost:8004`. 
+
 
-    The first run will download the `clip-ViT-B-32` model by default (approx. 600MB) and store it in the persistent `model_cache` volume.
 
-2.  **Access the API:** The service will be available at `http://localhost:8000`. You can view the interactive documentation (Swagger UI) at `http://localhost:8000/docs`.
 
 ## 💡 API Endpoints
 

docker/compose_load_balanced.yml

@@ -0,0 +1,24 @@
+services:
+  embedding-service:
+    build:
+      context: ../
+      dockerfile: Dockerfile
+    #image: ghcr.io/moda20/mes:latest
+    ports:
+      - "8000"
+    environment:
+      TRANSFORMERS_CACHE: ${MODEL_CACHE_DIR:-/app/model_cache}
+    volumes:
+      - model_cache:/app/model_cache
+    restart: unless-stopped
+    deploy:
+      replicas: 1
+  nginx:
+    image: nginx:latest
+    ports:
+      - "8004:80"
+    volumes:
+      - ./nginx.conf:/etc/nginx/nginx.conf:ro # Nginx configuration mount
+volumes:
+  model_cache:
+    driver: local

docker/nginx.conf

@@ -0,0 +1,24 @@
+events {
+    worker_connections 1024;
+}
+
+
+http {
+  resolver 127.0.0.11 valid=10s;
+  upstream backend {
+    zone backend 64k;
+    least_conn;
+    server embedding-service:8000 resolve;
+  }
+
+  server {
+    listen 80;
+
+    location / {
+      proxy_pass http://backend;
+      proxy_set_header Host $host;
+      proxy_set_header X-Real-IP $remote_addr;
+      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    }
+  }
+}