Updates to K8s manifests and services for K8s compatibility (#117)

## Description
This update improves the K8s manifest files and makes changes to the
services for better compatibility when running in K8s.

k8s datadog config:
* set `DD_ENV`
* Add `DD_CLOUD_PROVIDER_METADATA` setting

Cluster files:
* updated NGINX ingress controller version

Storedog k8s files:
* remove unnecessary envars
* add and update labels
* add files to setup AB testing

README files:
- Update AB Testing with K8s instructions
- Update Ads readme details
- K8s readme: review for clarity and flow

Docker Compose:
* Remove unnecessary envars
* Add `DD_CLOUD_PROVIDER_METADATA` setting
* Update Ruby commands to match Profiling docs

Services files:
- Discounts: remove variable for port
- Update dd-trace library versions

Author: spottsdd

README.md

@@ -206,7 +206,24 @@ There are several features that can be enabled by setting environment variables
 
 Run two Ads services and split traffic between them. The amount of traffic sent to each service is set with a percent value.
 
+This requires running a second Ads service in addition to the default Java Ads service and setting environment variables in the `service-proxy` service. The Python Ads service is typically used as the secondary service.
+
+1. Set an environment variable for the Python Ads service version.
+
+    ```bash
+    export DD_VERSION_ADS_PYTHON=1.0.0
+    ```
+
+1. These environment variables need to be set for the `service-proxy` service.
+
+    - `ADS_A_UPSTREAM`: Host and port for the primary (A) ads service (default: `ads:3030`)
+    - `ADS_B_UPSTREAM`: Host and port for the secondary (B) ads service (default: `ads-python:3030`)
+    - `ADS_B_PERCENT`: Percentage of traffic to route to the B (Python) ads service (default: `0`). The remainder goes to the A ads (Java) service.
+      - Set a value between `0` and `100` to control the split.
+
 **How to use**
+
+#### Docker Compose
 1. Add a second Ads service to the `docker-compose.yml`
 
     ```yaml
@@ -223,15 +240,19 @@ Run two Ads services and split traffic between them. The amount of traffic sent
           - POSTGRES_USER=${POSTGRES_USER:-postgres}
           - POSTGRES_HOST=postgres
           - DD_AGENT_HOST=dd-agent
+          - DD_ENV=${DD_ENV:-production}
           - DD_SERVICE=store-ads-python
           - DD_VERSION=${DD_VERSION_ADS_PYTHON:-1.0.0}
+          - DD_PROFILING_ENABLED=true
+          - DD_PROFILING_TIMELINE_ENABLED=true
+          - DD_PROFILING_ALLOCATION_ENABLED=true
         networks:
           - storedog-network
         labels:
           com.datadoghq.ad.logs: '[{"source": "python"}]'
     ```
 
-1. Add and set these environment variables to the `service-proxy` service:
+1. Add these environment variables to the `service-proxy` service in the `docker-compose.yml` file:
 
     ```yaml
     environment:
@@ -240,13 +261,40 @@ Run two Ads services and split traffic between them. The amount of traffic sent
       - ADS_B_PERCENT=${ADS_B_PERCENT:-0}
     ```
 
-    - `ADS_A_UPSTREAM`: Host and port for the primary (A) ads service (default: `ads:3030`)
-    - `ADS_B_UPSTREAM`: Host and port for the secondary (B) ads service (default: `ads-python:3030`)
-    - `ADS_B_PERCENT`: Percentage of traffic to route to the B (Python) ads service (default: `0`). The remainder goes to the A (Java) ads service.
-      - Set to a value between `0` and `100` to control the split.
 1. Start the app via `docker compose up -d`
 
+#### Kubernetes
+
+A Kubernetes manifest for the Python Ads service is available in the `services/ads/k8s-manifests/` directory.
+
+1. Add the `ads-python.yaml` file to the `k8s-manifests/storedog-app/deployments/` directory.
+
+1. Add the following environment variables to the `nginx.yaml` file and adjust as needed:
+
+    ```yaml
+    # A/B testing ads services
+    - name: ADS_A_UPSTREAM
+      value: "ads:3030"
+    - name: ADS_B_UPSTREAM
+      value: "ads-python:3030"
+    - name: ADS_B_PERCENT
+      value: "50"
+    ```
+
+1. Follow the instructions in the [Kubernetes README](./k8s-manifests/README.md) to run Storedog in Kubernetes.
+
+1. If the Storedog is already running, apply the manifests to the cluster:
+
+    ```bash
+    envsubst < k8s-manifests/storedog-app/deployments/ads-python.yaml | kubectl apply -n storedog -f -
+    envsubst < k8s-manifests/storedog-app/deployments/nginx.yaml | kubectl apply -n storedog -f -
+    ```
+
+> [!IMPORTANT]
+> Be sure to set the `DD_VERSION_ADS_PYTHON` environment variable so that it will be applied to the file by `envsubst`.
+
 ### Feature flags
+
 Some capabilities are hidden behind feature flags, which can be controlled via `services/frontend/site/featureFlags.config.json`. 
 
 > [!NOTE]
@@ -258,6 +306,7 @@ Some capabilities are hidden behind feature flags, which can be controlled via `
 > ```
 
 #### dbm 
+
 Enables a product ticker on the homepage with a long-running query to demonstrate DBM. 
 
 **How to use**:
@@ -270,6 +319,7 @@ Enables a product ticker on the homepage with a long-running query to demonstrat
 You can modify the ticker functionality in `services/frontend/components/common/NavBar.tsx`.
 
 #### error-tracking 
+
 Introduces an exception in the Ads services to demonstrate Error Tracking by setting a header in to a value that is not expected by the Ads service.
 
 **How to use**:
@@ -281,6 +331,7 @@ Introduces an exception in the Ads services to demonstrate Error Tracking by set
 Modify this functionality in `services/frontend/components/common/Ad/Ad.tsx` and respective Ads service being used.
 
 #### api-errors
+
 This introduces random errors that occur in the frontend service's `/api` routes.
 
 **How to use**:
@@ -291,6 +342,7 @@ This introduces random errors that occur in the frontend service's `/api` routes
 Modify this functionality in `services/frontend/pages/api/*`.
 
 #### product-card-frustration
+
 This will swap out the product card component with a version that doesn't have the thumbnails linked to the product page. When paired with the Puppeteer service, this can be used to demonstrate Frustration Signals in RUM.
 
 **How to use**:
@@ -301,6 +353,7 @@ This will swap out the product card component with a version that doesn't have t
 Modify this functionality in `services/frontend/components/Product/ProductCard.tsx` and `services/frontend/components/Product/ProductCard-v2.tsx`.
 
 ## Image publication
+
 Images are stored in GHCR. On PR merges, only the affected services will be pushed to GHCR, using the `latest` tag. For example, if you only made changes to the `backend` service, then only the `backend` Github workflow will trigger and publish `ghcr.io/datadog/storedog/backend:latest`. 
 
 Separately, we tag and publish *all* images when a new release is created with the corresponding release tag e.g. `ghcr.io/datadog/storedog/backend:1.0.1`. New releases are made on an ad-hoc basis, depending on the recent features that are added.
@@ -312,6 +365,7 @@ All of the services in the Storedog application are Dockerized and run in contai
 Below is a breakdown of services and some instructions on how to use them.
 
 ### Ads
+
 There are two advertisement services, the default service is built in Java and there is another option available in Python. These services do the same thing, have the same endpoints, run on the same port (`3030`), and have the same failure modes. These ads are served through the `Ads.tsx` component in the frontend service.
 
 To switch between the Java and Python services, see the instructions in the [Ads service README](./services/ads/README.md).
@@ -340,6 +394,7 @@ sh ./scripts/backup-db.sh
 This will create a new `restore.sql` file in the `services/postgres/db/` directory and get it set up with all of necessary SQL statements to prepare the database for Datadog monitoring. When done running, you'll want to rebuild the Postgres database image with the new restore point. 
 
 #### Worker
+
 The Spree application has a worker process that runs in the background. There is a specific Datadog tracer configuration for it in the `services/worker/` directory and is mounted into the worker container.
 
 ### Discounts

docker-compose.dev.yml

@@ -11,6 +11,7 @@ services:
       - DD_LOGS_ENABLED=true
       - DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true
       - DD_CONTAINER_EXCLUDE=image:agent name:puppeteer
+      - DD_CLOUD_PROVIDER_METADATA=[]
       - DD_HOSTNAME=${DD_HOSTNAME-development-host}
       - DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true
     networks:
@@ -38,9 +39,9 @@ services:
       - storedog-network
     environment:
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-development}
       - DD_SERVICE=store-frontend-api # used for Next.js API routes
       - DD_VERSION=${NEXT_PUBLIC_DD_VERSION_FRONTEND-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_RUNTIME_METRICS_ENABLED=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
@@ -59,13 +60,13 @@ services:
       - NEXT_PUBLIC_SPREE_IMAGE_HOST=${NEXT_PUBLIC_SPREE_IMAGE_HOST:-/services/backend}
       - NEXT_PUBLIC_SPREE_ALLOWED_IMAGE_DOMAIN=${NEXT_PUBLIC_SPREE_ALLOWED_IMAGE_DOMAIN:-service-proxy}
     labels:
-      com.datadoghq.ad.logs: '[{"source": "nodejs", "auto_multi_line_detection":true }]'
+      com.datadoghq.ad.logs: '[{"source": "nodejs", "auto_multi_line_detection": true }]'
 
   # Backend service (Ruby on Rails/Spree)
   backend:
     build:
       context: ./services/backend
-    command: wait-for-it postgres:5432 -- bundle exec rails s -b 0.0.0.0 -p 4000 --pid /app/tmp/pids/server.pid
+    command: wait-for-it postgres:5432 -- bundle exec ddprofrb exec rails s -b 0.0.0.0 -p 4000 --pid /app/tmp/pids/server.pid
     depends_on:
       - 'postgres'
       - 'redis'
@@ -85,21 +86,21 @@ services:
       - DB_POOL=${DB_POOL:-25}
       - MAX_THREADS=${MAX_THREADS:-5}
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-development}
       - DD_SERVICE=store-backend
       - DD_VERSION=${DD_VERSION_BACKEND:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_RUNTIME_METRICS_ENABLED=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_ALLOCATION_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
     labels:
-      com.datadoghq.ad.logs: '[{"source": "ruby", "auto_multi_line_detection":true }]'
+      com.datadoghq.ad.logs: '[{"source": "ruby", "auto_multi_line_detection": true }]'
 
   # Background job processor (Sidekiq)
   worker:
     build:
       context: ./services/backend
-    command: wait-for-it postgres:5432 -- bundle exec sidekiq -C config/sidekiq.yml
+    command: wait-for-it postgres:5432 -- bundle exec ddprofrb exec sidekiq -C config/sidekiq.yml
     depends_on:
       - 'postgres'
       - 'redis'
@@ -121,15 +122,15 @@ services:
       - DB_POOL=${DB_POOL:-25}
       - MAX_THREADS=${MAX_THREADS:-5}
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-development}
       - DD_SERVICE=store-worker
       - DD_VERSION=${DD_VERSION_BACKEND:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_RUNTIME_METRICS_ENABLED=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
       - DD_PROFILING_ALLOCATION_ENABLED=true
     labels:
-      com.datadoghq.ad.logs: '[{"source": "ruby", "auto_multi_line_detection":true }]'
+      com.datadoghq.ad.logs: '[{"source": "ruby", "auto_multi_line_detection": true }]'
 
   # Discounts service (Python/Flask)
   discounts:
@@ -140,15 +141,13 @@ services:
       - postgres
       - dd-agent
     environment:
-      - FLASK_APP=discounts.py
-      - FLASK_DEBUG=0
       - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
       - POSTGRES_USER=${POSTGRES_USER:-postgres}
       - POSTGRES_HOST=postgres
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-development}
       - DD_SERVICE=store-discounts
       - DD_VERSION=${DD_VERSION_DISCOUNTS:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_RUNTIME_METRICS_ENABLED=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
@@ -172,9 +171,9 @@ services:
       - POSTGRES_USER=${POSTGRES_USER:-postgres}
       - POSTGRES_HOST=postgres
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-development}
       - DD_SERVICE=store-ads
       - DD_VERSION=${DD_VERSION_ADS:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
       - DD_PROFILING_ALLOCATION_ENABLED=true
@@ -197,6 +196,7 @@ services:
       - dd-agent
     environment:
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-development}
       - DD_SERVICE=service-proxy
       - DD_VERSION=${DD_VERSION_NGINX:-1.28.0}
     labels:
@@ -225,7 +225,7 @@ services:
       com.datadoghq.ad.check_names: '["postgres"]'
       com.datadoghq.ad.init_configs: '[{}]'
       com.datadoghq.ad.instances: '[{"host":"%%host%%", "port":5432, "username":"datadog", "password":"datadog"}]'
-      com.datadoghq.ad.logs: '[{"source": "postgresql", "auto_multi_line_detection":true, "path": "/var/log/pg_log/postgresql*.json", "type": "file"}]'
+      com.datadoghq.ad.logs: '[{"source": "postgresql", "service":"store-db", "auto_multi_line_detection": true, "path": "/var/log/pg_log/postgresql*.json", "type": "file"}]'
 
   # Cache and job queue
   redis:
@@ -238,7 +238,6 @@ services:
       - storedog-network
     labels:
       com.datadoghq.tags.service: 'redis'
-      com.datadoghq.tags.env: '${DD_ENV:-development}'
       com.datadoghq.tags.version: '${DD_VERSION_REDIS:-6.2}'
       com.datadoghq.ad.check_names: '["redisdb"]'
       com.datadoghq.ad.init_configs: '[{}]'

docker-compose.yml

@@ -8,6 +8,7 @@ services:
       - DD_LOGS_ENABLED=true
       - DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true
       - DD_CONTAINER_EXCLUDE=image:agent name:puppeteer
+      - DD_CLOUD_PROVIDER_METADATA=[]
       - DD_HOSTNAME=${DD_HOSTNAME:-production-host}
       - DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true
     networks:
@@ -29,9 +30,9 @@ services:
       - storedog-network
     environment:
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-production}
       - DD_SERVICE=store-frontend-api # used for Next.js API routes
       - DD_VERSION=${NEXT_PUBLIC_DD_VERSION_FRONTEND:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_RUNTIME_METRICS_ENABLED=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
@@ -43,10 +44,10 @@ services:
       - NEXT_PUBLIC_DD_ENV=${DD_ENV:-production}
       - NEXT_PUBLIC_DD_VERSION=${NEXT_PUBLIC_DD_VERSION_FRONTEND:-1.0.0}
     labels:
-      com.datadoghq.ad.logs: '[{"source": "nodejs", "auto_multi_line_detection":true }]'
+      com.datadoghq.ad.logs: '[{"source": "nodejs", "auto_multi_line_detection": true }]'
   backend:
     image: ghcr.io/datadog/storedog/backend:${STOREDOG_IMAGE_VERSION:-latest}
-    command: wait-for-it postgres:5432 -- bundle exec rails s -b 0.0.0.0 -p 4000 --pid /app/tmp/pids/server.pid
+    command: wait-for-it postgres:5432 -- bundle exec ddprofrb exec rails s -b 0.0.0.0 -p 4000 --pid /app/tmp/pids/server.pid
     depends_on:
       - 'postgres'
       - 'redis'
@@ -57,18 +58,18 @@ services:
       - './services/backend:/app'
     environment:
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-production}
       - DD_SERVICE=store-backend
       - DD_VERSION=${DD_VERSION_BACKEND:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_RUNTIME_METRICS_ENABLED=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
       - DD_PROFILING_ALLOCATION_ENABLED=true
     labels:
-      com.datadoghq.ad.logs: '[{"source": "ruby", "auto_multi_line_detection":true }]'
+      com.datadoghq.ad.logs: '[{"source": "ruby", "auto_multi_line_detection": true }]'
   worker:
     image: ghcr.io/datadog/storedog/backend:${STOREDOG_IMAGE_VERSION:-latest}
-    command: wait-for-it postgres:5432 -- bundle exec sidekiq -C config/sidekiq.yml
+    command: wait-for-it postgres:5432 -- bundle exec ddprofrb exec sidekiq -C config/sidekiq.yml
     depends_on:
       - 'postgres'
       - 'redis'
@@ -79,15 +80,15 @@ services:
     environment:
       - WORKER=true
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-production}
       - DD_SERVICE=store-worker
       - DD_VERSION=${DD_VERSION_BACKEND:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_RUNTIME_METRICS_ENABLED=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
       - DD_PROFILING_ALLOCATION_ENABLED=true
     labels:
-      com.datadoghq.ad.logs: '[{"source": "ruby", "auto_multi_line_detection":true }]'
+      com.datadoghq.ad.logs: '[{"source": "ruby", "auto_multi_line_detection": true }]'
   discounts:
     image: ghcr.io/datadog/storedog/discounts:${STOREDOG_IMAGE_VERSION:-latest}
     command: wait-for-it postgres:5432 -- ddtrace-run flask run --port=2814 --host=0.0.0.0
@@ -96,9 +97,9 @@ services:
       - dd-agent
     environment:
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-production}
       - DD_SERVICE=store-discounts
       - DD_VERSION=${DD_VERSION_DISCOUNTS:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_RUNTIME_METRICS_ENABLED=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
@@ -117,9 +118,9 @@ services:
       - POSTGRES_USER=${POSTGRES_USER:-postgres}
       - POSTGRES_HOST=postgres
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-production}
       - DD_SERVICE=store-ads
       - DD_VERSION=${DD_VERSION_ADS:-1.0.0}
-      - DD_LOGS_INJECTION=true
       - DD_PROFILING_ENABLED=true
       - DD_PROFILING_TIMELINE_ENABLED=true
       - DD_PROFILING_ALLOCATION_ENABLED=true
@@ -141,6 +142,7 @@ services:
       - dd-agent
     environment:
       - DD_AGENT_HOST=dd-agent
+      - DD_ENV=${DD_ENV:-production}
       - DD_SERVICE=service-proxy
       - DD_VERSION=${DD_VERSION_NGINX:-1.28.0}
     labels:
@@ -166,7 +168,7 @@ services:
       com.datadoghq.ad.check_names: '["postgres"]'
       com.datadoghq.ad.init_configs: '[{}]'
       com.datadoghq.ad.instances: '[{"host":"%%host%%", "port":5432, "username":"datadog", "password":"datadog"}]'
-      com.datadoghq.ad.logs: '[{"source": "postgresql", "auto_multi_line_detection":true, "path": "/var/log/pg_log/postgresql*.json", "type": "file"}]'
+      com.datadoghq.ad.logs: '[{"source": "postgresql", "service":"store-db", "auto_multi_line_detection": true, "path": "/var/log/pg_log/postgresql*.json", "type": "file"}]'
   redis:
     image: redis:6.2-alpine
     depends_on:
@@ -177,7 +179,6 @@ services:
       - storedog-network
     labels:
       com.datadoghq.tags.service: 'redis'
-      com.datadoghq.tags.env: '${DD_ENV:-production}'
       com.datadoghq.tags.version: '${DD_VERSION_REDIS:-6.2}'
       com.datadoghq.ad.check_names: '["redisdb"]'
       com.datadoghq.ad.init_configs: '[{}]'