feat(openshift): add Jaeger distributed tracing integration

Add comprehensive distributed tracing support for OpenShift demos:

- Add Jaeger all-in-one deployment manifest
- Add deployment script with configuration instructions
- Add toggle script to enable/disable tracing easily
- Update DEMO-README.md with tracing documentation
- Add DEMO-URLS.md quick reference for all demo URLs

The tracing integration uses OpenTelemetry with OTLP exporters
and provides end-to-end visibility into request flows including:
- Request ingress through Envoy
- ExtProc classification pipeline
- Security checks (jailbreak, PII)
- Category classification
- Model routing decisions
- Cache hits/misses
- Per-span latency breakdown

This leverages the tracing capabilities added in PR vllm-project#434.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Signed-off-by: Yossi Ovadia <[email protected]>

Author: yossiovadia

deploy/openshift/demo/DEMO-README.md

@@ -35,11 +35,12 @@ python3 deploy/openshift/demo/demo-semantic-router.py
 
 **Features:**
 
-1. **Single Classification** - Tests random prompt from golden set
+1. **Single Classification** - Tests cache with same prompt (fast repeated runs)
 2. **All Classifications** - Tests all 10 golden prompts
-3. **PII Detection Test** - Tests personal information filtering
-4. **Jailbreak Detection Test** - Tests security filtering
-5. **Run All Tests** - Executes all tests sequentially
+3. **Reasoning Showcase** - Chain-of-Thought vs Standard routing
+4. **PII Detection Test** - Tests personal information filtering
+5. **Jailbreak Detection Test** - Tests security filtering
+6. **Run All Tests** - Executes all tests sequentially
 
 **Requirements:**
 
@@ -55,6 +56,55 @@ python3 deploy/openshift/demo/demo-semantic-router.py
 
 ---
 
+### 3. Distributed Tracing with Jaeger
+
+Visualize the complete request flow with distributed tracing:
+
+#### Deploy Jaeger
+
+```bash
+./deploy/openshift/demo/deploy-jaeger.sh
+```
+
+This deploys Jaeger all-in-one with:
+
+- 📊 **Jaeger UI** for visualizing traces
+- 🔗 **OTLP collector** (gRPC and HTTP)
+- 💾 **In-memory storage** (demo-friendly)
+
+#### Enable/Disable Tracing
+
+```bash
+# Enable tracing
+./deploy/openshift/demo/toggle-tracing.sh enable
+
+# Disable tracing
+./deploy/openshift/demo/toggle-tracing.sh disable
+
+# Check status
+./deploy/openshift/demo/toggle-tracing.sh status
+```
+
+#### What You'll See in Jaeger
+
+After enabling tracing and running some requests:
+
+1. Open Jaeger UI (URL shown by toggle-tracing.sh status)
+2. Select service: **vllm-semantic-router**
+3. Click **Find Traces**
+4. Click on a trace to see:
+   - 📥 Request ingress through Envoy
+   - 🔄 ExtProc classification pipeline
+   - 🛡️ Security checks (jailbreak, PII)
+   - 🎯 Category classification
+   - 🧭 Model routing decisions
+   - 💾 Cache hits/misses
+   - ⏱️ End-to-end latency breakdown
+
+**Tip:** Run some requests with `./deploy/openshift/demo/curl-examples.sh all` to generate multiple traces!
+
+---
+
 ## Demo Flow Suggestion
 
 ### Setup (Before Demo)
@@ -67,41 +117,57 @@ python3 deploy/openshift/demo/demo-semantic-router.py
 # (don't run yet)
 
 # Browser Tab 1: Open Grafana
-# http://grafana-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com
+# http://grafana-vllm-semantic-router-system.apps.cluster-xxx.opentlc.com
 
-# Browser Tab 2: Open OpenWebUI
-# http://openwebui-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com
+# Browser Tab 2: Open Jaeger (if tracing enabled)
+# http://jaeger-vllm-semantic-router-system.apps.cluster-xxx.opentlc.com
+
+# Browser Tab 3: Open Flow Visualization
+# http://flow-visualization-vllm-semantic-router-system.apps.cluster-xxx.opentlc.com
+
+# Browser Tab 4: Open OpenWebUI
+# http://openwebui-vllm-semantic-router-system.apps.cluster-xxx.opentlc.com
 ```
 
 ### During Demo
 
 1. **Show the system overview**
+   - Open Flow Visualization (Browser Tab 3)
+   - Click "Start Animation" to show request flow
    - Explain semantic routing concept
-   - Show the architecture diagram
 
 2. **Run interactive demo** (Terminal 2)
 
    ```bash
    python3 deploy/openshift/demo/demo-semantic-router.py
    ```
 
-   Choose option 2 (All Classifications)
+   - Choose option 3 (Reasoning Showcase) to demonstrate CoT
+   - Then option 2 (All Classifications)
 
 3. **Point to live logs** (Terminal 1)
    - Show real-time classification
    - Highlight security checks (jailbreak: BENIGN)
    - Show routing decisions (Model-A vs Model-B)
-   - Point out cache hits
+   - Point out cache hits and reasoning mode activation
 
 4. **Switch to Grafana** (Browser Tab 1)
    - Show request metrics appearing
    - Show classification category distribution
    - Show model usage breakdown
 
-5. **Show OpenWebUI integration** (Browser Tab 2)
+5. **Show Jaeger traces** (Browser Tab 2) - *Optional but impressive!*
+   - Select service: vllm-semantic-router
+   - Click "Find Traces"
+   - Click on a trace to show:
+     - Full request flow timeline
+     - Security checks, classification, routing
+     - Latency breakdown per step
+
+6. **Show OpenWebUI integration** (Browser Tab 4)
    - Type one of the golden prompts
    - Watch it appear in logs (Terminal 1)
-   - Show the same routing happening
+   - Check the trace in Jaeger (Browser Tab 2)
 
 ---
 
@@ -135,7 +201,16 @@ python3 deploy/openshift/demo/demo-semantic-router.py
 
 - Real-time logs with structured JSON
 - Grafana metrics and dashboards
-- Request tracing and debugging
+- **Distributed tracing** with Jaeger (OpenTelemetry)
+- End-to-end request flow visualization
+- Per-span latency breakdown
+
+### Reasoning Capabilities
+
+- **Chain-of-Thought (CoT)** for complex problems
+- Enabled for math, chemistry, physics
+- Standard routing for factual queries
+- Automatic reasoning mode detection
 
 ---
 
@@ -231,6 +306,8 @@ Restarts semantic-router deployment to clear in-memory cache (~30 seconds).
 - `cache-management.sh` - Cache management helper
 - `flow-visualization.html` - **Interactive flow visualization** (open in browser)
 - `deploy-flow-viz.sh` - Deploy flow visualization to OpenShift
+- `deploy-jaeger.sh` - Deploy Jaeger distributed tracing
+- `toggle-tracing.sh` - Enable/disable tracing in semantic-router
 - `CATEGORY-MODEL-MAPPING.md` - Category to model routing reference
 - `demo-classification-results.json` - Test results (auto-generated)
 

deploy/openshift/demo/DEMO-URLS.md

@@ -0,0 +1,86 @@
+# Demo URLs - Quick Reference
+
+**Last Updated:** 2025-10-16
+
+## Quick Access Links
+
+Copy and paste these URLs to access all demo components:
+
+### Core Services
+
+- **Envoy Gateway (HTTP):**
+  `http://envoy-http-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com`
+
+- **Semantic Router API:**
+  `http://semantic-router-api-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com`
+
+### Observability & Monitoring
+
+- **Grafana Dashboard:**
+  `https://grafana-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com`
+
+- **Jaeger Tracing UI:**
+  `https://jaeger-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com`
+
+- **Envoy Admin:**
+  `https://envoy-admin-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com`
+
+### Demo Visualizations
+
+- **Flow Visualization (Interactive):**
+  `https://flow-visualization-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com`
+
+- **OpenWebUI:**
+  `http://openwebui-vllm-semantic-router-system.apps.cluster-pbd96.pbd96.sandbox5333.opentlc.com`
+
+---
+
+## Quick Commands
+
+### Get All URLs Dynamically
+
+```bash
+# List all routes
+oc get routes -n vllm-semantic-router-system
+
+# Get specific URL
+oc get route jaeger -n vllm-semantic-router-system -o jsonpath='{.spec.host}'
+```
+
+### Check Tracing Status
+
+```bash
+./deploy/openshift/demo/toggle-tracing.sh status
+```
+
+### Run Quick Test
+
+```bash
+# Test all 10 golden prompts
+./deploy/openshift/demo/curl-examples.sh all
+
+# Or run interactive demo
+python3 deploy/openshift/demo/demo-semantic-router.py
+```
+
+---
+
+## Pre-Demo Checklist
+
+- [ ] All pods running: `oc get pods -n vllm-semantic-router-system`
+- [ ] Tracing enabled: `./deploy/openshift/demo/toggle-tracing.sh status`
+- [ ] Grafana accessible and showing data
+- [ ] Jaeger UI accessible (if tracing enabled)
+- [ ] Flow visualization deployed and accessible
+- [ ] Terminal 1 ready with log viewer
+- [ ] Terminal 2 ready with demo script
+- [ ] Browser tabs open for all observability tools
+
+---
+
+## Notes
+
+- URLs are specific to your OpenShift cluster
+- Use `oc login` before running demo scripts
+- All scripts automatically discover routes dynamically
+- See [DEMO-README.md](DEMO-README.md) for detailed demo flow

deploy/openshift/demo/deploy-jaeger.sh

@@ -0,0 +1,109 @@
+#!/bin/bash
+#
+# Deploy Jaeger Tracing to OpenShift
+#
+# This script deploys Jaeger all-in-one for distributed tracing visualization
+# and updates the semantic-router configuration to enable tracing.
+#
+
+set -e
+
+# Colors
+GREEN='\033[0;32m'
+CYAN='\033[0;36m'
+YELLOW='\033[1;33m'
+RED='\033[0;31m'
+BOLD='\033[1m'
+NC='\033[0m'
+
+NAMESPACE="vllm-semantic-router-system"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+echo -e "${CYAN}${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
+echo -e "${CYAN}${BOLD}  Deploying Jaeger Distributed Tracing${NC}"
+echo -e "${CYAN}${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}\n"
+
+# Check if logged into OpenShift
+if ! oc whoami &>/dev/null; then
+    echo -e "${RED}Error: Not logged into OpenShift${NC}"
+    echo -e "${YELLOW}Please run: oc login${NC}"
+    exit 1
+fi
+
+# Deploy Jaeger
+echo -e "${CYAN}📊 Deploying Jaeger all-in-one...${NC}"
+oc apply -f "${SCRIPT_DIR}/../observability/jaeger-deployment.yaml" -n "$NAMESPACE"
+
+# Wait for Jaeger to be ready
+echo -e "${CYAN}⏳ Waiting for Jaeger to be ready...${NC}"
+oc rollout status deployment/jaeger -n "$NAMESPACE" --timeout=90s
+
+# Get Jaeger URL
+JAEGER_URL=$(oc get route jaeger -n "$NAMESPACE" -o jsonpath='{.spec.host}' 2>/dev/null)
+
+if [ -z "$JAEGER_URL" ]; then
+    echo -e "${RED}Warning: Could not retrieve Jaeger route${NC}"
+    exit 1
+fi
+
+echo -e "\n${GREEN}✅ Jaeger deployed successfully!${NC}\n"
+
+# Check current tracing status
+CURRENT_CONFIG=$(oc get configmap semantic-router-config -n "$NAMESPACE" -o jsonpath='{.data.config\.yaml}' 2>/dev/null | grep -A 1 "tracing:" | grep "enabled:" | awk '{print $2}' || echo "false")
+
+if [ "$CURRENT_CONFIG" = "true" ]; then
+    echo -e "${GREEN}✓ Tracing is already enabled in config${NC}\n"
+else
+    echo -e "${YELLOW}⚠️  Tracing is currently disabled in semantic-router config${NC}"
+    echo -e "${YELLOW}   To enable tracing, update the ConfigMap:${NC}\n"
+    cat << EOF
+${CYAN}# Edit the ConfigMap:${NC}
+oc edit configmap semantic-router-config -n ${NAMESPACE}
+
+${CYAN}# Change observability.tracing.enabled to true:${NC}
+observability:
+  tracing:
+    enabled: true  # Change from false to true
+    provider: "opentelemetry"
+    exporter:
+      type: "otlp"
+      endpoint: "jaeger:4317"  # Jaeger OTLP gRPC endpoint
+      insecure: true
+    sampling:
+      type: "always_on"
+    resource:
+      service_name: "vllm-semantic-router"
+      service_version: "v0.1.0"
+      deployment_environment: "openshift-demo"
+
+${CYAN}# Then restart semantic-router to apply changes:${NC}
+oc rollout restart deployment/semantic-router -n ${NAMESPACE}
+EOF
+    echo ""
+fi
+
+# Display URLs
+echo -e "${GREEN}${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
+echo -e "${GREEN}${BOLD}  Jaeger Tracing URLs${NC}"
+echo -e "${GREEN}${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}\n"
+
+echo -e "${CYAN}🔍 Jaeger UI:${NC}"
+echo -e "   ${BOLD}https://${JAEGER_URL}${NC}\n"
+
+echo -e "${YELLOW}💡 How to use for your demo:${NC}"
+echo -e "   1. Enable tracing in semantic-router config (see above)"
+echo -e "   2. Run some requests through the semantic router"
+echo -e "   3. Open Jaeger UI and select 'vllm-semantic-router' service"
+echo -e "   4. Click 'Find Traces' to see request flows"
+echo -e "   5. Click on a trace to see the detailed span timeline\n"
+
+echo -e "${CYAN}📋 What you'll see in traces:${NC}"
+echo -e "   • Request ingress through Envoy"
+echo -e "   • ExtProc classification pipeline"
+echo -e "   • Security checks (jailbreak, PII)"
+echo -e "   • Category classification"
+echo -e "   • Model routing decisions"
+echo -e "   • Cache hits/misses"
+echo -e "   • End-to-end latency breakdown\n"
+
+echo -e "${GREEN}${BOLD}Ready for your demo! 🎉${NC}\n"