Skip to content

Commit 8838cbf

Browse files
szedan-rhclaude
szedan-rh
and
claude
authored
fix(openshift): add ChatUI (HuggingChat) deployment with MongoDB support (vllm-project#637)
* fix(openshift): add ChatUI (HuggingChat) deployment with MongoDB support Fixes vllm-project#635 ## Problem The dashboard /huggingchat endpoint was returning an error: {"error":"HuggingChat not configured","message":"TARGET_CHATUI_URL environment variable is not set"} This was because the OpenShift deployment was missing the ChatUI service and the corresponding TARGET_CHATUI_URL environment variable in the dashboard configuration. ## Solution This commit adds complete ChatUI (HuggingChat) support to the OpenShift deployment by: 1. **MongoDB Deployment** (deploy/openshift/mongo/): - PersistentVolumeClaim for 5Gi data storage - MongoDB 7 deployment with proper OpenShift security contexts - Health checks using mongosh ping command - ClusterIP service for internal access - Comprehensive README with backup/recovery guidance 2. **ChatUI Deployment** (deploy/openshift/chatui/): - HuggingFace ChatUI container (ghcr.io/huggingface/chat-ui-db:latest) - Configured to use semantic-router backend for LLM inference - MongoDB connection for data persistence - OpenShift Route with TLS for external access - Health probes for liveness and readiness - Comprehensive README with troubleshooting guide 3. **Dashboard Configuration Update**: - Added TARGET_CHATUI_URL="http://chatui:3000" to dashboard ConfigMap - This environment variable is required by the dashboard backend to configure the ChatUI proxy at /embedded/chatui/ ## Components Added **deploy/openshift/chatui/**: - deployment.yaml: ChatUI deployment, service, and route - README.md: Comprehensive documentation **deploy/openshift/mongo/**: - deployment.yaml: MongoDB deployment with PVC, service - README.md: MongoDB administration and troubleshooting guide ## Technical Details - **ChatUI** connects to semantic-router at port 8801 for LLM inference - **MongoDB** provides data persistence for ChatUI conversations - **Dashboard** proxies ChatUI requests from /huggingchat to /embedded/chatui/ - All components use OpenShift-compatible security contexts: - allowPrivilegeEscalation: false - capabilities dropped - seccompProfile: RuntimeDefault ## Testing After deployment, HuggingChat will be accessible at: - Dashboard integration: https://<dashboard-route>/huggingchat - Direct access: https://<chatui-route> ## Next Steps The deploy-to-openshift.sh script needs to be updated to include these new components in the automated deployment process. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> Signed-off-by: szedan <[email protected]> * fix(openshift): update MongoDB image and deployment script for ChatUI This commit completes the ChatUI/HuggingChat integration for OpenShift by: ## Changes 1. **MongoDB Image Fix** (deploy/openshift/mongo/deployment.yaml): - Changed image from `mongo:7` to `docker.io/library/mongo:7` - Fixes ImagePullBackOff error caused by private registry redirect - Removed `fsGroup: 999` from securityContext to comply with OpenShift security context constraints 2. **Updated Deployment Script** (deploy/openshift/deploy-to-openshift.sh): - Added MongoDB deployment step - Added ChatUI deployment step - Added Dashboard configuration with ChatUI integration - Updated help output to include ChatUI access instructions - Ensures automated deployment of all HuggingChat components ## Testing Verified working on OpenShift cluster: - MongoDB: 1/1 Running - ChatUI: 1/1 Running - Dashboard: Returns HTTP 200 for /huggingchat endpoint - Test URL: https://dashboard-vllm-semantic-router-system.apps.cluster-94lzc.94lzc.sandbox732.opentlc.com/huggingchat Fixes vllm-project#635 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> Signed-off-by: szedan <[email protected]> * fix(docs): add blank lines around fenced code blocks for markdown lint Signed-off-by: szedan <[email protected]> --------- Signed-off-by: szedan <[email protected]> Co-authored-by: Claude <[email protected]>
1 parent 5ec9f43 commit 8838cbf

File tree

6 files changed

+550
-0
lines changed

6 files changed

+550
-0
lines changed

‎deploy/openshift/chatui/README.md‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# HuggingChat ChatUI Deployment for OpenShift
2+
3+
This directory contains OpenShift deployment manifests for HuggingFace ChatUI, which provides the HuggingChat interface accessible through the dashboard.
4+
5+
## Overview
6+
7+
ChatUI is deployed as a containerized web application that provides a chat interface similar to HuggingChat. It connects to the semantic-router service for LLM inference and uses MongoDB for data persistence.
8+
9+
## Components
10+
11+
- **Deployment**: ChatUI application container
12+
- **Service**: ClusterIP service for internal access
13+
- **Route**: OpenShift route for external access with TLS
14+
- **Dependencies**: MongoDB (deployed separately)
15+
16+
## Configuration
17+
18+
The ChatUI deployment is configured via environment variables:
19+
20+
- `OPENAI_BASE_URL`: Points to semantic-router service (http://semantic-router:8801/v1)
21+
- `MONGODB_URL`: MongoDB connection string (mongodb://mongo:27017)
22+
- `PUBLIC_APP_NAME`: Display name ("HuggingChat")
23+
- `LOG_LEVEL`: Logging level (info)
24+
25+
## Deployment
26+
27+
ChatUI is deployed automatically when running the full OpenShift deployment script:
28+
29+
```bash
30+
cd deploy/openshift
31+
./deploy-to-openshift.sh
32+
```
33+
34+
### Manual Deployment
35+
36+
If deploying manually, ensure MongoDB is deployed first:
37+
38+
```bash
39+
# Deploy MongoDB
40+
oc apply -f deploy/openshift/mongo/deployment.yaml
41+
42+
# Deploy ChatUI
43+
oc apply -f deploy/openshift/chatui/deployment.yaml
44+
```
45+
46+
## Accessing ChatUI
47+
48+
### Through Dashboard
49+
50+
Access ChatUI through the semantic-router dashboard at `/huggingchat`:
51+
52+
```bash
53+
DASHBOARD_URL=$(oc get route dashboard -n vllm-semantic-router-system -o jsonpath='{.spec.host}')
54+
echo "HuggingChat: https://$DASHBOARD_URL/huggingchat"
55+
```
56+
57+
### Direct Access
58+
59+
Get the ChatUI route URL:
60+
61+
```bash
62+
CHATUI_URL=$(oc get route chatui -n vllm-semantic-router-system -o jsonpath='{.spec.host}')
63+
echo "ChatUI direct URL: https://$CHATUI_URL"
64+
```
65+
66+
## Integration with Dashboard
67+
68+
The dashboard backend proxies ChatUI at `/embedded/chatui/` and the frontend displays it in an iframe at `/huggingchat`. The integration requires:
69+
70+
1. **TARGET_CHATUI_URL environment variable** in dashboard ConfigMap pointing to `http://chatui:3000`
71+
2. **ChatUI service** running and accessible within the cluster
72+
3. **MongoDB service** running for ChatUI data persistence
73+
74+
## Troubleshooting
75+
76+
### ChatUI Not Loading
77+
78+
1. Check if ChatUI pod is running:
79+
80+
```bash
81+
oc get pods -l app=chatui -n vllm-semantic-router-system
82+
```
83+
84+
2. Check ChatUI logs:
85+
86+
```bash
87+
oc logs -f deployment/chatui -n vllm-semantic-router-system
88+
```
89+
90+
3. Verify MongoDB connection:
91+
92+
```bash
93+
oc get pods -l app=mongo -n vllm-semantic-router-system
94+
```
95+
96+
### Dashboard Shows "HuggingChat not configured" Error
97+
98+
This means the `TARGET_CHATUI_URL` environment variable is not set in the dashboard deployment. Verify the dashboard ConfigMap:
99+
100+
```bash
101+
oc get configmap dashboard-config -n vllm-semantic-router-system -o yaml | grep CHATUI
102+
```
103+
104+
If missing, redeploy the dashboard with the updated configuration.
105+
106+
## Security
107+
108+
- **OpenShift Security Contexts**: Runs with restricted security contexts (no privilege escalation, dropped capabilities)
109+
- **HTTPS Access**: Secure external access via OpenShift routes with TLS edge termination
110+
- **Network Policies**: Internal cluster communication only (no direct external access to backend services)
111+
112+
## Resource Requirements
113+
114+
- **CPU**: 100m request, 500m limit
115+
- **Memory**: 256Mi request, 1Gi limit
116+
- **Depends on**: MongoDB (additional resources)
117+
118+
## Notes
119+
120+
- ChatUI uses the same semantic-router backend as OpenWebUI
121+
- Both chat interfaces (ChatUI and OpenWebUI) are available through the dashboard
122+
- ChatUI requires MongoDB while OpenWebUI has its own data storage
123+
- The dashboard automatically configures the proxy routing based on the TARGET_CHATUI_URL environment variable

‎deploy/openshift/chatui/deployment.yaml‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
---
2+
apiVersion: apps/v1
3+
kind: Deployment
4+
metadata:
5+
name: chatui
6+
namespace: vllm-semantic-router-system
7+
labels:
8+
app: chatui
9+
component: frontend
10+
spec:
11+
replicas: 1
12+
selector:
13+
matchLabels:
14+
app: chatui
15+
template:
16+
metadata:
17+
labels:
18+
app: chatui
19+
component: frontend
20+
spec:
21+
containers:
22+
- name: chatui
23+
image: ghcr.io/huggingface/chat-ui-db:latest
24+
imagePullPolicy: IfNotPresent
25+
ports:
26+
- name: http
27+
containerPort: 3000
28+
protocol: TCP
29+
env:
30+
- name: OPENAI_BASE_URL
31+
value: "http://semantic-router.vllm-semantic-router-system.svc.cluster.local:8801/v1"
32+
- name: OPENAI_API_KEY
33+
value: "changeme"
34+
- name: MONGODB_URL
35+
value: "mongodb://mongo.vllm-semantic-router-system.svc.cluster.local:27017"
36+
- name: MONGODB_DB_NAME
37+
value: "chatui"
38+
- name: PUBLIC_APP_NAME
39+
value: "HuggingChat"
40+
- name: PUBLIC_APP_ASSETS
41+
value: "chatui"
42+
- name: LOG_LEVEL
43+
value: "info"
44+
resources:
45+
requests:
46+
cpu: 100m
47+
memory: 256Mi
48+
limits:
49+
cpu: 500m
50+
memory: 1Gi
51+
securityContext:
52+
allowPrivilegeEscalation: false
53+
capabilities:
54+
drop:
55+
- ALL
56+
seccompProfile:
57+
type: RuntimeDefault
58+
livenessProbe:
59+
httpGet:
60+
path: /
61+
port: 3000
62+
initialDelaySeconds: 30
63+
periodSeconds: 30
64+
timeoutSeconds: 10
65+
failureThreshold: 3
66+
readinessProbe:
67+
httpGet:
68+
path: /
69+
port: 3000
70+
initialDelaySeconds: 10
71+
periodSeconds: 10
72+
timeoutSeconds: 10
73+
failureThreshold: 3
74+
75+
---
76+
apiVersion: v1
77+
kind: Service
78+
metadata:
79+
name: chatui
80+
namespace: vllm-semantic-router-system
81+
labels:
82+
app: chatui
83+
spec:
84+
selector:
85+
app: chatui
86+
ports:
87+
- name: http
88+
port: 3000
89+
targetPort: http
90+
protocol: TCP
91+
type: ClusterIP
92+
93+
---
94+
apiVersion: route.openshift.io/v1
95+
kind: Route
96+
metadata:
97+
name: chatui
98+
namespace: vllm-semantic-router-system
99+
labels:
100+
app: chatui
101+
component: frontend
102+
annotations:
103+
haproxy.router.openshift.io/timeout: "300s"
104+
spec:
105+
to:
106+
kind: Service
107+
name: chatui
108+
weight: 100
109+
port:
110+
targetPort: http
111+
tls:
112+
termination: edge
113+
insecureEdgeTerminationPolicy: Redirect
114+
wildcardPolicy: None

‎deploy/openshift/dashboard/dashboard-deployment.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ data:
1111
TARGET_ROUTER_API_URL: "http://semantic-router:8080"
1212
TARGET_ROUTER_METRICS_URL: "http://semantic-router:9190/metrics"
1313
TARGET_OPENWEBUI_URL: "http://openwebui:3000"
14+
TARGET_CHATUI_URL: "http://chatui:3000"
1415
TARGET_JAEGER_URL: "http://jaeger:16686"
1516
ROUTER_CONFIG_PATH: "/app/config/config.yaml"
1617

‎deploy/openshift/deploy-to-openshift.sh‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,21 @@ rm -f "$TEMP_CONFIG"
217217

218218
success "Deployment manifests applied"
219219

220+
# Deploy MongoDB (required for ChatUI)
221+
log "Deploying MongoDB for ChatUI..."
222+
oc apply -f "$SCRIPT_DIR/mongo/deployment.yaml" -n "$NAMESPACE"
223+
success "MongoDB deployment applied"
224+
225+
# Deploy ChatUI (HuggingChat interface)
226+
log "Deploying ChatUI (HuggingChat)..."
227+
oc apply -f "$SCRIPT_DIR/chatui/deployment.yaml" -n "$NAMESPACE"
228+
success "ChatUI deployment applied"
229+
230+
# Deploy Dashboard with ChatUI integration
231+
log "Deploying Dashboard with ChatUI integration..."
232+
oc apply -f "$SCRIPT_DIR/dashboard/dashboard-deployment.yaml" -n "$NAMESPACE"
233+
success "Dashboard deployment applied"
234+
220235
# Create routes
221236
log "Creating OpenShift routes..."
222237
cat <<EOF | oc apply -n "$NAMESPACE" -f -
@@ -397,3 +412,16 @@ echo " oc logs -f deployment/semantic-router -c semantic-router -n $NAMESPACE"
397412
echo ""
398413
echo " # Check logs for Envoy"
399414
echo " oc logs -f deployment/semantic-router -c envoy-proxy -n $NAMESPACE"
415+
echo ""
416+
echo " # Check logs for MongoDB"
417+
echo " oc logs -f deployment/mongo -n $NAMESPACE"
418+
echo ""
419+
echo " # Check logs for ChatUI (HuggingChat)"
420+
echo " oc logs -f deployment/chatui -n $NAMESPACE"
421+
echo ""
422+
echo " # Check logs for Dashboard"
423+
echo " oc logs -f deployment/dashboard -n $NAMESPACE"
424+
echo ""
425+
echo " # Access ChatUI through Dashboard"
426+
echo " DASHBOARD_URL=\$(oc get route dashboard -n $NAMESPACE -o jsonpath='{.spec.host}')"
427+
echo " echo \"HuggingChat: https://\$DASHBOARD_URL/huggingchat\""

0 commit comments

Comments
 (0)