Skip to content

Commit ca2159c

Browse files
committed
complete chatkit server and ui integration
1 parent 4082a38 commit ca2159c

30 files changed

+4631
-65
lines changed

.github/workflows/deploy.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ jobs:
6161
AUTH_URL: ${{ secrets.AUTH_URL }}
6262
OAUTH_CLIENT_ID: ${{ secrets.OAUTH_CLIENT_ID }}
6363
BASE_URL: ${{ secrets.BASE_URL }}
64+
BACKEND_URL: ${{ secrets.BACKEND_URL }}
65+
CHATKIT_DOMAIN_KEY: ${{ secrets.CHATKIT_DOMAIN_KEY }}
6466
# GEMINI_API_KEY is optional - LibreTranslate (free) is used by default
6567
# Only set this if you want to use Gemini instead of LibreTranslate
6668
# GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}

.gitignore

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,9 @@ htmlcov/
4242
.ruff_cache/
4343
# Translation cache (temporary, regenerated)
4444
.translation-cache/
45-
# i18n/ - REMOVED: We commit translated files so CI doesn't need to translate
45+
# i18n/ - REMOVED: We commit translated files so CI doesn't need to translate
46+
47+
openai-chatkit-starter-app-main/
48+
business_leads_agent/
49+
50+
car-fixer-platform/

rag-agent/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,3 +43,6 @@ htmlcov/
4343
# Misc
4444
.DS_Store
4545
*.log
46+
47+
# Local chatkit directory (conflicts with installed package)
48+
chatkit/

rag-agent/CHATKIT_SETUP.md

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# ChatKit Integration Setup
2+
3+
This document describes how the ChatKit server is integrated with the RoboLearn interface.
4+
5+
## Architecture
6+
7+
- **Backend**: `rag-agent` - FastAPI server with ChatKit endpoint at `/chatkit`
8+
- **Frontend**: `robolearn-interface` - Docusaurus site with ChatKit widget component
9+
10+
## Backend Configuration
11+
12+
### Environment Variables
13+
14+
The ChatKit server requires a PostgreSQL database connection. Configure one of these:
15+
16+
**Option 1 (Recommended):**
17+
```bash
18+
CHATKIT_STORE_DATABASE_URL=postgresql+asyncpg://user:password@host:port/database
19+
```
20+
21+
**Option 2 (Fallback):**
22+
```bash
23+
DATABASE_URL=postgresql+asyncpg://user:password@host:port/database
24+
```
25+
26+
### Database Schema
27+
28+
The ChatKit store automatically creates the required schema on startup:
29+
- Schema name: `chatkit` (configurable via `CHATKIT_STORE_SCHEMA_NAME`)
30+
- Tables: `threads`, `items`, `attachments`
31+
32+
### Endpoints
33+
34+
- `POST /chatkit` - Main ChatKit API endpoint
35+
- Requires `X-User-ID` header for user identification
36+
- Optional `X-Request-ID` header for tracing
37+
38+
## Frontend Configuration
39+
40+
### Environment Variables
41+
42+
In `robolearn-interface/.env` or via `docusaurus.config.ts`:
43+
44+
```bash
45+
BACKEND_URL=http://localhost:8000 # URL of rag-agent server
46+
CHATKIT_DOMAIN_KEY=domain_pk_local_dev # Optional, dummy value for custom backend
47+
```
48+
49+
### Widget Component
50+
51+
The ChatKit widget (`src/components/ChatKitWidget/index.tsx`) is automatically loaded via `Root.tsx`.
52+
53+
**Features:**
54+
- Floating chat button (bottom-right)
55+
- Text selection with "Ask" button
56+
- Page context awareness
57+
- User authentication via `X-User-ID` header
58+
59+
### User Identification
60+
61+
The widget uses the authenticated user's ID from `AuthContext`:
62+
- If logged in: Uses `session.user.id`
63+
- If anonymous: Uses `'anonymous'`
64+
65+
## Testing
66+
67+
1. **Start the backend:**
68+
```bash
69+
cd rag-agent
70+
uv sync
71+
# Set DATABASE_URL or CHATKIT_STORE_DATABASE_URL in .env
72+
uvicorn app:app --reload --port 8000
73+
```
74+
75+
2. **Start the frontend:**
76+
```bash
77+
cd robolearn-interface
78+
npm install
79+
# Set BACKEND_URL in .env or docusaurus.config.ts
80+
npm start
81+
```
82+
83+
3. **Test the widget:**
84+
- Click the chat button (bottom-right)
85+
- Send a message
86+
- Check backend logs for `/chatkit` requests
87+
88+
## Troubleshooting
89+
90+
### ChatKit widget not appearing
91+
- Check browser console for errors
92+
- Verify ChatKit script is loaded (check Network tab for `chatkit.js`)
93+
- Check `BACKEND_URL` is correctly configured
94+
95+
### Backend errors
96+
- Verify database connection string is correct
97+
- Check database is accessible
98+
- Review backend logs for initialization errors
99+
100+
### Authentication issues
101+
- Verify `X-User-ID` header is being sent (check Network tab)
102+
- Check `AuthContext` is providing user session
103+
104+
## Differences from car-fixer-platform
105+
106+
The robolearn implementation:
107+
- Uses simpler authentication (no client secret endpoints)
108+
- Direct `/chatkit` endpoint communication
109+
- No domain key validation (uses dummy value for custom backend)
110+
- Integrated with Docusaurus instead of Next.js
111+

rag-agent/README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@ DOCS_PATH=../robolearn-interface/docs
5757
FRONTEND_BASE_URL=https://mjunaidca.github.io/robolearn
5858
```
5959

60+
uv run uvicorn app:app --port 8000 --reload
61+
6062
### 3. Ingest Content
6163

6264
**First-time ingestion (complete re-index):**
@@ -113,7 +115,7 @@ uvicorn app:app --port 8000 --reload
113115

114116
```bash
115117
# Copy your ChatKit server code
116-
cp -r /path/to/your/chatkit/* rag-agent/chatkit/
118+
cp -r /path/to/your/chatkit/* rag-agent/chatkit_integration/
117119
```
118120

119121
### Step 2: Register ChatKit Router

rag-agent/app.py

Lines changed: 90 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -6,47 +6,25 @@
66
- /rag/* - RAG retrieval and ingestion
77
- /chatkit/* - ChatKit server (agent works within ChatKit)
88
- /core/* - Shared utilities and config
9-
10-
To integrate your ChatKit code:
11-
1. Copy your chatkit/ folder here
12-
2. Register router in this app
13-
3. Add search_tool from rag.tools to your agent (within ChatKit)
149
"""
1510

11+
import json
1612
import logging
17-
from contextlib import asynccontextmanager
18-
from fastapi import FastAPI
13+
from fastapi import FastAPI, Request, HTTPException, Header
1914
from fastapi.middleware.cors import CORSMiddleware
15+
from fastapi.responses import Response, StreamingResponse
16+
from chatkit.server import StreamingResult
2017

2118
# Import API routes
2219
from api import search, health
20+
from lifespan import lifespan
21+
from chatkit_store import RequestContext
2322

2423
# Simple logging
2524
logging.basicConfig(level=logging.INFO)
2625
logger = logging.getLogger(__name__)
2726

2827

29-
@asynccontextmanager
30-
async def lifespan(app: FastAPI):
31-
"""Lifespan manager - resilient to Qdrant failures."""
32-
logger.info("Starting RoboLearn Backend")
33-
34-
# Initialize RAG search client (non-blocking, fails gracefully)
35-
try:
36-
from rag.tools import get_search_client
37-
client = get_search_client()
38-
if client is not None:
39-
logger.info("RAG search client initialized")
40-
else:
41-
logger.warning("RAG search client unavailable (Qdrant connection failed)")
42-
except Exception as e:
43-
logger.warning(f"RAG initialization failed (non-critical): {e}")
44-
45-
yield
46-
47-
logger.info("Shutting down RoboLearn Backend")
48-
49-
5028
app = FastAPI(
5129
title="RoboLearn Backend",
5230
description="RAG search, ChatKit, and Agent integration for educational content",
@@ -67,23 +45,93 @@ async def lifespan(app: FastAPI):
6745
app.include_router(search.router)
6846
app.include_router(health.router)
6947

70-
# Register ChatKit router (agent works within ChatKit)
71-
_chatkit_registered = False
72-
try:
73-
from chatkit import router as chatkit_router
74-
app.include_router(chatkit_router)
75-
_chatkit_registered = True
76-
logger.info("ChatKit router registered")
77-
except ImportError:
78-
logger.info("ChatKit router not found (expected if not yet integrated)")
79-
except Exception as e:
80-
logger.warning(f"Failed to register ChatKit router: {e}")
48+
49+
def get_request_context(
50+
user_id: str = Header(..., alias="X-User-ID"),
51+
request_id: str | None = Header(None, alias="X-Request-ID"),
52+
) -> RequestContext:
53+
"""
54+
Extract request context from headers.
55+
56+
In production, you should validate authentication here.
57+
"""
58+
return RequestContext(
59+
user_id=user_id,
60+
request_id=request_id,
61+
)
62+
63+
64+
@app.post("/chatkit")
65+
async def chatkit_endpoint(request: Request):
66+
"""
67+
Main ChatKit endpoint for conversational interaction.
68+
69+
Requires X-User-ID header for user identification.
70+
"""
71+
payload = await request.body()
72+
logger.debug(f"chatkit_endpoint received payload: {len(payload)} bytes")
73+
74+
# Get server from app state
75+
chatkit_server = getattr(request.app.state, "chatkit_server", None)
76+
if not chatkit_server:
77+
raise HTTPException(
78+
status_code=503,
79+
detail="ChatKit server not initialized. Check database configuration.",
80+
)
81+
82+
# Extract user ID from header
83+
user_id = request.headers.get("X-User-ID")
84+
if not user_id:
85+
raise HTTPException(status_code=401, detail="Missing X-User-ID header")
86+
87+
try:
88+
# Process ChatKit request
89+
payload = await request.body()
90+
# decode payload to dict from bytes and add in metadata
91+
payload_dict = json.loads(payload)
92+
93+
# Extract metadata from the correct location in ChatKit request
94+
metadata = {}
95+
if "params" in payload_dict and "input" in payload_dict["params"]:
96+
metadata = payload_dict["params"]["input"].get("metadata", {})
97+
98+
logger.debug(f"Extracted metadata: {metadata}")
99+
100+
# Create request context
101+
context = RequestContext(
102+
user_id=user_id,
103+
request_id=request.headers.get("X-Request-ID"),
104+
metadata=metadata,
105+
)
106+
result = await chatkit_server.process(payload, context)
107+
108+
# Return appropriate response type
109+
if isinstance(result, StreamingResult):
110+
return StreamingResponse(
111+
result,
112+
media_type="text/event-stream",
113+
headers={
114+
"Cache-Control": "no-cache",
115+
"Connection": "keep-alive",
116+
"X-Accel-Buffering": "no",
117+
},
118+
)
119+
else:
120+
return Response(
121+
content=result.json,
122+
media_type="application/json",
123+
)
124+
except Exception as e:
125+
logger.exception(f"Error processing ChatKit request: {e}")
126+
raise HTTPException(
127+
status_code=500, detail=f"Error processing request: {str(e)}"
128+
)
81129

82130

83131
@app.get("/")
84132
async def root():
85133
"""Root endpoint."""
86-
chatkit_status = "✅ Active" if _chatkit_registered else "⏳ Not integrated"
134+
chatkit_status = "✅ Active" if hasattr(app.state, "chatkit_server") and app.state.chatkit_server else "⏳ Not initialized"
87135
return {
88136
"service": "RoboLearn Backend",
89137
"version": "1.0.0",
@@ -94,11 +142,12 @@ async def root():
94142
"endpoints": {
95143
"search": "POST /search",
96144
"health": "GET /health",
145+
"chatkit": "POST /chatkit",
97146
},
98147
"tool": {
99148
"name": "search_robolearn_content",
100149
"import": "from rag.tools import search_tool, SEARCH_TOOL_SCHEMA",
101-
"note": "Add to your agent within ChatKit",
150+
"note": "Integrated with ChatKit agent",
102151
},
103152
}
104153

rag-agent/chatkit/__init__.py

Lines changed: 0 additions & 14 deletions
This file was deleted.

0 commit comments

Comments
 (0)