Skip to content

Commit f910800

Browse files
harish876sophiequynncelinejptheopan8sandhyabirajdar
authored
GraphQ-LLM MCP server (#227)
* Initial commit * Initial commit * Initial commit * Initial commit * setup * setup * Update docker-compose.dev.yml * Update docker-compose.dev.yml * setup update * setup update * fixed setup * fixed setup * Nexus integration * Nexus integration * nexus update * nexus update * setup update * setup update * feat: Dockerize MCP server and update documentation - Add separate Docker service for MCP server in docker-compose.dev.yml - MCP server runs as independent container using stdio transport - Configured with stdin_open and tty for stdio communication - Shares environment variables with backend service - Update TEAM_SETUP.md with comprehensive MCP server setup instructions - Add Step 5.1: Start MCP Server with Docker and local options - Clarify separation between HTTP API backend and MCP server - Update configuration summary and verification checklist - Add MCP server to Quick Start Commands section - Improve code clarity in src/index.ts - Update console output to distinguish HTTP API Server from MCP Server - Clarify port configuration display - Add QUERY_TUTOR_EXAMPLES.md - Comprehensive guide with GraphQL query examples - Question-based prompts for RAG testing - Recommended testing order and expected results * feat: Dockerize MCP server and update documentation - Add separate Docker service for MCP server in docker-compose.dev.yml - MCP server runs as independent container using stdio transport - Configured with stdin_open and tty for stdio communication - Shares environment variables with backend service - Update TEAM_SETUP.md with comprehensive MCP server setup instructions - Add Step 5.1: Start MCP Server with Docker and local options - Clarify separation between HTTP API backend and MCP server - Update configuration summary and verification checklist - Add MCP server to Quick Start Commands section - Improve code clarity in src/index.ts - Update console output to distinguish HTTP API Server from MCP Server - Clarify port configuration display - Add QUERY_TUTOR_EXAMPLES.md - Comprehensive guide with GraphQL query examples - Question-based prompts for RAG testing - Recommended testing order and expected results * test: Add comprehensive Docker services testing and MCP protocol verification - Add TEST_DOCKER_SERVICES.md with complete testing guide - Step-by-step verification procedures for all Docker services - Network communication tests - End-to-end flow testing - Troubleshooting tips and common issues - Add test-mcp-protocol.ts for MCP server protocol testing - Tests MCP protocol communication via JSON-RPC over stdio - Verifies all 5 core MCP operations: * Initialize protocol * List tools (9 tools verified) * Check connection tool * Execute GraphQL query tool * Introspect schema tool - All tests passing (5/5) - Confirms MCP server is responding correctly to protocol calls * test: Add comprehensive Docker services testing and MCP protocol verification - Add TEST_DOCKER_SERVICES.md with complete testing guide - Step-by-step verification procedures for all Docker services - Network communication tests - End-to-end flow testing - Troubleshooting tips and common issues - Add test-mcp-protocol.ts for MCP server protocol testing - Tests MCP protocol communication via JSON-RPC over stdio - Verifies all 5 core MCP operations: * Initialize protocol * List tools (9 tools verified) * Check connection tool * Execute GraphQL query tool * Introspect schema tool - All tests passing (5/5) - Confirms MCP server is responding correctly to protocol calls * Reslens integration * my changes * Apply changes after removing secret * Clean up unnecessary files and add Docker setup for ResLens - Removed temporary fix scripts and redundant documentation - Added Docker support for ResLens Frontend and ResLens Middleware - Updated docker-compose.dev.yml with ResLens services - Fixed middleware connection URLs for browser compatibility - Added check-docker-services.sh utility script - Kept essential setup scripts and documentation * Update README.md * Update README.md * Update README.md * Fix backend container command and GraphQL URL configuration - Fix docker-compose.dev.yml: Change backend command from 'npm run dev -- --http-api' to 'npm run http-api' for correct HTTP API mode - Fix ingest-graphql-docs.ts: Remove incorrect RESILIENTDB_GRAPHQL_URL override pointing to HTTP wrapper (port 18001) - HTTP wrapper only provides REST KV endpoints, not GraphQL server - GraphQL operations must use port 5001 - Added clarifying comments about service separation - Update environment.ts: Fix default RESILIENTDB_GRAPHQL_URL from port 18000 to 5001 with clarifying comment - Enable ResLens Middleware and Frontend services in docker-compose.dev.yml * Merge ResLens Docker setup into TEAM_SETUP.md and update TEST_DOCKER_SERVICES.md - Merged DOCKER_RESLENS_SETUP.md content into TEAM_SETUP.md Step 5.2 - Added comprehensive ResLens Docker setup instructions including: - Docker architecture details (multi-stage builds, Nginx setup) - Complete environment variables documentation - Networking configuration - Service management commands (logs, stop, remove) - Development vs Production comparison - Enhanced troubleshooting sections - Updated TEST_DOCKER_SERVICES.md to include ResLens services testing - Added ResLens Middleware and Frontend to service architecture - Added Step 5: Test ResLens Services with 6 sub-tests - Added ResLens troubleshooting sections - Updated verification checklists and success criteria - Deleted standalone DOCKER_RESLENS_SETUP.md file - Updated all documentation references * Update README.md * Update README.md * Update README.md * Update TEAM_SETUP.md to reference forked Nexus repository with GraphQ-LLM integration * Fix remaining Nexus repository URLs to point to fork * Add Nexus fork setup guide and script for team reference * Add nexus/ to .gitignore and update scripts to use forked Nexus repository * Remove nexus directory from git tracking (now using separate fork) * Update README.md with Nexus fork information and setup instructions * Add Nexus Fork Information section to README * Simplify conclusion sentence in README * Update README.md * Add ResLens fork setup guide and script * Configure ResLens to use separate fork repositories - Update docker-compose.dev.yml to use absolute paths to forks - Remove ResLens directories from git tracking (165 files) - Update TEAM_SETUP.md with fork information - Add ResLens to .gitignore - Fork URLs: - Frontend: sophiequynn/incubator-resilientdb-ResLens - Middleware: sophiequynn/incubator-resilientdb-ResLens-Middleware * Add ResLens fork clone commands to Quick Start section * Update documentation for ResLens fork setup - Add ResLens fork information to docs/README.md - Update Quick Start to include ResLens fork cloning steps - Add ResLens fork section similar to Nexus fork documentation - Update TEAM_SETUP.md with detailed ResLens fork instructions - Clarify that forks include GraphQ-LLM-specific modifications * Clarify ResLens fork modifications in TEAM_SETUP.md * Add ResLens fork cloning commands to prerequisites in Step 5.2 * Revise performance metrics section and add URLs Updated section titles and added access URLs for various services. * chore: Update docker-compose.dev.yml with local ResLens paths - Update ResLens Middleware path to /Users/CelineJohnPhilip/DDS/ResLens-Middleware/middleware - Update ResLens Frontend path to /Users/CelineJohnPhilip/DDS/ResLens - Configured for local development environment * Removed optimization related content. Updated README to improve clarity and consistency in descriptions of features and sections. * update * update * update * Update README.md * Fix ResLens frontend connection and enhance explanation quality - Fix ResLens frontend middleware URL to use localhost instead of Docker service name (browser can't resolve service names) - Increase token limits for detailed explanations (2000 for Gemini, 3000 for other models) - Enhance system and detailed explanation prompts for more comprehensive responses - Remove ResLens frontend dependency on middleware health check to allow startup * Update docs/README.md with latest features and fixes - Fix Nexus port inconsistency (3002 -> 3000) - Document query stats button feature in efficiency tab - Add note about ResLens frontend localhost configuration - Enhance performance metrics section with ResLens integration details * Fix ResLens paths to match current user setup * Document chatbot UI route in README --------- Co-authored-by: sophiequynn <39390333+sophiequynn@users.noreply.github.com> Co-authored-by: celinejp <celinejp.03@gmail.com> Co-authored-by: Theodore Pan <77665596+theopan8@users.noreply.github.com> Co-authored-by: sophiequynn <sjquynn@ucdavis.edu> Co-authored-by: Sandhya <sghanathe@ucdavis.edu> Co-authored-by: Ahadke <hadkeaayusha@gmail.com>
1 parent 2203ed4 commit f910800

File tree

110 files changed

+33983
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

110 files changed

+33983
-0
lines changed
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Dependencies
2+
node_modules/
3+
npm-debug.log*
4+
yarn-debug.log*
5+
yarn-error.log*
6+
7+
# Build outputs
8+
dist/
9+
*.tsbuildinfo
10+
11+
# Environment files
12+
.env
13+
.env.local
14+
.env.*.local
15+
16+
# IDE
17+
.vscode/
18+
.idea/
19+
*.swp
20+
*.swo
21+
*~
22+
23+
# OS
24+
.DS_Store
25+
Thumbs.db
26+
27+
# Git
28+
.git/
29+
.gitignore
30+
31+
# Documentation (not needed in container)
32+
*.md
33+
# But allow docs/ for ingestion
34+
!docs/
35+
36+
# Test files
37+
**/*.test.ts
38+
**/*.spec.ts
39+
test-*.ts
40+
test-*.sh
41+
42+
# Scripts (not needed in production container)
43+
scripts/
44+
*.sh
45+
# But allow these specific scripts for database container
46+
!healthcheck.sh
47+
!healthcheck-all.sh
48+
!healthcheck-unified.sh
49+
!init-postgres.sh
50+
!setup-graphql.sh
51+
52+
# Docker files
53+
Dockerfile*
54+
docker-compose*.yml
55+
.dockerignore
56+
57+
# Logs
58+
logs/
59+
*.log
60+
61+
# Misc
62+
coverage/
63+
.nyc_output/
64+
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# GraphQ-LLM Environment Configuration
2+
# Copy this file to .env and fill in your values
3+
4+
# ============================================
5+
# ResilientDB Configuration (Required)
6+
# ============================================
7+
RESILIENTDB_GRAPHQL_URL=http://localhost:5000/graphql
8+
RESILIENTDB_API_KEY=
9+
10+
# ============================================
11+
# ResLens Configuration (Optional - for Live Mode)
12+
# ============================================
13+
# Enable Live Mode for real-time stats accessible by LLM
14+
RESLENS_API_URL=
15+
RESLENS_API_KEY=
16+
RESLENS_LIVE_MODE=false
17+
RESLENS_POLL_INTERVAL=5000
18+
19+
# ============================================
20+
# Nexus Configuration (Optional)
21+
# ============================================
22+
NEXUS_API_URL=
23+
NEXUS_API_KEY=
24+
25+
# ============================================
26+
# LLM Configuration (Required for AI features)
27+
# ============================================
28+
# LLM Provider: 'deepseek', 'openai', or 'anthropic'
29+
LLM_PROVIDER=deepseek
30+
LLM_API_KEY=
31+
LLM_MODEL=deepseek-chat
32+
# Enable Live Stats access for LLM (requires ResLens)
33+
LLM_ENABLE_LIVE_STATS=false
34+
35+
# ============================================
36+
# Embedding Configuration (Optional - Hugging Face)
37+
# ============================================
38+
# Hugging Face works without API key (public access, rate limited)
39+
# Get free token from https://huggingface.co/settings/tokens for higher limits
40+
HUGGINGFACE_API_KEY=
41+
42+
# ============================================
43+
# MCP Server Configuration
44+
# ============================================
45+
MCP_SERVER_PORT=3001
46+
MCP_SERVER_HOST=0.0.0.0
47+
48+
# ============================================
49+
# Logging & Environment
50+
# ============================================
51+
LOG_LEVEL=info
52+
NODE_ENV=development
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"parser": "@typescript-eslint/parser",
3+
"extends": [
4+
"eslint:recommended",
5+
"plugin:@typescript-eslint/recommended"
6+
],
7+
"parserOptions": {
8+
"ecmaVersion": 2022,
9+
"sourceType": "module"
10+
},
11+
"rules": {
12+
"@typescript-eslint/no-explicit-any": "warn",
13+
"@typescript-eslint/explicit-function-return-type": "off",
14+
"@typescript-eslint/no-unused-vars": ["warn", { "argsIgnorePattern": "^_" }]
15+
},
16+
"env": {
17+
"node": true,
18+
"es2022": true
19+
}
20+
}
21+
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Dependencies
2+
node_modules/
3+
package-lock.json
4+
5+
# Build outputs
6+
dist/
7+
*.tsbuildinfo
8+
9+
# Environment variables
10+
.env
11+
.env.local
12+
.env.*.local
13+
14+
# IDE
15+
.vscode/
16+
.idea/
17+
*.swp
18+
*.swo
19+
*~
20+
21+
# External repositories (cloned locally for Docker builds)
22+
nexus/
23+
ResLens/
24+
ResLens-Middleware/
25+
26+
# Logs
27+
logs/
28+
*.log
29+
npm-debug.log*
30+
31+
# OS
32+
.DS_Store
33+
Thumbs.db
34+
35+
# Testing
36+
coverage/
37+
.nyc_output/
38+
39+
# Temporary files
40+
tmp/
41+
temp/
42+
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Dockerfile for GraphQ-LLM Backend
2+
# Multi-stage build for optimized production image
3+
4+
# Stage 1: Build
5+
FROM node:20-alpine AS builder
6+
7+
WORKDIR /app
8+
9+
# Copy package files
10+
COPY package*.json ./
11+
COPY tsconfig.json ./
12+
13+
# Install dependencies
14+
RUN npm ci
15+
16+
# Copy source code
17+
COPY src/ ./src/
18+
19+
# Build TypeScript
20+
RUN npm run build
21+
22+
# Stage 2: Production
23+
FROM node:20-alpine
24+
25+
WORKDIR /app
26+
27+
# Copy package files
28+
COPY package*.json ./
29+
30+
# Install production dependencies only
31+
RUN npm ci --only=production
32+
33+
# Copy built files from builder
34+
COPY --from=builder /app/dist ./dist
35+
36+
# Create non-root user
37+
RUN addgroup -g 1001 -S nodejs && \
38+
adduser -S nodejs -u 1001
39+
40+
# Change ownership
41+
RUN chown -R nodejs:nodejs /app
42+
43+
USER nodejs
44+
45+
# Expose port (adjust if needed)
46+
EXPOSE 3001
47+
48+
# Health check
49+
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
50+
CMD node -e "require('http').get('http://localhost:3001/health', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})" || exit 1
51+
52+
# Start the application
53+
CMD ["npm", "start"]
54+
Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
# All-in-One Dockerfile: ResilientDB + PostgreSQL + GraphQ-LLM Backend
2+
# Multi-stage build: Uses ResilientDB image and adds PostgreSQL + Node.js
3+
4+
ARG ARCH=arm64
5+
6+
# Stage 1: Get ResilientDB binaries
7+
FROM expolab/resdb:${ARCH} AS resilientdb-stage
8+
# Just to have the image available
9+
10+
# Stage 2: Base with PostgreSQL and Node.js
11+
FROM ubuntu:22.04 AS base
12+
13+
ENV DEBIAN_FRONTEND=noninteractive
14+
15+
# Install base dependencies
16+
RUN apt-get update && apt-get install -y \
17+
curl \
18+
wget \
19+
git \
20+
build-essential \
21+
python3 \
22+
make \
23+
g++ \
24+
supervisor \
25+
ca-certificates \
26+
&& rm -rf /var/lib/apt/lists/*
27+
28+
# Install Node.js 20
29+
RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && \
30+
apt-get install -y nodejs && \
31+
rm -rf /var/lib/apt/lists/*
32+
33+
# Install PostgreSQL 14
34+
RUN apt-get update && apt-get install -y \
35+
postgresql-14 \
36+
postgresql-contrib-14 \
37+
postgresql-server-dev-14 \
38+
&& rm -rf /var/lib/apt/lists/*
39+
40+
# Install pgvector extension
41+
RUN git clone --branch v0.5.1 https://github.com/pgvector/pgvector.git /tmp/pgvector && \
42+
cd /tmp/pgvector && \
43+
make && \
44+
make install && \
45+
cd / && \
46+
rm -rf /tmp/pgvector && \
47+
apt-get purge -y git build-essential postgresql-server-dev-14 && \
48+
apt-get autoremove -y && \
49+
rm -rf /var/lib/apt/lists/*
50+
51+
# Copy ResilientDB binaries from the image (if we can access them)
52+
# For now, we'll use a volume mount or run ResilientDB in a sidecar
53+
# Actually, let's use docker-compose to run ResilientDB separately but in the same network
54+
# OR: Use the ResilientDB HTTP wrapper approach
55+
56+
# Create directories
57+
RUN mkdir -p /var/lib/postgresql/data \
58+
&& mkdir -p /app/graphq-llm \
59+
&& mkdir -p /data \
60+
&& chown -R postgres:postgres /var/lib/postgresql/data
61+
62+
# Copy GraphQ-LLM application files
63+
WORKDIR /app/graphq-llm
64+
COPY package*.json ./
65+
COPY tsconfig.json ./
66+
67+
# Install Node.js dependencies
68+
RUN npm ci
69+
70+
# Copy source code and build
71+
COPY src/ ./src/
72+
RUN npm run build
73+
74+
# Copy supervisor configuration
75+
COPY supervisord-all.conf /etc/supervisor/conf.d/supervisord.conf
76+
77+
# Copy scripts
78+
COPY healthcheck-all.sh /app/healthcheck-all.sh
79+
COPY init-postgres.sh /app/init-postgres.sh
80+
RUN chmod +x /app/healthcheck-all.sh /app/init-postgres.sh
81+
82+
# Copy docs
83+
COPY docs/ /app/graphq-llm/docs/
84+
85+
# Note: ResilientDB will run in a separate container but same network
86+
# We'll use docker-compose to link them, or use a sidecar pattern
87+
# For true all-in-one, we'd need to install ResilientDB from source (complex)
88+
89+
# Expose ports
90+
EXPOSE 5432 3001
91+
92+
# Health check (only PostgreSQL and GraphQ-LLM, ResilientDB is separate)
93+
HEALTHCHECK --interval=30s --timeout=10s --retries=5 --start-period=120s \
94+
CMD /bin/bash -c "pg_isready -U nexus && curl -f http://localhost:3001/health || exit 1"
95+
96+
# Start supervisor
97+
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Dockerfile for Nexus (Next.js Application)
2+
# This builds the Nexus application for production deployment
3+
4+
FROM node:20-alpine AS base
5+
6+
# Install dependencies only when needed
7+
FROM base AS deps
8+
RUN apk add --no-cache libc6-compat
9+
WORKDIR /app
10+
11+
# Copy package files
12+
COPY package.json package-lock.json* ./
13+
RUN npm ci
14+
15+
# Rebuild the source code only when needed
16+
FROM base AS builder
17+
WORKDIR /app
18+
COPY --from=deps /app/node_modules ./node_modules
19+
COPY . .
20+
21+
# Set environment variables for build
22+
ENV NEXT_TELEMETRY_DISABLED=1
23+
24+
# Build Next.js application
25+
RUN npm run build
26+
27+
# Production image
28+
FROM base AS runner
29+
WORKDIR /app
30+
31+
ENV NODE_ENV=production
32+
ENV NEXT_TELEMETRY_DISABLED=1
33+
34+
RUN addgroup --system --gid 1001 nodejs
35+
RUN adduser --system --uid 1001 nextjs
36+
37+
# Copy necessary files from builder
38+
COPY --from=builder /app/public ./public
39+
COPY --from=builder /app/.next ./.next
40+
COPY --from=builder /app/node_modules ./node_modules
41+
COPY --from=builder /app/package.json ./package.json
42+
43+
# Copy other necessary files
44+
COPY --from=builder /app/next.config.* ./
45+
COPY --from=builder /app/tsconfig.json ./
46+
47+
USER nextjs
48+
49+
EXPOSE 3000
50+
51+
ENV PORT=3000
52+
ENV HOSTNAME="0.0.0.0"
53+
54+
# Start Next.js in production mode
55+
CMD ["npm", "start"]

0 commit comments

Comments
 (0)