Merge branch 'development'

Author: atupem

README.md

@@ -46,7 +46,10 @@ _For an in-depth guide see [here](https://docs.bytebot.ai/deployment/railway)._
 
 - Docker ≥ 20.10
 - Docker Compose
-- Anthropic API key ([get one here](https://console.anthropic.com))
+- AI API key from one of these providers:
+  - Anthropic ([get one here](https://console.anthropic.com)) - Claude models
+  - OpenAI ([get one here](https://platform.openai.com/api-keys)) - GPT models
+  - Google ([get one here](https://makersuite.google.com/app/apikey)) - Gemini models
 
 ### Start Your Desktop Agent (2 minutes)
 
@@ -55,7 +58,11 @@ _For an in-depth guide see [here](https://docs.bytebot.ai/deployment/railway)._
 ```bash
 git clone https://github.com/bytebot-ai/bytebot.git
 cd bytebot
-echo "ANTHROPIC_API_KEY=your_api_key_here" > docker/.env
+
+# Configure your AI provider (choose one):
+echo "ANTHROPIC_API_KEY=your_api_key_here" > docker/.env    # For Claude
+# echo "OPENAI_API_KEY=your_api_key_here" > docker/.env     # For OpenAI
+# echo "GOOGLE_API_KEY=your_api_key_here" > docker/.env     # For Gemini
 ```
 
 2. **Start the agent stack:**
@@ -80,6 +87,16 @@ That's it! Start chatting with your AI desktop agent. Watch it work in real-time
 - "Download all PDFs from this website and organize them by date"
 - "Monitor this webpage and alert me when the price drops below $50"
 
+## 🤖 Supported AI Models
+
+Bytebot supports multiple AI providers to power your desktop agent:
+
+- **Anthropic Claude**: Claude 3.5 Sonnet (default) - Best for complex reasoning and visual tasks
+- **OpenAI**: GPT-4, GPT-4o - Excellent for general automation tasks
+- **Google Gemini**: Gemini 1.5 Pro, Flash - Fast and efficient for routine tasks
+
+Choose the model that best fits your needs and budget. Simply set the appropriate API key in your environment configuration.
+
 ## 🏗️ Architecture Overview
 
 Bytebot consists of four main components working together:
@@ -99,7 +116,7 @@ Bytebot consists of four main components working together:
                       │ WebSocket
 ┌─────────────────────▼───────────────────────────────────────┐
 │                 Bytebot Agent (NestJS)                       │
-│              • LLM integration                               │
+│              • Multi-LLM integration (Claude/GPT/Gemini)     │
 │              • Task orchestration                            │
 │              • Action planning                               │
 └─────────────────────┬───────────────────────────────────────┘
@@ -158,8 +175,10 @@ Bytebot consists of four main components working together:
 Create `docker/.env`:
 
 ```bash
-# Required
-ANTHROPIC_API_KEY=sk-ant-...
+# Required - Choose one of these AI providers:
+ANTHROPIC_API_KEY=sk-ant-...      # For Claude models
+# OPENAI_API_KEY=sk-...           # For OpenAI models
+# GOOGLE_API_KEY=...              # For Google Gemini models
 ```
 
 ### Desktop Customization
@@ -182,7 +201,7 @@ COPY configs/.config /home/user/.config
 
 ## 🔒 Security Considerations
 
-- **API Key**: Keep your Anthropic API key secure and never commit it
+- **API Keys**: Keep your AI provider API keys secure and never commit them
 - **Network**: By default, services are only accessible from localhost
 - **VNC**: Change the default VNC password for production use
 - **Updates**: Regularly update the container images for security patches
@@ -325,6 +344,8 @@ Built with amazing open source projects:
 
 - [nutjs](https://github.com/nut-tree/nut.js) - Desktop automation framework
 - [Anthropic Claude](https://www.anthropic.com) - AI reasoning engine
+- [OpenAI](https://openai.com) - GPT models for automation
+- [Google AI](https://ai.google.dev) - Gemini models for efficient tasks
 - [noVNC](https://novnc.com) - Browser-based VNC client
 - Inspired by Anthropic's [computer-use demo](https://github.com/anthropics/anthropic-quickstarts)
 

docker/docker-compose.yml

@@ -49,6 +49,8 @@ services:
       - DATABASE_URL=${DATABASE_URL:-postgresql://postgres:postgres@postgres:5432/bytebotdb}
       - BYTEBOT_DESKTOP_BASE_URL=${BYTEBOT_DESKTOP_BASE_URL:-http://bytebot-desktop:9990}
       - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
+      - OPENAI_API_KEY=${OPENAI_API_KEY}
+      - GOOGLE_API_KEY=${GOOGLE_API_KEY}
     depends_on:
       - postgres
     networks:

docs/core-concepts/agent-system.mdx

@@ -15,9 +15,26 @@ The Bytebot Agent System transforms a simple desktop container into an intellige
 
 ## How the AI Agent Works
 
-### The Brain: Claude AI Integration
+### The Brain: Multi-Model AI Integration
 
-At the heart of Bytebot is Claude, Anthropic's advanced AI assistant. The agent:
+At the heart of Bytebot is a flexible AI integration that supports multiple models. Choose the AI that best fits your needs:
+
+**Anthropic Claude** (Default):
+- Best for complex reasoning and visual understanding
+- Excellent at following detailed instructions
+- Superior performance on desktop automation tasks
+
+**OpenAI GPT Models**:
+- Fast and reliable for general automation
+- Strong code understanding and generation
+- Cost-effective for routine tasks
+
+**Google Gemini**:
+- Efficient for high-volume tasks
+- Good balance of speed and capability
+- Excellent multilingual support
+
+The agent with any model:
 
 1. **Understands Context**: Processes your natural language requests with full conversation history
 2. **Plans Actions**: Breaks down complex tasks into executable computer actions
@@ -31,7 +48,7 @@ At the heart of Bytebot is Claude, Anthropic's advanced AI assistant. The agent:
     "Research competitors for my SaaS product and create a comparison table"
   </Step>
   <Step title="AI Plans the Approach">
-    Claude understands the request and plans: open browser → search → visit sites → extract data → create document
+    The AI model understands the request and plans: open browser → search → visit sites → extract data → create document
   </Step>
   <Step title="Executes Actions">
     The agent controls the desktop: clicking, typing, taking screenshots, reading content
@@ -119,7 +136,7 @@ The agent processes tasks intelligently:
 ### Core Components
 
 1. **NestJS Agent Service**
-   - Integrates with Anthropic API
+   - Integrates with multiple AI provider APIs (Anthropic, OpenAI, Google)
    - Handles WebSocket connections
    - Coordinates with desktop API
 
@@ -194,7 +211,7 @@ The web interface provides:
 
 ### Data Isolation
 - All processing happens in your infrastructure
-- No data sent to external services (except Claude API)
+- No data sent to external services (except your chosen AI provider API)
 - Conversations stored locally
 - Complete audit trail
 
@@ -223,17 +240,18 @@ The web interface provides:
 
 <AccordionGroup>
   <Accordion title="Agent not responding">
-    - Check Anthropic API key is valid
+    - Check your AI provider API key is valid
     - Verify agent service is running
     - Review logs for errors
-    - Ensure sufficient API credits
+    - Ensure sufficient API credits/quota with your provider
   </Accordion>
 
   <Accordion title="Slow task execution">
     - Monitor system resources
     - Check network latency
     - Reduce screenshot frequency
-    - Optimize Claude prompts
+    - Optimize AI prompts for your chosen model
+    - Consider switching to a faster model (e.g., Gemini Flash)
   </Accordion>
 </AccordionGroup>
 

docs/core-concepts/architecture.mdx

@@ -24,7 +24,7 @@ The foundation of the system - a containerized Linux desktop that provides:
 - **XFCE4 Desktop** for a lightweight, responsive UI
 - **bytebotd Daemon** - The automation service built on nutjs that executes computer actions
 - **Pre-installed Applications**: Firefox ESR, Thunderbird, text editors, and development tools
-- **VNC & noVNC** for remote desktop access
+- **noVNC** for remote desktop access
 
 **Key Features:**
 - Runs completely isolated from your host system
@@ -139,7 +139,6 @@ graph LR
  - **Desktop API**: No authentication by default (localhost only). Supports REST and MCP.
 - **Agent API**: Can be secured with API keys
 - **Database**: Password protected, not exposed externally
-- **VNC Access**: Optional password protection
 
 <Warning>
   Default configuration is for development. For production:

docs/core-concepts/desktop-environment.mdx

@@ -39,27 +39,24 @@ The Bytebot Desktop Environment is a fully functional Linux desktop running insi
 - **Ubuntu 22.04 LTS**: Stable, well-supported Linux distribution
 - **XFCE4 Desktop**: Lightweight, responsive desktop environment
 - **X11 Display Server**: Standard Linux graphics system
-- **SystemD**: Modern service management
+- **supervisord**: Service management
 
 ### Pre-installed Software
 
 <CardGroup cols={2}>
   <Card title="Web Browsers" icon="globe">
     - Firefox ESR (Extended Support Release)
-    - Chrome/Chromium available
     - Pre-configured for automation
     - Ad blocker extensions
   </Card>
   <Card title="Office Tools" icon="file-lines">
     - Text editors (nano, vim, gedit)
-    - LibreOffice suite (optional)
-    - PDF viewers
+    - PDF viewer
     - File managers
   </Card>
   <Card title="Communication" icon="envelope">
     - Thunderbird email client
     - Chat applications
-    - Video conferencing tools
     - Terminal emulators
   </Card>
   <Card title="Development" icon="code">
@@ -78,17 +75,12 @@ The Bytebot Desktop Environment is a fully functional Linux desktop running insi
    - Built on nutjs framework
    - Provides REST API
 
-2. **VNC Server**
-   - TigerVNC for remote access
-   - Configurable resolution
-   - Multiple connection support
-
-3. **noVNC Web Client**
+2. **noVNC Web Client**
    - Browser-based desktop access
    - No client installation needed
    - WebSocket proxy included
 
-4. **Supervisor**
+3. **Supervisor**
    - Process management
    - Service monitoring
    - Automatic restarts
@@ -164,7 +156,7 @@ curl -X POST http://localhost:9990/api/computer \
 # Move mouse
 curl -X POST http://localhost:9990/api/computer \
   -H "Content-Type: application/json" \
-  -d '{"action": "move_mouse", "coordinate": [500, 300]}'
+  -d '{"action": "move_mouse", "coordinate": {"x": 500, "y": 300}}'
 ```
 
 ## Customization

docs/deployment/railway.mdx

@@ -3,7 +3,7 @@ title: "Deploying Bytebot on Railway"
 description: "Comprehensive guide to deploying the full Bytebot stack on Railway using the official 1-click template"
 ---
 
-> **TL;DR –** Click the button below, add your Anthropic key, and your personal Bytebot instance will be live in ~2 minutes.
+> **TL;DR –** Click the button below, add your AI API key (Anthropic, OpenAI, or Google), and your personal Bytebot instance will be live in ~2 minutes.
 
 [![Deploy on Railway](https://railway.com/button.svg)](https://railway.com/deploy/bytebot?referralCode=L9lKXQ)
 
@@ -35,7 +35,12 @@ All internal traffic flows through Railway’s [private networking](https://docs
     Click the **Deploy on Railway** button above or visit [https://railway.com/deploy/bytebot?referralCode=L9lKXQ](https://railway.com/deploy/bytebot?referralCode=L9lKXQ).
   </Step>
   <Step title="2. Configure Environment">
-    Railway automatically detects required variables. Paste your **Anthropic API key** into `ANTHROPIC_API_KEY` and keep other defaults.
+    Railway automatically detects required variables. Add your AI API key (choose one):
+    - **Anthropic**: Paste into `ANTHROPIC_API_KEY` for Claude models
+    - **OpenAI**: Paste into `OPENAI_API_KEY` for GPT models
+    - **Google**: Paste into `GOOGLE_API_KEY` for Gemini models
+    
+    Keep other defaults as is.
   </Step>
   <Step title="3. Kick off the Deployment">
     Press **Deploy**. Railway will pull the pre-built images, create the Postgres database and link all services on a private network.
@@ -73,7 +78,7 @@ All internal traffic flows through Railway’s [private networking](https://docs
 | Symptom | Likely Cause | Fix |
 | ------- | ------------ | ---- |
 | Web UI shows “connecting…” | Desktop not ready or private networking mis-config | Wait for `bytebot-desktop` container to finish starting, or restart service |
-| Agent errors `401 Anthropic` | Missing/invalid API key | Re-enter `ANTHROPIC_API_KEY` in Railway variables |
+| Agent errors `401` or `403` | Missing/invalid API key | Re-enter your AI provider's API key in Railway variables |
 | Slow desktop video | Free Railway plan throttling | Upgrade plan or reduce screen resolution in desktop settings |
 
 ---

docs/images/agent-architecture.png

@@ -95,7 +95,7 @@ Bytebot consists of four integrated components working together:
     Ubuntu 22.04 with XFCE4, pre-installed apps, and the automation daemon
   </Card>
   <Card title="AI Agent" icon="brain" href="/core-concepts/agent-system">
-    NestJS service that uses an LLM to plan and execute tasks
+    NestJS service that uses an LLM (Claude, GPT, or Gemini) to plan and execute tasks
   </Card>
   <Card
     title="Task Interface"

docs/images/core-container.png

@@ -34,7 +34,10 @@ description: "Get your AI desktop agent running in 2 minutes"
 - Docker ≥ 20.10
 - Docker Compose
 - 4GB+ RAM available
-- Anthropic API key ([get one here](https://console.anthropic.com))
+- AI API key from one of these providers:
+  - Anthropic ([get one here](https://console.anthropic.com)) - Claude models
+  - OpenAI ([get one here](https://platform.openai.com/api-keys)) - GPT models
+  - Google ([get one here](https://makersuite.google.com/app/apikey)) - Gemini models
 
 ## 🚀 2-Minute Setup
 
@@ -43,7 +46,13 @@ Get your self-hosted AI desktop agent running with just three commands:
 <Steps>
   <Step title="Clone and Configure">
     ```bash 
-    git clone https://github.com/bytebot-ai/bytebot.git cd bytebot echo "ANTHROPIC_API_KEY=your_api_key_here" > docker/.env 
+    git clone https://github.com/bytebot-ai/bytebot.git
+    cd bytebot
+    
+    # Configure your AI provider (choose one):
+    echo "ANTHROPIC_API_KEY=your_api_key_here" > docker/.env    # For Claude
+    # echo "OPENAI_API_KEY=your_api_key_here" > docker/.env     # For OpenAI
+    # echo "GOOGLE_API_KEY=your_api_key_here" > docker/.env     # For Gemini
     ```
   </Step>
 
@@ -224,6 +233,7 @@ curl -X POST http://localhost:9990/api/computer \
     cat docker/.env
     docker-compose -f docker/docker-compose.yml logs bytebot-agent 
     ```
+    Ensure you're using a valid API key from Anthropic, OpenAI, or Google.
   </Accordion>
 </AccordionGroup>