Voice Vibe Coding 🎙️→💻

Transform natural phone conversations into deployed web applications using Claude Code's multi-agent system with ephemeral storage and phone-based multi-tenancy.

🚀 Overview

Voice Vibe Coding enables users to call an ElevenLabs phone agent, describe what they want to build in natural language, and automatically receive a fully functional web application. The system uses ephemeral storage with GitHub as the source of truth, ensuring zero disk bloat and complete user isolation.

Key Features

• Zero Local Persistence: Projects exist only during builds, then pushed to GitHub
• Phone-Based Multi-Tenancy: Complete user isolation via phone number prefixes
• Multi-Project Support: Users can update existing projects naturally
• Self-Healing UI Testing: Browser automation with Playwright MCP
• Dynamic Pipeline Selection: Only runs necessary agents for efficiency
• Git-Based Version Control: Full history and rollback capabilities

🎬 Demo Video

See Voice Vibe Coding in action:

Checkout the fog game on Vercel (since improved!): https://vvc-ehsw4l-golden-gate-fog-navigator-4d7jsoxev.vercel.app/

🏗️ Architecture

Ephemeral Storage Workflow

Phone Call → Extract Phone → Create Temp Workspace → Build Project → 
Push to GitHub (phone-prefixed) → Delete Local → Update Registry (GitHub URL only)

System Components

Phone Call → ElevenLabs → Webhook → Orchestrator → Agent Pipeline → GitHub → Deployed App → SMS

Key Principles

1. Ephemeral Workspaces: /projects/.temp/{phone}/{timestamp}/ (auto-deleted)
2. Anonymized Repos: github.com/breagent/vvc-{user_id}-{project-name} (privacy-first)
3. GitHub as Truth: All code persisted remotely, no local storage
4. Stateless Processing: Local system is just a build server

⚡ Quick Start

Get Voice Vibe Coding running in 5 minutes.

Prerequisites

• Python 3.10+
• Node.js and npx (for Playwright MCP)
• Git configured with GitHub access
• Claude CLI installed and authenticated (claude login)

1. Automated Setup (2 minutes)

# Clone repository
git clone <repository-url>
cd "Claude Code Voice Vibe Coding"

# Run automated setup
./setup.sh

The setup script automatically:

• ✅ Checks Python/Node.js versions
• ✅ Creates virtual environment
• ✅ Installs all dependencies
• ✅ Installs Playwright MCP
• ✅ Creates .env template
• ✅ Sets up project directories

2. Configure Credentials (2 minutes)

Edit .env with your API keys:

# Required (system won't run without these)
GITHUB_TOKEN=ghp_xxxxxxxxxxxx        # Get from: https://github.com/settings/tokens
GITHUB_USERNAME=yourusername         # Your GitHub username

# Optional (features gracefully disabled if missing)
TWILIO_ACCOUNT_SID=ACxxxx            # For SMS notifications
TWILIO_AUTH_TOKEN=xxxx               # For SMS authentication
TWILIO_PHONE_NUMBER=+1555555xxxx    # Your Twilio number
VERCEL_TOKEN=xxxx                    # For automatic deployment
ELEVENLABS_API_KEY=xxxx              # For webhook security

Important: Do NOT set ANTHROPIC_API_KEY - Claude CLI uses OAuth.

3. Validate Setup (30 seconds)

source venv/bin/activate
python scripts/validate_credentials.py

Expected output:

✅ All required credentials are configured
   You're ready to run the orchestrator!

4. Start Everything (10 seconds)

# Single command to start all services
python run.py

This starts:

• 🌐 Webhook server (http://localhost:5001)
• 📞 Pre-call webhook endpoint
• 📞 Post-call webhook endpoint
• 🤖 Automatic orchestrator triggering

5. Configure ElevenLabs (See detailed section below)

Set your webhook URLs and you're ready to build with voice!

📞 ElevenLabs Setup

Configure your ElevenLabs phone agent to trigger Voice Vibe Coding builds.

Step 1: Create Phone Agent

1. Log into ElevenLabs Dashboard
2. Navigate to Conversational AI → Agents
3. Click Create New Agent
4. Configure:
   • Name: Voice Vibe Coding Agent
   • Voice: Choose your preferred voice
   • Language: English

Step 2: Configure Agent Prompt

In the Agent Prompt field, add:

You are Voice Vibe Coding, a system that helps users build web applications through natural phone conversations.

{{projects_context}}

If the user has existing projects listed above:
- Greet them warmly: "Hey! I see you have your [project name]..."
- Ask if they want to update it or build something new

If no projects exist:
- Welcome them: "Welcome to Voice Vibe Coding! Let's build something amazing."
- Ask what they want to create

Your role is to:
1. Understand what they want to build or change
2. Ask clarifying questions about features, style, and functionality
3. Provide up-to-date design ideas and suggestions by searching the web for current trends
4. Look up examples, color palettes, or technical details if helpful
5. Confirm you understand before ending the call

Keep the conversation natural and friendly. You're helping them bring ideas to life!

Key: The {{projects_context}} variable is automatically populated by the pre-call webhook - this happens BEFORE the conversation starts, so the agent already knows about the user's existing projects when they answer the call.

Step 2b: Enable Web Search (Important!)

In ElevenLabs Agent Settings:

1. Navigate to Tools section
2. Enable Web Search tool
3. This allows the agent to:
   • Look up current design trends in real-time
   • Find color palette inspiration
   • Search for examples and references
   • Provide up-to-date suggestions during the call

Step 3: Configure Webhooks

For Local Development (ngrok)

1. Start webhook server:

   python run.py

2. In another terminal, start ngrok:

   ngrok http 5001

3. Copy the HTTPS URL from ngrok output:

   Forwarding  https://abc123def.ngrok.io -> http://localhost:5001
   

4. In ElevenLabs Agent Settings, add webhooks:

   • Post-Call Webhook: https://abc123def.ngrok.io/elevenlabs-webhook
   • Pre-Call Webhook: https://abc123def.ngrok.io/precall
   • Method: POST
   • Click Save

Step 4: Get Your Phone Number

1. Navigate to Phone Numbers section
2. Click Get a Number
3. Choose your country and area code
4. Assign number to your agent

Step 5: Test Your Setup

1. Call your ElevenLabs number

2. Have a conversation about building something

3. Check your terminal - you should see:

   Pre-call webhook received from +1555555xxxx
   Post-call webhook received from +1555555xxxx
   Orchestrator triggered for phone: 1555555xxxx
   

4. Monitor logs: tail -f logs/pipeline_*.log

Troubleshooting

Problem: Dynamic variable shows as {{projects_context}} Solution: Pre-call webhook not configured or not responding. Check ngrok is running and webhook URL is correct.

Problem: Webhook not triggering orchestrator Solution: Verify ngrok URL is publicly accessible. Test with: curl https://your-ngrok-url.ngrok.io/health

Problem: No projects appear in context Solution: Expected for first-time callers. Context says "No projects yet" (this is normal).

📱 Twilio Setup (Optional)

Enable SMS notifications when projects are complete.

Why Twilio?

Get SMS alerts with deployment URLs and project status.

Setup Steps

1. Create Twilio account at twilio.com/console

2. Get your credentials from dashboard:

   • Account SID
   • Auth Token
   • Twilio phone number (SMS-capable)

3. Add to .env:

   TWILIO_ACCOUNT_SID=ACxxx
   TWILIO_AUTH_TOKEN=xxx
   TWILIO_PHONE_NUMBER=+1555555xxxx

4. Test SMS delivery:

   python twilio_messaging.py +15555551234 "Test message"

What happens if not configured: SMS notifications are skipped (system continues normally).

🚀 Vercel Deployment (Optional)

Enable automatic deployment of projects to Vercel.

Why Vercel?

Automatic hosting with every GitHub push - your projects go live instantly.

Setup Steps

1. Create Vercel account at vercel.com

2. Generate token:

   • Go to Settings → Tokens
   • Click Create Token
   • Copy the token

3. Add to .env:

   VERCEL_TOKEN=xxx

4. How it works:

   • System creates Vercel project linked to GitHub repo
   • Automatically disables deployment protection for public access
   • Every push to main branch triggers automatic deployment
   • Returns production URL: https://{project-name}.vercel.app

What happens if not configured: Projects stay in GitHub only (still accessible via git clone).

🎯 Usage

How It Works

1. Start the system: python run.py (starts webhook server)
2. Call your ElevenLabs number and describe what you want to build
3. Webhook triggers orchestrator automatically
4. Pipeline executes and builds your app
5. GitHub repo created with your code
6. SMS notification sent (if Twilio configured)

Command-Line Options

The orchestrator supports:

• --phone <number> - Process specific phone number (optional, reads from webhook by default)
• --verbose - Show detailed agent output (enabled by default)

# Run manually with specific phone number
python orchestrator.py --phone 1555555xxxx

# Run with verbose output (default)
python orchestrator.py --verbose

Note: The webhook server automatically triggers the orchestrator when calls come in, so you typically don't need to run it manually.

Multi-Project Support

Users can naturally update existing projects:

• First call: "Build me a banana delivery website"
• Second call: "Make the buttons blue on my banana site"
• Third call: "Add a contact form to the banana delivery page"

The system maintains context via VOICE_CONTEXT.md and GitHub repositories.

🤖 Implemented Agents

Core Pipeline Agents

1. voice-requirements-analyst - Extracts structured requirements from voice transcript
2. github-manager - Handles git operations and maintains voice context
3. content-strategist - Generates all text content and copy
4. ui-ux-designer - Creates comprehensive design specifications

Next.js/React Pipeline

5. project-scaffolder - Initializes Next.js project with TypeScript and Tailwind
6. component-architect - Plans React component structure
7. component-builder - Builds actual React components
8. page-assembler - Assembles pages and configures routing
9. styling-finisher - Applies final styling and animations
10. nextjs-validator - Validates build and fixes TypeScript errors
11. browser-functionality-validator - Browser UI/UX testing with self-healing

Output & Deployment Agents

12. static-html-generator - Creates viewable HTML with inline CSS
13. vercel-deployer - Deploys projects to Vercel with automatic GitHub integration
14. notification-agent - Sends SMS notifications via Twilio

📁 Project Structure

Ephemeral Architecture

voice-vibe-coding/
├── run.py                       # Single-command runner (starts all services)
├── orchestrator.py              # Main orchestrator using Claude Agent SDK
├── setup.sh                     # Automated setup script
├── .claude/
│   └── agents/                 # Agent prompt files (discovery location)
│       ├── voice-requirements-analyst.md
│       ├── workflow-orchestrator.md
│       ├── github-manager.md
│       └── ... (13 more agents)
├── webhooks/
│   └── latest_raw.json         # Latest ElevenLabs transcript
├── projects/
│   ├── .temp/                  # Ephemeral workspaces (auto-deleted)
│   │   └── {phone}/{timestamp}/ # Temporary build directory
│   ├── .project-registry.json  # GitHub-centric registry
│   └── {phone}/
│       └── VOICE_CONTEXT.md    # Conversational context for voice agent
├── webhook_server.py           # Webhook receiver
├── requirements.txt            # Python dependencies
├── .env.example               # Environment template
└── CLAUDE.md                  # System architecture docs

GitHub Repository Structure

All repos use anonymized user IDs for complete privacy and user isolation:

github.com/breagent/vvc-{user_id}-{project-name}
Example: github.com/breagent/vvc-a7x9k2-banana-delivery

Privacy Note: Phone numbers are mapped to random alphanumeric IDs (e.g., "a7x9k2") so repositories never expose personal information.

🔄 Pipeline Execution

New Project Pipeline

1. Requirements Analysis → requirements.json
2. GitHub Setup → Create phone-prefixed repo
3. Workflow Planning → Determine agents needed
4. Content Strategy → content.json
5. UI/UX Design → design_specs.json
6. Component Architecture → components_map.json
7. Project Scaffolding → Next.js setup
8. Component Building → React components
9. Page Assembly → Routes and pages
10. Styling Finisher → Final polish
11. Next.js Validation → Build verification
12. Browser Testing → UI/UX validation
13. GitHub Finalize → Commit and push

Update Pipelines (Dynamic)

• Content Only: content → builder → validator → commit
• Styling Only: styling → validator → commit
• New Feature: relevant agents → validator → commit
• Major Change: full pipeline → commit

💾 Recovery & Management

Clone Any Project

# Clone specific project (use your actual user ID from .project-registry.json)
git clone https://github.com/breagent/vvc-a7x9k2-banana-delivery

# Run locally
cd vvc-a7x9k2-banana-delivery/src
npm install
npm run dev
# Open http://localhost:3000

View Static HTML

open vvc-a7x9k2-banana-delivery/src/index.html

Clone All Your Projects

# Check .project-registry.json for your actual repository names
cat projects/.project-registry.json

# Clone all projects for your user ID
for repo in vvc-a7x9k2-banana-delivery vvc-a7x9k2-arc-phone vvc-a7x9k2-garden-glory; do
  git clone https://github.com/breagent/$repo
done

🧪 Testing

Prerequisites for Testing

1. Environment Variables - Ensure .env file contains:

   ANTHROPIC_API_KEY=xxx       # Required
   ANTHROPIC_MODEL=opus        # Recommended
   GITHUB_TOKEN=xxx            # Required
   GITHUB_USERNAME=xxx         # Required
   VERCEL_TOKEN=xxx            # Required for deployment
   TWILIO_ACCOUNT_SID=xxx      # For SMS (optional)
   TWILIO_AUTH_TOKEN=xxx       # For SMS (optional)
   TWILIO_PHONE_NUMBER=xxx     # For SMS (optional)

2. Dependencies Installed:

   pip install -r requirements.txt

3. Playwright MCP (for browser testing):

   npx -y @playwright/mcp@latest

Testing Options

Option 1: Direct Orchestrator Run

# Run with sample transcript data
python orchestrator.py

Option 2: Webhook Server (Automatic)

1. Start the webhook server:

   python webhook_server.py

2. The server automatically:

   • Receives webhooks at http://localhost:5001/elevenlabs-webhook
   • Saves transcript to webhooks/latest_raw.json
   • Triggers orchestrator.py automatically
   • Logs output to logs/pipeline_TIMESTAMP.log

3. Check status:

   curl http://localhost:5001/status

Expected Output

The pipeline will:

1. Extract requirements from voice transcript
2. Create/update GitHub repository
3. Execute necessary agents based on requirements
4. Build and validate the application
5. Commit and push to GitHub
6. Deploy to Vercel (if configured)
7. Send SMS notification (if configured)

Monitor Execution

Watch for:

1. Temp directory created in /projects/.temp/
2. Agents executing in sequence/parallel
3. GitHub repo created/updated with phone prefix
4. Temp directory deleted after successful push
5. Registry updated with GitHub URL

Success Criteria

✅ Pipeline completes without errors
✅ GitHub repo created/updated with new code
✅ Vercel deployment is publicly accessible (if deployed)
✅ Application functions as described in requirements
✅ SMS received with deployment URL (if configured)

Verify Results

# Check GitHub for repo
open https://github.com/breagent/vvc-{user_id}-{project-name}

# Check local registry
cat projects/.project-registry.json

# Verify minimal disk usage
du -sh projects/

# Check logs for details
ls -la logs/pipeline_*.log

🔍 Troubleshooting

Common Issues

1. Missing API Key: Ensure all required environment variables are set
2. GitHub Permission: Token needs repo scope for creation/push
3. Agent Not Found: Agents must be in .claude/agents/ directory
4. Build Failures: Check Node.js/npm installation for Next.js projects
5. Cleanup Failed: Manually delete /projects/.temp/ if needed

Debug Mode

# Enable verbose logging
python orchestrator.py --verbose

# Check orchestrator log
tail -f orchestrator.log

# Check pipeline logs
ls -la logs/pipeline_*.log
tail -f logs/pipeline_latest.log

# Check agent log
tail -f .claude/subagent.log

# View pipeline status
python orchestrator.py --status

🚀 Advanced Features

How Pre-Call Context Works (Multi-Project Magic)

The system uses pre-call webhooks to inject project context before the conversation even starts:

The Flow:

1. User dials the ElevenLabs phone number
2. Before answering, ElevenLabs sends a pre-call webhook to /precall with caller's phone number
3. System responds with the user's VOICE_CONTEXT.md content as {{projects_context}}
4. ElevenLabs injects this context into the agent's prompt
5. Agent answers already knowing about the user's projects: "Hey! I see you have your banana delivery site and your arc phone landing page..."
6. Natural conversation about updating existing projects or building something new

Why This Matters:

• No manual project selection - Agent automatically knows what you've built
• Conversational updates - "Make the buttons blue on my banana site" (agent knows which site)
• Context awareness - Agent remembers visual style, features, and recent changes
• Seamless UX - Feels like talking to someone who knows your work

This is what enables the multi-project magic where users can naturally say "update my hat store" without explaining which project they mean.

Self-Healing Browser Tests

The browser-functionality-validator agent:

• Tests all forms and interactions
• Validates responsive design
• Automatically fixes UI issues
• Re-tests to confirm fixes

Dynamic Pipeline Selection

The workflow-orchestrator intelligently chooses agents:

• Analyzes requirements complexity
• Skips unnecessary agents
• Optimizes for speed
• Documents decisions in execution_plan.json

📈 Monitoring

State Tracking

Each build maintains state in ephemeral workspace:

{
  "project_id": "vvc-a7x9k2-banana-delivery",
  "current_phase": "building",
  "completed_agents": ["requirements", "github-setup"],
  "pending_agents": ["content", "design"],
  "errors": [],
  "start_time": "2025-01-12T10:00:00Z"
}

Registry Structure

{
  "projects": {
    "1555555xxxx": [
      {
        "name": "Banana Delivery",
        "repo": "vvc-a7x9k2-banana-delivery",
        "github_url": "https://github.com/breagent/vvc-a7x9k2-banana-delivery",
        "created": "2025-01-11",
        "last_updated": "2025-01-12T10:00:00",
        "local_cache": null
      }
    ]
  }
}

Note: Phone number "1555555xxxx" is mapped internally to anonymized ID "a7x9k2".

🔐 Privacy & Security

Privacy-First Multi-Tenancy

Voice Vibe Coding implements zero-PII exposure through phone number anonymization:

How It Works

1. Phone Number Reception: ElevenLabs sends caller's phone number (e.g., "+1555555xxxx")

2. Anonymous ID Mapping:

   • First-time callers get a random 6-character alphanumeric ID (e.g., "a7x9k2")
   • Mapping stored locally in .phone-id-mapping.json (never committed to git)
   • Same phone always maps to same ID for consistency

3. Repository Naming:

   • Format: vvc-{user_id}-{project-name}
   • Example: vvc-a7x9k2-banana-delivery
   • Phone number never appears in GitHub URLs, repo names, or public artifacts

4. Complete Isolation:

   • Each user ID has separate repositories
   • No way to link user ID back to phone number without the mapping file
   • Mapping file excluded from version control (in .gitignore)

Privacy Guarantees

• ✅ GitHub repos - No phone numbers in URLs or metadata
• ✅ Public artifacts - All examples use anonymized IDs
• ✅ Version control - .phone-id-mapping.json in .gitignore
• ✅ Multi-user isolation - Each ID namespace is completely separate
• ✅ Vercel deployments - Use anonymized repo names automatically

Where Phone Numbers ARE Stored (Locally Only)

• projects/.phone-id-mapping.json - Phone → ID mapping (git-ignored)
• projects/{phone}/VOICE_CONTEXT.md - Local context files (git-ignored)
• projects/.project-registry.json - Registry by phone for local lookups (git-ignored)

Critical: All projects/ directory contents are git-ignored for privacy.

Security Best Practices

• Never commit .env file
• Use environment variables for all credentials
• GitHub repos are public by default (configure for private if needed)
• Implement rate limiting for webhook endpoints in production
• Rotate API tokens periodically

🛠️ Development

Adding New Agents

1. Create agent in .claude/agents/{agent-name}.md
2. Follow the template structure
3. Update orchestrator pipeline if needed
4. Test with sample transcript

Agent Template

# {Agent Name} Agent

You are a {role} agent in the Voice Vibe Coding system.

## Context
You are part of a pipeline that transforms voice conversations into deployed web applications.

## Your Specific Task
[Clear description]

## Inputs
- [List of files/artifacts]

## Expected Outputs
- [List of outputs]

[Instructions...]

📚 Additional Documentation

• CLAUDE.md - Detailed system architecture and agent specifications

---

Built with Claude Agent SDK - Transforming voice into code, one conversation at a time.