AI Provider Implementation Summary

Overview

VisionForge now supports two AI providers for the chatbot functionality:

Gemini (Google's Generative AI) - Original provider
Claude (Anthropic's Claude AI) - New addition

Users can switch between providers using a simple environment variable configuration.

Files Created

1. Claude Service (`project/block_manager/services/claude_service.py`)

Purpose: Implements Claude AI integration mirroring Gemini's functionality
Key Features:
- Chat with conversation history
- Workflow modification suggestions
- File upload support (images, PDFs, text)
- Architecture generation from files
- Improvement suggestions

Model Used: claude-3-5-sonnet-20241022 (Latest Claude Sonnet)

Key Differences from Gemini:

Uses Anthropic SDK (anthropic package)
File handling: Base64 encoding for images/PDFs instead of File API
Message format: Direct user/assistant roles (no conversion needed)
Response format: response.content[0].text instead of response.text

2. AI Service Factory (`project/block_manager/services/ai_service_factory.py`)

Purpose: Provider selection and instantiation
Methods:
- create_service(): Returns appropriate service based on AI_PROVIDER env var
- get_provider_name(): Returns human-readable provider name

Error Handling:

Validates AI_PROVIDER value (must be 'gemini' or 'claude')
Propagates API key errors from individual services

Files Modified

1. Requirements (`project/requirements.txt`)

Added:

anthropic>=0.39.0

2. Environment Configuration

`.env`

# AI Provider Configuration
AI_PROVIDER=gemini  # or 'claude'

# Gemini AI Configuration
GEMINI_API_KEY=your-key-here

# Claude AI Configuration
ANTHROPIC_API_KEY=your-key-here

`.env.example`

Same structure with placeholder values.

3. Chat Views (`project/block_manager/views/chat_views.py`)

Changes:

Replaced direct GeminiChatService import with AIServiceFactory
Updated chat_message() endpoint:
- Uses factory to create service
- Handles file uploads differently per provider:
  - Gemini: Uploads to Gemini File API → passes gemini_file param
  - Claude: Reads file content locally → passes file_content param
- Provider-agnostic error messages
Updated get_suggestions() endpoint:
- Uses factory instead of direct service instantiation
- Generic error messages

4. Documentation (`docs/CHATBOT_SETUP.md`)

Additions:

Provider selection guide
Separate setup instructions for Gemini and Claude
API key acquisition for both providers
Provider comparison section
Troubleshooting for provider switching
Updated security and privacy sections

Configuration Guide

Using Gemini (Default)

Get API Key:
- Visit: https://aistudio.google.com/app/apikey
- Create free API key

Configure .env:

AI_PROVIDER=gemini
GEMINI_API_KEY=AIzaSy...

Restart server:
```
python manage.py runserver
```

Using Claude

Get API Key:
- Visit: https://console.anthropic.com/
- Create account and generate API key

Configure .env:

AI_PROVIDER=claude
ANTHROPIC_API_KEY=sk-ant-...

Install package:
```
pip install anthropic
```
Restart server:
```
python manage.py runserver
```

Switching Providers

Simply update AI_PROVIDER in .env and restart the server. No code changes needed!

Technical Implementation Details

Architecture Pattern: Factory + Strategy

User Request
    ↓
chat_views.py
    ↓
AIServiceFactory.create_service()
    ↓
    ├─→ GeminiChatService (if AI_PROVIDER=gemini)
    └─→ ClaudeChatService (if AI_PROVIDER=claude)
    ↓
AI Provider API
    ↓
Response → Frontend

Common Interface

Both services implement the same interface:

class ChatServiceInterface:
    def chat(message, history, modification_mode, workflow_state, **kwargs)
    def generate_suggestions(workflow_state)
    def _format_workflow_context(workflow_state)
    def _build_system_prompt(modification_mode, workflow_state)
    def _extract_modifications(response_text)

File Upload Handling

Gemini Approach:

Save uploaded file to temp location
Upload to Gemini File API using genai.upload_file()
Pass file object to model
Clean up temp file

Claude Approach:

Read file content directly from Django's UploadedFile
Encode images/PDFs as base64
Include in message content array
No temp file needed

Response Parsing

Both services use identical regex pattern to extract JSON modifications:

json_pattern = r'```json\s*(\{.*?\})\s*```'

This ensures consistent modification format regardless of provider.

API Compatibility

Request Format (Same for Both)

{
  "message": "Add a Conv2D layer",
  "history": [{"role": "user", "content": "..."}],
  "modificationMode": true,
  "workflowState": {"nodes": [...], "edges": [...]}
}

Response Format (Same for Both)

{
  "response": "AI response text...",
  "modifications": [
    {
      "action": "add_node",
      "details": {...},
      "explanation": "..."
    }
  ]
}

Frontend compatibility: No changes needed! The response format is identical.

Error Handling

Configuration Errors

Invalid Provider:

ValueError: Invalid AI_PROVIDER: 'gpt4'. Must be 'gemini' or 'claude'.

Missing API Key (Gemini):

ValueError: GEMINI_API_KEY environment variable is not set

Missing API Key (Claude):

ValueError: ANTHROPIC_API_KEY environment variable is not set

Runtime Errors

Both services handle:

API communication failures
Rate limiting
Invalid file uploads
Malformed responses

Errors are logged and returned as user-friendly messages.

Testing Checklist

Provider Comparison

Feature	Gemini	Claude
Model	gemini-2.0-flash	claude-3-5-sonnet-20241022
Speed	Very Fast	Fast
Free Tier	✅ Yes	❌ No
Image Support	✅ Yes	✅ Yes
PDF Support	✅ Yes	✅ Yes
Max Tokens	8192	4096 (configurable)
Reasoning	Good	Excellent
Code Understanding	Good	Excellent
Rate Limit (Free)	15 RPM	N/A

Future Enhancements

Potential Additions:

OpenAI GPT-4 support
Provider-specific features:
- Gemini: Grounding with Google Search
- Claude: Extended context (200k tokens)
Provider fallback: If one fails, try another
Cost tracking: Monitor API usage per provider
A/B testing: Compare response quality
Provider-specific prompts: Optimize for each model's strengths

Extension Pattern:

# Add new provider:
# 1. Create service class: NewProviderChatService
# 2. Update AIServiceFactory.create_service()
# 3. Add env vars: NEW_PROVIDER_API_KEY
# 4. Update documentation

Security Considerations

API Keys:
- Stored in .env (git-ignored)
- Never exposed to frontend
- Validated at service initialization
Data Privacy:
- Workflow data sent to external APIs
- User should review provider privacy policies
- No sensitive data should be in workflows
Rate Limiting:
- Implement request throttling in production
- Monitor costs (especially for Claude)
- Consider caching common responses

Troubleshooting

Issue: "AI service not properly configured"

Cause: Missing or invalid AI_PROVIDER or API key Solution:

Check .env file has AI_PROVIDER=gemini or AI_PROVIDER=claude
Verify corresponding API key is set
Restart Django server

Issue: Provider not switching

Cause: Server not restarted after .env change Solution: Always restart Django after changing environment variables

Issue: File uploads failing with Claude

Cause: Unsupported file type or size Solution:

Check file is image (PNG, JPG, WEBP, GIF) or PDF
Ensure file is under 10MB
Review error logs for details

Summary

This implementation provides:

✅ Flexibility: Easy provider switching via config
✅ Consistency: Same API interface for both providers
✅ Maintainability: Factory pattern for easy extension
✅ Reliability: Comprehensive error handling
✅ Documentation: Complete setup and usage guides

No frontend changes required - the implementation is completely transparent to the client.

Users can now choose the AI provider that best fits their needs, budget, and preferences!

FilesExpand file tree

AI_PROVIDER_IMPLEMENTATION.md

Latest commit

History