Skip to content

Latest commit

 

History

History
476 lines (335 loc) · 13.7 KB

PROJECT_DOCS.md

File metadata and controls

476 lines (335 loc) · 13.7 KB

ESP32-CAM MCP Server - Project Documentation

Project Structure

ESP32-CAM-AI/
├── src/
│   └── main.cpp                 # Main MCP server implementation
├── include/
│   └── camera_config.h          # Camera hardware configurations  
├── lib/
│   └── mcp/                     # MCP protocol library
│       ├── mcp.h                # MCP class definitions
│       └── mcp.cpp              # MCP implementation
├── .vscode/
│   └── mcp.json                 # MCP client configuration
├── platformio.ini               # Build configuration
├── README.md                    # Comprehensive documentation
├── QUICK_REFERENCE.md          # Quick reference guide
└── PROJECT_DOCS.md             # This file

Project Overview

The ESP32-CAM MCP Server is a comprehensive IoT solution that transforms an ESP32-CAM module into a remotely controllable camera system using the Model Context Protocol (MCP). This project demonstrates advanced embedded systems programming, network protocols, and IoT integration.

Important: All images are automatically optimized to stay below 4KB base64 encoding to comply with AI client data limitations.

Key Technologies

  • Hardware: ESP32-CAM with OV2640 camera sensor
  • Framework: Arduino/ESP-IDF via PlatformIO
  • Protocol: Model Context Protocol (MCP) 2024-11-05
  • Networking: WiFi with HTTP server
  • Data Format: JSON-RPC 2.0 with base64 image encoding (under 4KB)

Architecture Deep Dive

1. Hardware Layer

// Camera initialization with AI-Thinker settings
camera_config_t config = esp32cam_aithinker_settings;
esp_err_t err = esp_camera_init(&config);

2. Network Layer

// WiFi management with auto-reconnection
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
server.begin();

3. Protocol Layer

// MCP request handling
String method = doc["method"];
if (method == "initialize") {
    handle_initialize(doc, response);
}

4. Application Layer

// Tool implementations
void handle_led_tool(const JsonObject& arguments, JsonDocument& response);
void handle_capture_tool(const JsonObject& arguments, JsonDocument& response);

Implementation Details

MCP Protocol Implementation

The server implements a complete MCP 2024-11-05 protocol stack:

Core Methods

  1. initialize: Establishes MCP connection and capabilities
  2. tools/list: Returns available tool definitions with JSON schemas
  3. tools/call: Executes specific tools with parameter validation
  4. notifications/initialized: Confirms initialization completion

Tool Architecture

Each tool follows a consistent pattern:

void handle_[tool]_tool(const JsonObject& arguments, JsonDocument& response) {
    // Parameter validation
    // Hardware interaction  
    // Response formatting
    // Error handling
}

Camera Management

Initialization Sequence

  1. Configure camera pins and settings
  2. Initialize camera driver
  3. Verify camera functionality
  4. Set up frame buffer management

Image Capture Process

  1. Prepare camera for capture
  2. Optional flash activation
  3. Capture frame buffer
  4. Convert to JPEG format
  5. Encode as base64
  6. Clean up resources

Camera Orientation Best Practices

Critical Setup: Position the ESP32-CAM with the flash LED facing downward for optimal results:

  • Proper Lighting: Flash LED illuminates subjects from above, providing natural lighting direction
  • Reflection Prevention: Downward orientation prevents flash reflection off surfaces
  • Image Quality: Ensures consistent, well-lit photographs without harsh shadows
  • Use Case Alignment: Matches expected perspective for security cameras, monitoring, and photography applications

The camera lens should face the target while the small flash LED (adjacent to the lens) points toward the ground or subject surface.

Network Reliability

WiFi Management

  • Event-driven connection handling
  • Automatic reconnection with exponential backoff
  • Connection status monitoring
  • Graceful degradation on network issues

HTTP Server

  • Single-threaded request processing
  • JSON request/response handling
  • Proper HTTP status codes
  • Error response formatting
  • CORS Support: Cross-Origin Resource Sharing headers for browser compatibility
    • Access-Control-Allow-Origin: *
    • Access-Control-Allow-Methods: POST, OPTIONS
    • Access-Control-Allow-Headers: Content-Type, Authorization
    • Automatic OPTIONS preflight request handling

Development Workflow

Build Process

  1. Environment Setup: PlatformIO with ESP32 platform
  2. Dependency Management: ArduinoJson, Base64, ESP32 Camera libraries
  3. Configuration: WiFi credentials (via .env file) and GPIO pin assignments
  4. Compilation: C++ compilation with Arduino framework
  5. Upload: Serial upload to ESP32-CAM module

Testing Strategy

  1. Unit Testing: Individual tool function validation
  2. Integration Testing: Complete MCP protocol flow
  3. Hardware Testing: Camera and GPIO functionality
  4. Network Testing: WiFi connectivity and HTTP responses
  5. Performance Testing: Memory usage and response times

Debugging Approach

Serial debugging throughout the application using the log_ macros

System Status Tool Documentation

Complete Parameter Reference

The system_status tool provides comprehensive diagnostic data for monitoring ESP32-CAM health and performance:

Hardware Metrics

  • CPU Frequency: Current processor speed (240 MHz typical)
  • Flash Size: Total flash memory (4,194,304 bytes = 4MB typical)
  • Flash Speed: Flash memory interface speed (40,000,000 Hz = 40MHz typical)
  • Internal Temperature: ESP32 die temperature in Celsius (calculated from internal sensor)

Memory Management

  • Free Heap: Currently available dynamic memory in bytes
  • Min Free Heap: Lowest free heap recorded since boot (memory pressure indicator)
  • Max Alloc Heap: Largest contiguous memory block available for allocation
  • Sketch Size: Size of compiled firmware in flash memory
  • Free Sketch Space: Remaining flash space available for OTA updates

System Information

  • Uptime: Milliseconds since boot converted to seconds
  • SDK Version: ESP-IDF framework version string
  • Reset Reason: Numeric code indicating last restart cause:
    • 1 = POWERON_RESET (normal power-on)
    • 2 = EXT_RESET (external reset pin)
    • 3 = SW_RESET (software-initiated restart)
    • 12 = BROWNOUT_RESET (power supply voltage drop)
    • 14 = RTCWDT_RESET (RTC watchdog timeout)

Camera Status

  • Camera initialized: Boolean status with error details
    • Success: "Yes" (camera driver loaded and functional)
    • Failure: "No (code = 0x[hex])" with specific ESP camera error code

Temperature Analysis

auto internal_temperature = (temprature_sens_read() - 32) / 1.8;
  • Normal Operation: 40-60°C during typical WiFi and camera operations
  • Heavy Load: 60-75°C during intensive processing (continuous capture, high WiFi traffic)
  • Warning Zone: 75-85°C (check ventilation, power supply quality)
  • Critical: >85°C (thermal protection may activate, performance degradation)

Memory Health Assessment

  • Healthy State: Free Heap > 100KB (sufficient for all operations)
  • Moderate Usage: 50-100KB free (normal operation, monitor trends)
  • High Pressure: 20-50KB free (limit concurrent operations)
  • Critical State: <20KB free (risk of crashes, restart recommended)

Diagnostic Use Cases

Performance Monitoring

  • Track memory leaks via Min Free Heap trends over time
  • Monitor temperature during extended operation
  • Verify stable camera initialization after power cycles

Troubleshooting

  • Check reset reason to identify power or software issues
  • Analyze memory pressure during camera capture operations
  • Verify system stability through uptime tracking

Capacity Planning

  • Determine optimal capture frequency based on memory usage
  • Plan OTA update timing based on available sketch space
  • Monitor thermal performance for enclosure design

Performance Analysis

Memory Usage Breakdown

  • Program Flash: ~1.2MB (code + libraries)
  • Dynamic RAM: ~100KB (variables + buffers)
  • Camera Buffers: ~50KB per frame buffer
  • JSON Processing: ~10KB for request/response
  • Network Stack: ~30KB for WiFi/HTTP

CPU Utilization

  • Idle State: <5% CPU usage
  • Image Capture: ~30% CPU usage (2-3 seconds)
  • Network Processing: ~10% CPU usage
  • Tool Execution: <1% CPU usage

Network Performance

  • Request Processing: <100ms typical
  • Image Transfer: ~2-5 seconds (depends on size/quality, optimized for 4KB)
  • Reconnection Time: ~10-15 seconds
  • Packet Loss: Handled by TCP retransmission

Security Analysis

Current Security Model

  • Authentication: None (open HTTP server)
  • Authorization: None (all tools accessible)
  • Encryption: None (plain HTTP)
  • Input Validation: JSON schema validation only

Security Recommendations

  1. Add API Key Authentication
bool validate_api_key(const String& key) {
    return key == STORED_API_KEY;
}
  1. Implement HTTPS
WiFiClientSecure client;
client.setCACert(ca_cert);
  1. Rate Limiting
unsigned long last_request = 0;
if (millis() - last_request < MIN_REQUEST_INTERVAL) {
    return error_response("Rate limit exceeded");
}

Performance Optimizations

  1. Compressed Responses: Reduce network overhead
  2. Power Management: Sleep modes for battery operation

Testing Documentation

Hardware Testing Checklist

  • Camera initialization successful
  • LED control functional
  • Flash timing accurate
  • WiFi connection stable
  • Memory usage within limits
  • Temperature monitoring

Protocol Testing Checklist

  • MCP initialization handshake
  • Tool listing complete and accurate
  • Parameter validation working
  • Error handling comprehensive
  • Response format compliance
  • Notification support

Integration Testing Scenarios

  1. Cold Start: Device boot to ready state
  2. Network Recovery: WiFi reconnection after outage
  3. Concurrent Requests: Multiple simultaneous tool calls
  4. Error Conditions: Invalid requests and hardware failures
  5. Long Running: Extended operation stability

Monitoring and Diagnostics

Key Metrics

  • Uptime: System operational time
  • Memory Usage: Heap utilization trends
  • Network Statistics: Requests, errors, response times
  • Camera Performance: Capture success rate, quality metrics, 4KB compliance

Diagnostic Tools

  1. Serial Monitor: Real-time logging output
  2. System Status Tool: Comprehensive health check
  3. WiFi Status Tool: Network connectivity details
  4. Memory Analysis: Heap and stack usage

Alerting Conditions

  • Memory usage >90%
  • WiFi disconnection >5 minutes
  • Camera initialization failure
  • System restart/watchdog timeout

Contribution Guidelines

Code Standards

  • Follow Arduino coding style
  • Use descriptive variable names
  • Comment complex algorithms
  • Implement proper error handling
  • Include debug output for troubleshooting

Testing Requirements

  • Test on actual hardware
  • Verify all MCP tools function
  • Check memory usage limits
  • Validate network reliability
  • Document performance characteristics

Documentation Updates

  • Update README for new features
  • Add tool descriptions to reference
  • Include configuration examples
  • Provide troubleshooting steps

Release Notes

Version 1.0.0 (Current)

  • Complete MCP protocol implementation
  • Five core tools (LED, flash, capture, WiFi status, system status)
  • Robust WiFi management with auto-reconnection
  • Comprehensive error handling and reporting
  • Base64 image encoding and transfer (optimized for 4KB limit)
  • Watchdog timer and system stability features

Planned Version 1.1.0

  • Motion detection integration
  • Scheduled capture functionality
  • Enhanced security with API key authentication
  • Performance optimizations
  • Extended configuration options

Troubleshooting Guide

Common Issues and Solutions

"Camera init failed"

Cause: Hardware connection or power issues

Solution:

  • Check camera ribbon cable connection
  • Verify 5V power supply (not 3.3V)
  • Try different camera module

"WiFi connection timeout"

Cause: Network credentials or signal issues

Solution:

  • Verify SSID and password
  • Check 2.4GHz network availability
  • Move closer to router

"Out of memory"

Cause: Memory leak or insufficient heap

Solution:

  • Reduce image quality/size
  • Check for proper buffer cleanup
  • Monitor heap usage over time

"Tool call timeout"

Cause: Network latency or processing delay

Solution:

  • Increase client timeout values
  • Check network connectivity
  • Monitor system performance

Configuration Management

WiFi Credentials via .env File

The project uses a .env file for secure WiFi credential management:

# .env file format
WIFI_SSID="YourWiFiNetwork"
WIFI_PASSWORD="YourPassword"

Build System Integration:

  • env-extra.py script automatically reads .env file during build
  • Credentials are passed as compile-time definitions
  • Build fails with clear error messages if .env is missing

Security Considerations:

  • .env file should be excluded from version control
  • Credentials are compiled into firmware (not runtime configurable)
  • No plain-text credential storage in source code

This project demonstrates the power of combining embedded systems, network protocols, and modern IoT architectures to create sophisticated, remotely controllable hardware solutions.