Skip to content

Latest commit

 

History

History
112 lines (83 loc) · 3.28 KB

README.md

File metadata and controls

112 lines (83 loc) · 3.28 KB

Emotion-Reactive Web Interface

A web application that captures webcam input, analyzes facial expressions using Google's Gemini API, and dynamically adjusts the interface colors based on detected emotions.

Features

  • Real-time emotion detection using Google Gemini 1.5 flash model
  • Dynamic UI styling based on detected emotions with color themes
  • Emotion-specific content recommendations and suggestions
  • In-browser webcam integration with manual and auto-capture options
  • Smooth transitions between emotional states
  • Debug console for troubleshooting

Supported Emotions

The application detects and responds to the following emotions:

  • Happy
  • Sad
  • Angry
  • Surprised
  • Neutral
  • Fearful
  • Disgusted

Setup and Installation

Prerequisites

  • Python 3.8 or higher
  • Google API key for Gemini Pro Vision

Installation

  1. Clone this repository:

    git clone <repository-url>
cd emotion-reactive-app

  2. Create a virtual environment and activate it:

    python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

  3. Install the required packages:

    pip install -r requirements.txt

  4. Set up your Google API key:

    • Get a Google API key for Gemini from the Google AI Studio
    • Add your API key to the .env file: 
      GOOGLE_API_KEY=your_api_key_here

Usage

  1. Run the Flask application:

    python app.py

  2. Open your browser and navigate to http://localhost:5000

  3. Click the "Start Camera" button to begin capturing webcam input

  4. Click "Analyze Emotion" or toggle "Auto-Capture" to detect emotions

  5. Express different emotions to see the interface change in real-time

How It Works

  1. The application uses your browser's webcam API to capture video
  2. When you click "Analyze Emotion" or enable auto-capture, it:
    • Captures a frame from the webcam
    • Sends the image to the server as a base64-encoded JPEG
    • Server processes the image with Google Gemini API to detect emotions
    • Results are sent back to update the UI colors and content

Project Structure

emotion_reactive_app/
├── app.py                 # Main Flask application
├── gemini_emotion.py      # Gemini API integration
├── utils.py               # Helper functions
├── .env                   # API keys (gitignored)
├── README.md              # Documentation
├── templates/             # HTML templates
│   └── index.html         # Main application page
└── static/                # Static assets folder

Key Features

  • Browser-based webcam capture: Uses the HTML5 Webcam API for reliable camera access
  • Manual and automatic capture modes: Analyze on-demand or set to auto-capture every few seconds
  • Real-time UI updates: Changes colors, content, and recommendations based on detected emotions
  • Built-in debugging: Visual debug console shows what's happening at each step
  • Smooth visual transitions: Gradual color changes between emotional states

Limitations

  • Emotion detection requires good lighting conditions
  • The application requires webcam access and an internet connection
  • The Gemini API has rate limits and usage costs

License

MIT License