AI Video Dub Generator

An innovative AI-driven platform that seamlessly dubs videos across languages. Powered by Deepgram for speech recognition, OpenAI for translation, and ElevenLabs for lifelike voiceovers, this application helps content creators, educators, and businesses reach global audiences effortlessly.

Badges

Demo

Architecture Diagram

✨ Features

• Automatic Video Dubbing: Convert YouTube videos from one language to another with AI-powered translation and voice synthesis
• Real-time Job Monitoring: Track video dubbing progress with live status updates
• Multiple Language Support: Support for 30+ target languages
• High-Quality Audio: Advanced TTS engines for natural-sounding voice synthesis
• Modern Web Interface: Clean, responsive UI built with React and TypeScript
• RESTful API: FastAPI-based backend with automatic documentation

🚀 Quick Start

Prerequisites

• Node.js v18+ (for frontend)
• Python 3.11+ with UV (for backend)
• API Keys: Deepgram, OpenAI, and ElevenLabs

Getting Started

1. Clone the repository

   git clone https://github.com/ysskrishna/ai-video-dub-generator.git
   cd ai-video-dub-generator

2. Set up the Backend

   • Navigate to backend/ directory
   • Follow the detailed setup instructions in backend/README.md
   • The API server runs on http://localhost:8085

3. Set up the Frontend

   • Navigate to client/ directory
   • Follow the detailed setup instructions in client/README.md
   • The frontend runs on http://localhost:8080

📚 Documentation

• Backend README - Backend setup, API documentation, and architecture details
• Frontend README - Frontend setup, project structure, and development guide

Technology Stack

• Frontend: React, TypeScript, Vite, Tailwind CSS, shadcn/ui
• Backend: FastAPI (Python), SQLite, SQLAlchemy
• AI Services:
  • Deepgram - Speech-to-text transcription
  • OpenAI - Language translation
  • ElevenLabs - Text-to-speech synthesis
• Video Processing: MoviePy, yt-dlp

📺 Example Output

Check out example dubbed videos in the media/examples/ directory:

Example 1: English to Hindi

• Source Video: YouTube Video
• Dubbed Output: youtubeid_JcfOdtBk55Y_en_to_hi_dub.mp4

Example 2: English to Chinese

• Source Video: YouTube Video
• Dubbed Output: youtubeid_nbf0qPC3Wwk_en_to_zh_dub.mp4

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

👨‍💻 Author

Y. Siva Sai Krishna (@ysskrishna)