Based on Open-LLM-VTuber

⚠️ This project is in its early stages and is currently under active development.

⚠️ If you want to run the server remotely and access it on a different machine, such as running the server on your computer and access it on your phone, you will need to configure https, because the microphone on the front end will only launch in a secure context (a.k.a. https or localhost). See MDN Web Doc. Therefore, you should configure https with a reverse proxy to access the page on a remote machine (non-localhost).

⭐️ What is this project?

Agent-Avatar is an enhanced fork of Open-LLM-VTuber, a unique voice-interactive AI companion that not only supports real-time voice conversations and visual perception but also features a lively Live2D avatar. All functionalities can run completely offline on your computer!

You can treat it as your personal AI companion — whether you want a virtual girlfriend, boyfriend, cute pet, or any other character, it can meet your expectations. The project fully supports Windows, macOS, and Linux, and offers two usage modes: web version and desktop client (with special support for transparent background desktop pet mode, allowing the AI companion to accompany you anywhere on your screen).

This project is in active development with additional features and improvements being added regularly.

✨ Features & Highlights

• 💻 Attractive and powerful web and desktop clients: Offers both web version and desktop client usage modes, supporting rich interactive features and personalization settings. The desktop client can switch freely between window mode and desktop pet mode, allowing the AI companion to be by your side at all times.

• 🎯 Advanced interaction features:

  • 👁️ Visual perception, supporting camera, screen recording and screenshots, allowing your AI companion to see you and your screen
  • 🎤 Voice interruption without headphones (AI won't hear its own voice)
  • 🫱 Touch feedback, interact with your AI companion through clicks or drags
  • 😊 Live2D expressions, set emotion mapping to control model expressions from the backend
  • 🐱 Pet mode, supporting transparent background, global top-most, and mouse click-through - drag your AI companion anywhere on the screen
  • 💭 Display AI's inner thoughts, allowing you to see AI's expressions, thoughts and actions without them being spoken
  • 🗣️ AI proactive speaking feature
  • 💾 Chat log persistence, switch to previous conversations anytime
  • 🌍 TTS translation support (e.g., chat in Chinese while AI uses Japanese voice)

• 🧠 Extensive model support:

  • 🤖 Large Language Models (LLM): Ollama, OpenAI (and any OpenAI-compatible API), Gemini, Claude, Mistral, DeepSeek, Zhipu AI, GGUF, LM Studio, vLLM, etc.
  • 🎙️ Automatic Speech Recognition (ASR): sherpa-onnx, FunASR, Faster-Whisper, Whisper.cpp, Whisper, Groq Whisper, Azure ASR, etc.
  • 🔊 Text-to-Speech (TTS): sherpa-onnx, pyttsx3, MeloTTS, Coqui-TTS, GPTSoVITS, Bark, CosyVoice, Edge TTS, Fish Audio, Azure TTS, etc.

• 🔧 Highly customizable:

  • ⚙️ Simple module configuration: Switch various functional modules through simple configuration file modifications, without delving into the code
  • 🎨 Character customization: Import custom Live2D models to give your AI companion a unique appearance. Shape your AI companion's persona by modifying the Prompt. Perform voice cloning to give your AI companion the voice you desire
  • 🧩 Flexible Agent implementation: Inherit and implement the Agent interface to integrate any Agent architecture, such as HumeAI EVI, OpenAI Her, Mem0, etc.
  • 🔌 Good extensibility: Modular design allows you to easily add your own LLM, ASR, TTS, and other module implementations, extending new features at any time

🚀 Quick Start

Prerequisites

• Docker and Docker Compose installed on your system

Installation with Docker

1. Clone the repository

   git clone https://github.com/netixc/Agent-Avatar.git
   cd Agent-Avatar

2. Configure the application

   • Copy the default configuration template:

     cp config_templates/conf.default.yaml conf.yaml

   • Edit conf.yaml to configure your preferred LLM, ASR, and TTS providers
   • Add your API keys if using cloud services

3. Build and run with Docker Compose

   docker compose up -d --build

4. Access the web interface

   • Open your browser and navigate to http://localhost:12393
   • Allow microphone access when prompted

Stop the server

docker compose down

📜 Third-Party Licenses

Live2D Sample Models Notice

This project includes Live2D sample models provided by Live2D Inc. These assets are licensed separately under the Live2D Free Material License Agreement and the Terms of Use for Live2D Cubism Sample Data. They are not covered by the MIT license of this project.

This content uses sample data owned and copyrighted by Live2D Inc. The sample data are utilized in accordance with the terms and conditions set by Live2D Inc. (See Live2D Free Material License Agreement and Terms of Use).

Note: For commercial use, especially by medium or large-scale enterprises, the use of these Live2D sample models may be subject to additional licensing requirements. If you plan to use this project commercially, please ensure that you have the appropriate permissions from Live2D Inc., or use versions of the project without these models.