Skip to content

Betanu701/atlas-cortex

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

345 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

🧠 Atlas Cortex

A self-evolving AI assistant that learns, adapts, and grows with your household.

Python 3.11+ License: MIT Tests Open WebUI

Hardware-agnostic Β· Privacy-first Β· Family-safe Β· Self-learning


Atlas Cortex transforms a local LLM into an intelligent home assistant that understands who's speaking, adapts to each family member, controls your smart home, and gets smarter every day β€” all running on your hardware, with zero cloud dependencies.

✨ Key Features

πŸš€ Intelligent Response Pipeline

Layer Latency What Happens
Context Assembly ~1ms Identifies speaker, room, sentiment, time-of-day
Instant Answers ~5ms Date, time, math, greetings β€” no LLM needed
Plugin Dispatch ~100ms Smart home control, lists, knowledge search
LLM Generation ~500–4000ms Full reasoning with filler streaming for zero perceived wait

🏠 Smart Home Integration

  • Natural language control β€” "Turn off the bedroom lights" executes directly via Home Assistant API
  • Spatial awareness β€” knows which room you're in via satellite mics and presence sensors
  • Scene automation β€” "Good night" triggers your bedtime routine
  • Self-learning β€” commands that go to the LLM are analyzed nightly and converted into fast regex patterns

πŸ—£οΈ Voice & Speech Engine

  • Multi-TTS stack β€” Qwen3-TTS (primary), Fish Audio S2 (story character voices), Orpheus (emotional), Kokoro (CPU), Piper (fast fallback)
  • Voice identification β€” recognizes family members by voice, personalizes responses per person
  • TTS hot-swap β€” swap voice models at runtime for character voices in stories
  • Sentence-boundary streaming β€” starts speaking before the full response is generated
  • Speaker age estimation β€” hybrid age detection for content tier assignment

🎭 Avatar System

  • Animated face β€” Nick Jr. Face-inspired SVG avatar with real-time lip-sync and expressions
  • Audio-synced visemes β€” mouth shapes timed to actual TTS audio duration, not guesswork
  • Content-aware expressions β€” silly grin after joke punchlines, thinking face during processing
  • Pre-cached joke bank β€” 25 kid-friendly jokes with instant TTS playback, weekly rotation
  • Interactive button β€” #show-joke-button URL hash enables a "Tell me a joke" button on the display
  • WebSocket streaming β€” TTS audio, visemes, and expressions all delivered in real-time

Avatar Expressions Neutral Β· Talking Β· Silly grin (after a joke punchline)

⏰ Alarms, Timers & Reminders

  • Natural language time parsing β€” "Wake me up at 7am", "Set a 15-minute timer"
  • Recurring alarms with RRULE recurrence rules
  • Notification routing β€” deliver to the right satellite speaker or HA media player
  • Snooze/dismiss via voice β€” "Snooze for 5 minutes" / "Stop"

πŸ”„ Routines & Automations

  • Built-in routines β€” "Good morning" β†’ lights, coffee, weather, calendar
  • Custom routines β€” "When I say 'movie time', dim lights and turn on TV"
  • Multiple triggers β€” voice command, cron schedule, HA events
  • Conversational builder β€” Atlas asks clarifying questions to build routines
  • Templates β€” pre-built routines for common scenarios

πŸ“‘ Proactive Intelligence

  • Rule engine with notification throttling
  • Weather, energy, anomaly, and calendar providers
  • Daily briefing β€” personalized morning summary
  • Pattern detection β€” learns household habits over time

πŸ“š Learning & Education

  • Socratic tutoring β€” guides understanding, never gives direct answers
  • Quiz generator β€” math curriculum through Calculus III
  • 3 STEM games β€” Number Quest, Science Safari, Word Wizard
  • Progress tracking and parent reports per child

πŸ“’ Intercom & Broadcasting

  • Targeted announce β€” "Tell the kids dinner is ready" β†’ children's rooms
  • Whole-house broadcast β€” all satellites simultaneously
  • Two-way calling β€” "Atlas, talk to the garage"
  • Drop-in monitoring β€” listen to a room without announcement
  • Zone groups β€” broadcast to custom room groups

🎡 Media & Entertainment

  • YouTube Music, Plex, Audiobookshelf, Podcasts
  • Local music library with ID3 tag scanning
  • Multi-room playback via satellite β†’ Chromecast β†’ HA media players
  • Playback router β€” picks the best output for each room

πŸ“– Story Time

  • Interactive branching stories with user choices
  • Character voices via Fish Audio S2 with TTS hot-swap
  • Story library β€” save, resume, and replay stories
  • Age-appropriate content per user profile

πŸ€– Self-Evolution

  • Conversation quality analysis β€” identifies areas for improvement
  • LoRA training pipeline on AMD GPUs (ROCm) β€” runs overnight
  • Model scout with safety gates β€” finds and evaluates new models
  • A/B testing for model comparisons
  • Personality drift monitoring β€” ensures consistency over time

πŸ’» Atlas CLI Agent

  • Interactive chat: atlas chat
  • Autonomous agent: atlas agent "build an API"
  • 31 agent tools β€” file, shell, git, web, vision, memory
  • ReAct reasoning loop with context management and sessions

🌐 Standalone Web App

  • Browser chat with WebSocket streaming
  • Voice input/output in the browser
  • Animated avatar with lip-sync
  • Unified dashboard for all Atlas features

πŸ›‘οΈ Safety & Content Policy

  • Age-appropriate responses β€” automatically adapts vocabulary and content for toddlers, children, teens, and adults
  • Educational mode β€” uses scientific terminology for biology/anatomy at all ages β€” never evasive
  • 5-layer jailbreak defense β€” regex patterns, semantic analysis, system prompt hardening, output monitoring, adaptive learning
  • PII protection β€” SSN, credit card, phone, email auto-redacted from logs and memory
  • Crisis detection β€” recognizes self-harm/emergency language and responds with appropriate resources

🧠 Memory & Learning

  • HOT/COLD architecture β€” ChromaDB vector search + SQLite FTS5 with reciprocal rank fusion
  • Persistent memory β€” remembers conversations, preferences, and facts across sessions
  • Nightly evolution β€” analyzes patterns, learns from mistakes, evolves personality profiles
  • Anti-hallucination β€” confidence scoring, grounding loops, and mistake tracking

πŸ‘€ User Profiles & Personality

  • Per-user adaptation β€” vocabulary level, preferred tone, communication style
  • Honest personality β€” pushes back on bad ideas, challenges in tutoring mode, never sycophantic
  • Emotional evolution β€” builds unique rapport with each household member over time
  • Parental controls β€” content filtering, allowed hours, restricted actions per child

πŸ–₯️ Admin Web Panel

  • Dark-themed dashboard β€” real-time stats, recent activity, system health at a glance
  • 20 views β€” Chat, Dashboard, Users, UserDetail, Parental, Safety, Voice, Avatar, Devices, Satellites, SatelliteDetail, Plugins, Scheduling, Routines, Learning, Proactive, Media, Intercom, Evolution, Stories, System
  • User management β€” profiles, age settings, vocabulary levels, parental controls
  • Safety monitoring β€” guardrail event log, jailbreak pattern management, content tier overrides
  • Plugin management β€” enable/disable 21 built-in plugins with per-plugin settings
  • Voice enrollment β€” view and manage speaker profiles, confidence thresholds
  • Device management β€” Home Assistant devices, aliases, command patterns
  • Evolution tracking β€” rapport scores, emotional profiles, nightly evolution logs
  • System overview β€” hardware info, GPU assignment, model configs, discovered services

Admin Dashboard Admin Safety

πŸ“– Full Admin Panel Guide β†’ β€” walkthrough with screenshots of every page

πŸ—οΈ Architecture

                        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                        β”‚    User Interface      β”‚
                        β”‚  Open WebUI / Voice    β”‚
                        β”‚  / Satellite Mics      β”‚
                        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                    β”‚
                        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                        β”‚    Atlas Cortex        β”‚
                        β”‚    Server (:5100)      β”‚
                        β”‚  OpenAI-compatible     β”‚
                        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                    β”‚
            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β”‚                       β”‚                       β”‚
  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚   Input Pipeline    β”‚ β”‚  Safety Guardrails  β”‚ β”‚   Voice Engine      β”‚
  β”‚                     β”‚ β”‚                     β”‚ β”‚                     β”‚
  β”‚ L0: Context  (1ms)  β”‚ β”‚ β€’ Content tiers     β”‚ β”‚ β€’ Qwen3-TTS         β”‚
  β”‚ L1: Instant  (5ms)  β”‚ β”‚ β€’ Jailbreak defense β”‚ β”‚ β€’ Fish Audio S2      β”‚
  β”‚ L2: Plugins (100ms) β”‚ β”‚ β€’ PII redaction     β”‚ β”‚ β€’ Orpheus / Kokoro   β”‚
  β”‚ L3: LLM   (500ms+)  β”‚ β”‚ β€’ Crisis detection  β”‚ β”‚ β€’ Piper fallback     β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
            β”‚
  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚                         Integrations                           β”‚
  β”‚                                                                β”‚
  β”‚  🏠 Home Assistant  πŸ“‹ Lists  πŸ“š Knowledge  πŸ” Memory        β”‚
  β”‚  πŸ”§ Discovery  πŸ“¦ Backup  πŸŽ“ Learning  ⏰ Scheduling         β”‚
  β”‚  πŸ”„ Routines  πŸ“‘ Proactive  πŸ“’ Intercom  🎡 Media            β”‚
  β”‚  πŸ“– Stories  πŸ€– Evolution  πŸ’» CLI Agent  🌐 Web App           β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                β”‚
                     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                     β”‚  SQLite + ChromaDB    β”‚
                     β”‚  (WAL mode, 50+ tbl)  β”‚
                     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                β”‚
                     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                     β”‚  Nightly Evolution    β”‚
                     β”‚  LoRA training        β”‚
                     β”‚  Model scouting       β”‚
                     β”‚  Pattern learning     β”‚
                     β”‚  Profile evolution    β”‚
                     β”‚  Device discovery     β”‚
                     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Multi-GPU Support

Atlas detects all GPUs at startup and assigns optimal roles:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  GPU 0 (Largest)β”‚     β”‚  GPU 1 (Second) β”‚     β”‚  iGPU (Fallback)β”‚
β”‚  ═══════════════│     β”‚  ═══════════════│     β”‚  ═══════════════│
β”‚  LLM Inference  β”‚     β”‚  Voice / TTS    β”‚     β”‚  Lightweight    β”‚
β”‚  Transformers   β”‚     β”‚  Qwen3-TTS      β”‚     β”‚  tasks only     β”‚
β”‚  20GB+ VRAM     β”‚     β”‚  8-12GB VRAM    β”‚     β”‚                 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  • Supported GPUs: AMD (ROCm), NVIDIA (CUDA), Intel (oneAPI/IPEX), Apple (Metal)
  • Auto-sizing: Models selected based on available VRAM β€” from 1.7B (4GB) to 72B (48GB+)
  • Mixed vendors: Run AMD + Intel GPUs in the same system via separate containers

πŸš€ Getting Started

Prerequisites

Note: Ollama is still supported as a legacy fallback β€” set LLM_PROVIDER=ollama to use it.

Quick Start (Docker)

# Clone the repository
git clone https://github.com/Betanu701/atlas-cortex.git
cd atlas-cortex

# Start with Docker Compose (includes Qwen3-TTS, Whisper, Kokoro, Piper, and more)
# HuggingFace models download automatically via cached volume
docker compose -f docker/docker-compose.yml up -d

# For NVIDIA GPU support:
docker compose -f docker/docker-compose.yml -f docker/docker-compose.gpu-nvidia.yml up -d

# Atlas is now running at http://localhost:5100
# Chat UI: http://localhost:5100/chat
# Admin panel: http://localhost:5100/admin/

Quick Start (Manual)

# Clone and set up
git clone https://github.com/Betanu701/atlas-cortex.git
cd atlas-cortex

# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install dependencies (includes transformers, accelerate, sentence-transformers)
pip install -r requirements.txt

# Run the interactive installer
python -m cortex.install

# Or start the server directly (HuggingFace models download on first use)
CAG_MODEL=Qwen/Qwen3-4B EMBED_MODEL=all-MiniLM-L6-v2 python -m cortex.server

Connect to Open WebUI

  1. Open your Open WebUI instance
  2. Go to Admin β†’ Settings β†’ Connections
  3. Add a new OpenAI-compatible connection:
    • URL: http://<atlas-host>:5100/v1
    • Model: atlas-cortex
  4. Start chatting β€” Atlas handles the rest

Admin Panel Setup

The admin panel is a Vue 3 SPA served directly from the Atlas Cortex server.

# Build the admin panel (requires Node.js 18+)
cd admin
npm install
npx vite build
cd ..

# Start the server (admin panel is now available)
python -m cortex.server

Open http://localhost:5100/admin/ in your browser.

Default username admin
Default password atlas-admin

⚠️ Change the default password immediately via the admin panel.

πŸ“– Full Admin Panel Guide β†’ β€” walkthrough of every page with screenshots

πŸ“– API Reference (144+ endpoints) β†’ β€” complete endpoint documentation

Discover Your Services

# Scan your network for Home Assistant, Nextcloud, MQTT, etc.
python -m cortex.discover

Atlas finds available services on your network and configures integrations automatically.

βš™οΈ Configuration

Environment Variables

Variable Default Description
CORTEX_HOST 0.0.0.0 Server bind address
CORTEX_PORT 5100 Server port
CORTEX_DATA_DIR ./data Database and state directory
LLM_PROVIDER transformers LLM backend (transformers, ollama, openai_compatible)
CAG_MODEL Qwen/Qwen3-4B HuggingFace model for inference (Transformers provider)
CAG_DEVICE auto Device for model inference (auto, cuda, cpu)
CAG_DTYPE auto Model dtype (auto, float16, bfloat16)
EMBED_MODEL all-MiniLM-L6-v2 Sentence-transformers embedding model
OLLAMA_BASE_URL http://localhost:11434 Ollama API URL (legacy fallback)
OPENAI_BASE_URL β€” Custom OpenAI-compatible endpoint
OPENAI_API_KEY β€” API key for OpenAI-compatible backends
MODEL_FAST qwen2.5:14b Model for quick factual answers (Ollama/OpenAI providers)
MODEL_THINKING qwen3:30b-a3b Model for complex reasoning (Ollama/OpenAI providers)
HA_URL β€” Home Assistant URL (e.g., http://192.168.1.100:8123)
HA_TOKEN β€” Home Assistant long-lived access token
CORTEX_JWT_SECRET atlas-cortex-change-me Secret key for admin JWT tokens (change in production!)
CORTEX_JWT_EXPIRY 86400 Admin session duration in seconds (default: 24 hours)

πŸ“‘ API Reference

Atlas exposes an OpenAI-compatible API so any client that works with OpenAI/Ollama works with Atlas.

Chat Completions

# Streaming
curl http://localhost:5100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "atlas-cortex",
    "messages": [{"role": "user", "content": "Turn off the living room lights"}],
    "stream": true
  }'

Text-to-Speech

# Generate speech with emotion
curl http://localhost:5100/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kokoro",
    "input": "Good morning! The weather looks beautiful today.",
    "voice": "af_bella"
  }' --output speech.wav

# List available voices
curl http://localhost:5100/v1/audio/voices

Health Check

curl http://localhost:5100/health

Admin API

All admin endpoints require a JWT token. Quick example:

# Login and get token
TOKEN=$(curl -s http://localhost:5100/admin/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "atlas-admin"}' | python3 -c "import sys,json; print(json.load(sys.stdin)['token'])")

# Use with any admin endpoint
curl http://localhost:5100/admin/dashboard \
  -H "Authorization: Bearer $TOKEN"

πŸ“– Full endpoint list (144+ endpoints) β†’

πŸ“ Project Structure

atlas-cortex/
β”œβ”€β”€ admin/                         # Vue 3 Admin Panel (SPA, 20 views)
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ views/                 #   20 admin views (dashboard, users, safety, etc.)
β”‚   β”‚   β”œβ”€β”€ components/            #   Reusable components (NavBar, DataTable, etc.)
β”‚   β”‚   β”œβ”€β”€ router/                #   Vue Router with auth guards
β”‚   β”‚   β”œβ”€β”€ stores/                #   Pinia auth store
β”‚   β”‚   └── api.js                 #   API client with JWT handling
β”‚   β”œβ”€β”€ package.json
β”‚   └── vite.config.js
β”œβ”€β”€ cortex/                        # Core Python package
β”‚   β”œβ”€β”€ server.py                  # OpenAI-compatible FastAPI server
β”‚   β”œβ”€β”€ auth.py                    # JWT authentication (bcrypt + PyJWT)
β”‚   β”œβ”€β”€ pipe.py                    # Open WebUI Pipe function
β”‚   β”œβ”€β”€ db.py                      # SQLite schema (50+ tables, WAL mode)
β”‚   β”œβ”€β”€ admin/                     # Admin API domain routers (9 sub-routers)
β”‚   β”œβ”€β”€ pipeline/                  # 4-layer processing pipeline
β”‚   β”‚   β”œβ”€β”€ layer0_context.py      #   Context assembly, sentiment, spatial
β”‚   β”‚   β”œβ”€β”€ layer1_instant.py      #   Instant answers (math, date, identity)
β”‚   β”‚   β”œβ”€β”€ layer2_plugins.py      #   Plugin dispatch (21 built-in plugins)
β”‚   β”‚   └── layer3_llm.py          #   Filler streaming + LLM generation
β”‚   β”œβ”€β”€ providers/                 # LLM backend abstraction
β”‚   β”‚   β”œβ”€β”€ transformers.py        #   HuggingFace Transformers provider (default)
β”‚   β”‚   β”œβ”€β”€ ollama.py              #   Ollama provider (legacy fallback)
β”‚   β”‚   └── openai_compat.py       #   Any OpenAI-compatible backend
β”‚   β”œβ”€β”€ speech/                    # All audio synthesis/transcription
β”‚   β”‚   β”œβ”€β”€ tts.py                 #   Multi-provider TTS with hot-swap
β”‚   β”‚   β”œβ”€β”€ stt.py                 #   Whisper + Wyoming STT
β”‚   β”‚   β”œβ”€β”€ voices.py              #   Voice resolution (satelliteβ†’userβ†’system)
β”‚   β”‚   β”œβ”€β”€ cache.py               #   Unified audio cache
β”‚   β”‚   β”œβ”€β”€ fish_audio.py          #   Fish Audio S2 character voices
β”‚   β”‚   └── hotswap.py             #   Runtime TTS model swapping
β”‚   β”œβ”€β”€ orchestrator/              # Request coordination (STTβ†’pipelineβ†’TTS)
β”‚   β”œβ”€β”€ voice/                     # Legacy voice module (β†’ cortex.speech)
β”‚   β”œβ”€β”€ safety/                    # Safety guardrails + jailbreak defense
β”‚   β”œβ”€β”€ plugins/                   # Plugin framework (CortexPlugin, registry)
β”‚   β”œβ”€β”€ integrations/              # HA, knowledge, lists, learning
β”‚   β”œβ”€β”€ memory/                    # HOT/COLD memory (BM25 + vector, RRF)
β”‚   β”œβ”€β”€ profiles/                  # User profiles & parental controls
β”‚   β”œβ”€β”€ context/                   # Context window management
β”‚   β”œβ”€β”€ avatar/                    # SVG avatar, lip-sync, expressions
β”‚   β”œβ”€β”€ filler/                    # Sentiment-aware filler streaming
β”‚   β”œβ”€β”€ grounding/                 # Anti-hallucination engine
β”‚   β”œβ”€β”€ backup/                    # Automated backup/restore + offsite
β”‚   β”œβ”€β”€ scheduling/                # Alarms, timers, reminders (NL time parsing)
β”‚   β”œβ”€β”€ routines/                  # Routine automations, triggers, templates
β”‚   β”œβ”€β”€ proactive/                 # Proactive intelligence, daily briefing
β”‚   β”œβ”€β”€ learning/                  # Self-learning (fallthrough analysis)
β”‚   β”œβ”€β”€ intercom/                  # Announce, broadcast, two-way calling
β”‚   β”œβ”€β”€ media/                     # YouTube Music, Plex, ABS, podcasts
β”‚   β”œβ”€β”€ stories/                   # Story generator, character voices
β”‚   β”œβ”€β”€ evolution/                 # LoRA training, model scout, drift
β”‚   β”œβ”€β”€ cli/                       # Atlas CLI agent (REPL, 31 tools)
β”‚   β”œβ”€β”€ notifications/             # Alert and notification routing
β”‚   β”œβ”€β”€ selfmod/                   # Self-evolution security gates
β”‚   β”œβ”€β”€ content/                   # Pre-cached content (jokes)
β”‚   β”œβ”€β”€ scheduler/                 # Background task management
β”‚   β”œβ”€β”€ satellite/                 # Satellite WebSocket + provisioning
β”‚   β”œβ”€β”€ install/                   # Hardware detection & installer
β”‚   β”œβ”€β”€ discovery/                 # Network service discovery
β”‚   └── integrity/                 # Data integrity checks
β”œβ”€β”€ satellite/                     # Satellite agent (Pi, ESP32)
β”œβ”€β”€ docker/                        # Docker deployment configs
β”œβ”€β”€ docs/                          # Architecture, roadmap, guides (35+ files)
β”œβ”€β”€ mocks/                         # GPU-free development servers
β”œβ”€β”€ seeds/                         # Initial data (command patterns)
β”œβ”€β”€ tests/                         # 3,660+ tests
β”œβ”€β”€ requirements.txt
└── pytest.ini

πŸ§ͺ Testing

# Run all tests
source .venv/bin/activate
python -m pytest tests/ -q

# Run specific module
python -m pytest tests/test_pipeline.py -v
python -m pytest tests/test_safety.py -v
python -m pytest tests/test_voice.py -v

Current status: 3,660+ tests passing across pipeline, providers, safety, voice, discovery, integrations, filler, memory, learning, evolution, avatar, admin, spatial, Wyoming, WebDAV, CalDAV, backup, multi-room, scheduling, routines, proactive, intercom, media, stories, CLI, and web app modules.

πŸ“Š Implementation Status

Part 1: Core Engine

Phase Module Status Description
C0 Installer & Backend βœ… Complete LLM provider abstraction, GPU detection, CLI installer
C1 Core Pipeline βœ… Complete 4-layer pipeline, sentiment, instant answers, filler streaming
C3a Voice Identity βœ… Complete Speaker recognition, enrollment, hybrid age estimation
C4 Emotional Evolution βœ… Complete Rapport tracking, personality drift, proactive suggestions
C5 Memory System βœ… Complete HOT/COLD paths, vector search, BM25, RRF fusion
C6 User Profiles βœ… Complete Age-awareness, onboarding, parental controls
C7 Avatar System βœ… Complete SVG face, lip-sync visemes, emotion expressions, joke bank, TTS caching
C9 Backup & Restore βœ… Complete Automated nightly backups, one-command restore
C10 Context Management βœ… Complete Context windows, compaction, overflow recovery
C11 Voice & Speech βœ… Complete Multi-TTS (Qwen3-TTS, Fish Audio S2, Orpheus, Kokoro, Piper), hot-swap, streaming
C12 Safety Guardrails βœ… Complete Content tiers, jailbreak defense, PII redaction
β€” Admin Web Panel βœ… Complete Vue 3 dashboard (20 views), JWT auth, 30+ REST endpoints

Part 2: Integration Layer

Phase Module Status Description
I1 Service Discovery βœ… Complete HTTP-probe scanner, service registry, config wizard
I2 Home Assistant βœ… Complete REST + WebSocket client, device bootstrap, patterns
I3 Voice Pipeline βœ… Complete Wyoming STT/TTS, spatial awareness, multi-room commands
I4 Self-Learning βœ… Complete Fallthrough analysis, pattern lifecycle, nightly evolution
I5 Knowledge Sources βœ… Complete WebDAV/Nextcloud, CalDAV calendars, sync scheduler
I6 List Management βœ… Complete Multi-backend lists, HA to-do discovery, permissions
I7 Offsite Backup βœ… Complete NAS rsync/SMB, voice commands, retention policy

Part 2.5–2.7: Satellite & Plugins

Phase Module Status Description
S2.5 Satellite System ⏸️ Wake word deferred Distributed speakers/mics, Pi + ESP32, provisioning
P2.7 Fast-Path Plugins βœ… Complete 21 built-in plugins (weather, dictionary, wikipedia, conversions, movie, cooking, news, translation, stocks, sports, sound library, scheduling, routines, daily briefing, STEM games, stories, intercom, media + core 3)

Parts 3–12: Extended Features

Phase Module Status Description
P3 Alarms, Timers & Reminders βœ… Complete NL time parser, RRULE recurrence, notification routing
P4 Routines & Automations βœ… Complete Voice/cron/HA triggers, templates, conversational builder
P5 Proactive Intelligence βœ… Complete Rule engine, throttle, weather/energy/anomaly/calendar providers, daily briefing
P6 Learning & Education βœ… Complete Socratic tutoring, quiz gen (through Calc III), 3 STEM games, progress tracking
P7 Intercom & Broadcasting βœ… Complete Announce, broadcast, zones, two-way calling, drop-in monitoring
P8 Media & Entertainment βœ… Complete YouTube Music, Plex, Audiobookshelf, podcasts, local library, playback router
P9 Self-Evolution βœ… Complete Conversation analysis, LoRA training (ROCm/AMD), model scout, A/B testing, drift monitor
P10 Story Time βœ… Complete Generator, character voices (Fish Audio S2), TTS hot-swap, interactive stories, library
P11 Atlas CLI Agent βœ… Complete REPL, 31 agent tools, ReAct loop, context management, sessions
P12 Standalone Web App βœ… Complete Browser chat, WebSocket streaming, voice I/O, avatar, unified dashboard

Coming Next

Part Name Description
13 Legacy Protocol Backward compatibility for older integrations
14 Household Management Pet care, cooking assistant, inventory, chore tracking
15 Security & Monitoring Camera summaries, motion alerts, visitor history
16 Health & Wellness Medication reminders, sleep analysis, air quality
17 Multi-Language Support Real-time language detection, per-user preferences
18 Visual Media & Casting Screen casting, photo display, visual content

πŸ“‹ Full roadmap with detailed plans: docs/roadmap.md

πŸ“– Documentation

Comprehensive design documentation lives in the docs/ directory:

Document Description
Quick Start 5-minute setup guide (Docker, bare metal, satellite)
Configuration All environment variables and settings
Architecture System design, pipeline layers, evolution engine
Data Model 50+ SQLite tables, normalized schema, relationships
Voice Engine TTS providers, emotion composition, streaming
Safety Guardrails Content tiers, jailbreak defense, crisis protocol
Context Management Context windows, compaction, multi-GPU detection
Memory System HOT/COLD architecture, vector search, RRF fusion
User Profiles Age-awareness, onboarding, parental controls
Personality Honesty system, pushback, tutoring mode
Grounding Anti-hallucination, confidence scoring
Knowledge Access Document indexing, privacy gates
Lists Multi-backend lists, permissions
Avatar System Lip-sync, visemes, emotion expressions
Backup & Restore Automated backups, one-command restore
Admin Panel Guide Full walkthrough with screenshots
API Reference All 144+ API endpoints
Plugin Guide Plugin system for admins and developers
Persistent Storage Docker volumes and data paths
Satellite System Satellite speaker/mic architecture
Alarms & Timers NL time parsing, recurrence, notification routing
Routines Triggers, templates, conversational builder
Proactive Intelligence Rule engine, providers, daily briefing
Learning & Education Tutoring, quizzes, STEM games
Intercom Announce, broadcast, two-way calling
Media YouTube Music, Plex, Audiobookshelf, podcasts
Roadmap Future features and implementation plan
Phases Implementation roadmap and dependency graph
Installation Installer flow, backend abstraction

🀝 Contributing

Atlas Cortex is open source and welcomes contributions!

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/my-feature
  3. Make your changes and add tests
  4. Run the test suite: python -m pytest tests/ -q
  5. Submit a Pull Request

Development Setup

git clone https://github.com/Betanu701/atlas-cortex.git
cd atlas-cortex
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python -m pytest tests/ -q  # verify everything works

πŸ“„ License

MIT β€” use it, modify it, build on it.

About

Self-evolving AI assistant with HOT/COLD memory, spatial awareness, voice identity, and age-appropriate responses

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors