Skip to content

SPEC.md

Latest commit

 

History

History
148 lines (125 loc) · 5.86 KB

SPEC.md

File metadata and controls

148 lines (125 loc) · 5.86 KB

Eater - Individual Cat Identification System

Project Overview

Project name: Eater
Type: Computer vision / individual identification system
Core functionality: Identify individual cats from camera feeds using body/coat pattern recognition
Target users: TNR caregivers, wildlife monitors, multi-cat households

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         SYSTEM ARCHITECTURE                     │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────────┐  │
│  │   Camera    │───▶│  Detection  │───▶│  Identification │  │
│  │   Input     │    │   (YOLO)    │    │   (Embeddings)  │  │
│  │  (RTSP)     │    │   "cat"     │    │  PPGNet-Cat     │  │
│  └─────────────┘    └─────────────┘    └────────┬────────┘  │
│                                                  │             │
│                   ┌─────────────┐               │             │
│                   │   TUI for   │◀──────────────┘             │
│                   │ annotation  │                              │
│                   └──────┬──────┘                              │
│                          │                                     │
│                   ┌──────▼──────┐                              │
│                   │  Postgres   │                              │
│                   │ +pgvector   │                              │
│                   └─────────────┘                              │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Technology Stack

Component Technology
Language Python 3.11+
Inference ONNX Runtime + OpenVINO
Database Postgres 18 + pgvector
Detection YOLOv8 (COCO "cat" class)
Embeddings PPGNet-Cat (converted to ONNX)
TUI Textual
Message Bus MQTT (for Frigate integration)
Camera UniFi RTSP (via UniFi Protect)

Functionality Specification

Core Features

  1. Camera Stream Ingestion

    • Connect to UniFi camera RTSP streams
    • Poll at configurable FPS (default: 1 FPS)
    • Support multiple camera feeds

  2. Cat Detection

    • Use YOLOv8 with COCO "cat" class (class 15)
    • Extract bounding boxes of detected cats
    • Filter by confidence threshold

  3. Cat Identification

    • Extract embeddings from cat body/flank using PPGNet-Cat
    • Compare against catalog using pgvector similarity search
    • Return match or flag for annotation

  4. Catalog Management

    • Store cat profiles with embeddings
    • Associate multiple photos per cat
    • Track sightings (timestamp, camera, confidence)

  5. Manual Annotation (TUI)

    • Prompt when new cat detected
    • Display cat image for identification
    • Accept name input
    • Option to skip/ignore

Data Models

-- Cats table
CREATE TABLE cats (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name TEXT NOT NULL,
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW()
);

-- Cat embeddings (one per photo)
CREATE TABLE cat_embeddings (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    cat_id UUID REFERENCES cats(id),
    embedding vector(512) NOT NULL,
    image_path TEXT,
    camera_id TEXT,
    captured_at TIMESTAMP DEFAULT NOW(),
    is_primary BOOLEAN DEFAULT FALSE
);

-- Sightings (detections)
CREATE TABLE sightings (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    cat_id UUID REFERENCES cats(id),
    camera_id TEXT NOT NULL,
    confidence FLOAT,
    captured_at TIMESTAMP DEFAULT NOW()
);

API / Interface

  • MQTT subscriber: Listen for Frigate cat detection events
  • REST API: Query catalog, get sightings
  • TUI: Manual annotation interface

Acceptance Criteria

  1. ✅ Can connect to UniFi RTSP streams
  2. ✅ Detects cats in frame (YOLO)
  3. ✅ Extracts embeddings from detected cats
  4. ✅ Matches new detections against catalog
  5. ✅ Stores cat profiles in Postgres
  6. ✅ TUI prompts for new cat names
  7. ✅ Runs inference on Intel Arc A770 via OpenVINO
  8. ✅ 1 FPS processing is achievable

File Structure

eater/
├── src/
│   ├── eater/
│   │   ├── __init__.py
│   │   ├── config.py           # Configuration
│   │   ├── database.py         # Postgres/pgvector operations
│   │   ├── detector.py        # YOLO cat detection
│   │   ├── identifier.py       # PPGNet embedding extraction
│   │   ├── camera.py           # RTSP stream handling
│   │   ├── matcher.py          # Vector similarity matching
│   │   ├── tui.py              # Annotation TUI
│   │   └── main.py             # Entry point
│   └── tests/
├── models/                     # YOLO, PPGNet models
├── data/                       # Cat images
├── SPEC.md
└── README.md