Skip to content

Latest commit

 

History

History
326 lines (291 loc) · 16.5 KB

ARCHITECTURE.md

File metadata and controls

326 lines (291 loc) · 16.5 KB

GenieHelper Media Hub - Architecture

System Overview

GenieHelper Media Hub is a monolithic Flask application designed for media processing. The architecture prioritizes simplicity, self-hosting capability, and efficient resource usage on modest hardware.

┌─────────────────────────────────────────────────────────────────────────┐
│                           CLIENT LAYER                                   │
├─────────────────┬─────────────────┬─────────────────────────────────────┤
│   Web Browser   │ Chrome Extension│         Mobile Browser              │
│   (Desktop)     │   (extension-ui)│      (Responsive UI)                │
└────────┬────────┴────────┬────────┴────────────────┬────────────────────┘
         │                 │                          │
         ▼                 ▼                          ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         PRESENTATION LAYER                               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────────────┐  │
│  │  Jinja2      │  │   HTMX       │  │   Tailwind CSS               │  │
│  │  Templates   │  │   Dynamic    │  │   Styling                    │  │
│  └──────────────┘  └──────────────┘  └──────────────────────────────┘  │
└────────────────────────────────┬────────────────────────────────────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         APPLICATION LAYER                                │
│                         Flask 3.1 (app.py)                               │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │  Routes                                                            │  │
│  │  ├── Page Routes (/, /dashboard, /convert, /resize, etc.)        │  │
│  │  ├── API Routes (/api/files, /api/tasks, /api/dashboard/stats)   │  │
│  │  └── Operation Routes (/convert/image, /resize/video, etc.)      │  │
│  └──────────────────────────────────────────────────────────────────┘  │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │  Middleware                                                        │  │
│  │  ├── File Upload Handler (werkzeug)                               │  │
│  │  ├── Request Validation                                           │  │
│  │  └── Error Handling                                               │  │
│  └──────────────────────────────────────────────────────────────────┘  │
└────────────────────────────────┬────────────────────────────────────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          SERVICE LAYER                                   │
│  ┌────────────────────────────────────────────────────────────────────┐│
│  │  services/                                                          ││
│  │  ├── file_manager.py    → File upload, storage, retrieval          ││
│  │  ├── database.py        → SQLite connection management             ││
│  │  └── task_queue/        → Background task processing               ││
│  │      └── threading_backend.py                                       ││
│  └────────────────────────────────────────────────────────────────────┘│
│  ┌────────────────────────────────────────────────────────────────────┐│
│  │  services/media/                                                    ││
│  │  ├── image_ops.py       → crop, resize, convert (Pillow)           ││
│  │  ├── video_ops.py       → compress, crop, resize, trim (FFmpeg)    ││
│  │  ├── doc_ops.py         → PDF, DOCX conversion                     ││
│  │  └── media_info.py      → Metadata extraction (ffprobe)            ││
│  └────────────────────────────────────────────────────────────────────┘│
└────────────────────────────────┬────────────────────────────────────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                           DATA LAYER                                     │
│  ┌─────────────────────┐  ┌─────────────────────┐                       │
│  │     files.db        │  │     tasks.db        │                       │
│  │     (SQLite)        │  │     (SQLite)        │                       │
│  │  ├── files table    │  │  ├── tasks table    │                       │
│  │  └── metadata       │  │  └── status/logs    │                       │
│  └─────────────────────┘  └─────────────────────┘                       │
│  ┌─────────────────────────────────────────────────────────────────┐   │
│  │  File System                                                      │   │
│  │  ├── files/staging/     → Temporary uploaded files               │   │
│  │  ├── files/output/      → Processed output files                 │   │
│  │  ├── media-output/      → Public download directory              │   │
│  │  └── logs/              → Application logs                       │   │
│  └─────────────────────────────────────────────────────────────────┘   │
└────────────────────────────────┬────────────────────────────────────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                       EXTERNAL DEPENDENCIES                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────────────┐  │
│  │   FFmpeg     │  │ ImageMagick  │  │        yt-dlp               │  │
│  │   (Video)    │  │   (Images)   │  │   (YouTube Downloads)       │  │
│  └──────────────┘  └──────────────┘  └──────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────┘

Component Details

1. Flask Application (app.py)

The main application file contains:

app.py (~1200 lines)
├── Configuration & Initialization
│   ├── Flask app setup
│   ├── Logging configuration
│   ├── Database initialization
│   └── File manager instance
│
├── Helper Functions
│   ├── allowed_file() - Extension validation
│   ├── get_upload_path() - Staging directory
│   ├── get_output_path() - Output directory
│   ├── save_uploaded_file() - File persistence
│   └── get_dashboard_stats() - Statistics
│
├── Page Routes
│   ├── / → workflow.html
│   ├── /dashboard → dashboard_interactive.html
│   ├── /convert, /resize, /crop, etc.
│   └── /files, /tasks, /queue
│
├── Operation Routes
│   ├── /convert/image (POST)
│   ├── /resize/image, /resize/video (POST)
│   ├── /crop/image, /crop/video (POST)
│   ├── /compress/video (POST)
│   ├── /download/video (POST)
│   ├── /trim/video (POST)
│   └── /join/videos (POST)
│
└── API Routes
    ├── /api/files (GET/POST)
    ├── /api/files/<id> (GET/DELETE)
    ├── /api/tasks (GET)
    └── /api/dashboard/stats (GET)

2. Service Layer

File Manager (services/file_manager.py)

FileManager
├── list_files()      → Get all tracked files
├── add_file()        → Register new file
├── get_file()        → Retrieve file info
├── delete_file()     → Remove file
└── cleanup_old()     → Remove expired files

Database (services/database.py)

Database Functions
├── init_db()              → Create tables if not exist
├── get_db_connection()    → Get SQLite connection
└── execute_query()        → Run SQL queries

Media Operations (services/media/)

image_ops.py
├── convert_image()   → Format conversion
├── resize_image()    → Scale by percentage
├── crop_image()      → Crop to coordinates
└── get_image_info()  → Metadata extraction

video_ops.py
├── compress_video()   → CRF-based compression
├── resize_video()     → Resolution scaling
├── crop_video()       → Frame cropping
├── trim_video()       → Segment extraction
├── join_videos()      → Concatenation
├── convert_video()    → Format conversion
└── download_youtube() → yt-dlp wrapper

doc_ops.py
├── convert_document() → PDF/DOCX/MD/HTML conversion

3. Database Schema

files.db

CREATE TABLE files (
    id TEXT PRIMARY KEY,
    filename TEXT NOT NULL,
    path TEXT NOT NULL,
    size INTEGER,
    mime_type TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    metadata TEXT  -- JSON blob
);

tasks.db

CREATE TABLE tasks (
    id TEXT PRIMARY KEY,
    type TEXT NOT NULL,           -- 'convert', 'resize', etc.
    status TEXT DEFAULT 'pending', -- pending, processing, completed, failed
    input_file TEXT,
    output_file TEXT,
    params TEXT,                   -- JSON blob
    error TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    completed_at TIMESTAMP
);

4. Request Flow

User Action → HTTP Request
    │
    ▼
Flask Route Handler
    │
    ├── Validate Input
    │   ├── Check file extension
    │   ├── Validate parameters
    │   └── Check file size
    │
    ├── Save Uploaded File
    │   ├── Generate unique filename
    │   └── Save to staging/
    │
    ├── Process Media
    │   ├── Call appropriate service
    │   ├── FFmpeg/ImageMagick execution
    │   └── Generate output file
    │
    ├── Cleanup
    │   └── Remove staging file
    │
    └── Return Response
        ├── HTML fragment (HTMX)
        ├── JSON (API)
        └── Download link

5. UI Architecture

Dashboard Hub (Level 0)
    │
    ├── Convert → Convert Page
    │   ├── Image Conversion
    │   └── Document Conversion
    │
    ├── Resize → Resize Page
    │   ├── Image Resize
    │   └── Video Resize
    │
    ├── Crop → Crop Page
    │   ├── Image Crop (Interactive)
    │   └── Video Crop
    │
    ├── Compress → Compress Page
    │   └── Video Compression
    │
    └── Download → Download Page
        └── YouTube Download

Navigation Pattern:
- Hub → Operation Page → Action → Result
- Back button returns to Hub
- HTMX for dynamic updates without page reload

6. Chrome Extension

extension-ui/
├── manifest.json       → Extension configuration
├── background.js       → Service worker
├── contentScript.js    → Page injection
├── index.html          → Popup UI
└── assets/             → Icons

Design Decisions

Why Flask (not FastAPI)?

  • Simpler deployment on shared hosting
  • Jinja2 templates for SSR
  • Lower memory footprint
  • Sufficient for file processing workloads

Why SQLite (not PostgreSQL)?

  • Zero configuration
  • Single file per database
  • Sufficient for single-server deployment
  • Easy backup (just copy file)

Why Threading (not Celery)?

  • Simpler architecture
  • No Redis/RabbitMQ dependency
  • Adequate for light task queues
  • Can upgrade later if needed

Why HTMX (not React/Vue)?

  • Progressive enhancement
  • Server-side rendering
  • Minimal JavaScript
  • Faster initial load

Scaling Considerations

Current Capacity

  • Single server deployment
  • ~100 concurrent users
  • ~500MB max file size
  • Files cleaned after 24 hours

Future Scaling Options

  1. Add Redis - Session storage, task queue
  2. Add Celery - Distributed task processing
  3. S3 Storage - Offload file storage
  4. CDN - Static asset delivery
  5. Load Balancer - Multiple Flask instances

Security Measures

  1. File Validation - Extension whitelist
  2. Size Limits - Max upload size enforced
  3. Secure Filenames - Sanitized via werkzeug
  4. HTTPS - Enforced via Plesk/nginx
  5. No Execution - Uploaded files never executed

Version: 1.0.0 Last Updated: January 25, 2026