SIEM Dashboard

SIEM Dashboard

Full-stack Security Information and Event Management platform with real-time log correlation, alerting, and attack playbooks.

Overview

A SIEM platform that ingests log events from multiple sources, normalizes them into a common schema, classifies severity using pattern matching, and runs a real-time correlation engine to generate alerts. Includes a React dashboard for monitoring, investigation, and attack scenario playback using MITRE ATT&CK techniques.

Status: Complete | Difficulty: Intermediate

Tech Stack

Backend

Technology	Version	Purpose
Flask	-	Web framework (app factory pattern)
MongoDB	-	Log and alert persistence
Redis Streams	-	Real-time event delivery
MongoEngine	-	MongoDB ODM
Pydantic	-	Request/response validation
JWT + Argon2id	-	Authentication

Frontend

Technology	Version	Purpose
React	-	UI framework
TypeScript	-	Type safety
Zustand	-	Client state management
TanStack Query	v5	Server state management
visx	-	Chart visualization
SSE	-	Real-time streaming

Infrastructure

• Docker Compose (dev + prod configs)
• Nginx reverse proxy (SSE-aware)
• Gunicorn WSGI server

Features

Log Processing

• Multi-source ingestion (firewall, IDS, auth, endpoint, DNS, proxy)
• Common schema normalization
• Regex-based severity classification
• Event pivot API for investigation workflows

Correlation Engine

• Sliding window rule evaluation
• Threshold counting (e.g., 20 failed logins in 5 minutes)
• Ordered sequence detection (brute force → successful login)
• Distinct-value aggregation
• Rule testing against historical data

Alert Management

• Full lifecycle: detection → acknowledgment → investigation → resolution
• False positive classification
• Severity-based prioritization

Attack Playbooks

• YAML-based scenario definitions
• MITRE ATT&CK technique mapping
• Brute force (T1110.001), DNS tunneling (T1048.003), and more
• Threaded scenario execution

Real-Time Dashboard

• Server-Sent Events for live updates
• Protocol distribution charts
• Severity breakdown timelines
• Top source analysis
• MongoDB aggregation pipelines

Architecture

┌─────────────────────────────────────────────────────────┐
│                   Frontend (React + TS)                   │
│  Zustand stores | TanStack Query | SSE streaming         │
└───────────────────────────┬─────────────────────────────┘
                            │
┌───────────────────────────▼─────────────────────────────┐
│                    Nginx (SSE-aware proxy)                │
└───────────────────────────┬─────────────────────────────┘
                            │
┌───────────────────────────▼─────────────────────────────┐
│                   Backend (Flask)                         │
│  ┌──────────────────────────────────────────────────┐   │
│  │                     Engine                        │   │
│  │  normalizer.py | severity.py | correlation.py    │   │
│  └──────────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────────┐   │
│  │                   Scenarios                       │   │
│  │  playbook.py (YAML parser) | runner.py (thread)  │   │
│  └──────────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────────┐   │
│  │           Routes / Controllers / Schemas          │   │
│  │  ~30 endpoints under /v1/                         │   │
│  └──────────────────────────────────────────────────┘   │
└────────┬──────────────────┬─────────────────────────────┘
         │                  │
         ▼                  ▼
┌─────────────┐    ┌─────────────┐
│   MongoDB   │    │    Redis    │
│  Logs/Alerts│    │   Streams   │
│  Rules/Users│    │  + Consumer │
└─────────────┘    │    Groups   │
                   └─────────────┘

Quick Start

cd PROJECTS/intermediate/siem-dashboard

# Start development environment
docker compose -f dev.compose.yml up --build

# Access at http://localhost:8431
# API at http://localhost:8431/api/v1/

# Create admin account
docker exec -it siem-backend-dev flask admin create \
  --username admin --email admin@example.com

# Ingest a test event
curl -X POST http://localhost:8431/api/v1/logs/ingest \
  -H "Content-Type: application/json" \
  -d '{"source_type":"auth","event_type":"login_failure","source_ip":"10.0.0.1","username":"root"}'

Project Structure

siem-dashboard/
├── backend/
│   ├── app/
│   │   ├── __init__.py           # Flask app factory
│   │   ├── config.py             # Pydantic settings (60+ values)
│   │   ├── core/                 # Auth, streaming, errors, decorators
│   │   ├── engine/               # Normalizer, severity, correlation
│   │   ├── models/               # MongoEngine documents
│   │   ├── routes/               # Flask blueprints
│   │   ├── controllers/          # Business logic
│   │   ├── schemas/              # Pydantic validation
│   │   └── scenarios/            # Attack playbooks (YAML)
│   └── pyproject.toml
├── frontend/
│   ├── src/
│   │   ├── api/                  # TanStack Query hooks + Zod types
│   │   ├── core/                 # Shell, routing, stores, charts
│   │   └── routes/               # Page components (lazy loaded)
│   └── package.json
├── conf/                         # Docker + Nginx configs
├── compose.yml                   # Production
└── dev.compose.yml               # Development

Development

# Backend
make backend-lint
make backend-test

# Frontend
make frontend-lint
make frontend-build

# Both
make lint
make test

Source Code

View on GitHub