Skip to content

Latest commit

 

History

History
412 lines (319 loc) · 16.4 KB

CLAUDE.md

File metadata and controls

412 lines (319 loc) · 16.4 KB

Parse DMARC - Project Guide

A Go application that fetches DMARC reports from IMAP mailboxes, parses them, and displays them in a Vue.js dashboard. Also supports MCP (Model Context Protocol) for AI assistant integration.

Tech Stack

  • Backend: Go 1.25.4 (see go.mod for exact version)
  • Frontend: Vue.js 3 with Vite
  • Database: SQLite (supports both CGO and pure-Go variants)
  • Package Manager: Bun (for frontend)
  • Task Runner: Just (Justfile)
  • CLI Framework: urfave/cli/v3
  • JSON Library: goccy/go-json (high-performance)
  • Logging: rs/zerolog (structured logging)
  • Metrics: Prometheus client_golang
  • MCP SDK: modelcontextprotocol/go-sdk

Project Structure

parse-dmarc/
├── main.go                    # Main application entry point
├── internal/
│   ├── api/                   # REST API server and embedded frontend
│   │   └── server.go          # HTTP server, routes, metrics middleware
│   ├── config/                # Configuration management (JSON + env vars)
│   │   └── config.go          # Config loading and validation
│   ├── imap/                  # IMAP client for fetching emails
│   │   └── client.go          # Email fetching logic
│   ├── logger/                # Structured logging setup
│   │   └── logger.go          # Zerolog configuration
│   ├── mcp/                   # MCP (Model Context Protocol) server
│   │   ├── server.go          # MCP server (stdio and HTTP/SSE)
│   │   ├── tools.go           # MCP tool implementations
│   │   └── oauth/             # OAuth2 authentication for MCP
│   │       ├── config.go      # OAuth configuration
│   │       ├── middleware.go  # Bearer auth middleware
│   │       ├── metadata.go    # RFC 9728 metadata endpoint
│   │       └── verifier.go    # Token verification
│   ├── metrics/               # Prometheus metrics
│   │   └── metrics.go         # Metrics definitions and HTTP middleware
│   ├── parser/                # DMARC XML parser
│   │   ├── dmarc.go           # Parsing logic
│   │   └── dmarc_test.go      # Parser tests
│   └── storage/               # SQLite database layer
│       ├── common.go          # Shared SQL queries and types
│       ├── sqlite_cgo.go      # CGO SQLite (mattn/go-sqlite3)
│       └── sqlite_no_cgo.go   # Pure Go SQLite (modernc.org/sqlite)
├── src/                       # Vue.js 3 frontend source
│   ├── App.vue                # Main application component
│   ├── main.js                # Vue entry point
│   ├── assets/
│   │   └── base.css           # Base styles
│   ├── stores/                # Pinia-like state management
│   │   ├── index.js           # Store exports
│   │   ├── theme.js           # Theme (dark/light/system) store
│   │   └── settings.js        # API endpoint settings store
│   └── components/
│       ├── dashboard/
│       │   ├── DashboardHero.vue   # Dashboard header/hero section
│       │   ├── RecentReports.vue   # Recent reports list
│       │   └── ReportDrawer.vue    # Report detail drawer
│       ├── settings/
│       │   └── SettingsModal.vue   # Settings modal (theme, API endpoint)
│       └── tools/
│           └── DnsGenerator.vue    # DMARC DNS record generator
├── public/                    # Static frontend assets (favicons, logos)
├── assets/                    # Project assets (screenshots, images)
├── deploy/                    # Deployment configurations
│   ├── coolify.yaml           # Coolify deployment
│   ├── captain-definition     # CapRover deployment
│   ├── digitalocean/          # DigitalOcean Droplet/Marketplace
│   │   ├── packer.pkr.hcl     # Packer image build
│   │   ├── marketplace.yaml   # DO Marketplace metadata
│   │   └── scripts/           # Setup scripts
│   └── dokploy/               # Dokploy deployment
│       ├── template.toml
│       └── docker-compose.yml
├── grafana/                   # Grafana monitoring
│   ├── dashboard.json         # Pre-built dashboard
│   └── provisioning.yaml      # Auto-provisioning config
├── scripts/                   # Utility scripts
├── Justfile                   # Build commands
├── Dockerfile                 # Multi-stage Docker build
├── compose.yml                # Docker Compose for local dev
├── parse-dmarc.service        # systemd service file
├── zeabur.yml                 # Zeabur deployment template
├── render.yaml                # Render deployment config
├── Northflank.json            # Northflank deployment config
├── ROADMAP.md                 # Product roadmap
├── CONTRIBUTING.md            # Contribution guidelines
├── .goreleaser.yml            # Release automation
└── .github/workflows/ci.yml   # CI/CD pipeline

Development Commands

# Install all dependencies (Go + Node)
just install-deps

# Build full application (frontend + backend)
just build

# Build with CGO (for native SQLite)
just build-cgo

# Run development server with hot reload (uses air)
just dev

# Run frontend dev server only
just frontend-dev

# Run tests
just test

# Generate config file
just config

# Clean build artifacts
just clean

# Install binary to /usr/local/bin
just install

# Update Zeabur template
just update-zeabur-template

Building

The build process:

  1. Frontend is built with bun run build
  2. Frontend dist is copied to internal/api/dist/
  3. Go binary embeds the frontend and serves it
  4. Output binary: bin/parse-dmarc

Two build modes:

  • just build - Pure Go (CGO_ENABLED=0), uses modernc.org/sqlite
  • just build-cgo - CGO enabled, uses mattn/go-sqlite3

Testing

# Run all Go tests
go test -v ./...

# Run tests for specific package
go test -v ./internal/parser/...

Running the Application

Standard Mode (Dashboard + IMAP Fetching)

# Continuous fetching and dashboard
./parse-dmarc --config config.json

# Fetch once and exit
./parse-dmarc --config config.json --fetch-once

# Dashboard only (no IMAP fetching)
./parse-dmarc --config config.json --serve-only

MCP Mode (AI Assistant Integration)

# MCP over stdio (for Claude Desktop, etc.)
./parse-dmarc --mcp

# MCP over HTTP/SSE
./parse-dmarc --mcp-http :8081

# MCP with OAuth2 authentication
./parse-dmarc --mcp-http :8081 --mcp-oauth \
  --mcp-oauth-issuer https://auth.example.com \
  --mcp-oauth-audience https://mcp.example.com

CLI Flags

Flag Env Var Description
--config, -c PARSE_DMARC_CONFIG Config file path (default: config.json)
--gen-config PARSE_DMARC_GEN_CONFIG Generate sample config
--fetch-once PARSE_DMARC_FETCH_ONCE Fetch reports once and exit
--serve-only PARSE_DMARC_SERVE_ONLY Dashboard only, no fetching
--fetch-interval PARSE_DMARC_FETCH_INTERVAL Fetch interval in seconds (default: 300)
--metrics PARSE_DMARC_METRICS Enable Prometheus metrics (default: true)
--mcp PARSE_DMARC_MCP Run as MCP server over stdio
--mcp-http PARSE_DMARC_MCP_HTTP Run MCP over HTTP at address
--mcp-oauth PARSE_DMARC_MCP_OAUTH Enable OAuth2 for MCP HTTP
--mcp-oauth-issuer PARSE_DMARC_MCP_OAUTH_ISSUER OAuth2/OIDC issuer URL
--mcp-oauth-audience PARSE_DMARC_MCP_OAUTH_AUDIENCE Expected token audience
--mcp-oauth-client-id PARSE_DMARC_MCP_OAUTH_CLIENT_ID OAuth2 client ID for token introspection
--mcp-oauth-client-secret PARSE_DMARC_MCP_OAUTH_CLIENT_SECRET OAuth2 client secret for token introspection
--mcp-oauth-scopes PARSE_DMARC_MCP_OAUTH_SCOPES Required scopes (comma-separated, default: mcp:tools)
--mcp-oauth-introspection-endpoint PARSE_DMARC_MCP_OAUTH_INTROSPECTION_ENDPOINT Token introspection endpoint URL
--mcp-oauth-resource-name PARSE_DMARC_MCP_OAUTH_RESOURCE_NAME Human-readable name for MCP server metadata
--mcp-oauth-insecure PARSE_DMARC_MCP_OAUTH_INSECURE Skip TLS certificate verification (dev only)

Code Style

  • Go: Standard gofmt formatting, golangci-lint for linting
  • Frontend: Vue 3 Composition API, Prettier for formatting
  • Pre-commit hooks configured in .pre-commit-config.yaml

Key Files

Backend

  • main.go - CLI entry point with flag parsing, signal handling
  • internal/api/server.go - HTTP server, API routes, metrics middleware
  • internal/config/config.go - Configuration loading (JSON + env vars)
  • internal/parser/dmarc.go - DMARC XML parsing (gzip, zip, raw XML)
  • internal/imap/client.go - IMAP email fetching
  • internal/storage/common.go - SQL queries, data types
  • internal/mcp/server.go - MCP server implementation
  • internal/mcp/tools.go - MCP tool handlers
  • internal/metrics/metrics.go - Prometheus metrics definitions

Frontend

  • src/App.vue - Main Vue.js dashboard component
  • src/stores/theme.js - Theme state management (light/dark/system)
  • src/stores/settings.js - API endpoint settings management
  • src/components/dashboard/DashboardHero.vue - Statistics overview
  • src/components/dashboard/RecentReports.vue - Reports list
  • src/components/dashboard/ReportDrawer.vue - Report detail view
  • src/components/settings/SettingsModal.vue - Settings dialog (theme, API endpoint)
  • src/components/tools/DnsGenerator.vue - DMARC DNS record generator

API Endpoints

REST API

  • GET /api/statistics - Dashboard statistics
  • GET /api/reports - List reports (paginated: ?limit=50&offset=0)
  • GET /api/reports/:id - Single report details
  • GET /api/top-sources - Top sending source IPs

Metrics

  • GET /metrics - Prometheus metrics endpoint

MCP Tools

When running in MCP mode, the following tools are available:

Tool Description
get_statistics Overall DMARC compliance statistics
get_reports List reports with pagination
get_report_by_id Get detailed report by ID
get_top_source_ips Top sending IP addresses
get_domain_stats Per-domain compliance stats
get_org_stats Stats by reporting organization
get_spf_stats SPF authentication result stats
get_dkim_stats DKIM authentication result stats
parse_dmarc_report Parse raw DMARC XML (base64 encoded)

Prometheus Metrics

Key metrics exposed at /metrics:

Report Processing

  • parse_dmarc_reports_fetched_total - Reports fetched from IMAP
  • parse_dmarc_reports_parsed_total - Successfully parsed reports
  • parse_dmarc_reports_stored_total - Reports saved to database
  • parse_dmarc_reports_fetch_duration_seconds - Fetch operation duration

DMARC Statistics

  • parse_dmarc_dmarc_reports_total - Total reports in database
  • parse_dmarc_dmarc_messages_total - Total messages processed
  • parse_dmarc_dmarc_compliance_rate - Overall compliance rate
  • parse_dmarc_dmarc_messages_by_domain{domain} - Per-domain message count
  • parse_dmarc_dmarc_compliance_rate_by_domain{domain} - Per-domain compliance

HTTP Server

  • parse_dmarc_http_requests_total{method,path,status} - Request count
  • parse_dmarc_http_request_duration_seconds{method,path} - Request latency

Frontend Features

The Vue.js dashboard includes:

  • Dashboard Hero - Overview statistics with compliance rate
  • Recent Reports - Paginated list of DMARC reports
  • Report Drawer - Detailed view of individual reports
  • Top Sources - Visualization of top sending source IPs
  • DMARC DNS Generator - Interactive tool to generate DMARC DNS TXT records
  • Settings Modal - Theme switching (light/dark/system) and custom API endpoint configuration

Configuration

Config via JSON file or environment variables (using caarlos0/env):

{
  "imap": {
    "host": "imap.example.com",
    "port": 993,
    "username": "dmarc@example.com",
    "password": "your-password",
    "mailbox": "INBOX",
    "use_tls": true
  },
  "database": {
    "path": "~/.parse-dmarc/db.sqlite"
  },
  "server": {
    "host": "0.0.0.0",
    "port": 8080
  }
}

Environment variables: IMAP_HOST, IMAP_PORT, IMAP_USERNAME, IMAP_PASSWORD, IMAP_MAILBOX, IMAP_USE_TLS, DATABASE_PATH, SERVER_HOST, SERVER_PORT

Deployment Options

Docker

docker run -d -p 8080:8080 \
  -e IMAP_HOST=imap.example.com \
  -e IMAP_USERNAME=dmarc@example.com \
  -e IMAP_PASSWORD=secret \
  ghcr.io/meysam81/parse-dmarc:latest

Docker Compose

See compose.yml for a complete example with persistence.

Systemd

See parse-dmarc.service for systemd service configuration.

Platform-Specific

  • DigitalOcean: deploy/digitalocean/ - Packer template for Marketplace
  • Dokploy: deploy/dokploy/ - Docker Compose template
  • Coolify: deploy/coolify.yaml
  • CapRover: deploy/captain-definition
  • Zeabur: zeabur.yml - Zeabur platform template
  • Render: render.yaml - Render.com configuration
  • Northflank: Northflank.json - Northflank configuration

CI/CD

  • GitHub Actions workflow in .github/workflows/ci.yml
  • Prettier for code formatting (auto-commit in PRs)
  • golangci-lint for Go linting
  • Docker build with cosign signing
  • Kubescape security scanning
  • Release automation via release-please and goreleaser
  • Multi-platform Docker images (amd64, arm64)

Roadmap

See ROADMAP.md for the comprehensive product roadmap including:

  • Phase 1: Delightful Defaults (dark mode, DNS generator, health score, exports)
  • Phase 2: Proactive Intelligence (alerting, trends, GeoIP maps, DNS validator)
  • Phase 3: Enterprise Ready (auth, multi-org, RBAC, Prometheus metrics)
  • Phase 4: AI-Powered Security (AI assistant, anomaly detection, forensic reports)

Contributing

See CONTRIBUTING.md for development setup and contribution guidelines. Key areas:

  • Forensic Reports (RUF support)
  • OAuth2 for IMAP authentication
  • CSV/JSON export functionality
  • Email alerts for compliance issues
  • Historical trend analysis
  • Test coverage improvements

Architecture Notes

Database Schema

  • reports table: Stores report metadata and raw JSON
  • records table: Stores individual record data per report
  • Build tags (cgo/!cgo) select SQLite driver at compile time

Frontend Embedding

The Vue.js frontend is built to dist/, copied to internal/api/dist/, and embedded via Go's embed directive. The binary is self-contained.

State Management

The frontend uses a custom reactive store pattern (similar to Pinia):

  • theme.js - Manages light/dark/system theme with localStorage persistence
  • settings.js - Manages custom API endpoint with validation and connection testing

MCP Integration

The MCP server uses the official modelcontextprotocol/go-sdk. It supports:

  • stdio transport: For desktop apps like Claude Desktop
  • HTTP/SSE transport: For web-based MCP clients
  • OAuth2: Optional authentication via OIDC/token introspection