Naglfar Analytics

The ship made of dead men's nails - A Norse mythology reference representing collection and analysis of threat data for abuse protection.

Overview

Naglfar Analytics is an abuse protection system built with .NET 10.0 that acts as a defensive layer between your API gateway and backend services. It analyzes incoming requests, detects malicious patterns, and blocks abusive traffic before it reaches your core application.

Current Status: Monorepo structure established. Architecture and design phase with comprehensive documentation and diagrams.

Repository Structure: This is a monorepo containing multiple microservices including the naglfar-validation service (.NET 10.0), with planned additions for worker services, authentication, and protected demo applications.

Features

Core Functionality

• ✅ RESTful API with .NET 10.0 Minimal APIs
• ✅ API Versioning (URL, query string, and header-based)
• ✅ Health check endpoints (/healthz, /readyz) for Kubernetes
• ✅ Prometheus metrics endpoint (/metrics)
• ✅ Swagger/OpenAPI documentation

Infrastructure

• ✅ Traefik API Gateway (v3.6) integration
• ✅ Docker support with multi-stage Alpine builds
• ✅ Docker Compose orchestration with custom networking
• ✅ Makefile automation (17+ commands)

Quality & Documentation

• ✅ Comprehensive integration tests (10 tests, all passing)
• ✅ Architecture diagrams (9 diagrams with automated generation)
• ✅ Detailed system design and architecture documentation
• ✅ Automated diagram validation and regeneration

Documentation

📚 Comprehensive documentation available:

• System Design - High-level architecture and design philosophy
• Naglfar Layer Architecture - Detailed component architecture with diagrams
• API Endpoints - Quick reference for all endpoints
• Architecture Diagrams - Diagram generation workflow and usage
• CHANGELOG - Complete version history and changes

Prerequisites

• .NET 10.0 SDK or later
• Docker (required for diagram generation and containerized deployment)
• Docker Compose (for multi-service orchestration)

Quick Start

Local Development (without Docker)

# Restore dependencies
make restore

# Build the application
make build

# Run the application
make run

The application will be available at http://localhost:5000

Using Docker

# Build Docker image
make docker-build

# Run in Docker container
make docker-run

The application will be available at http://localhost:8080

Using Docker Compose

# Start all services (API + Traefik)
make compose-up

# View logs
make compose-logs

# Stop all services
make compose-down

Testing Endpoints

Direct Access (Port 8000)

# Health check
curl http://localhost:8000/healthz

# Readiness check
curl http://localhost:8000/readyz

# API info (versioned)
curl http://localhost:8000/api/v1/info

# Prometheus metrics
curl http://localhost:8000/metrics

Via Traefik (Port 80)

# Access API through Traefik
curl -H "Host: api.local" http://localhost/healthz
curl -H "Host: api.local" http://localhost/api/v1/info

Makefile Commands

Run make help to see all available commands with descriptions.

Local Development

• make restore - Restore .NET dependencies
• make build - Build the application
• make run - Run the application locally
• make test - Run all tests
• make test-watch - Run tests in watch mode
• make test-coverage - Run tests with code coverage
• make clean - Clean build artifacts

Docker Compose

• make compose-up - Start all services with docker-compose
• make compose-down - Stop and remove all services
• make compose-logs - Show logs for all services
• make validation-rebuild - Rebuild only the naglfar-validation service
• make apigw-restart - Rebuild and restart Traefik API Gateway

Diagram Generation

• make diagrams - Generate all SVG diagrams from Mermaid sources
• make diagrams-validate - Validate all diagram syntax
• make diagrams-clean - Remove generated SVG files
• make diagrams-check - Check if Docker is available

Service-Specific Commands

The project uses modular helper makefiles for service-specific commands. Each service maintains its own helpers.mk file with build and deployment commands.

Book Store Service:

• make docker-build-book-store - Build book-store Docker image
• make docker-run-book-store - Run book-store container (port 8090)
• make lock-dependencies-book-store - Generate Pipfile.lock using Docker (no local Python/pipenv needed)
• make compose-rebuild-book-store - Rebuild book-store via docker-compose

Naglfar Validation Service:

• make docker-build-naglfar - Build naglfar-validation Docker image
• make docker-run-naglfar - Run naglfar-validation container
• make docker-stop-naglfar - Stop and remove container
• make docker-clean-naglfar - Remove Docker image

Benefits of Modular Makefiles:

• Each service maintains its own build commands in services/<service>/helpers.mk
• Helper makefiles are automatically included by the root Makefile
• Services know their own directory location (location-aware)
• Easy to add new services without modifying the root Makefile
• Run make help to see all available commands from all services

Testing

The project includes three comprehensive testing frameworks:

End-to-End Testing (Python CLI)

Python-based CLI tool for testing user journeys through the system.

# Run all E2E tests
make e2e-all

# Run individual tests
make e2e-browse        # Browse books journey
make e2e-purchase      # Purchase book journey
make e2e-full-flow     # Complete user flow

# View results
make e2e-results

📖 Documentation: testing/e2e/README.md

Performance Testing (k6)

Load and stress testing using Grafana k6.

# Run all performance tests
make perf-all

# Run individual tests
make perf-browse        # Browse books load test
make perf-full-flow     # Full user flow test
make perf-stress        # Stress test (up to 300 VUs)

# View and compare results
make perf-results
make perf-compare

📖 Documentation: testing/performance/README.md

Capacity Testing (Gatling)

YAML-driven capacity testing with Gatling for defining test scenarios without code.

# Run all capacity tests
make capacity-all

# Run individual tests
make capacity-browse        # Browse capacity test
make capacity-full-flow     # Full flow capacity test
make capacity-stress        # System stress test

# View results and reports
make capacity-results
make capacity-report        # Open HTML report

Key Features:

• Define test scenarios in YAML files (no Scala code needed)
• Flexible load injection profiles (ramp, constant, spike)
• Built-in authentication flow support (E-TOKEN/AUTH-TOKEN)
• Rich HTML reports with charts and metrics
• Performance thresholds and assertions

📖 Documentation: testing/capacity/README.md

Example YAML Scenario:

name: "Browse Books Test"
injection:
  - type: rampUsers
    users: 10
    duration: 30s
scenarios:
  - name: "Browse Journey"
    steps:
      - http:
          method: GET
          path: "/api/books"
          checks:
            - status: 200

Project Structure

naglfar-analytics/                          # Monorepo root
├── services/                               # Microservices directory
│   ├── book-store/                         # Book store service (Python FastAPI)
│   │   ├── src/                            # Source code
│   │   ├── Pipfile                         # Python dependencies
│   │   ├── Pipfile.lock                    # Locked dependencies
│   │   ├── Dockerfile                      # Multi-stage Docker build
│   │   └── helpers.mk                      # Service-specific Makefile commands
│   │
│   └── naglfar-validation/                 # Validation service (.NET 10.0)
│       ├── src/
│       │   └── NaglfartAnalytics/
│       │       ├── Program.cs              # Application entry point
│       │       ├── NaglfartAnalytics.csproj # .NET 10.0 project file
│       │       ├── appsettings.json        # Production configuration
│       │       └── appsettings.Development.json # Development configuration
│       ├── tests/
│       │   └── NaglfartAnalytics.Tests/
│       │       ├── IntegrationTests.cs     # Integration tests (9 tests)
│       │       ├── MetricsTests.cs         # Metrics endpoint tests (1 test)
│       │       └── NaglfartAnalytics.Tests.csproj # Test project file
│       ├── Dockerfile                      # Multi-stage Docker build (Alpine)
│       └── helpers.mk                      # Service-specific Makefile commands
│
├── infrastructure/                         # Infrastructure configuration
│   ├── docker-compose.yml                  # Service orchestration (API + Traefik)
│   ├── helpers.mk                          # Infrastructure Makefile commands
│   ├── traefik/                            # API Gateway configuration (future)
│   ├── kafka/                              # Message broker configuration (future)
│   ├── neo4j/                              # Graph database configuration (future)
│   └── prometheus/                         # Monitoring configuration (future)
│
├── shared/                                 # Shared libraries (future)
│   ├── dotnet/                             # Shared .NET libraries
│   └── python/                             # Shared Python libraries
│
├── testing/                                # Testing frameworks
│   ├── e2e/                                # End-to-end tests (Python CLI)
│   ├── performance/                        # Performance tests (k6)
│   └── capacity/                           # Capacity tests (Gatling + YAML)
│
├── scripts/                                # Automation scripts (future)
│   └── ...
│
├── docs/
│   ├── system-design.md                    # High-level system architecture
│   ├── naglfar-layer-architecture.md       # Detailed component architecture
│   ├── endpoints.md                        # API endpoint reference
│   └── assets/
│       └── diagrams/
│           ├── README.md                   # Diagram workflow documentation
│           ├── *.mmd                       # Mermaid source files (9 diagrams)
│           └── *.svg                       # Generated SVG diagrams
│
├── Makefile                                # Build automation (17+ commands)
├── CHANGELOG.md                            # Complete version history
└── README.md                               # This file

Development

The application uses .NET 10.0 Minimal APIs for a lightweight and performant web service with API versioning.

Adding New Endpoints

Edit services/naglfar-validation/src/NaglfartAnalytics/Program.cs to add new versioned endpoints:

// Add to v1 API group
v1Group.MapGet("/your-endpoint", () => Results.Ok(new { message = "Hello" }))
    .WithName("YourEndpoint")
    .WithDescription("Your endpoint description");

Configuration

Application settings can be configured in:

• appsettings.json - Production settings
• appsettings.Development.json - Development settings
• Environment variables (prefixed with ASPNETCORE_)

Health Checks

The application provides two types of health checks:

1. Health Check (/healthz): Indicates if the application is running and healthy
2. Readiness Check (/readyz): Indicates if the application is ready to accept traffic

These endpoints are commonly used by:

• Kubernetes liveness and readiness probes
• Load balancers
• Monitoring systems
• Container orchestrators

Resources

• Reference Compose
• Traefik Expose Services
• Traefik Examples
• Traefik Routes
• Mermaid diagrams cli
• Diagrams as code
• Dapr
• Redis
• Redis mod
• Neo4J Driver
• Neo4j examples
• Access Neptune

License

This project is part of the ik-workshop organization.

Contributing

See CHANGELOG.md for version history and changes.