API Gateway & Load Balancer

Production-grade distributed API gateway and HTTP reverse proxy written in Go. This project demonstrates clean architecture, idiomatic Go patterns, concurrency-safe design, and production-ready features including multiple load balancing strategies, health checks, rate limiting, JWT authentication, dynamic configuration with hot reload, and Prometheus metrics.

Why This Project Matters

This gateway implements a production-style distributed API gateway that fronts multiple backend services. It features pluggable load-balancing strategies (round-robin, least-connections, weighted round-robin, consistent hashing), active health checking with automatic failover, middleware-based cross-cutting concerns (logging, rate limiting, JWT auth), dynamic configuration with hot reload, an admin API for live backend management, and first-class observability via structured logs and Prometheus metrics. Designed to showcase idiomatic Go, clean architecture, concurrency-safe design, and DevOps readiness (Docker, docker-compose, CI) in a single, cohesive codebase.

Features

• Load Balancing: Round-robin, least-connections, weighted round-robin, and consistent hashing strategies
• Health Checks: Automatic backend health monitoring with failover
• Configuration: YAML-based config with hot reload (no restart required)
• Middleware Architecture: Request ID, structured logging, per-IP rate limiting (token bucket), JWT authentication
• Admin API: REST API for dynamic backend management at runtime
• Observability: Prometheus-compatible /metrics endpoint and structured logging
• Production Ready: Graceful shutdown, Docker support, CI/CD integration

Highlights for Reviewers

• Reverse Proxy & Load Balancing:
  • internal/proxy/gateway.go - Core HTTP reverse proxy with integration to balancer, health checker, and dynamic config
  • internal/balancer/balancer.go - Four load balancing algorithms implemented with thread-safe backends
• Resilience & Health:
  • internal/health/health.go - Periodic HTTP health checks with automatic backend liveness tracking
• Cross-Cutting Concerns:
  • internal/middleware/middleware.go - Request ID, structured logging, per-IP rate limiting, JWT authentication
  • internal/auth/jwt.go - JWT validation with HMAC signing
  • internal/ratelimit/ratelimit.go - Concurrency-safe token bucket rate limiter per key (IP)
• Configuration & Admin:
  • internal/config/config.go - YAML config model with hot-reload file watcher
  • internal/adminapi/admin.go - Admin REST API for listing/adding/removing backends at runtime
• Observability & Quality:
  • internal/metrics/metrics.go - Prometheus text-format metrics implementation
  • test/ - Unit tests for balancers, rate limiter, middleware, and admin API
  • .github/workflows/ci.yml - GitHub Actions CI running go test ./...

Tech Stack & Patterns

• Language: Go (standard library first, minimal external dependencies)
• HTTP: net/http, httputil.ReverseProxy
• Load Balancing: Multiple algorithms (RR, least-connections, weighted RR, consistent hashing)
• Concurrency: sync.Mutex, sync.RWMutex, atomic operations for thread-safe state
• Configuration: YAML parsing with hot reload via file watcher
• Observability: Prometheus metrics format, structured logging
• DevOps: Docker, docker-compose, GitHub Actions CI
• Architecture: Clean architecture with internal/pkg separation, middleware pattern

Quickstart

Local (without Docker)

# Clone the repository
git clone https://github.com/zenhs/api-gateway.git
cd api-gateway

# Run the gateway
make run
# or
go run ./cmd/gateway

Gateway listens on :8080 by default and reads configuration from config.yaml.

Docker / Compose

# Start gateway + sample backends
make docker-run
# or
docker-compose up --build

This starts:

• backend1 on :8081
• backend2 on :8082
• gateway on :8080, proxying to the backends

Walkthrough

Assuming the gateway is running on localhost:8080:

1. Basic Request & Load Balancing

# Make requests - they'll be distributed across backends
curl http://localhost:8080/
curl http://localhost:8080/
curl http://localhost:8080/

2. View Metrics

# Prometheus-compatible metrics endpoint
curl http://localhost:8080/metrics

Example output:

# TYPE http_requests_total counter
http_requests_total{code="200",method="GET"} 3
# TYPE http_request_duration_seconds summary
http_request_duration_seconds_sum{method="GET"} 0.015
http_request_duration_seconds_count{method="GET"} 3

3. Admin API - List Backends

# Get current backend state
curl http://localhost:8080/admin/backends | jq

Example output:

[
  {
    "name": "backend1",
    "url": "http://backend1:8081",
    "alive": true,
    "weight": 1,
    "conns": 5
  },
  {
    "name": "backend2",
    "url": "http://backend2:8082",
    "alive": true,
    "weight": 1,
    "conns": 3
  }
]

4. Admin API - Add Backend Dynamically

# Add a new backend at runtime (no restart needed)
curl -X POST http://localhost:8080/admin/backends/add \
  -H "Content-Type: application/json" \
  -d '{"name": "backend3", "url": "http://backend3:8083", "weight": 2}'

5. Admin API - Remove Backend

# Remove a backend at runtime
curl -X POST http://localhost:8080/admin/backends/remove \
  -H "Content-Type: application/json" \
  -d '{"name": "backend2"}'

6. Rate Limiting

# Make rapid requests - after burst limit, you'll get 429
for i in {1..25}; do curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8080/; done

7. JWT Authentication

First, enable JWT in config.yaml:

auth:
  enable_jwt: true
  jwt_secret: "your-secret-key"

Then test:

# Without token - 401 Unauthorized
curl http://localhost:8080/

# With valid JWT token
curl -H "Authorization: Bearer YOUR_JWT_TOKEN" http://localhost:8080/

Configuration

See config.yaml for configuration options:

server:
  listen_address: ":8080"

upstream:
  name: "main"
  strategy: "round_robin"  # round_robin, least_connections, weighted_round_robin, consistent_hashing
  health_url: "/health"
  backends:
    - name: "backend1"
      url: "http://backend1:8081"
      weight: 1
      enabled: true
    - name: "backend2"
      url: "http://backend2:8082"
      weight: 1
      enabled: true

rate_limit:
  requests_per_second: 10
  burst: 20

auth:
  enable_jwt: false
  jwt_secret: "change-me-in-production"

health:
  interval_seconds: 5
  timeout_seconds: 2

Configuration is hot-reloaded automatically when config.yaml changes (checked every 3 seconds).

Architecture

• Entrypoint: cmd/gateway/main.go wires config, proxy, middleware, metrics, and admin API
• Core Proxy: internal/proxy/gateway.go - HTTP reverse proxy with load balancer integration
• Load Balancing: internal/balancer/balancer.go - Multiple balancing strategies
• Health Checks: internal/health/health.go - Periodic backend health monitoring
• Middleware: internal/middleware/middleware.go - Request ID, logging, rate limiting, JWT auth
• Configuration: internal/config/config.go - YAML config with hot reload
• Admin API: internal/adminapi/admin.go - REST API for backend management
• Observability: internal/metrics/metrics.go - Prometheus metrics, pkg/logger - Structured logging

Testing

# Run all tests
go test ./...

# Run tests with race detector
go test -race ./...

# Run tests with coverage
go test -cover ./...

Project Structure

api-gateway/
├── cmd/gateway/main.go          # Application entrypoint
├── internal/
│   ├── config/                  # YAML config & hot reload
│   ├── proxy/                   # Reverse proxy & gateway
│   ├── balancer/                # Load balancing strategies
│   ├── health/                  # Backend health checks
│   ├── ratelimit/               # Token bucket rate limiter
│   ├── auth/                    # JWT authentication
│   ├── middleware/              # HTTP middleware chain
│   ├── metrics/                 # Prometheus metrics
│   └── adminapi/                # Admin REST API
├── pkg/
│   ├── logger/                  # Structured logger
│   └── utils/                   # Utility functions
├── test/                        # Unit tests
├── deployments/                 # Docker & compose files
├── Dockerfile                   # Production Dockerfile
├── docker-compose.yml           # Local development setup
├── Makefile                     # Build & run commands
├── config.yaml                  # Configuration file
└── README.md                    # This file

Contributing

See CONTRIBUTING.md for development guidelines.

License

MIT