Go Clean Boilerplate - Enterprise Modular REST API

Enterprise-ready Go boilerplate implementing Clean Architecture, RBAC with Casbin, Modular Audit Logging, and Distributed WebSocket scaling.

---

🚀 Core Features

• Clean Architecture: Strict separation of concerns (Delivery, UseCase, Repository, Entity).
• Advanced RBAC with Casbin: Fine-grained access control using GORM adapter. Policies are stored in the database for dynamic updates.
• Multi-Tenancy (Organization Module): Complete organization-based data isolation with member management, tenant middleware, and GORM scopes for secure multi-tenant operations.
• Distributed WebSockets: Scalable WebSocket management using Redis Pub/Sub backplane, allowing multi-node synchronization.
• Modular Audit Logging: Synchronous activity tracking (LOGIN, REGISTER, UPDATE, DELETE) integrated directly into business UseCases.
• Multi-Provider File Storage: Pluggable storage abstraction supporting Local Disk, AWS S3, MinIO, and Cloudflare R2.
• Automated Cleanup Jobs: Integrated background worker scheduler for database maintenance (token pruning, soft-delete cleanup, log rotation).
• Distributed Tracing (OTEL): Full visibility with OpenTelemetry integration, tracking request flow across HTTP, Database, and Workers.
• Dynamic Search & Filtering: Secure, reusable query builder supporting complex clauses, range filters, and dynamic sorting.
• Secure Authentication: JWT-based auth with stateful session management in Redis for instant token revocation.
• Real-time SSE: Server-Sent Events manager for live one-way data streaming.
• Hardened Security:
  • UseCase-level validation (Regex email, password strength).
  • Automatic HTTP security headers.
  • Go 1.25.5 for critical security fixes.
• Comprehensive Testing:
  • Unit Tests: Fast, mock-based verification of logic.
  • Integration Tests: Lightweight testing using Singleton Testcontainers pattern.
  • E2E Tests: Full HTTP lifecycle validation.

---

🎛️ Toggleable Features & Configuration

This project is designed with high flexibility. Many core features can be enabled/disabled via environment variables (.env).

Core Features

Feature	Env Variable	Default	Description
RBAC Authorization	CASBIN_ENABLED	false	Enables Casbin authorization checks. If false, authorization is bypassed.
Casbin Sync	CASBIN_WATCHER_ENABLED	false	Enables policy sync across instances via Redis. Required for multi-replica setups.
Rate Limiter	RATE_LIMIT_ENABLED	true	Limits requests per second (RPS) per IP to prevent DoS/Brute Force.
Distributed WS	WEBSOCKET_DISTRIBUTED_ENABLED	false	Enables WebSocket message sync via Redis Pub/Sub. Required for horizontal scaling.

Security & Network

Configuration	Env Variable	Default	Description
Trusted Proxies	SERVER_TRUSTED_PROXIES	Empty	Comma-separated list of trusted Load Balancer IPs/CIDRs.
CORS Origins	CORS_ALLOWED_ORIGINS	*	Allowed domains for CORS.
JWT Secrets	JWT_ACCESS_SECRET JWT_REFRESH_SECRET	-	Critical: Must be random strings (min 32 chars).

Telemetry & Observability

Configuration	Env Variable	Default	Description
OTEL Tracing	OTEL_ENABLED	false	Enables OpenTelemetry tracing.
OTEL Service	OTEL_SERVICE_NAME	queue-base-api	Service name shown in Jaeger/Tempo.
Collector URL	OTEL_COLLECTOR_URL	localhost:4317	OTLP gRPC collector endpoint (e.g. Jaeger).

Storage

Configuration	Env Variable	Default	Description
Driver	STORAGE_DRIVER	local	Storage strategy: local or s3.
Root Path	STORAGE_LOCAL_ROOT_PATH	./uploads	Local directory for file storage.
S3 Endpoint	STORAGE_S3_ENDPOINT	-	Custom S3 endpoint (required for MinIO/R2).

Performance

Configuration	Env Variable	Default	Description
Rate Limit Store	RATE_LIMIT_STORE	memory	Counter storage: memory (single instance) or redis (distributed).
WS Ping Period	WEBSOCKET_PING_PERIOD	Auto	Keep-alive ping interval (default: 90% of Pong Wait).

---

📦 Deployment Scenarios

Choose the configuration that matches your infrastructure scale.

1. Single Instance (Monolith)

Suitable for development, small VPS, or simple deployments.

# No need for distributed sync
RATE_LIMIT_STORE=memory
WEBSOCKET_DISTRIBUTED_ENABLED=false
CASBIN_WATCHER_ENABLED=false

2. Distributed Cluster (Kubernetes/Load Balanced)

Suitable for high-availability setups with multiple API replicas. Requires a shared Redis instance.

# Sync state via Redis
RATE_LIMIT_STORE=redis
WEBSOCKET_DISTRIBUTED_ENABLED=true
CASBIN_WATCHER_ENABLED=true

# Security behind Load Balancer
SERVER_TRUSTED_PROXIES=10.0.0.0/8,172.16.0.0/12  # IPs of your LB/Ingress

---

🛠️ Technology Stack

Category	Technology	Description
Language	Go 1.25.5+	Core programming language
Framework	Gin Gonic	High-performance HTTP framework
Database	MySQL 8.0	Primary relational database
Cache/Session	Redis 7	Session storage & WS Pub/Sub backplane
Authorization	Casbin	RBAC model & Policy enforcement
Migrations	golang-migrate	Database schema management
Testing	Testcontainers	Real instances for integration tests

---

🏁 Getting Started

Prerequisites

1. Go: Version 1.25.5 or higher.
2. Docker & Docker Compose: For running MySQL and Redis services easily.
3. Make: For running automation commands defined in Makefile.
4. Air (Optional): For live reloading during development.

   go install github.com/air-verse/air@latest

5. Swag CLI (Optional): For regenerating API docs.

   go install github.com/swaggo/swag/cmd/swag@latest

6. Golang Migrate (Optional): If you want to run migrations manually without the Makefile helper.

Installation

1. Clone & Configure:

   git clone https://github.com/Roisfaozi/queue-base.git
   cd queue-base
   cp .env.example .env.local

2. Recommended Branch Model:
   • main = production-ready
   • staging = release candidate / pre-release integration
   • dev = daily integration branch
3. Create Worktree For Daily Development:

   make wt-new feat/my-feature

   Notes:
   • branch name can be passed directly after wt-new
   • base branch defaults to current checked out branch
   • optional arg kedua override base branch, contoh make wt-new feat/my-feature staging
   • default worktree root is .worktrees/ inside repo
   • each worktree gets its own .env.local
   • if you prefer sibling folders, override WORKTREE_ROOT
4. Start Infrastructure:

   make dev-up

   Auto behavior:
   • initializes .env.local when missing
   • syncs missing keys from .env.example
   • uses worktree-specific compose project name and ports
5. Run Migrations & Seeding:

   make migrate-up-local
   make seed-up

6. Run Application:

   make run

Legacy Non-Worktree Flow

If you want old single-checkout flow:

cp .env.example .env.local
make docker-dev
make migrate-up
make run

---

🌿 Worktree Development Flow

This repo supports worktree-based parallel development.

Primary use cases:

• split frontend and caller logic into separate branches
• run multiple feature streams from dev
• isolate docker ports and local env per branch
• avoid branch switching churn in one checkout

Default Behavior

• default worktree root is .worktrees/ inside repo
• each worktree gets branch-specific .env.local
• compose project name is unique per worktree
• local ports are derived per worktree slug
• wt-new, wt-enter, dev-up, and dev-status auto-manage env setup

Command Reference

Command	Function	Main Use
make wt-new feat/x [base]	Create new git worktree and bootstrap env	Start new feature stream from current branch or explicit base
make wt-list	List all git worktrees	Inspect active worktrees
make wt-path feat/x	Print worktree path for branch	Quick path lookup
make wt-enter feat/x	Ensure env for target worktree and print path	Re-open existing worktree safely
make wt-rm feat/x	Stop local stack if needed and remove worktree	Clean finished feature stream
make wt-prune	Prune stale worktree metadata	Cleanup broken or removed entries
make env-init	Create .env.local and assign isolated ports	Bootstrap env in current worktree
make env-sync	Append missing keys from .env.example	Keep local env aligned after template changes
make dev-up	Start docker compose stack for current worktree	Daily local development start
make dev-down	Stop docker compose stack for current worktree	Stop only current branch stack
make dev-reset	Stop stack and remove local volumes	Reset local DB/cache state
make dev-status	Show branch, compose project, ports, and container state	Debug local worktree environment
make migrate-up-local	Run DB migrations against current .env.local	Prepare schema for current worktree
make migrate-down-local	Roll back one migration against current .env.local	Local rollback
make test-local	Run narrow local tests	Fast loop in current worktree
make doctor	Check git, docker, pnpm, go, and env state	Validate development readiness

Example Parallel Streams

• feat/frontend-dashboard
  • focus: apps/web, apps/client, packages/*
• feat/caller-runtime
  • focus: counter/station runtime, queue serving, realtime
• feat/queue-core
  • focus: queue lifecycle, queue journeys, domain rules

Example Daily Flow

make wt-new feat/frontend-dashboard
cd .worktrees/feat-frontend-dashboard
make dev-up
make migrate-up-local
make test-local

Override base branch:

make wt-new feat/caller-runtime staging
cd .worktrees/feat-caller-runtime
make dev-up

For existing worktree:

make wt-enter feat/frontend-dashboard
cd .worktrees/feat-frontend-dashboard
make dev-up

To use sibling worktree root instead of .worktrees/:

make wt-new feat/frontend-dashboard

---

🧪 Testing Strategy

We use a layered testing strategy optimized for both speed and reliability.

Command	Type	Description
make test-unit	Unit	Runs mock-based tests for internal/pkg logic.
make test-integration	Integration	Uses Singleton Containers for DB/Redis logic.
make test-e2e	E2E	Validates full HTTP request/response flows.
make test-all	Full Suite	Executes all test layers sequentially.
make test-coverage	Coverage	Generates an interactive HTML coverage report.

Note: Integration and E2E tests require Docker. We use a Singleton Container Pattern to reuse a single database/redis instance across the entire suite, drastically reducing execution time and resource usage.

---

📂 Project Structure

The project follows a standard Go project layout suitable for scalable microservices or monolithic APIs.

.
├── .air.toml           # Configuration for Air (live reloading)
├── Makefile            # Automation commands (build, test, migrate, run, mocks)
├── README.md           # Main project documentation
├── docker-compose.yml  # Docker services definition (MySQL, Redis)
├── go.mod              # Go dependency definitions
│
├── apps/
│   ├── web/            # Next.js Frontend (Legacy)
│   └── client/         # React Router 7 Frontend (New)
│
├── packages/
│   └── ui/             # Shared React UI components (@casbin/ui)
│
├── cmd/
│   └── api/            # Application entry point (main.go)
│
├── db/
│   ├── migrations/     # Database schema migration files (.sql)
│   └── seeds/          # Initial data seeding scripts (e.g. bootstrapping)
│
├── docs/               # Auto-generated Swagger/OpenAPI documentation files
│
├── documentation/      # Project guides and additional documentation
│   ├── architecture/   # System design and architecture blueprints
│   ├── guides/         # Developer guides (API, Storage, Testing, SSE/WS, etc.)
│   ├── ops/            # Operations runbooks and project roadmaps
│   └── productplan/    # PRDs, wireframes, and UI specs
│
├── postman/            # Postman collections for testing
│   ├── Casbin Project API.postman_collection.json         # Main collection
│   ├── Casbin Project API - Dynamic Search.postman_collection.json # Dynamic search tests
│   ├── Casbin Project API - Realtime.postman_collection.json # Realtime features (WS, SSE)
│   └── ...
│
└───internal/           # Private application code (not importable by other apps)
    ├── config/         # Configuration loading & app initialization wiring
    ├── middleware/     # HTTP Middlewares (Auth, Casbin Enforcer, CORS, OTEL)
    ├── router/         # Gin router setup and route registration
    ├── worker/         # Background tasks, handlers & scheduler
    │
    └── modules/        # Domain-specific modules following Clean Architecture
        ├── auth/       # Authentication logic & JWT handling
        ├── user/       # User management (CRUD) & Avatar Upload
        ├── role/       # Role management
        ├── permission/ # Permission/Policy management (Casbin)
        └── access/     # Access Right & Endpoint management

📜 Documentation Links

• Documentation Index
• System Architecture
• Developer Flow
• Getting Started
• API Usage Guide
• API Access & RBAC
• Multi-Tenancy Architecture
• Testing Strategy
• Real-time (WS & SSE)
• Dynamic Search
• Multi-Provider Storage
• Observability (Tracing/Metrics)
• Maintenance & Scheduler
• Frontend Structure

---

📄 License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.