Skip to content

matutetandil/mycel

Repository files navigation

Mycel

CI Go Version Release License: MIT Docker

Declarative microservices through configuration, not code.

Define HCL2 files. Run Mycel. Get a production-ready microservice.

How It Works

Mycel is a single binary runtime (like nginx). Same binary, different configuration = different microservice.

Mycel has two core building blocks: connectors and flows. Everything else builds on top of them.

A connector is anything Mycel can talk to — a database, a REST API, a message queue, a gRPC service, a file system. Every connector is bidirectional: it can be a source (receives data that triggers a flow) or a target (destination where a flow writes data).

A flow wires two connectors together, moving data from one to the other:

Connector (source) ──→ Flow ──→ Connector (target)

On top of this, you can add transforms (reshape data), types (validate schemas), steps (multi-step orchestration), sagas (distributed transactions), auth, aspects, security, and more. But every feature ultimately serves the same pattern: data enters through a connector, optionally gets transformed, and exits through another connector.

Every Mycel service automatically includes health checks (/health, /health/live, /health/ready), Prometheus metrics (/metrics), and hot reload — no configuration needed. Change a .mycel file and the service reloads with zero downtime.

That's the whole model. Everything else is configuration. Learn more in Core Concepts.

Quick Start

Create a directory with three .mycel files — that's your entire microservice:

mkdir my-api && cd my-api

config.mycel — Name and version your service:

service {
  name    = "users-api"
  version = "1.0.0"
}

connectors.mycel — Define what your service talks to:

connector "api" {
  type = "rest"
  port = 3000
}

connector "db" {
  type     = "database"
  driver   = "sqlite"
  database = "./data/app.db"
}

flows.mycel — Wire them together:

flow "list_users" {
  from {
    connector = "api"
    operation = "GET /users"
  }
  to {
    connector = "db"
    target    = "users"
  }
}

flow "create_user" {
  from {
    connector = "api"
    operation = "POST /users"
  }
  transform {
    id         = "uuid()"
    email      = "lower(input.email)"
    created_at = "now()"
  }
  to {
    connector = "db"
    target    = "users"
  }
}

Now run it — Mycel reads the directory and starts the service:

docker run -v $(pwd):/etc/mycel -p 3000:3000 ghcr.io/matutetandil/mycel

Test it:

curl -X POST http://localhost:3000/users \
  -H "Content-Type: application/json" \
  -d '{"email":"test@example.com","name":"Test"}'

curl http://localhost:3000/users

That's it. REST API + database, zero code.

See the Quick Start Guide for a complete tutorial, or explore the full documentation.

Purpose

  • What: An open-source runtime that reads HCL configuration and exposes microservices. Same binary, different config = different service.
  • Why: Most microservice code is plumbing — routing, database queries, data transformations, protocol translation, error handling, retries. It's the same patterns repeated across every service, in every team, in every company. Mycel extracts that into configuration so teams can focus on what's actually unique to their service.
  • Who: Backend teams building CRUD APIs, event processors, integrations, or protocol bridges.

Features

Feature Description
REST API Expose and consume REST endpoints
SQLite / PostgreSQL / MySQL Relational database connectors
MongoDB NoSQL document database
GraphQL Server & Client Schema-based GraphQL API
GraphQL Query Optimization Field selection, step skipping, DataLoader
GraphQL Federation Federation v2, entity resolution, gateway-compatible subgraphs (docs)
GraphQL Subscriptions Real-time push via WebSocket, per-user filtering (docs)
GraphQL Subscription Client Subscribe to external GraphQL events via WebSocket (docs)
gRPC Server & Client Protocol Buffers based RPC
gRPC Load Balancing Round-robin and weighted balancing
RabbitMQ / Kafka / Redis Pub/Sub Message queue producers and consumers
MQTT IoT messaging protocol (QoS 0/1/2, TLS, auto-reconnect) (docs)
FTP / SFTP Remote file transfer (FTP, FTPS, SFTP with key auth) (docs)
WebSocket Bidirectional real-time communication with rooms and per-user targeting (docs)
CDC (Change Data Capture) Real-time database change streaming with wildcard matching (docs)
SSE (Server-Sent Events) Unidirectional HTTP push with rooms and per-user targeting (docs)
Elasticsearch Full-text search and analytics over Elasticsearch REST API (docs)
OAuth (Social Login) Declarative social login: Google, GitHub, Apple, OIDC, custom (docs)
Batch Processing Chunked data processing for migrations, ETL, reindexing (docs)
Sagas Distributed transactions with automatic compensation, delay/await steps, workflow persistence (docs)
State Machines Entity lifecycle with guards, actions, final states (docs)
Long-Running Workflows Persistent workflows with delay timers, await/signal events, timeout enforcement, REST API (docs)
SOAP Call or expose SOAP/XML web services (SOAP 1.1/1.2) (docs)
Format Declarations Multi-format support (JSON, XML) at connector, flow, and step level (docs)
TCP Server & Client JSON, msgpack, and NestJS protocols
Files / S3 Local filesystem and AWS S3 / MinIO
Cache (Memory / Redis) In-memory and Redis caching (docs)
Multi-step Flow Orchestration Sequential and conditional step execution (docs)
Accept Gate Business-level message routing with on_reject policy (ack/reject/requeue) (docs)
Source Fan-Out Multiple flows from the same connector+operation, concurrent execution (docs)
Named Operations Reusable parameterized operations
Data Enrichment Combine data from multiple sources
Auth (JWT, MFA, WebAuthn) Authentication with presets and MFA (docs)
Rate Limiting / Circuit Breaker Traffic control and fault tolerance
Connector Profiles Multiple backends with fallback
Read Replicas Route reads to replica databases
Synchronization Distributed locks, semaphores, coordination (docs)
Notifications Email, Slack, Discord, SMS, Push, Webhook (docs)
Aspects (AOP) Cross-cutting concerns via pattern matching (docs)
Validators Regex, CEL, and custom validation rules (docs)
WASM Custom functions and validators via WebAssembly (docs)
Mocks Mock data for development and testing (docs)
Plugins Extend Mycel with WASM plugins (docs)
Exec Execute shell commands from flows
Error Handling Retry, DLQ, circuit breaker, custom error responses, on_error aspects (docs)
Security Secure-by-default input sanitization, XXE/injection protection, WASM sanitizers (docs)
Scheduled Jobs Cron expressions and interval-based flow triggers
Hot Reload Apply HCL changes without restart
Health Checks / Prometheus /health, /metrics endpoints
Debugging Trace flows, interactive breakpoints, dry-run, IDE integration (VS Code, IntelliJ, Neovim)

Debugging

Mycel has a built-in debugging toolkit for tracing data through your flows — no log statements needed.

# See what a flow does, step by step
mycel trace get_users

# Simulate a write without touching the database
mycel trace create_user --input '{"email":"test@x.com"}' --dry-run

# Interactive breakpoints — pause at each pipeline stage
mycel trace create_user --input '{"email":"test@x.com"}' --breakpoints

# Pause only at specific stages
mycel trace create_user --input '{"email":"test@x.com"}' --break-at=transform,write

# IDE debugging (VS Code, IntelliJ, Neovim) via DAP
mycel trace create_user --input '{"email":"test@x.com"}' --dap=4711

# Per-request pipeline logging in a running service
mycel start --verbose-flow

All debug features are development-only — automatically disabled in staging/production with zero overhead.

See the Debugging Guide for full documentation including IDE setup.

CLI

mycel start [--config=<path>] [--env=<env>] [--log-level=<level>] [--log-format=<format>] [--hot-reload] [--verbose-flow]
mycel validate [--config=<path>]
mycel check [--config=<path>]
mycel version

mycel trace <flow-name> [--input=<json>] [--params=<k=v>] [--dry-run] [--breakpoints] [--break-at=<stages>] [--dap=<port>] [--list]

mycel plugin install [name]
mycel plugin list
mycel plugin remove <name>
mycel plugin update [name]

Environment: MYCEL_ENV (default: development), MYCEL_LOG_LEVEL (default: info), MYCEL_LOG_FORMAT (default: text). Flags take precedence.

See the Debugging Guide for mycel trace usage and examples.

Plugins

Plugins extend Mycel with additional connectors, validators, and sanitizers via WASM. Declare them in your config and they work like built-in features — no extra wiring needed.

Declare the plugin:

plugin "salesforce" {
  source  = "github.com/acme/mycel-salesforce"
  version = "^1.0"
}

Use it like any built-in connector:

connector "sf" {
  type         = "salesforce"
  instance_url = env("SF_URL")
  api_key      = env("SF_KEY")
}

flow "sync_accounts" {
  from { connector.api = "POST /accounts" }
  to   { connector.sf  = "accounts" }
}

Plugin validators and sanitizers are also available immediately after declaration — use them in type definitions or security rules just like native ones.

Plugins are auto-installed on mycel start. Sources: local paths (./plugins/my-plugin), GitHub, GitLab, Bitbucket, or any git URL. Versions use semver constraints (^1.0, ~2.0, >= 1.0, < 3.0). Cache stored in mycel_plugins/.

See the plugin example for a complete walkthrough.

Installation

Docker (recommended):

# From GitHub Container Registry
docker run -v $(pwd):/etc/mycel -p 3000:3000 ghcr.io/matutetandil/mycel

# Or from Docker Hub
docker run -v $(pwd):/etc/mycel -p 3000:3000 mdenda/mycel

Go:

go install github.com/matutetandil/mycel/cmd/mycel@latest

Kubernetes (Helm):

helm install my-api oci://ghcr.io/matutetandil/charts/mycel

See helm/mycel/README.md for full Helm documentation including values, autoscaling, and ingress configuration.

Requirements: Docker (recommended) or Go 1.24+ (for building from source).

Documentation

Full documentation is at docs/index.md. Quick links:

Getting Started

Core Concepts

  • Connectors — All connector types
  • Flows — Complete flow reference
  • Transforms — CEL functions and expressions
  • Types — Schema validation and field constraints
  • Environments — Environment variables and overlays

Guides

Reference

Deployment

Advanced

Project

  • Architecture — Design decisions: why HCL, why CEL, why WASM, why Go
  • Roadmap — Implementation status and future plans
  • Connector Catalog — Individual documentation for every connector type

More Examples

Variants and integration patterns beyond the individual features above:

Example Description
s3 AWS S3 / MinIO (variant of Files)
redis-cluster Redis cluster setup (variant of Cache)
dynamic-api-key Dynamic API key auth (variant of Auth)
wasm-validator WASM validators (variant of WASM)
integration Multi-connector integration patterns

Support

If you find this project useful, consider supporting its development:

Buy Me A Coffee

Contributing

Contributions are welcome! Please read the contributing guidelines before submitting a pull request.

License

MIT

About

Declarative microservice runtime — define HCL config, get a production-ready service. REST, GraphQL, gRPC, SOAP, message queues, databases, and more.

Topics

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages