GCP Secret Manager Emulator

IAM-enforced Secret Manager emulator — Test secrets AND permissions locally, fail like production would.

A production-grade Secret Manager implementation with optional pre-flight IAM enforcement. Unlike standard emulators that allow everything, this can deny unauthorized requests using real IAM policies.

Dual protocol support: Native gRPC + REST/HTTP. No GCP credentials required.

Quick Start

Prerequisites: Go 1.24+ or Docker

Get the emulator running in under 60 seconds. No GCP account or credentials needed.

Option A: Docker (recommended -- no Go required)

docker run -p 9090:9090 -p 8080:8080 ghcr.io/blackwell-systems/gcp-secret-manager-emulator:dual

Option B: Go install

go install github.com/blackwell-systems/gcp-secret-manager-emulator/cmd/server-dual@latest
server-dual

Both options start the emulator with gRPC on :9090 and REST on :8080.

Verify it works:

# Create a secret
curl -X POST "http://localhost:8080/v1/projects/my-project/secrets?secretId=my-secret" \
  -H "Content-Type: application/json" \
  -d '{"replication":{"automatic":{}}}'

# Add a value to it
curl -X POST "http://localhost:8080/v1/projects/my-project/secrets/my-secret:addVersion" \
  -H "Content-Type: application/json" \
  -d '{"payload":{"data":"'$(echo -n "hello-world" | base64)'"}}'

# Read it back
curl "http://localhost:8080/v1/projects/my-project/secrets/my-secret/versions/1:access"

In production: Used by enterprise teams for hermetic CI/CD testing

Why This Emulator Is Different

Most Secret Manager emulators skip authorization. This one can enforce production-style IAM authorization policies using the IAM Emulator as a control plane.

Approach	Example	When	Behavior
Mock	Standard emulators	Never	Always allows
Observer	Post-execution analysis	After	Records what you used
Control Plane	Blackwell (this)	Before	Denies unauthorized

Pre-flight enforcement catches permission bugs in development/CI, not production.

The Hermetic Seal

Before Blackwell, "GCP Hermetic Testing" was essentially impossible.

Google's official emulators have a critical flaw: they ignore authorization. Your tests pass locally because the emulator allows everything, then fail in production when IAM denies the request.

The two bad options:

Fake Auth - Emulator ignores permissions (fast, but catches zero IAM bugs)
Staging Leak - Call real GCP IAM API (hermetic seal broken, tests become flaky)

Blackwell closes the hermetic seal:

With IAM enforcement enabled, your tests:

Fail for the same authorization reasons production would (PermissionDenied errors)
Run completely offline (no network, no GCP credentials)
Execute deterministically (0ms IAM propagation delay vs 1-60s in real GCP)

This is true hermetic testing - all dependencies sealed inside the boundary, no external leaks.

Enforcement Modes

Scope note: IAM enforcement in this emulator is scoped for CI authorization testing. It validates authorization intent and high-risk permissions, not full GCP IAM semantic parity.

Off (default) - No IAM checks, fast iteration
Permissive - Enforce when IAM available, allow on connectivity errors (fail-open)
Strict - Always enforce, deny on connectivity errors (fail-closed, CI-ready)

The Security Paradox:

"A test that cannot fail due to a permission error is a test that has not fully validated the code's production readiness."

Use strict mode in CI to catch IAM bugs before deployment, not during Friday night incidents.

Usage Modes

Standalone - Run independently for Secret Manager-only testing:

server-dual
# Single service, no IAM enforcement (mode=off)

With IAM Enforcement - Run standalone with IAM checks:

# Start IAM emulator first
cd ../gcp-iam-emulator && ./bin/server --config policy.yaml

# Start Secret Manager with enforcement
IAM_MODE=strict IAM_EMULATOR_HOST=localhost:8080 server-dual
# Now requires valid permissions for all operations

Orchestrated Ecosystem - Use with GCP IAM Control Plane for multi-service testing:

gcp-emulator start
# Secret Manager + KMS + IAM emulator
# Single policy file, unified authorization

Choose standalone for simple workflows, IAM-enforced for production-like testing.

Features

Core Functionality

Dual Protocol Support - Native gRPC + REST/HTTP APIs (choose what fits your workflow)
SDK Compatible - Drop-in replacement for official cloud.google.com/go/secretmanager (gRPC)
curl Friendly - Full REST API with JSON, test from any language or terminal
Complete API - 12 core methods implemented (all Secret Manager operations)
High Test Coverage - 90.8% coverage with comprehensive integration tests

IAM Enforcement (Optional)

Pre-Flight Authorization - Checks permissions before data access
Deterministic Policy Evaluation - Uses IAM Emulator control plane for decisions
Three Modes - Off (default), Permissive (fail-open), Strict (fail-closed)
Production Semantics - Same permission names as real GCP (secretmanager.secrets.get)
Fail Like Production - Catch permission bugs in CI, not production

Operations

No GCP Credentials - Works entirely offline without authentication
Fast & Lightweight - In-memory storage, starts in milliseconds
Docker Support - Pre-built containers (gRPC-only, REST-only, or dual)
Thread-Safe - Concurrent access with proper synchronization

Supported Operations

Secrets

CreateSecret - Create new secrets with labels
GetSecret - Retrieve secret metadata
UpdateSecret - Modify secret metadata (labels, annotations)
ListSecrets - List all secrets with pagination
DeleteSecret - Remove secrets

Secret Versions

AddSecretVersion - Add new version with payload
GetSecretVersion - Retrieve version metadata
AccessSecretVersion - Retrieve version payload (respects version state)
ListSecretVersions - List all versions with pagination and filtering
EnableSecretVersion - Enable a disabled version
DisableSecretVersion - Disable a version (prevents access)
DestroySecretVersion - Permanently destroy a version (irreversible)

Server Variants

Not sure which to pick? Use server-dual. It does everything the others do.

Variant	Protocols	Best For
`server-dual`	gRPC + REST	Most users -- works with SDKs and curl
`server`	gRPC only	Go/Python/Java SDK users who want minimal overhead
`server-rest`	REST/HTTP only	Shell scripts, curl, non-Go languages without gRPC

Install

# gRPC only (recommended for SDK users)
go install github.com/blackwell-systems/gcp-secret-manager-emulator/cmd/server@latest

# REST API only
go install github.com/blackwell-systems/gcp-secret-manager-emulator/cmd/server-rest@latest

# Both protocols
go install github.com/blackwell-systems/gcp-secret-manager-emulator/cmd/server-dual@latest

Run Server

gRPC server:

# Start on default port 9090
server

# Custom port
server --port 8080

REST server:

# Start on default ports (gRPC: 9090, HTTP: 8080)
server-rest

# Custom ports
server-rest --grpc-port 9090 --http-port 8080

Dual protocol server:

# Start both protocols (gRPC: 9090, HTTP: 8080)
server-dual

# Custom ports
server-dual --grpc-port 9090 --http-port 8080

Use with GCP SDKs

Point your existing GCP SDK code at the emulator instead of real GCP. No code changes needed beyond the connection setup.

Note: Unlike some GCP emulators, this one does not currently support the SECRETMANAGER_EMULATOR_HOST environment variable for automatic SDK redirection. You need to configure the connection explicitly as shown below.

Go:

ctx := context.Background()

// Connect to emulator instead of real GCP
conn, _ := grpc.NewClient(
    "localhost:9090",
    grpc.WithTransportCredentials(insecure.NewCredentials()),
)

client, _ := secretmanager.NewClient(ctx, option.WithGRPCConn(conn))
defer client.Close()

// Use client normally -- API is identical to real GCP

Python:

from google.cloud import secretmanager
import grpc

channel = grpc.insecure_channel("localhost:9090")
client = secretmanager.SecretManagerServiceClient(
    transport=secretmanager.transports.SecretManagerServiceGrpcTransport(
        channel=channel
    )
)

# Use client normally
response = client.access_secret_version(
    name="projects/my-project/secrets/my-secret/versions/latest"
)

Node.js:

const {SecretManagerServiceClient} = require('@google-cloud/secret-manager');

const client = new SecretManagerServiceClient({
  apiEndpoint: 'localhost:9090',  // Point at emulator
  // No credentials needed
});

const [version] = await client.accessSecretVersion({
  name: 'projects/my-project/secrets/my-secret/versions/latest',
});

Use with REST API

Start REST server:

server-rest
# HTTP gateway listening at :8080
# Example: curl http://localhost:8080/v1/projects/test-project/secrets

Create a secret:

curl -X POST "http://localhost:8080/v1/projects/my-project/secrets?secretId=my-secret" \
  -H "Content-Type: application/json" \
  -d '{"replication":{"automatic":{}}}'

Add a secret version:

curl -X POST "http://localhost:8080/v1/projects/my-project/secrets/my-secret:addVersion" \
  -H "Content-Type: application/json" \
  -d '{"payload":{"data":"'$(echo -n "my-secret-data" | base64)'"}}'

Access secret data:

curl "http://localhost:8080/v1/projects/my-project/secrets/my-secret/versions/1:access"

List secrets:

curl "http://localhost:8080/v1/projects/my-project/secrets"

Disable a version:

curl -X POST "http://localhost:8080/v1/projects/my-project/secrets/my-secret/versions/1:disable"

REST API matches GCP's official REST endpoints - same paths, same JSON format, same behavior.

Docker

Build Docker Images

# Build all variants
make docker

# Or build individually
docker build --build-arg VARIANT=grpc -t emulator:grpc .  # gRPC only (default)
docker build --build-arg VARIANT=rest -t emulator:rest .  # REST only
docker build --build-arg VARIANT=dual -t emulator:dual .  # Both protocols

Run Docker Containers

gRPC only:

docker run -p 9090:9090 gcp-secret-manager-emulator:grpc

REST only:

docker run -p 8080:8080 gcp-secret-manager-emulator:rest
# Access via: curl http://localhost:8080/v1/projects/test/secrets

Dual protocol (both gRPC + REST):

docker run -p 9090:9090 -p 8080:8080 gcp-secret-manager-emulator:dual
# gRPC on :9090, REST on :8080

In CI/CD

GitHub Actions:

services:
  gcp-emulator:
    image: gcp-secret-manager-emulator:dual
    ports:
      - 9090:9090
      - 8080:8080

Docker Compose:

services:
  gcp-emulator:
    image: gcp-secret-manager-emulator:dual
    ports:
      - "9090:9090"  # gRPC
      - "8080:8080"  # REST
    environment:
      - GCP_MOCK_LOG_LEVEL=debug

Use Cases

Local Development - Test GCP Secret Manager integration without cloud access
CI/CD Pipelines - Fast integration tests without GCP credentials
Unit Testing - Deterministic test environment
Demos & Prototyping - Showcase GCP integrations offline
Cost Reduction - Avoid GCP API charges during development

IAM Integration

The Secret Manager emulator supports optional permission checks using the GCP IAM Emulator.

Configuration

Environment Variables:

IAM_MODE - Controls permission enforcement (default: off)
- off - No permission checks (legacy behavior)
- permissive - Check permissions, fail-open on connectivity errors
- strict - Check permissions, fail-closed on connectivity errors (for CI)
IAM_EMULATOR_HOST - IAM emulator address (default: localhost:8080)

Usage

Without IAM (default):

server-dual

With IAM (permissive mode):

IAM_MODE=permissive IAM_EMULATOR_HOST=localhost:8080 server-dual

With IAM (strict mode for CI):

IAM_MODE=strict IAM_EMULATOR_HOST=localhost:8080 server-dual

Principal Injection

Specify the calling principal for permission checks:

gRPC:

ctx := metadata.AppendToOutgoingContext(ctx, "x-emulator-principal", "user:admin@example.com")
resp, err := client.CreateSecret(ctx, req)

REST:

curl -H "X-Emulator-Principal: user:admin@example.com" \
  -X POST "http://localhost:8080/v1/projects/my-project/secrets" \
  -H "Content-Type: application/json" \
  -d '{"secretId":"my-secret"}'

Permissions

Secret Manager operations map to GCP IAM permissions:

Operation	Permission	Resource
CreateSecret	`secretmanager.secrets.create`	Parent project
GetSecret	`secretmanager.secrets.get`	Secret
UpdateSecret	`secretmanager.secrets.update`	Secret
DeleteSecret	`secretmanager.secrets.delete`	Secret
ListSecrets	`secretmanager.secrets.list`	Parent project
AddSecretVersion	`secretmanager.versions.add`	Secret
AccessSecretVersion	`secretmanager.versions.access`	Secret version
GetSecretVersion	`secretmanager.versions.get`	Secret version
ListSecretVersions	`secretmanager.versions.list`	Secret
EnableSecretVersion	`secretmanager.versions.enable`	Secret version
DisableSecretVersion	`secretmanager.versions.disable`	Secret version
DestroySecretVersion	`secretmanager.versions.destroy`	Secret version

Mode Differences

Scenario	`off`	`permissive`	`strict`
No IAM emulator	Allow	Allow	Deny
IAM unavailable	Allow	Allow	Deny
No principal	Allow	Deny	Deny
Permission denied	Allow	Deny	Deny

Use off for local dev, permissive for integration tests, strict for CI.

Why IAM Enforcement Uses Curated Permissions (On Purpose)

IAM enforcement in this emulator is deliberately scoped for authorization testing, not comprehensive permission modeling. The underlying IAM emulator checks a small set of built-in roles (primitives + Secret Manager + KMS) plus unlimited custom role definitions to catch the bugs that actually break production: missing permissions, wrong role assignments, and unauthorized destructive operations (delete, destroy, setIamPolicy). This curated-first approach catches 95% of real-world authorization bugs while maintaining hermetic execution (no GCP credentials required), deterministic behavior (0ms propagation delay vs 1-60s in real GCP), and zero maintenance burden from tracking GCP's evolving role catalog. If you need to test additional permissions, define them explicitly in the IAM emulator's policy.yaml as custom roles — this explicit approach is simpler, more reliable, and avoids the catalog staleness problem that plagues comprehensive IAM emulation. We optimize for authorization failures that matter, not theoretical IAM completeness.

Configuration

Environment Variables

Variable	Default	Description
`GCP_MOCK_PORT`	`9090`	gRPC port (`server` only)
`GCP_MOCK_GRPC_PORT`	`9090`	gRPC port (`server-rest`, `server-dual`)
`GCP_MOCK_HTTP_PORT`	`8080`	HTTP port (`server-rest`, `server-dual`)
`GCP_MOCK_LOG_LEVEL`	`info`	Log level: debug, info, warn, error
`IAM_MODE`	`off`	IAM enforcement: off, permissive, strict
`IAM_EMULATOR_HOST`	`localhost:8080`	IAM emulator address

Command Line Flags

gRPC server (server):

server --help

Flags:
  --port int           Port to listen on (default 9090)
  --log-level string   Log level (default "info")

REST and dual servers (server-rest, server-dual):

server-dual --help

Flags:
  --grpc-port int      gRPC port to listen on (default 9090)
  --http-port int      HTTP port to listen on (default 8080)
  --log-level string   Log level (default "info")

Documentation

📚 View Full Documentation

API Reference - Complete API documentation with examples
Architecture Guide - System design, components, and diagrams
Roadmap - Planned features and future direction
Changelog - Version history and release notes
Security Policy - Security guidelines and reporting
Brand Guidelines - Trademark and logo usage
Maintainers - Project maintainers and contact info

Testing

# Run all tests
go test ./...

# With coverage
go test -cover ./...

# With race detector
go test -race ./...

API Coverage

Implemented (12 of 12 core methods): All Secret Manager operations are implemented.

Not Implemented:

IAM methods (SetIamPolicy, GetIamPolicy, TestIamPermissions) -- these manage per-resource policies

Rationale: This emulator uses the IAM Emulator as a centralized control plane instead of per-resource policy storage. Authorization is enforced pre-flight via the IAM emulator's policy engine.

Differences from Real GCP

Intentional Simplifications:

Optional IAM enforcement (off by default, strict mode available for CI)
Centralized policy evaluation (via IAM Emulator, not per-resource policies)
No encryption at rest (in-memory storage)
No replication or regional constraints
Simplified error responses (no retry-after headers)

Perfect for:

Development and testing workflows
CI/CD environments
Local integration testing

Not for:

Production use
Security testing
Performance benchmarking

Project Status

Extracted from vaultmux where it powers GCP backend integration tests.

Disclaimer

This project is not affiliated with, endorsed by, or sponsored by Google LLC or Google Cloud Platform. "Google Cloud", "Secret Manager", and related trademarks are property of Google LLC. This is an independent open-source implementation for testing and development purposes.

Maintained By

Maintained by Dayna Blackwell — founder of Blackwell Systems, building reference infrastructure for cloud-native development.

GitHub · LinkedIn · Blog

Related Projects

GCP IAM Control Plane - CLI to orchestrate the Local IAM Control Plane (this emulator + IAM + others)
GCP IAM Emulator - Policy engine (the brain) for IAM enforcement
GCP KMS Emulator - IAM-enforced KMS data plane
gcp-emulator-auth - Enforcement proxy library (the guard)

Who's Using This?

If you're using this Secret Manager emulator — in CI, locally, or in a test harness — I'd love to hear how you're using it.

What secret management bugs did you catch? (unauthorized access, missing version permissions, replication issues)
Are you using IAM enforcement? (integrated with gcp-iam-emulator, or running in permissive mode)
Which API are you using? (gRPC, REST, or both)
What's still friction? (missing methods, IAM integration complexity, performance issues)

Open an issue, start a discussion, or reach out directly:

📬 dayna@blackwell-systems.com

This helps shape the roadmap and ensures the project stays aligned with real-world needs.

Trademarks

Blackwell Systems™ and the Blackwell Systems logo are trademarks of Dayna Blackwell. You may use the name "Blackwell Systems" to refer to this project, but you may not use the name or logo in a way that suggests endorsement or official affiliation without prior written permission. See BRAND.md for usage guidelines.

License

Apache License 2.0 - See LICENSE for details.

Name		Name	Last commit message	Last commit date
Latest commit History 122 Commits
.github		.github
cmd		cmd
docs		docs
internal		internal
.gitignore		.gitignore
.golangci.yml		.golangci.yml
BRAND.md		BRAND.md
CHANGELOG.md		CHANGELOG.md
Dockerfile		Dockerfile
LICENSE		LICENSE
MAINTAINERS.md		MAINTAINERS.md
Makefile		Makefile
README.md		README.md
SECURITY.md		SECURITY.md
doc.go		doc.go
docker-compose.standalone.yml		docker-compose.standalone.yml
docker-compose.yml		docker-compose.yml
example_test.go		example_test.go
go.mod		go.mod
go.sum		go.sum
iam_integration_test.go		iam_integration_test.go

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

GCP Secret Manager Emulator

Quick Start

Why This Emulator Is Different

The Hermetic Seal

Enforcement Modes

Usage Modes

Features

Core Functionality

IAM Enforcement (Optional)

Operations

Supported Operations

Secrets

Secret Versions

Server Variants

Install

Run Server

Use with GCP SDKs

Use with REST API

Docker

Build Docker Images

Run Docker Containers

In CI/CD

Use Cases

IAM Integration

Configuration

Usage

Principal Injection

Permissions

Mode Differences

Why IAM Enforcement Uses Curated Permissions (On Purpose)

Configuration

Environment Variables

Command Line Flags

Documentation

Testing

API Coverage

Differences from Real GCP

Project Status

Disclaimer

Maintained By

Related Projects

Who's Using This?

Trademarks

License

About

Topics

Resources

License

Security policy

Uh oh!

Stars

Watchers

Forks

Releases 8

Sponsor this project

Uh oh!

Packages 0

Uh oh!

Contributors 1

Languages

Packages