Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Documentation

Documentation here is a developer level view of concepts and how the system is implemented.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence starts a bit strange, missing a "The" perhaps?

While system operators and end users may benefit from this information, there are other docs which are more relevant to those audiences.

In particular:
* For Kubernetes Operator/OpenShift deployments, see the [CAO Documentation on CNG](https://docs.couchbase.com/operator/current/concept-cloud-native-gateway.html)
* For Capella Data API, see the [Runbook for Data API for Provisioned Clusters](https://confluence.issues.couchbase.com/wiki/x/S4ACv)

## Concepts

**Stellar Gateway** is a Couchbase Protostellar implementation that provides both a gRPC "protostellar" and a RESTful interface to Couchbase. It is deployed 1:1 with a cluster, but may be deployed different ways.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stellar Gateway is a service that provides both a gRPC "protostellar" implementation and a RESTful interface to Couchbase.


### Architecture Overview

The system uses a modular architecture where each service (data, service discovery, Data API) can be independently configured and scaled.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence is a slightly confusing statement, it seems to be mixing the concept of scalability of the cluster with the scalability of CNG itself.


### Main Entry Points
- **`cmd/gateway/main.go`** - Main gateway service entry point with CLI interface
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be limited to just cmd/gateway/main.go, the other 2 are likely to be eventually removed (they are not maintained anymore).

- **`cmd/dev/main.go`** - Development environment setup
- **`cmd/bridge/main.go`** - Legacy bridge service (deprecated, never deployed)

### Core Modules

#### **Gateway Core (`gateway/`)**
- **`gateway.go`** - Main Gateway struct and orchestration logic
- **`system/`** - System-level server management (gRPC, HTTP, Data API servers)
- **`clustering/`** - Cluster membership and service discovery
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While "cluster membership and service discovery" is what this folder does, this statement is a bit confusing without mentioning the distinction between CNG service discovery and cluster service discovery. This particular folder may also be deprecated in the future, since we don't currently have a fully implemented implementation of CNG service discovery.

- **`topology/`** - Network topology management

#### **Protocol Implementations**
- **`dataimpl/`** - Protostellar data API service (KV, Query, Search) implementation
- **`sdimpl/`** - Service discovery implementation
- **`dapiimpl/`** - Data API v1 HTTP REST implementation with proxy services
- **`legacybridge/`** - Legacy protocol bridge for backwards compatibility

#### **Client**
- **`client/`** - Client front end for connecting to clusters
- Connection management, routing, topology watching

#### **Data API Specification**
- **`dataapiv1/`** - OpenAPI spec and generated code for REST Data API
- Auto-generated from `spec.yaml` using oapi-codegen

#### **Infrastructure Modules**
- **`auth/`** - Authentication (cbauth integration)
- **`ratelimiting/`** - Request rate limiting
- **`hooks/`** - Lifecycle hooks and barriers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hooks library enables hooking the GRPC implementation in various ways for testing. It's "testing hooks" rather than "lifecycle hooks".

- **`apiversion/`** - API versioning and gRPC interceptors

#### **Contrib/Utilities**
- **`contrib/`** - External components (etcd, goclustering, protostellar extensions)
- **`utils/`** - Utility packages (TLS, secrets management, networking)
- **`pkg/`** - Shared packages (metrics, interceptors, web API)

### Public Interfaces

1. **gRPC Protostellar API** - Main protocol interface (port 18098/18099)
2. **Data API REST** - HTTP REST API (configurable port, spec in `dataapiv1/`)
3. **Web/Metrics API** - Health checks and metrics (port 9091)
4. **Client Library** - Go client in `client/` package

### Key Features
- Multiple deployment modes (standalone, clustered)
- TLS support with configurable certificates
- Rate limiting and request throttling
- OpenTelemetry integration for observability
- Cloud provider secrets integration (AWS, Azure, GCP)
- Graceful shutdown and configuration reloading

## Detailed Documentation

* [Tuneables](tuneables.md)

20 changes: 20 additions & 0 deletions docs/tuneables.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Tuneables

## Rate Limiting

In a Capella deployment, we wish to have a rate limit as a simple method of avoiding accidental overload, resource exhaustion and pathological system failure.
Comprehensive DDoS prevention is more complex and outside the scope of what Data API can provide, but this feature provides some reasonable limit.

**Field:** `rate-limit`

**Format:** Integer, unsigned

**Default:** 0 (unlimited). Capella deployments default this to 10000.

**Description:** A per-process limit which, when exceeded, will return HTTP 429 errors.

This is service independent and resource usage/complexity of underlying services called may not align. For example, KV operations with options like durability, complex queries, and FTS requests may exhaust resources before this rate limit is hit.

**When to adjust:**
- If a deployment wants to be more conservative on Data API or CNG resource usage, this may be tuned lower which will reject requests to keep load lower.
- If a deployment wants to allow more resource usage, increase this to a higher level or to the value 0 to be unlimited.