Skip to content

Commit d321656

Browse files
sarthypartysarthyparty
committed
Add references document defining NGF permissions
1 parent e88fbeb commit d321656

File tree

3 files changed

+167
-0
lines changed

3 files changed

+167
-0
lines changed

‎docs/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@ _Please note: You can find the user documentation for NGINX Gateway Fabric in th
99
- [Developer](developer/): Docs for developers of the project. Contains guides relating to processes and workflows.
1010
- [Proposals](proposals/): Enhancement proposals for new features.
1111
- [API](api/): The files for generating API documentation, used in the Makefile
12+
- [Reference](reference/): Reference documentation for NGINX Gateway Fabric components and configurations.

‎docs/reference/README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Reference Documentation
2+
3+
This directory contains reference documentation for NGINX Gateway Fabric components and configurations.
4+
5+
## Contents
6+
7+
- [Permissions](permissions.md): Security contexts, volumes, and RBAC permissions required by NGF components.

‎docs/reference/permissions.md

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
# NGINX Gateway Fabric Permissions
2+
3+
This document outlines the permissions required by NGINX Gateway Fabric components.
4+
5+
## Overview
6+
7+
NGINX Gateway Fabric uses a **split-plane architecture** where control and data functions are separated for security and scalability:
8+
9+
### Architecture Split
10+
11+
- **Control Plane**: A single deployment that watches Kubernetes APIs, validates configurations, and manages NGINX data plane deployments. Requires broad Kubernetes API access but handles no user traffic.
12+
13+
- **Data Plane**: Multiple NGINX pods (created by the control plane) that process actual traffic. Requires minimal permissions since they only serve traffic and receive configuration from the control plane via secure gRPC.
14+
15+
- **Certificate Generator**: A one-time job that creates TLS certificates for secure communication between planes.
16+
17+
### Why Different Permissions?
18+
19+
This split requires different security contexts because:
20+
21+
- **Control plane** needs Kubernetes API access to manage resources but never handles user traffic
22+
- **Data plane** processes user traffic but never accesses Kubernetes APIs directly
23+
- Both planes communicate securely via TLS, eliminating the need for shared filesystem access or elevated privileges
24+
25+
## Control Plane
26+
27+
The control plane runs as a single container in the `nginx-gateway` deployment.
28+
29+
### Security Context
30+
31+
- **User ID**: 101 (non-root)
32+
- **Group ID**: 1001
33+
- **Capabilities**: All dropped (`drop: ALL`) - no capabilities are added
34+
- **Privilege Escalation**: Disabled (may need to be enabled in some environments for NGINX reload)
35+
- **Root Filesystem**: Read-only
36+
- **Seccomp**: Runtime default profile
37+
38+
### Required Volumes
39+
40+
| Volume | Type | Purpose |
41+
|--------|------|---------|
42+
| `nginx-agent-tls` | Secret | TLS certificates for communicating with NGINX Agent |
43+
44+
### RBAC Permissions
45+
46+
The control plane requires these Kubernetes API permissions:
47+
48+
- **Secrets, ConfigMaps, Services**: Create, update, delete, list, get, watch
49+
- **Deployments, DaemonSets**: Create, update, delete, list, get, watch
50+
- **ServiceAccounts**: Create, update, delete, list, get, watch
51+
- **Namespaces, Pods**: Get, list, watch
52+
- **Events**: Create, patch
53+
- **Gateway API resources**: Full access
54+
- **Custom resources** (NginxGateway, NginxProxy): Full access
55+
56+
## Data Plane
57+
58+
The data plane consists of NGINX containers dynamically created and managed by the control plane. These containers focus solely on traffic processing and require **fewer privileges** than the control plane since they:
59+
60+
- Don't access Kubernetes APIs (no RBAC needed)
61+
- Receive configuration via secure gRPC from control plane
62+
- Only need to bind to network ports and write to specific directories
63+
64+
### Security Context
65+
66+
- **User ID**: 101 (non-root)
67+
- **Group ID**: 1001
68+
- **Capabilities**: All dropped (`drop: ALL`)
69+
- **Privilege Escalation**: Disabled
70+
- **Root Filesystem**: Read-only
71+
- **Seccomp**: Runtime default profile
72+
- **Sysctl**: `net.ipv4.ip_unprivileged_port_start=0` (allows binding to ports < 1024)
73+
74+
### Required Volumes
75+
76+
| Volume | Type | Purpose |
77+
|--------|------|---------|
78+
| `nginx-agent` | EmptyDir | NGINX Agent configuration |
79+
| `nginx-agent-tls` | Secret | TLS certificates for control plane communication |
80+
| `token` | Projected | Service account token |
81+
| `nginx-agent-log` | EmptyDir | NGINX Agent logs |
82+
| `nginx-agent-lib` | EmptyDir | NGINX Agent runtime data |
83+
| `nginx-conf` | EmptyDir | Main NGINX configuration |
84+
| `nginx-stream-conf` | EmptyDir | Stream configuration |
85+
| `nginx-main-includes` | EmptyDir | Main context includes |
86+
| `nginx-secrets` | EmptyDir | TLS secrets for NGINX |
87+
| `nginx-run` | EmptyDir | Runtime files (PID, sockets) |
88+
| `nginx-cache` | EmptyDir | NGINX cache directory |
89+
| `nginx-includes` | EmptyDir | HTTP context includes |
90+
91+
### Volume Mounts
92+
93+
All volumes are mounted with appropriate read/write permissions. Critical runtime directories like `/var/run/nginx` and `/var/cache/nginx` use ephemeral storage.
94+
95+
## Certificate Generator
96+
97+
Runs as a Kubernetes Job to create initial TLS certificates.
98+
99+
### Security Context
100+
101+
- **User ID**: 101 (non-root)
102+
- **Group ID**: 1001
103+
- **Capabilities**: All dropped (`drop: ALL`)
104+
- **Root Filesystem**: Read-only
105+
- **Seccomp**: Runtime default profile
106+
107+
### RBAC Permissions
108+
109+
Limited to secret management in the NGINX Gateway Fabric namespace:
110+
111+
- **Secrets**: Create, update, get
112+
113+
## OpenShift Compatibility
114+
115+
NGINX Gateway Fabric includes Security Context Constraints (SCCs) for OpenShift:
116+
117+
### Control Plane SCC
118+
119+
- **Privilege Escalation**: Disabled
120+
- **Host Access**: Disabled (network, IPC, PID, ports)
121+
- **User ID Range**: 101-101 (fixed)
122+
- **Group ID Range**: 1001-1001 (fixed)
123+
- **Volumes**: Secret only
124+
125+
### Data Plane SCC
126+
127+
Same restrictions as control plane, plus additional volume types:
128+
129+
- **Additional Volumes**: EmptyDir, ConfigMap, Projected
130+
131+
## Linux Capabilities
132+
133+
**Current Requirements**: NGINX Gateway Fabric drops ALL Linux capabilities and adds none. This follows security best practices for containerized applications.
134+
135+
**Why No Capabilities Needed**:
136+
137+
- **NGINX Process Management**: Uses standard Unix signals that don't require elevated capabilities
138+
- **Port Binding**: Uses sysctl `net.ipv4.ip_unprivileged_port_start=0` to bind to ports < 1024 without privileges
139+
- **File Operations**: All necessary files are writable via volume mounts
140+
141+
**Historical Context**: Earlier versions or specific environments may have required the KILL capability for process management. The current implementation has been redesigned to eliminate this requirement through:
142+
143+
- Proper signal handling without elevated privileges
144+
- NGINX Agent managing process lifecycle
145+
- Secure communication between control and data planes
146+
147+
**Troubleshooting**: If you encounter "operation not permitted" errors during NGINX reload, you may need to temporarily enable `allowPrivilegeEscalation: true` in your environment while investigating the root cause.
148+
149+
## Security Features
150+
151+
The split-plane architecture enables a defense-in-depth security model:
152+
153+
- **Separation of concerns**: Control plane has API access but no traffic exposure; data plane has traffic exposure but no API access
154+
- **Non-root execution**: All components run as unprivileged user (UID 101)
155+
- **Zero capabilities**: All Linux capabilities dropped, none added
156+
- **Read-only root**: Prevents runtime modifications to container filesystems
157+
- **Ephemeral storage**: Writable data uses temporary volumes, not persistent storage
158+
- **Least privilege RBAC**: Control plane gets only required Kubernetes permissions; data plane needs no RBAC
159+
- **Secure inter-plane communication**: TLS-encrypted gRPC between control and data planes

0 commit comments

Comments
 (0)