Skip to content

Commit 394ae14

Browse files
kcp-ci-botkcp-ci-bot
authored
Merge pull request #3580 from mjudeikis/mjudeikis/workspace.mounts.docs
Add workspace mounts docs
2 parents c69a5a3 + ea3ab53 commit 394ae14

File tree

2 files changed

+242
-0
lines changed

2 files changed

+242
-0
lines changed

‎docs/content/concepts/workspaces/.pages‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ nav:
33
- workspace-types.md
44
- virtual-workspaces.md
55
- workspace-initialization.md
6+
- mounts.md

‎docs/content/concepts/workspaces/mounts.md‎

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
---
2+
description: >
3+
What are workspace mounts and how do they work?
4+
---
5+
6+
# Workspace Mounts
7+
8+
Workspace mounts allow you to mount external Kubernetes-like API endpoints onto a workspace, similar to how you mount remote filesystems in Linux using NFS. Just like a Linux directory can be a local folder or a mounted remote filesystem, a workspace can be either a local LogicalCluster or a mounted external endpoint.
9+
10+
When a workspace uses a mount, it does not have a LogicalCluster backing it. Instead, requests to the workspace are proxied to the external API endpoint specified by the mount object. This allows you to have a unified view of multiple clusters and workspaces under the same workspace tree/hierarchy.
11+
12+
**Analogy**: Think of workspaces as directories in a Linux filesystem:
13+
- **Regular workspace** = Local directory with files stored on the local filesystem
14+
- **Mounted workspace** = Directory that's an NFS mount pointing to a remote filesystem
15+
- **kcp** = The filesystem manager that routes requests to the right location
16+
17+
## Architecture Overview
18+
19+
```mermaid
20+
sequenceDiagram
21+
participant C as 🖥️ Client
22+
participant P as 🔄 Front Proxy
23+
participant LC as 🧠 Logical Cluster<br/>(root:org1:project-a)
24+
participant EK as ☸️ External Kube API<br/>(mounted cluster)
25+
26+
Note over C,EK: Request Routing Based on Mount Status
27+
28+
rect rgb(240, 248, 255)
29+
Note over C,EK: Scenario 1: Non-mounted workspace (project-a)
30+
C->>+P: GET /clusters/root:org1:project-a/api/v1/pods
31+
P->>+LC: Route to logical cluster
32+
Note right of P: No mount detected,<br/>use internal logical cluster
33+
LC-->>-P: Return apis from logical cluster
34+
P-->>-C: Forward response
35+
end
36+
37+
rect rgb(255, 248, 240)
38+
Note over C,EK: Scenario 2: Mounted workspace (project-b)
39+
C->>+P: GET /clusters/root:org1:project-b/api/v1/pods
40+
Note right of C: Request to mounted workspace
41+
P->>P: Check mount.ref to external-k8s
42+
P->>+EK: Proxy to https://ext-k8s.com
43+
Note right of P: Mount detected (status: Ready),<br/>proxy to external cluster
44+
EK-->>-P: Return pods from external cluster
45+
P-->>-C: Forward response
46+
end
47+
48+
Note over C,EK: Routing determined by workspace mount configuration
49+
```
50+
51+
52+
### Workspace Tree Structure
53+
54+
```
55+
root/
56+
└── org1/
57+
├── project-a/ # Traditional LogicalCluster workspace
58+
│ ├── LogicalCluster object # ✓ Has backing logical cluster
59+
│ ├── /api/v1/configmaps # ✓ Served by kcp directly
60+
│ └── /api/v1/secrets # ✓ Standard Kubernetes APIs
61+
│
62+
└── project-b/ # Mounted workspace
63+
├── spec.mount.ref # ✗ No LogicalCluster object
64+
│ └── "external-k8s" # → References mount object
65+
├── /api/v1/pods # → Proxied to https://ext-k8s.com/api/v1/pods . kcp does not have pods, but this is a mount.
66+
└── /apis/apps/v1/deployments # → Proxied to https://ext-k8s.com/api/v1/deployments
67+
```
68+
69+
## How it Works
70+
71+
### Prerequisites
72+
73+
1. **Feature Gate**: The `WorkspaceMounts=true` feature gate must be enabled on the kcp instance.
74+
2. **External Controller/Proxy**: You need to implement a controller that:
75+
- Creates and manages mount objects (with the required annotation and status fields)
76+
- Runs a proxy/server that implements the Kubernetes API and serves requests at the URL specified in `status.URL`
77+
- The controller can be any custom implementation as long as it follows the mount object contract. See [1] as an example.
78+
79+
**Important**: kcp provides the mounting machinery, but you must "Bring Your Own API" (BYO-API). This means you're responsible for implementing both the mount object management and the actual API server that will handle the proxied requests.
80+
81+
### Mount Objects
82+
83+
Workspace mounts follow a **"Bring Your Own API"** pattern. This means you can use any Kubernetes Custom Resource as a mount object, as long as it meets three simple requirements. The mounting machinery in kcp is generic and doesn't care about the specifics of your API or implementation.
84+
85+
```yaml title="Example Mount Object"
86+
apiVersion: mounts.contrib.kcp.io/v1alpha1
87+
kind: KubeCluster
88+
metadata:
89+
name: proxy-cluster
90+
annotations:
91+
experimental.tenancy.kcp.io/is-mount: "true"
92+
spec:
93+
mode: Delegated
94+
secretString: kTPlAYLMjKJDRly5
95+
status:
96+
URL: https://proxy-cluster.proxy-cluster.svc.cluster.local
97+
phase: Ready
98+
```
99+
100+
#### Requirements for Mount Objects
101+
102+
1. **Annotation**: Must have the `experimental.tenancy.kcp.io/is-mount: "true"` annotation
103+
2. **Status URL**: Must have a `status.URL` field containing the target endpoint URL
104+
3. **Status Phase**: Must have a `status.phase` field with one of the following values:
105+
- `Initializing`: The mount proxy is being initialized
106+
- `Connecting`: The mount proxy is waiting for connection
107+
- `Ready`: The mount proxy is ready and connected
108+
- `Unknown`: The mount proxy status is unknown
109+
110+
!!! note
111+
112+
Mount objects can be created and managed by users or by the system. For example, if a user has credentials for a delegated cluster, they can create a mount object and reference it in their workspace.
113+
114+
#### Controller Requirements
115+
116+
While the mount object can be any Custom Resource, you still need a controller to:
117+
- Create and manage the lifecycle of these mount objects
118+
- Set the required annotation and status fields
119+
- Implement and run the actual API server/proxy that serves requests at the `status.URL`
120+
- Handle authentication, authorization, and any request filtering if needed
121+
122+
The kcp mounting machinery handles the workspace-to-mount routing, but the actual API implementation is entirely up to you.
123+
124+
### Creating a Mounted Workspace
125+
126+
To create a workspace that uses a mount, specify the mount reference in the workspace spec:
127+
128+
```yaml
129+
apiVersion: tenancy.kcp.io/v1alpha1
130+
kind: Workspace
131+
metadata:
132+
name: mounted-workspace
133+
spec:
134+
mount:
135+
ref:
136+
apiVersion: mounts.contrib.kcp.io/v1alpha1
137+
kind: KubeCluster
138+
name: proxy-cluster
139+
```
140+
141+
#### Mount Field Requirements
142+
143+
- `ref.apiVersion`: The API version of the mount object
144+
- `ref.kind`: The kind of the mount object
145+
- `ref.name`: The name of the mount object
146+
- `ref.namespace`: (Optional) The namespace of the mount object if it's namespaced
147+
148+
!!! Important
149+
150+
The mount reference is immutable after workspace creation.
151+
152+
## Simple End-to-End Example
153+
154+
Here's a basic example to illustrate how workspace mounts work in practice:
155+
156+
### Step 1: Create a Mount Object
157+
Your controller creates a mount object (this could be any Custom Resource):
158+
159+
```yaml
160+
apiVersion: example.io/v1alpha1
161+
kind: RemoteCluster
162+
metadata:
163+
name: my-remote-k8s
164+
annotations:
165+
experimental.tenancy.kcp.io/is-mount: "true" # Required
166+
spec:
167+
endpoint: "https://my-k8s-cluster.com"
168+
status:
169+
URL: "https://my-proxy-service.com" # Required: where requests will be proxied
170+
phase: "Ready" # Required: mount status
171+
```
172+
173+
### Step 2: Create a Workspace with Mount Reference
174+
```yaml
175+
apiVersion: tenancy.kcp.io/v1alpha1
176+
kind: Workspace
177+
metadata:
178+
name: remote-workspace
179+
spec:
180+
mount:
181+
ref:
182+
apiVersion: example.io/v1alpha1
183+
kind: RemoteCluster
184+
name: my-remote-k8s
185+
```
186+
187+
### Step 3: Access the Mounted Workspace
188+
When you make requests to the workspace:
189+
190+
```bash
191+
kubectl --server=https://kcp.example.com/clusters/root:remote-workspace get pods
192+
```
193+
194+
**What happens**:
195+
1. kcp receives the request for `/clusters/root:remote-workspace/api/v1/pods`
196+
2. kcp sees `remote-workspace` has a mount reference
197+
3. kcp looks up the `my-remote-k8s` mount object
198+
4. kcp proxies the request to `https://my-proxy-service.com/api/v1/pods`
199+
5. Your controller's proxy service handles the request and returns the response
200+
201+
!!! Important
202+
203+
You need to implement `https://my-proxy-service.com` to actually serve Kubernetes API requests. kcp only handles the routing.
204+
205+
### How Mounted Workspaces Work
206+
207+
Once a workspace with a mount is created, the following process occurs:
208+
209+
1. **No LogicalCluster Creation**: The workspace will not have a LogicalCluster backing it. Instead, it relies entirely on the external proxy.
210+
211+
2. **Mount Resolution**: The kcp front proxy resolves the mount object referenced in the workspace spec.
212+
213+
3. **URL Resolution**: When requests are made to the workspace, the proxy:
214+
- Looks up the mount object
215+
- Extracts the `status.URL` from the mount object
216+
- Forwards requests only if the mount object is in `Ready` phase
217+
- Returns an error if the mount object is not found or not ready
218+
219+
4. **Request Routing**: The proxy rewrites the incoming request URL to target the mount's URL while preserving the Kubernetes API context (e.g., `/api/v1/pods` becomes `{mount.status.URL}/api/v1/pods`).
220+
221+
### Controllers and Management
222+
223+
The workspace mounts controller (`kcp-workspace-mounts`) manages the integration between workspaces and their mount objects:
224+
225+
- **Watches**: Both workspace objects and dynamically discovered mount resources
226+
- **Reconciliation**: Updates workspace annotations and status based on mount object state
227+
- **Indexing**: Maintains indexes to efficiently find workspaces that reference specific mount objects
228+
- **Status Updates**: Updates workspace conditions based on mount availability and readiness
229+
230+
### Limitations and Considerations
231+
232+
- Mount references are immutable after workspace creation
233+
- Only mount objects in `Ready` phase will serve traffic
234+
- The external proxy must be properly configured and accessible
235+
- Authentication and authorization are handled by the external proxy, not by kcp
236+
- Workspace mounts do not filter kubernetes view. If filtering is required, it must be implemented in the external proxy.
237+
238+
239+
## References
240+
241+
1. https://github.com/kcp-dev/contrib/tree/main/20241013-kubecon-saltlakecity/mounts-vw - Example mount controller and proxy implementation

0 commit comments

Comments
 (0)