-
Notifications
You must be signed in to change notification settings - Fork 353
docs(MADR): 094 zone proxies new deployment model #15460
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Reviewer Checklist🔍 Each of these sections need to be checked by the reviewer of the PR 🔍:
|
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR introduces MADR 094, which defines the deployment model for zone proxies as they transition from global-scoped to mesh-scoped resources. The architectural change addresses fundamental limitations with the current global zone proxy model, particularly around MeshIdentity issuance and policy application.
Changes:
- Adds comprehensive architecture decision record for zone proxy deployment model
- Proposes unified zone proxy approach where a single Dataplane type uses labels to determine ingress/egress/both capabilities
- Defines deployment patterns for Kubernetes, Universal, Helm, Terraform, and Konnect workflows
- Establishes defaults for Helm installations and migration paths from global to mesh-scoped zone proxies
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: Krzysztof Słonka <[email protected]>
Signed-off-by: Krzysztof Słonka <[email protected]>
|
General Q:
|
Signed-off-by: Krzysztof Słonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
Signed-off-by: slonka <[email protected]>
|
|
||
| | Tooling Decision | Choice | | ||
| |------------------|--------| | ||
| | Per-mesh Services | **Yes** - each mesh gets its own Service/LoadBalancer for mTLS isolation | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What means Per-mesh Services? What kind of services ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kubernetes Services - specifically the LoadBalancer or NodePort Service that exposes the zone proxy to other zones. Today there's one global Service for zone ingress and one for zone egress. With mesh-scoped zone proxies, each mesh gets its own Service (e.g., kuma-payments-mesh-zoneproxy) because each mesh has a different mTLS CA, so they can't share a single LoadBalancer without complex SNI-based cert selection. This is detailed in the "Per-Mesh Services (Not Shared)" section below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's just kubernetes specific what about universal not sure if naming Service is here a good option
| | Tooling Decision | Choice | | ||
| |------------------|--------| | ||
| | Per-mesh Services | **Yes** - each mesh gets its own Service/LoadBalancer for mTLS isolation | | ||
| | Namespace placement | **kuma-system** | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it need to be system namespace? In case of mulitple meshes is it possible to change it or we want to stick to the one only?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For single-mesh (this MADR's scope), kuma-system keeps everything together with the CP. For multi-mesh, standalone kuma-zone-proxy releases can target any namespace — the Helm --namespace flag controls that. We could add a dedicated namespace option per mesh later, but the single-mesh default of kuma-system keeps things simple. I'll clarify this in the namespace section.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to ensure there is transparent proxy disabled for a zoneproxy - even if it's enabled on the namespace we should have it disabled on the deployment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not good because that assumes kuma-system namespace will be marked as a "meshed" namespace which we don't want right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also kuma-system is sort of meant to be admin realm here there's almost a Mesh operator persona comes in no?
lahabana
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a PoC for this?
|
|
||
| 2. **Cannot apply policies on zone proxies**: Kuma policies (MeshTrafficPermission, MeshTimeout, etc.) are mesh-scoped. | ||
| A global zone proxy cannot be targeted by mesh-specific policies, limiting observability and traffic control for cross-zone communication. | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Observability story is tricky (how do you share to a mesh a subset of all the observability)
| **Scope of this document**: This MADR focuses on **deployment tooling** — how users deploy zone proxies via Helm, Konnect UI, and Terraform. | ||
|
|
||
| **Single-mesh focus**: This document assumes **single-mesh-per-zone as the default** deployment pattern. | ||
| For multi-mesh scenarios, deploy additional zone proxies using separate Helm releases with a dedicated `kuma-zone-proxy` chart. A multi-mesh deployment guide will be provided separately. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No let's not as we're just creating maintainance for a path we don't recommend.
I think for folks that NEED multi-mesh they can:
- Just write their own deployment
- Push for Mesh to be supported by KO
Our goal for K3 is really to streamline what folks can do. Adding a new helm short is a complete no go.
| | Tooling Decision | Choice | | ||
| |------------------|--------| | ||
| | Per-mesh Services | **Yes** - each mesh gets its own Service/LoadBalancer for mTLS isolation | | ||
| | Namespace placement | **kuma-system** | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not good because that assumes kuma-system namespace will be marked as a "meshed" namespace which we don't want right?
| | Tooling Decision | Choice | | ||
| |------------------|--------| | ||
| | Per-mesh Services | **Yes** - each mesh gets its own Service/LoadBalancer for mTLS isolation | | ||
| | Namespace placement | **kuma-system** | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also kuma-system is sort of meant to be admin realm here there's almost a Mesh operator persona comes in no?
| │ Mesh: [default ▼] │ | ||
| │ (if multiple meshes detected there will be info here on │ | ||
| │ how to handle that) │ | ||
| └─────────────────────────────────────────────────────────┘ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if no mesh?
| # mesh defaults to "default" | ||
| ``` | ||
|
|
||
| If the user selected a non-default mesh name: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not always do it?
Motivation
Zone proxies are becoming mesh-scoped to enable MeshIdentity and policy support. This requires revisiting the deployment model.
Implementation
MADR 094 addresses:
Key decisions: unified zone proxy as Dataplane with kuma.io/zone-proxy-role label, standalone deployment, mesh-scoped resources.
Supporting documentation
fix #9030