Skip to content

Commit fb2bfb4

Browse files
tssuryatssurya
committed
NPEP: Iron out Egress Support API Design
Signed-off-by: Surya Seetharaman <[email protected]>
1 parent bb2cafa commit fb2bfb4

File tree

1 file changed

+170
-5
lines changed

1 file changed

+170
-5
lines changed

‎npep/npep-126-egress-traffic-control.md‎

Lines changed: 170 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# NPEP-126: Add northbound traffic support in (B)ANP API
22

33
* Issue: [#126](https://github.com/kubernetes-sigs/network-policy-api/issues/126)
4-
* Status: Provisional
4+
* Status: Implementable
55

66
## TLDR
77

@@ -76,13 +76,178 @@ selected cluster workloads to k8s-apiservers for securing the server.
7676

7777
## API
7878

79-
(... details, can point to PR with changes)
80-
79+
Proof of Concept for the API design details can be found here: https://github.com/kubernetes-sigs/network-policy-api/pull/143
80+
81+
### Implementing egress traffic control towards cluster nodes
82+
83+
This NPEP proposes to add a new type of `AdminNetworkPolicyEgressPeer` called `Nodes`
84+
to be able to explicitly select nodes (based on the node's labels) in the cluster.
85+
This ensures that if the list of IPs on a node OR list of nodes change, the users
86+
don't need to manually intervene to include those new IPs. The label selectors will
87+
take care of this automatically. Note that the nodeIPs that this type of peer matches
88+
on are the IPs present in `Node.Status.Addresses` field of the node.
89+
90+
```
91+
// AdminNetworkPolicyEgressPeer defines a peer to allow traffic to.
92+
// Exactly one of the selector pointers must be set for a given peer. If a
93+
// consumer observes none of its fields are set, they must assume an unknown
94+
// option has been specified and fail closed.
95+
// +kubebuilder:validation:MaxProperties=1
96+
// +kubebuilder:validation:MinProperties=1
97+
type AdminNetworkPolicyEgressPeer struct {
98+
<snipped>
99+
// Nodes defines a way to select a set of nodes in
100+
// in the cluster. This field follows standard label selector
101+
// semantics; if present but empty, it selects all Nodes.
102+
//
103+
// Support: Core
104+
//
105+
// +optional
106+
Nodes *metav1.LabelSelector `json:"nodes,omitempty"`
107+
}
108+
```
109+
110+
Note that `AdminNetworkPolicyPeer` will be changed to
111+
`AdminNetworkPolicyEgressPeer` and `AdminNetworkPolicyIngressPeer` since ingress and
112+
egress peers have started to diverge at this point and it is easy to
113+
maintain it with two sets of peer definitions.
114+
This ensures nodes can be referred to only as "egress peers".
115+
116+
Example: Admin wants to deny egress traffic from tenants who don't have
117+
`restricted`, `confidential` or `internal` level security clearance
118+
to control-plane nodes at 443 and 6443 ports in the cluster
119+
120+
```
121+
apiVersion: policy.networking.k8s.io/v1alpha1
122+
kind: AdminNetworkPolicy
123+
metadata:
124+
name: node-as-egress-peers
125+
spec:
126+
priority: 55
127+
subject:
128+
namespaces:
129+
matchExpressions:
130+
- {key: security, operator: notIn, values: [restricted, confidential, internal]}
131+
egress:
132+
- name: "deny-all-egress-to-kapi-server"
133+
action: "Deny"
134+
to:
135+
- nodes:
136+
matchLabels:
137+
node-role.kubernetes.io/control-plane:
138+
ports:
139+
- portNumber:
140+
protocol: TCP
141+
port: 443
142+
- portNumber:
143+
protocol: TCP
144+
port: 6443
145+
```
146+
147+
### Implementing egress traffic control towards external destinations
148+
149+
This NPEP proposes to add a new type of `AdminNetworkPolicyEgressPeer` called `ExternalNetworks`
150+
to be able to explicitly select external destination CIDRs outside the cluster.
151+
This peer type will not be supported in `AdminNetworkPolicyIngressPeer`.
152+
153+
```
154+
// AdminNetworkPolicyEgressPeer defines a peer to allow traffic to.
155+
// Exactly one of the selector pointers must be set for a given peer. If a
156+
// consumer observes none of its fields are set, they must assume an unknown
157+
// option has been specified and fail closed.
158+
// +kubebuilder:validation:MaxProperties=1
159+
// +kubebuilder:validation:MinProperties=1
160+
type AdminNetworkPolicyEgressPeer struct {
161+
<snipped>
162+
// ExternalNetworks defines a way to select ExternalNetworkSets
163+
// that consist of network CIDRs that live outside the cluster as a peer.
164+
// It is the list of NetworkCIDR (both v4 & v6) that can be used to define
165+
// external destinations.
166+
// This field follows standard label selector semantics; if present
167+
// but empty, it selects all ExternalNetworkSets defined in the cluster.
168+
//
169+
// Support: Core
170+
//
171+
// +optional
172+
// +kubebuilder:validation:MinItems=1
173+
// +kubebuilder:validation:MaxItems=100
174+
ExternalNetworks []string `json:"externalNetworks,omitempty" validate:"omitempty,dive,cidr"`
175+
}
176+
```
177+
178+
Note that the API expects `externalNetworks` to be a set of "external" destinations.
179+
However if user puts a podCIDR, nodeCIDR, serviceCIDR or other intra-cluster
180+
networks, it is implementation-defined as to what the behavior will be. Not
181+
all implementations can correctly define the boundary between "internal" and
182+
"external" destinations with respect to a Kubernetes cluster.
183+
184+
Example: <TBD> -> after we reach consensus on externalNetwork versus network
185+
186+
Let's define ANP and BANP that refer to these external networks:
187+
```
188+
apiVersion: policy.networking.k8s.io/v1alpha1
189+
kind: AdminNetworkPolicy
190+
metadata:
191+
name: cluster-egress-controls
192+
spec:
193+
priority: 70
194+
subject:
195+
namespaces: {}
196+
egress:
197+
- name: "allow-all-egress-to-intranet"
198+
action: "Allow"
199+
to:
200+
- externalNetworks:
201+
matchLabels:
202+
network: intranet
203+
- name: "allow-egress-to-external-dns"
204+
action: "Allow"
205+
to:
206+
- externalNetworks:
207+
matchLabels:
208+
network: dns
209+
ports:
210+
- portNumber:
211+
protocol: UDP
212+
port: 53
213+
- name: "pass-egress-to-internet"
214+
action: "Pass"
215+
to:
216+
- externalNetworks:
217+
matchLabels:
218+
network: internet
219+
---
220+
apiVersion: policy.networking.k8s.io/v1alpha1
221+
kind: BaselineAdminNetworkPolicy
222+
metadata:
223+
name: default
224+
spec:
225+
subject:
226+
namespaces: {}
227+
egress:
228+
- name: "deny-egress-to-internet"
229+
action: "Deny"
230+
to:
231+
- externalNetworks:
232+
matchLabels:
233+
network: internet
234+
```
235+
This allows admins to specify all cluster workloads can talk to
236+
the company's internet CIDR's and the external DNS network. It
237+
also allows them to put up default guardrails of not allowing
238+
any other egress traffic towards internet in a BANP.
81239

82240
## Alternatives
83241

84-
(List other design alternatives and why we did not go in that
85-
direction)
242+
* Instead of adding CIDR peer directly into the main object, we can
243+
define a new object called `ExternalNetworkSet` and use selectors or
244+
name of that object to be referred to from AdminNetworkPolicy and
245+
BaselineAdminNetworkPolicy objects. This is particularly useful
246+
if CIDR ranges are prone to changes versus the current model is
247+
is better if the set of CIDRs are mostly a constant and are only referred
248+
to from one or two egress rules. It increases readability. However the
249+
drawback is if the CIDRs do change, then one has to ensure to update all
250+
the relevant ANPs and BANP accordingly.
86251

87252
## References
88253

0 commit comments

Comments
 (0)