Skip to content

Commit 288438d

Browse files
jab-rhjab-rh
authored
Merge pull request #62157 from jab-rh/OSDOCS-5280
 OSDOCS-5280: Add secondary external gateway support for ovn-k8s
2 parents 16de601 + 9110005 commit 288438d

File tree

6 files changed

+421
-0
lines changed

6 files changed

+421
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1355,6 +1355,8 @@ Topics:
13551355
File: logging-network-policy
13561356
- Name: Configuring IPsec encryption
13571357
File: configuring-ipsec-ovn
1358+
- Name: Configure an external gateway through a secondary network interface
1359+
File: configuring-secondary-external-gateway
13581360
- Name: Configuring an egress firewall for a project
13591361
File: configuring-egress-firewall-ovn
13601362
- Name: Viewing an egress firewall for a project

‎modules/nw-secondary-ext-gw-about.adoc

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ovn_kubernetes_network_provider/configuring-secondary-external-gateway.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nw-secondary-ext-gw-about_{context}"]
7+
= How {product-title} determines the external gateway IP address
8+
9+
You configure a secondary external gateway with the `AdminPolicyBasedExternalRoute` custom resource from the the `k8s.ovn.org` API group. The custom resource (CR) supports static and dynamic approaches to specifying an external gateway's IP address. Each namespace that a `AdminPolicyBasedExternalRoute` CR targets cannot be selected by any other `AdminPolicyBasedExternalRoute` CR. A namespace cannot have concurrent secondary external gateways.
10+
11+
Static assignment:: You specify an IP address directly.
12+
Dynamic assignment:: You specify an IP address indirectly, with namespace and pod selectors, and an optional network attachment definition.
13+
+
14+
--
15+
- If the name of a network attachment definition is provided, the external gateway IP address of the network attachment is used.
16+
- If the name of a network attachment definition is not provided, the external gateway IP address for the pod itself is used. However, this approach works only if the pod is configured with `hostNetwork` set to `true`.
17+
--

‎modules/nw-secondary-ext-gw-configure.adoc

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ovn_kubernetes_network_provider/configuring-secondary-external-gateway.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nw-secondary-ext-gw-configure_{context}"]
7+
= Configure a secondary external gateway
8+
9+
You can configure a secondary external gateway for a namespace in your cluster.
10+
11+
.Prerequisites
12+
13+
* You installed the OpenShift CLI (`oc`).
14+
* You are logged in to the cluster with a user with `cluster-admin` privileges.
15+
16+
.Procedure
17+
18+
. Create a YAML file that contains an `AdminPolicyBasedExternalRoute` object.
19+
. To create an admin policy based external route, enter the following command:
20+
+
21+
[source,terminal]
22+
----
23+
$ oc create -f <file>.yaml
24+
----
25+
+
26+
--
27+
where:
28+
29+
`<file>`:: Specifies the name of the YAML file that you created in the previous step.
30+
--
31+
+
32+
.Example output
33+
[source,text]
34+
----
35+
adminpolicybasedexternalroute.k8s.ovn.org/default-route-policy created
36+
----
37+
38+
. To confirm that the admin policy based external route was created, enter the following command:
39+
+
40+
[source,terminal]
41+
----
42+
$ oc describe apbexternalroute <name> | tail -n 6
43+
----
44+
+
45+
--
46+
where:
47+
48+
`<name>`:: Specifies the name of the `AdminPolicyBasedExternalRoute` object.
49+
--
50+
+
51+
.Example output
52+
[source,text]
53+
----
54+
Status:
55+
Last Transition Time: 2023-04-24T15:09:01Z
56+
Messages:
57+
Configured external gateway IPs: 172.18.0.8
58+
Status: Success
59+
Events: <none>
60+
----
61+
62+
////
63+
.Verification
64+
65+
If you created an `AdminPolicyBasedExternalRoute` object that selects a host-network pod IP address as the secondary external gateway, you can confirm that the next hop is correct for a pod with the following steps:
66+
67+
. To get the IP address of the pod, enter the following command:
68+
+
69+
[source,terminal]
70+
----
71+
oc get pods/<pod_name> -n <namespace> -o wide
72+
----
73+
+
74+
--
75+
where:
76+
77+
`<pod_name>`:: Specifies the name of the pod.
78+
`<namespace>`:: Specifies the namespace of the pod.
79+
--
80+
+
81+
.Example output
82+
[source,text]
83+
----
84+
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
85+
ns1 pod1 1/1 Running 1 (37m ago) 41m 10.130.0.8 node1 <none> <none>
86+
----
87+
88+
. Confirm that the IP address from the previous step is available as an external gateway.
89+
90+
.. To find the the OVN-Kubernetes control plane pod that manages the next hop for the pod, enter the following command:
91+
+
92+
[source,terminal]
93+
----
94+
$ oc get pod -n openshift-ovn-kubernetes \
95+
--field-selector spec.nodeName=<node_name> \
96+
-o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' | \
97+
grep ovnkube-node-
98+
----
99+
+
100+
--
101+
where:
102+
103+
`<node_name>`:: Specifies the name of the node from the `NODE` column that the pod from the previous step is running on.
104+
--
105+
+
106+
.Example output
107+
[source,text]
108+
----
109+
ovnkube-node-rpt55
110+
----
111+
112+
.. To confirm that the OVN-Kubernetes node pod includes the correct next hop, enter the following command:
113+
+
114+
[source,terminal]
115+
----
116+
$ oc exec -t <pod_name> -n openshift-ovn-kubernetes -c nbdb \
117+
-- ovn-nbctl lr-route-list GR_ovn-work | grep <pod_ip> -A 6 -B 4
118+
119+
oc exec -ti <pod_name> -n openshift-ovn-kubernetes -c nbdb -- ovn-nbctl lr-route-list GR_<node_name> | grep <pod_id> -A6 -B4
120+
----
121+
+
122+
--
123+
where:
124+
125+
`<pod_name>`:: Specifies the name of the OVN-Kubernetes node pod from the previous step.
126+
`<node_name>`:: Specifies the name of the cluster node that the OVN-Kubernetes node pod is running on.
127+
`<pod_ip>`:: Specifies the name of the pod IP address.
128+
--
129+
+
130+
.Example output
131+
[source,text]
132+
----
133+
IPv4 Routes
134+
Route Table
135+
136+
:
137+
10.128.2.206 172.18.0.10 src-ip rtoe-GR_worker-0-1 ecmp-symmetric-reply bfd
138+
10.128.3.229 172.18.0.10 src-ip rtoe-GR_worker-0-1 ecmp-symmetric-reply bfd
139+
169.254.169.0/29 169.254.169.4 dst-ip rtoe-GR_worker-0-1
140+
10.128.0.0/14 100.64.0.1 dst-ip
141+
0.0.0.0/0 192.168.123.1 dst-ip rtoe-GR_worker-0-1
142+
----
143+
////

‎modules/nw-secondary-ext-gw-object.adoc

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ovn_kubernetes_network_provider/configuring-secondary-external-gateway.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="nw-secondary-ext-gw-object_{context}"]
7+
= AdminPolicyBasedExternalRoute object configuration
8+
9+
You can define an `AdminPolicyBasedExternalRoute` object, which is cluster scoped, with the following properties. A namespace can be selected by only one `AdminPolicyBasedExternalRoute` CR at a time.
10+
11+
.`AdminPolicyBasedExternalRoute` object
12+
[cols=".^3,.^2,.^5a",options="header"]
13+
14+
|====
15+
|Field|Type|Description
16+
17+
|`metadata.name`
18+
|`string`
19+
|
20+
Specifies the name of the `AdminPolicyBasedExternalRoute` object.
21+
22+
|`spec.from`
23+
|`string`
24+
|
25+
Specifies a namespace selector that the routing polices apply to. Only `namespaceSelector` is supported for external traffic. For example:
26+
27+
[source,yaml]
28+
----
29+
from:
30+
namespaceSelector:
31+
matchLabels:
32+
kubernetes.io/metadata.name: novxlan-externalgw-ecmp-4059
33+
----
34+
35+
A namespace can be targeted by only one `AdminPolicyBasedExternalRoute` CR. If a namespace is selected by more than one `AdminPolicyBasedExternalRoute` CR, a `failed` error status occurs on the second and subsequent CRs targeting the same namespace.
36+
37+
|`spec.nextHops`
38+
|`object`
39+
|
40+
Specifies the destinations where the packets are forwarded to. Must be either or both of `static` and `dynamic`. You must have at least one next hop defined.
41+
42+
|====
43+
44+
45+
.`nextHops` object
46+
[cols=".^3,.^2,.^5a",options="header"]
47+
|====
48+
|Field|Type|Description
49+
50+
|`static`
51+
|`array`
52+
| Specifies an array of static IP addresses.
53+
54+
|`dynamic`
55+
|`array`
56+
| Specifies an array of pod selectors corresponding to pods configured with a network attachment definition to use as the external gateway target.
57+
58+
|====
59+
60+
61+
.`nextHops.static` object
62+
[cols=".^3,.^2,.^5a",options="header"]
63+
|====
64+
|Field|Type|Description
65+
66+
|`ip`
67+
|`string`
68+
| Specifies either an IPv4 or IPv6 address of the next destination hop.
69+
70+
|`bfdEnabled`
71+
|`boolean`
72+
|Optional: Specifies whether Bi-Directional Forwarding Detection (BFD) is supported by the network. The default value is `false`.
73+
74+
|====
75+
76+
.`nextHops.dynamic` object
77+
[cols=".^3,.^2,.^5a",options="header"]
78+
|====
79+
|Field|Type|Description
80+
81+
|`podSelector`
82+
|`string`
83+
|
84+
Specifies a [set-based](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#set-based-requirement) label selector to filter the pods in the namespace that match this network configuration.
85+
86+
|`namespaceSelector`
87+
|`string`
88+
| Specifies a `set-based` selector to filter the namespaces that the `podSelector` applies to. You must specify a value for this field.
89+
90+
91+
|`bfdEnabled`
92+
|`boolean`
93+
|Optional: Specifies whether Bi-Directional Forwarding Detection (BFD) is supported by the network. The default value is `false`.
94+
95+
|`networkAttachmentName`
96+
|`string`
97+
|
98+
Optional: Specifies the name of a network attachment definition. The name must match the list of logical networks associated with the pod. If this field is not specified, the host network of the pod is used. However, the pod must be configure as a host network pod to use the host network.
99+
100+
|====
101+
102+
[id="example-secondary-external-gateway-configurations_{context}"]
103+
== Example secondary external gateway configurations
104+
105+
In the following example, the `AdminPolicyBasedExternalRoute` object configures two static IP addresses as external gateways for pods in namespaces with the `kubernetes.io/metadata.name: novxlan-externalgw-ecmp-4059` label.
106+
107+
[source,yaml]
108+
----
109+
apiVersion: k8s.ovn.org/v1
110+
kind: AdminPolicyBasedExternalRoute
111+
metadata:
112+
name: default-route-policy
113+
spec:
114+
from:
115+
namespaceSelector:
116+
matchLabels:
117+
kubernetes.io/metadata.name: novxlan-externalgw-ecmp-4059
118+
nextHops:
119+
static:
120+
- ip: "172.18.0.8"
121+
- ip: "172.18.0.9"
122+
----
123+
124+
In the following example, the `AdminPolicyBasedExternalRoute` object configures a dynamic external gateway. The IP addresses used for the external gateway are derived from the additional network attachments associated with each of the selected pods.
125+
126+
[source,yaml]
127+
----
128+
apiVersion: k8s.ovn.org/v1
129+
kind: AdminPolicyBasedExternalRoute
130+
metadata:
131+
name: shadow-traffic-policy
132+
spec:
133+
from:
134+
namespaceSelector:
135+
matchLabels:
136+
externalTraffic: ""
137+
nextHops:
138+
dynamic:
139+
- podSelector:
140+
matchLabels:
141+
gatewayPod: ""
142+
namespaceSelector:
143+
matchLabels:
144+
shadowTraffic: ""
145+
networkAttachmentName: shadow-gateway
146+
- podSelector:
147+
matchLabels:
148+
gigabyteGW: ""
149+
namespaceSelector:
150+
matchLabels:
151+
gatewayNamespace: ""
152+
networkAttachmentName: gateway
153+
----
154+
155+
In the following example, the `AdminPolicyBasedExternalRoute` object configures both static and dynamic external gateways.
156+
157+
[source,yaml]
158+
----
159+
apiVersion: k8s.ovn.org/v1
160+
kind: AdminPolicyBasedExternalRoute
161+
metadata:
162+
name: multi-hop-policy
163+
spec:
164+
from:
165+
namespaceSelector:
166+
matchLabels:
167+
trafficType: "egress"
168+
nextHops:
169+
static:
170+
- ip: "172.18.0.8"
171+
- ip: "172.18.0.9"
172+
dynamic:
173+
- podSelector:
174+
matchLabels:
175+
gatewayPod: ""
176+
namespaceSelector:
177+
matchLabels:
178+
egressTraffic: ""
179+
networkAttachmentName: gigabyte
180+
----

‎modules/nw-secondary-ext-gw-status.adoc

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ovn_kubernetes_network_provider/configuring-secondary-external-gateway.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nw-secondary-ext-gw-status_{context}"]
7+
= View the status of a secondary external gateway
8+
9+
You can view the status of a secondary external gateway that is configured for your cluster. The `status` field for the `AdminPolicyBasedExternalRoute` custom resource reports recent status messages whenever you update the resource, subject to a few limitations:
10+
11+
- Namespaces impacted are not reported in status messages
12+
- Pods selected as part of a dynamic next hop configuration do not trigger status updates as a result of pod lifecycle events, such as pod termination
13+
14+
.Prerequisites
15+
16+
* You installed the OpenShift CLI (`oc`).
17+
* You are logged in to the cluster with a user with `cluster-admin` privileges.
18+
19+
.Procedure
20+
21+
* To access the status logs for a secondary external gateway, enter the following command:
22+
+
23+
[source,terminal]
24+
----
25+
$ oc get adminpolicybasedexternalroutes <name> -o yaml
26+
----
27+
+
28+
--
29+
where:
30+
31+
`<name>`:: Specifies the name of an `AdminPolicyBasedExternalRoute` object.
32+
--
33+
+
34+
.Example output
35+
[source,text]
36+
----
37+
...
38+
Status:
39+
Last Transition Time: 2023-04-24T14:49:45Z
40+
Messages:
41+
Configured external gateway IPs: 172.18.0.8,172.18.0.9
42+
Configured external gateway IPs: 172.18.0.8
43+
Status: Success
44+
Events: <none>
45+
----

0 commit comments

Comments
 (0)