Skip to content

Commit d0a8e3d

Browse files
rohennesrohennes
committed
TELCODOCS-1237: VRFs for consistent isolated routes
1 parent 2c49faf commit d0a8e3d

11 files changed

+766
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1323,6 +1323,8 @@ Topics:
13231323
File: configuring-egress-ips-ovn
13241324
- Name: Assigning an egress IP address
13251325
File: assigning-egress-ips-ovn
1326+
- Name: Configuring an egress service
1327+
File: configuring-egress-traffic-for-vrf-loadbalancer-services
13261328
- Name: Considerations for the use of an egress router pod
13271329
File: using-an-egress-router-ovn
13281330
- Name: Deploying an egress router pod in redirect mode
@@ -1451,6 +1453,8 @@ Topics:
14511453
File: metallb-configure-bfd-profiles
14521454
- Name: Configuring services to use MetalLB
14531455
File: metallb-configure-services
1456+
- Name: Managing symmetric routing with MetalLB
1457+
File: metallb-configure-return-traffic
14541458
- Name: MetalLB logging, troubleshooting, and support
14551459
File: metallb-troubleshoot-support
14561460
- Name: Associating secondary interfaces metrics to network attachments

‎images/357_OpenShift_MetalLB_VRF_0823.png

51.7 KB
Loading

‎modules/nw-egress-service-cr.adoc

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ovn_kubernetes_network_provider/configuring-egress-traffic-for-vrf-loadbalancer-services.adoc
4+
5+
:_content-type: REFERENCE
6+
[id="nw-egress-service-ovn-cr_{context}"]
7+
= Egress service custom resource
8+
9+
Define the configuration for an egress service in an `EgressService` custom resource. The following YAML describes the fields for the configuration of an egress service:
10+
11+
[source,yaml]
12+
----
13+
apiVersion: k8s.ovn.org/v1
14+
kind: EgressService
15+
metadata:
16+
name: <egress_service_name> <1>
17+
namespace: <namespace> <2>
18+
spec:
19+
sourceIPBy: <egress_traffic_ip> <3>
20+
nodeSelector: <4>
21+
matchLabels:
22+
node-role.kubernetes.io/<role>: ""
23+
network: <egress_traffic_network> <5>
24+
----
25+
<1> Specify the name for the egress service. The name of the `EgressService` resource must match the name of the load-balancer service that you want to modify.
26+
<2> Specify the namespace for the egress service. The namespace for the `EgressService` must match the namespace of the load-balancer service that you want to modify. The egress service is namespace-scoped.
27+
<3> Specify the source IP address of egress traffic for pods behind a service. Valid values are `LoadBalancerIP` or `Network`. Use the `LoadBalancerIP` value to assign the `LoadBalancer` service ingress IP address as the source IP address for egress traffic. Specify `Network` to assign the network interface IP address as the source IP address for egress traffic.
28+
<4> Optional: If you use the `LoadBalancerIP` value for the `sourceIPBy` specification, a single node handles the `LoadBalancer` service traffic. Use the `nodeSelector` field to limit which node can be assigned this task. When a node is selected to handle the service traffic, OVN-Kubernetes labels the node in the following format: `egress-service.k8s.ovn.org/<svc-namespace>-<svc-name>: ""`. When the `nodeSelector` field is not specified, any node can manage the `LoadBalancer` service traffic.
29+
<5> Optional: Specify the routing table for egress traffic. If you do not include the `network` specification, the egress service uses the default host network.
30+
31+
.Example egress service specification
32+
[source,yaml]
33+
----
34+
apiVersion: k8s.ovn.org/v1
35+
kind: EgressService
36+
metadata:
37+
name: test-egress-service
38+
namespace: test-namespace
39+
spec:
40+
sourceIPBy: "LoadBalancerIP"
41+
nodeSelector:
42+
matchLabels:
43+
vrf: "true"
44+
network: "2"
45+
----

‎modules/nw-egress-service-ovn.adoc

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ovn_kubernetes_network_provider/configuring-egress-traffic-for-vrf-loadbalancer-services.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nw-egress-service-ovn_{context}"]
7+
= Deploying an egress service
8+
9+
You can deploy an egress service to manage egress traffic for pods behind a `LoadBalancer` service.
10+
11+
The following example configures the egress traffic to have the same source IP address as the ingress IP address of the `LoadBalancer` service.
12+
13+
.Prerequisites
14+
15+
* Install the OpenShift CLI (`oc`).
16+
* Log in as a user with `cluster-admin` privileges.
17+
* You configured MetalLB `BGPPeer` resources.
18+
19+
.Procedure
20+
21+
. Create an `IPAddressPool` CR with the desired IP for the service:
22+
23+
.. Create a file, such as `ip-addr-pool.yaml`, with content like the following example:
24+
+
25+
[source,yaml]
26+
----
27+
apiVersion: metallb.io/v1beta1
28+
kind: IPAddressPool
29+
metadata:
30+
name: example-pool
31+
namespace: metallb-system
32+
spec:
33+
addresses:
34+
- 172.19.0.100/32
35+
----
36+
37+
.. Apply the configuration for the IP address pool by running the following command:
38+
+
39+
[source,terminal]
40+
----
41+
$ oc apply -f ip-addr-pool.yaml
42+
----
43+
44+
. Create `Service` and `EgressService` CRs.
45+
46+
.. Create a file, such as `service-egress-service.yaml`, with content like the following example:
47+
+
48+
[source,yaml,subs="+quotes,+macros"]
49+
----
50+
apiVersion: v1
51+
kind: Service
52+
metadata:
53+
name: example-service
54+
namespace: example-namespace
55+
annotations:
56+
metallb.universe.tf/address-pool: example-pool <1>
57+
spec:
58+
selector:
59+
app: example
60+
ports:
61+
- name: http
62+
protocol: TCP
63+
port: 8080
64+
targetPort: 8080
65+
type: LoadBalancer
66+
---
67+
apiVersion: k8s.ovn.org/v1
68+
kind: EgressService
69+
metadata:
70+
name: example-service
71+
namespace: example-namespace
72+
spec:
73+
sourceIPBy: "LoadBalancerIP" <2>
74+
nodeSelector: <3>
75+
matchLabels:
76+
node-role.kubernetes.io/worker: ""
77+
----
78+
<1> The `LoadBalancer` service uses the IP address assigned by MetalLB from the `example-pool` IP address pool.
79+
<2> This example uses the `LoadBalancerIP` value to assign the ingress IP address of the `LoadBalancer` service as the source IP address of egress traffic.
80+
<3> When you specify the `LoadBalancerIP` value, a single node handles the `LoadBalancer` service's traffic. In this example, only nodes with the `worker` label can be selected to handle the traffic. When a node is selected, OVN-Kubernetes labels the node in the following format `egress-service.k8s.ovn.org/<svc-namespace>-<svc-name>: ""`.
81+
+
82+
[NOTE]
83+
====
84+
If you use the `sourceIPBy: "LoadBalancerIP"` setting, you must specify the load-balancer node in the `BGPAdvertisement` custom resource (CR).
85+
====
86+
87+
.. Apply the configuration for the service and egress service by running the following command:
88+
+
89+
[source,terminal]
90+
----
91+
$ oc apply -f service-egress-service.yaml
92+
----
93+
94+
. Create a `BGPAdvertisement` CR to advertise the service:
95+
96+
.. Create a file, such as `service-bgp-advertisement.yaml`, with content like the following example:
97+
+
98+
[source,yaml]
99+
----
100+
apiVersion: metallb.io/v1beta1
101+
kind: BGPAdvertisement
102+
metadata:
103+
name: example-bgp-adv
104+
namespace: metallb-system
105+
spec:
106+
ipAddressPools:
107+
- example-pool
108+
nodeSelector:
109+
- matchLabels:
110+
egress-service.k8s.ovn.org/example-namespace-example-service: "" <1>
111+
----
112+
113+
<1> In this example, the `EgressService` CR configures the source IP address for egress traffic to use the load-balancer service IP address. Therefore, you must specify the load-balancer node for return traffic to use the same return path for the traffic originating from the pod.
114+
115+
.Verification
116+
117+
. Verify that you can access the application endpoint of the pods running behind the MetalLB service by running the following command:
118+
+
119+
[source,terminal]
120+
----
121+
$ curl <external_ip_address>:<port_number> <1>
122+
----
123+
<1> Update the external IP address and port number to suit your application endpoint.
124+
125+
. If you assigned the `LoadBalancer` service's ingress IP address as the source IP address for egress traffic, verify this configuration by using tools such as `tcpdump` to analyze packets received at the external client.

‎modules/nw-metallb-configure-return-traffic-proc.adoc

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ovn_kubernetes_network_provider/configuring-egress-traffic-for-vrf-loadbalancer-services.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nw-metallb-configure-return-traffic-proc_{context}"]
7+
= Configuring symmetric routing by using VRFs with MetalLB
8+
9+
You can configure symmetric network routing for applications behind a MetalLB service that require the same ingress and egress network paths.
10+
11+
This example associates a VRF routing table with MetalLB and an egress service to enable symmetric routing for ingress and egress traffic for pods behind a `LoadBalancer` service.
12+
13+
[NOTE]
14+
====
15+
* If you use the `sourceIPBy: "LoadBalancerIP"` setting in the `EgressService` CR, you must specify the load-balancer node in the `BGPAdvertisement` custom resource (CR).
16+
17+
* You can use the `sourceIPBy: "Network"` setting on clusters that use OVN-Kubernetes configured with the `gatewayConfig.routingViaHost` specification set to `true` only. Additionally, if you use the `sourceIPBy: "Network"` setting, you must schedule the application workload on nodes configured with the network VRF instance.
18+
====
19+
20+
.Prerequisites
21+
22+
* Install the OpenShift CLI (`oc`).
23+
* Log in as a user with `cluster-admin` privileges.
24+
25+
.Procedure
26+
27+
. Create a `NodeNetworkConfigurationPolicy` CR to define the VRF instance:
28+
29+
.. Create a file, such as `node-network-vrf.yaml`, with content like the following example:
30+
+
31+
[source,yaml]
32+
----
33+
apiVersion: nmstate.io/v1
34+
kind: NodeNetworkConfigurationPolicy
35+
metadata:
36+
name: vrfpolicy <1>
37+
spec:
38+
nodeSelector:
39+
vrf: "true" <2>
40+
maxUnavailable: 3
41+
desiredState:
42+
interfaces:
43+
- name: ens4vrf <3>
44+
type: vrf <4>
45+
state: up
46+
vrf:
47+
port:
48+
- ens4 <5>
49+
route-table-id: 2 <6>
50+
routes: <7>
51+
config:
52+
- destination: 0.0.0.0/0
53+
metric: 150
54+
next-hop-address: 192.168.130.1
55+
next-hop-interface: ens4
56+
table-id: 2
57+
route-rules: <8>
58+
config:
59+
- ip-to: 172.30.0.0/16
60+
priority: 998
61+
route-table: 254
62+
- ip-to: 10.132.0.0/14
63+
priority: 998
64+
route-table: 254
65+
----
66+
<1> The name of the policy.
67+
<2> This example applies the policy to all nodes with the label `vrf:true`.
68+
<3> The name of the interface.
69+
<4> The type of interface. This example creates a VRF instance.
70+
<5> The node interface that the VRF attaches to.
71+
<6> The name of the route table ID for the VRF.
72+
<7> Defines the configuration for network routes. The `next-hop-address` field defines the IP address of the next hop for the route. The `next-hop-interface` field defines the outgoing interface for the route. In this example, the VRF routing table is `2`, which references the ID that you define in the `EgressService` CR.
73+
<8> Defines additional route rules. The `ip-to` fields must match the `Cluster Network` CIDR and `Service Network` CIDR. You can view the values for these CIDR address specifications by running the following command: `oc describe network.config/cluster`.
74+
75+
.. Apply the policy by running the following command:
76+
+
77+
[source,terminal]
78+
----
79+
$ oc apply -f node-network-vrf.yaml
80+
----
81+
82+
. Create a `BGPPeer` custom resource (CR):
83+
84+
.. Create a file, such as `frr-via-vrf.yaml`, with content like the following example:
85+
+
86+
[source,yaml]
87+
----
88+
apiVersion: metallb.io/v1beta2
89+
kind: BGPPeer
90+
metadata:
91+
name: frrviavrf
92+
namespace: metallb-system
93+
spec:
94+
myASN: 100
95+
peerASN: 200
96+
peerAddress: 192.168.130.1
97+
vrf: ens4vrf <1>
98+
----
99+
<1> Specifies the VRF instance to associate with the BGP peer. MetalLB can advertise services and make routing decisions based on the routing information in the VRF.
100+
101+
.. Apply the configuration for the BGP peer by running the following command:
102+
+
103+
[source,terminal]
104+
----
105+
$ oc apply -f frr-via-vrf.yaml
106+
----
107+
108+
. Create an `IPAddressPool` CR:
109+
110+
.. Create a file, such as `first-pool.yaml`, with content like the following example:
111+
+
112+
[source,yaml]
113+
----
114+
apiVersion: metallb.io/v1beta1
115+
kind: IPAddressPool
116+
metadata:
117+
name: first-pool
118+
namespace: metallb-system
119+
spec:
120+
addresses:
121+
- 192.169.10.0/32
122+
----
123+
124+
.. Apply the configuration for the IP address pool by running the following command:
125+
+
126+
[source,terminal]
127+
----
128+
$ oc apply -f first-pool.yaml
129+
----
130+
131+
. Create a `BGPAdvertisement` CR:
132+
133+
.. Create a file, such as `first-adv.yaml`, with content like the following example:
134+
+
135+
[source,yaml]
136+
----
137+
apiVersion: metallb.io/v1beta1
138+
kind: BGPAdvertisement
139+
metadata:
140+
name: first-adv
141+
namespace: metallb-system
142+
spec:
143+
ipAddressPools:
144+
- first-pool
145+
peers:
146+
- frrviavrf <1>
147+
nodeSelectors:
148+
- matchLabels:
149+
egress-service.k8s.ovn.org/test-server1: "" <2>
150+
----
151+
<1> In this example, MetalLB advertises a range of IP addresses from the `first-pool` IP address pool to the `frrviavrf` BGP peer.
152+
<2> In this example, the `EgressService` CR configures the source IP address for egress traffic to use the load-balancer service IP address. Therefore, you must specify the load-balancer node for return traffic to use the same return path for the traffic originating from the pod.
153+
154+
.. Apply the configuration for the BGP advertisement by running the following command:
155+
+
156+
[source,terminal]
157+
----
158+
$ oc apply -f first-adv.yaml
159+
----
160+
161+
. Create an `EgressService` CR:
162+
163+
.. Create a file, such as `egress-service.yaml`, with content like the following example:
164+
+
165+
[source,yaml,options="nowrap",role="white-space-pre"]
166+
----
167+
apiVersion: k8s.ovn.org/v1
168+
kind: EgressService
169+
metadata:
170+
name: server1 <1>
171+
namespace: test <2>
172+
spec:
173+
sourceIPBy: "LoadBalancerIP" <3>
174+
nodeSelector:
175+
matchLabels:
176+
vrf: "true" <4>
177+
network: "2" <5>
178+
----
179+
<1> Specify the name for the egress service. The name of the `EgressService` resource must match the name of the load-balancer service that you want to modify.
180+
<2> Specify the namespace for the egress service. The namespace for the `EgressService` must match the namespace of the load-balancer service that you want to modify. The egress service is namespace-scoped.
181+
<3> This example assigns the `LoadBalancer` service ingress IP address as the source IP address for egress traffic.
182+
<4> If you specify `LoadBalancer` for the `sourceIPBy` specification, a single node handles the `LoadBalancer` service traffic. In this example, only a node with the label `vrf: "true"` can handle the service traffic. If you do not specify a node, OVN-Kubernetes selects a worker node to handle the service traffic. When a node is selected, OVN-Kubernetes labels the node in the following format: `egress-service.k8s.ovn.org/<svc_namespace>-<svc_name>: ""`.
183+
<5> Specify the routing table for egress traffic.
184+
185+
.. Apply the configuration for the egress service by running the following command:
186+
+
187+
[source,terminal]
188+
----
189+
$ oc apply -f egress-service.yaml
190+
----
191+
192+
.Verification
193+
194+
. Verify that you can access the application endpoint of the pods running behind the MetalLB service by running the following command:
195+
+
196+
[source,terminal]
197+
----
198+
$ curl <external_ip_address>:<port_number> <1>
199+
----
200+
<1> Update the external IP address and port number to suit your application endpoint.
201+
202+
. Optional: If you assigned the `LoadBalancer` service ingress IP address as the source IP address for egress traffic, verify this configuration by using tools such as `tcpdump` to analyze packets received at the external client.
203+

0 commit comments

Comments
 (0)