Skip to content

Commit c83b88c

Browse files
eduolivareseduolivares
committed
Adapt BGP DT01 documentation to Pure L3 Spine&Leaf
1 parent ceaa627 commit c83b88c

File tree

4 files changed

+111
-30
lines changed

4 files changed

+111
-30
lines changed

‎examples/dt/bgp/bgp_dt01/README.md‎

Lines changed: 21 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,9 @@ workers cannot be configured as OVN Gateways.
4141
The OCP and EDPM nodes deployed with this DT are distributed into three
4242
different racks. Each rack is connected to two leaves.
4343
Hence, the distribution of the nodes in the racks is the following one:
44-
* rack0: compute-0, networker-0, ocp-master-0, ocp-worker-0, leaf-0, leaf-1
45-
* rack1: compute-1, networker-1, ocp-master-1, ocp-worker-1, leaf-2, leaf-3
46-
* rack2: compute-2, networker-2, ocp-master-2, ocp-worker-2, leaf-4, leaf-5
44+
* rack0: r0-compute-0, r0-networker-0, ocp-master-0, ocp-worker-0, leaf-0, leaf-1
45+
* rack1: r1-compute-0, r1-networker-0, ocp-master-1, ocp-worker-1, leaf-2, leaf-3
46+
* rack2: r2-compute-0, r2-networker-0, ocp-master-2, ocp-worker-2, leaf-4, leaf-5
4747

4848
The OCP tester (ocp-worker-3) is not included into any rack. It is not
4949
connected to any leaves, but to a router connected to the spines, due to the
@@ -66,7 +66,9 @@ network).
6666

6767
| Name | Type | CIDR |
6868
| ------------------------ | -------- | ---------------- |
69-
| Provisioning | untagged | 192.168.122.0/24 |
69+
| Controlplane rack0 | untagged | 192.168.122.0/24 |
70+
| Controlplane rack1 | untagged | 192.168.123.0/24 |
71+
| Controlplane rack2 | untagged | 192.168.124.0/24 |
7072
| Provider network | untagged | 192.168.133.0/24 |
7173
| RH OSP | untagged | 192.168.111.0/24 |
7274
| edpm/ocp to left leaves | untagged | 100.64.x.y/30 |
@@ -89,21 +91,27 @@ network).
8991
2. All the VMs that are neither Openstack nor Openshift nodes, i.e. those that
9092
act as routers, need to be properly configured in order to support the BGP
9193
protocol.
92-
3. The spine/leaf topology separates the overcloud nodes into different L2
94+
3. The spine/leaf topology separates the nodes into different L2
9395
network segments, called racks. Each rack includes two leaves, some OCP
9496
nodes and some EDPM nodes.
95-
4. A separate provisioning network is necessary to install Openstack on those
96-
nodes.
97-
5. Once Openstack is installed on them, dataplane connectivity is achieved
98-
using the BGP protocol.
97+
4. The Openstack services running on the EDPM nodes are installed using the BGP
98+
network, i.e. the Openstack services running on OCP nodes connect to the
99+
Openstack services running on EDPM nodes using BGP. There is no direct L2
100+
network connectivity between them. OCP version 4.18 or higher is required
101+
because the Openstack Operators use the frr-k8s feature for this and frr-k8s
102+
is not available in OCP 4.16.
103+
5. Once Openstack is installed on them, both dataplane and controlplane
104+
connections are achieved using the BGP protocol.
99105
6. Tests are executed from the OCP worker to verify external connectivity.
100106

101107
## Stages
102108

103109
All stages must be executed in the order listed below. Everything is required unless otherwise indicated.
104110

105111
1. [Configure taints on the OCP worker](configure-taints.md)
106-
2. [Install the OpenStack K8S operators and their dependencies](../../../common/)
107-
3. [Apply metallb customization required to run a speaker pod on the OCP tester node](metallb/)
108-
4. [Configuring networking and deploy the OpenStack control plane](control-plane.md)
109-
5. [Configure and deploy the dataplane - networker and compute nodes](data-plane.md)
112+
2. [Disable RP filters on OCP nodes](disable-rp-filters.md)
113+
3. [Install the OpenStack K8S operators and their dependencies](../../../common/)
114+
4. [Apply metallb customization required to run a speaker pod on the OCP tester node](metallb/)
115+
5. [Configuring networking and deploy the OpenStack control plane](control-plane.md)
116+
6. [Create BGPConfiguration after controplane is deployed](bgp-configuration.md)
117+
7. [Configure and deploy the dataplane - networker and compute nodes](data-plane.md)

‎examples/dt/bgp/bgp_dt01/bgp-configuration.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Create BGPConfiguration after controplane is deployed
2+
3+
An empty BGPConfiguration Openshift resource needs to be created.
4+
The infra-operator will detect this resource is created and will automatically
5+
apply the required Openshift BGP configuration.
6+
OCP 4.18 release is necessary for this.
7+
8+
The following CR needs to be applied:
9+
```
10+
apiVersion: network.openstack.org/v1beta1
11+
kind: BGPConfiguration
12+
metadata:
13+
name: bgpconfiguration
14+
namespace: openstack
15+
spec: {}
16+
```

‎examples/dt/bgp/bgp_dt01/data-plane.md‎

Lines changed: 35 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -16,26 +16,36 @@ Change to the bgp_dt01/ directory
1616
```
1717
cd architecture/examples/dt/bgp/bgp_dt01/
1818
```
19-
Edit the [edpm/networkers/values.yaml](edpm/networkers/values.yaml) file to suit
20-
your environment.
19+
Edit the networker values.yaml files corresponding to each rack (r0, r1 and r2)
20+
to suit your environment:
21+
- [edpm/networkers/r0/values.yaml](edpm/networkers/r0/values.yaml)
22+
- [edpm/networkers/r1/values.yaml](edpm/networkers/r1/values.yaml)
23+
- [edpm/networkers/r2/values.yaml](edpm/networkers/r2/values.yaml)
2124
```
2225
vi values.yaml
2326
```
24-
Edit the [edpm/computes/values.yaml](edpm/computes/values.yaml) file to suit
25-
your environment.
27+
Edit the compute values.yaml files corresponding to each rack (r0, r1 and r2)
28+
to suit your environment:
29+
- [edpm/computes/r0/values.yaml](edpm/computes/r0/values.yaml)
30+
- [edpm/computes/r1/values.yaml](edpm/computes/r1/values.yaml)
31+
- [edpm/computes/r2/values.yaml](edpm/computes/r2/values.yaml)
2632
```
2733
vi values.yaml
2834
```
2935

3036
## Create Networker and Compute Nodeset CRs
3137

32-
Generate the networkers dataplane nodeset CR.
38+
Generate the networkers dataplane nodeset CRs for each rack.
3339
```
34-
kustomize build edpm/networkers > edpm-networker-nodeset.yaml
40+
kustomize build edpm/networkers/r0 > edpm-r0-networker-nodeset.yaml
41+
kustomize build edpm/networkers/r1 > edpm-r1-networker-nodeset.yaml
42+
kustomize build edpm/networkers/r2 > edpm-r2-networker-nodeset.yaml
3543
```
36-
Generate the computes dataplane nodeset CR.
44+
Generate the computes dataplane nodeset CRs for each rack.
3745
```
38-
kustomize build edpm/computes > edpm-compute-nodeset.yaml
46+
kustomize build edpm/computes/r0 > edpm-r0-compute-nodeset.yaml
47+
kustomize build edpm/computes/r1 > edpm-r1-compute-nodeset.yaml
48+
kustomize build edpm/computes/r2 > edpm-r2-compute-nodeset.yaml
3949
```
4050

4151
## Create EDPM Deployment CR
@@ -46,27 +56,35 @@ kustomize build edpm/deployment > edpm-deployment.yaml
4656

4757
## Apply the Nodeset CRs
4858

49-
Apply the Networker nodeset CR
59+
Apply the Networker nodeset CRs
5060
```
51-
oc apply -f edpm-networker-nodeset.yaml
61+
oc apply -f edpm/networkers/r0/edpm-r0-networker-nodeset.yaml
62+
oc apply -f edpm/networkers/r1/edpm-r1-networker-nodeset.yaml
63+
oc apply -f edpm/networkers/r2/edpm-r2-networker-nodeset.yaml
5264
```
53-
Wait for Networker dataplane nodeset setup to finish
65+
Wait for Networker dataplane nodesets setup to finish
5466
```
55-
oc wait osdpns networker-nodes --for condition=SetupReady --timeout=600s
67+
oc wait osdpns r0-networker-nodes --for condition=SetupReady --timeout=600s
68+
oc wait osdpns r1-networker-nodes --for condition=SetupReady --timeout=600s
69+
oc wait osdpns r2-networker-nodes --for condition=SetupReady --timeout=600s
5670
```
57-
Apply the Compute nodeset CR
71+
Apply the Compute nodeset CRs
5872
```
59-
oc apply -f edpm-compute-nodeset.yaml
73+
oc apply -f edpm/computes/r0/edpm-r0-compute-nodeset.yaml
74+
oc apply -f edpm/computes/r1/edpm-r1-compute-nodeset.yaml
75+
oc apply -f edpm/computes/r2/edpm-r2-compute-nodeset.yaml
6076
```
61-
Wait for Compute dataplane nodeset setup to finish
77+
Wait for Compute dataplane nodesets setup to finish
6278
```
63-
oc wait osdpns compute-nodes --for condition=SetupReady --timeout=600s
79+
oc wait osdpns r0-compute-nodes --for condition=SetupReady --timeout=600s
80+
oc wait osdpns r1-compute-nodes --for condition=SetupReady --timeout=600s
81+
oc wait osdpns r2-compute-nodes --for condition=SetupReady --timeout=600s
6482
```
6583

6684
## Apply the deployment
6785
Start the deployment
6886
```
69-
oc apply -f edpm-deployment.yaml
87+
oc apply -f edpm/deployment/edpm-deployment.yaml
7088
```
7189
Wait for dataplane deployment to finish
7290
```

‎examples/dt/bgp/bgp_dt01/disable-rp-filters.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Disable RP filters on OCP workers
2+
3+
Reverse path filters need to be disabled on OCP workers running Openstack services.
4+
This is needed to make OCP workers forward traffic properly based on the BGP
5+
advertisements received.
6+
7+
The following CR needs to be applied:
8+
```
9+
apiVersion: tuned.openshift.io/v1
10+
kind: Tuned
11+
metadata:
12+
name: openshift-no-reapply-sysctl
13+
namespace: openshift-cluster-node-tuning-operator
14+
spec:
15+
profile:
16+
- data: |
17+
[main]
18+
summary=Optimize systems running OpenShift (provider specific parent profile)
19+
include=-provider-${f:exec:cat:/var/lib/ocp-tuned/provider},openshift
20+
21+
[sysctl]
22+
net.ipv4.conf.enp7s0.rp_filter=0
23+
net.ipv4.conf.enp8s0.rp_filter=0
24+
name: openshift-no-reapply-sysctl
25+
recommend:
26+
- match:
27+
- label: kubernetes.io/hostname
28+
value: worker-0
29+
- label: kubernetes.io/hostname
30+
value: worker-1
31+
- label: kubernetes.io/hostname
32+
value: worker-2
33+
- label: node-role.kubernetes.io/master
34+
operand:
35+
tunedConfig:
36+
reapply_sysctl: false
37+
priority: 15
38+
profile: openshift-no-reapply-sysctl
39+
```

0 commit comments

Comments
 (0)