Skip to content

Commit 85a82df

Browse files
maxwelldbmaxwelldb
authored
Merge pull request #27508 from dulek/kuryr-tuning
Adding info on setting Kuryr configuration options
2 parents 220bc71 + c4d0df3 commit 85a82df

File tree

6 files changed

+179
-0
lines changed

6 files changed

+179
-0
lines changed

‎installing/installing_openstack/installing-openstack-installer-kuryr.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ You can add additional networks, including provider networks, to the `platform.o
4747
After you deploy your cluster, you can attach pods to additional networks. For more information, see xref:../../networking/multiple_networks/understanding-multiple-networks.adoc#understanding-multiple-networks[Understanding multiple networks].
4848
====
4949

50+
include::modules/installation-osp-kuryr-port-pools.adoc[leveloffset=+2]
51+
include::modules/installation-osp-kuryr-settings-installing.adoc[leveloffset=+2]
5052
include::modules/ssh-agent-using.adoc[leveloffset=+1]
5153
include::modules/installation-osp-accessing-api.adoc[leveloffset=+1]
5254
include::modules/installation-osp-accessing-api-floating.adoc[leveloffset=+2]

‎installing/installing_openstack/installing-openstack-user-kuryr.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,8 @@ You can add additional networks, including provider networks, to the `platform.o
5555
After you deploy your cluster, you can attach pods to additional networks. For more information, see xref:../../networking/multiple_networks/understanding-multiple-networks.adoc#understanding-multiple-networks[Understanding multiple networks].
5656
====
5757

58+
include::modules/installation-osp-kuryr-port-pools.adoc[leveloffset=+2]
59+
include::modules/installation-osp-kuryr-settings-installing.adoc[leveloffset=+2]
5860
include::modules/installation-osp-fixing-subnet.adoc[leveloffset=+2]
5961
include::modules/installation-osp-emptying-worker-pools.adoc[leveloffset=+2]
6062
include::modules/installation-osp-modifying-networktype.adoc[leveloffset=+2]

‎modules/installation-osp-kuryr-port-pools.adoc

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_openstack/installing-openstack-installer-kuryr.adoc
4+
// * installing/installing_openstack/installing-openstack-user-kuryr.adoc
5+
// * post_installation_configuration/network-configuration.adoc
6+
7+
[id="installation-osp-kuryr-port-pools_{context}"]
8+
= Kuryr ports pools
9+
10+
A Kuryr ports pool maintains a number of ports on standby for pod creation.
11+
12+
Keeping ports on standby minimizes pod creation time. Without ports pools, Kuryr must explicitly request port creation or deletion whenever a pod is created or deleted.
13+
14+
The Neutron ports that Kuryr uses are created in subnets that are tied to namespaces. These pod ports are also added as subports to the primary port of {product-title} cluster nodes.
15+
16+
Because Kuryr keeps each namespace in a separate subnet, a separate ports pool is maintained for each namespace-worker pair.
17+
18+
Prior to installing a cluster, you can set the following parameters in the `cluster-network-03-config.yml` manifest file to configure ports pool behavior:
19+
20+
* The `enablePortPoolsPrepopulation` parameter controls pool prepopulation, which forces Kuryr to add ports to the pool when it is created, such as when a new host is added, or a new namespace is created. The default value is `false`.
21+
* The `poolMinPorts` parameter is the minimum number of free ports that are kept in the pool. The default value is `1`.
22+
* The `poolMaxPorts` parameter is the maximum number of free ports that are kept in the pool. A value of `0` disables that upper bound. This is the default setting.
23+
+
24+
If your OpenStack port quota is low, or you have a limited number of IP addresses on the pod network, consider setting this option to ensure that unneeded ports are deleted.
25+
* The `poolBatchPorts` parameter defines the maximum number of Neutron ports that can be created at once. The default value is `3`.

‎modules/installation-osp-kuryr-settings-active.adoc

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * post_installation_configuration/network-configuration.adoc
4+
5+
[id="installation-osp-kuryr-settings-active_{context}"]
6+
= Adjusting Kuryr ports pool settings in active deployments on {rh-openstack}
7+
8+
You can use a custom resource (CR) to configure how Kuryr manages {rh-openstack-first} Neutron ports to control the speed and efficiency of pod creation on a deployed cluster.
9+
10+
.Procedure
11+
12+
. From a command line, open the Cluster Network Operator (CNO) CR for editing:
13+
+
14+
[source,terminal]
15+
----
16+
$ oc edit networks.operator.openshift.io cluster
17+
----
18+
19+
. Edit the settings to meet your requirements. The following file is provided as an example:
20+
+
21+
[source,yaml]
22+
----
23+
apiVersion: operator.openshift.io/v1
24+
kind: Network
25+
metadata:
26+
name: cluster
27+
spec:
28+
clusterNetwork:
29+
- cidr: 10.128.0.0/14
30+
hostPrefix: 23
31+
serviceNetwork:
32+
- 172.30.0.0/16
33+
defaultNetwork:
34+
type: Kuryr
35+
kuryrConfig:
36+
enablePortPoolsPrepopulation: false <1>
37+
poolMinPorts: 1 <2>
38+
poolBatchPorts: 3 <3>
39+
poolMaxPorts: 5 <4>
40+
----
41+
<1> Set `enablePortPoolsPrepopulation` to `true` to make Kuryr create new Neutron ports after a namespace is created or a new node is added to the cluster. This setting raises the Neutron ports quota but can reduce the time that is required to spawn pods. The default value is `false`.
42+
<2> Kuryr creates new ports for a pool if the number of free ports in that pool is lower than the value of `poolMinPorts`. The default value is `1`.
43+
<3> `poolBatchPorts` controls the number of new ports that are created if the number of free ports is lower than the value of `poolMinPorts`. The default value is `3`.
44+
<4> If the number of free ports in a pool is higher than the value of `poolMaxPorts`, Kuryr deletes them until the number matches that value. Setting the value to `0` disables this upper bound, preventing pools from shrinking. The default value is `0`.
45+
46+
. Save your changes and quit the text editor to commit your changes.
47+
48+
[IMPORTANT]
49+
====
50+
Modifying these options on a running cluster forces the kuryr-controller and kuryr-cni pods to restart. As a result, the creation of new pods and services will be delayed.
51+
====

‎modules/installation-osp-kuryr-settings-installing.adoc

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_openstack/installing-openstack-installer-kuryr.adoc
4+
// * installing/installing_openstack/installing-openstack-user-kuryr.adoc
5+
6+
[id="installation-osp-kuryr-settings-installing_{context}"]
7+
= Adjusting Kuryr ports pools during installation
8+
9+
During installation, you can configure how Kuryr manages {rh-openstack-first} Neutron ports to control the speed and efficiency of pod creation.
10+
11+
.Prerequisites
12+
13+
* Create and modify the `install-config.yaml` file.
14+
15+
.Procedure
16+
17+
. From a command line, create the manifest files:
18+
+
19+
[source,terminal]
20+
----
21+
$ ./openshift-install create manifests --dir=<installation_directory> <1>
22+
----
23+
<1> For `<installation_directory>`, specify the name of the directory that
24+
contains the `install-config.yaml` file for your cluster.
25+
26+
. Create a file that is named `cluster-network-03-config.yml` in the
27+
`<installation_directory>/manifests/` directory:
28+
+
29+
[source,terminal]
30+
----
31+
$ touch <installation_directory>/manifests/cluster-network-03-config.yml <1>
32+
----
33+
<1> For `<installation_directory>`, specify the directory name that contains the
34+
`manifests/` directory for your cluster.
35+
+
36+
After creating the file, several network configuration files are in the
37+
`manifests/` directory, as shown:
38+
+
39+
[source,terminal]
40+
----
41+
$ ls <installation_directory>/manifests/cluster-network-*
42+
----
43+
+
44+
.Example output
45+
[source,terminal]
46+
----
47+
cluster-network-01-crd.yml
48+
cluster-network-02-config.yml
49+
cluster-network-03-config.yml
50+
----
51+
52+
. Open the `cluster-network-03-config.yml` file in an editor, and enter a custom resource (CR) that describes the Cluster Network Operator configuration that you want:
53+
+
54+
[source,terminal]
55+
----
56+
$ oc edit networks.operator.openshift.io cluster
57+
----
58+
59+
. Edit the settings to meet your requirements. The following file is provided as an example:
60+
+
61+
[source,yaml]
62+
----
63+
apiVersion: operator.openshift.io/v1
64+
kind: Network
65+
metadata:
66+
name: cluster
67+
spec:
68+
clusterNetwork:
69+
- cidr: 10.128.0.0/14
70+
hostPrefix: 23
71+
serviceNetwork:
72+
- 172.30.0.0/16
73+
defaultNetwork:
74+
type: Kuryr
75+
kuryrConfig:
76+
enablePortPoolsPrepopulation: false <1>
77+
poolMinPorts: 1 <2>
78+
poolBatchPorts: 3 <3>
79+
poolMaxPorts: 5 <4>
80+
openstackServiceNetwork: 172.30.0.0/15 <5>
81+
----
82+
<1> Set the value of `enablePortPoolsPrepopulation` to `true` to make Kuryr create new Neutron ports after a namespace is created or a new node is added to the cluster. This setting raises the Neutron ports quota but can reduce the time that is required to spawn pods. The default value is `false`.
83+
<2> Kuryr creates new ports for a pool if the number of free ports in that pool is lower than the value of `poolMinPorts`. The default value is `1`.
84+
<3> `poolBatchPorts` controls the number of new ports that are created if the number of free ports is lower than the value of `poolMinPorts`. The default value is `3`.
85+
<4> If the number of free ports in a pool is higher than the value of `poolMaxPorts`, Kuryr deletes them until the number matches that value. Setting this value to `0` disables this upper bound, preventing pools from shrinking. The default value is `0`.
86+
<5> The `openStackServiceNetwork` parameter defines the CIDR range of the network from which IP addresses are allocated to {rh-openstack} Octavia's LoadBalancers.
87+
+
88+
If this parameter is used with the Amphora driver, Octavia takes two IP addresses from this network for each load balancer: one for OpenShift and the other for VRRP connections. Because these IP addresses are managed by {product-title} and Neutron respectively, they must come from different pools.
89+
Therefore, the value of `openStackServiceNetwork` must be at least twice the size of the value of `serviceNetwork`, and the value of `serviceNetwork` must overlap entirely with the range that is defined by `openStackServiceNetwork`.
90+
+
91+
The CNO verifies that VRRP IP addresses that are taken from the range that is defined by this parameter do not overlap with the range that is defined by the `serviceNetwork` parameter.
92+
+
93+
If this parameter is not set, the CNO uses an expanded value of `serviceNetwork` that is determined by decrementing the prefix size by 1.
94+
95+
. Save the `cluster-network-03-config.yml` file, and exit the text editor.
96+
97+
. Optional: Back up the `manifests/cluster-network-03-config.yml` file. The installation program deletes the `manifests/` directory while creating the cluster.

‎post_installation_configuration/network-configuration.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,3 +103,5 @@ include::modules/router-performance-optimizations.adoc[leveloffset=+2]
103103
You can configure some aspects of a {product-title} on {rh-openstack-first} cluster after installation.
104104

105105
include::modules/installation-osp-configuring-api-floating-ip.adoc[leveloffset=+2]
106+
include::modules/installation-osp-kuryr-port-pools.adoc[leveloffset=+2]
107+
include::modules/installation-osp-kuryr-settings-active.adoc[leveloffset=+2]

0 commit comments

Comments
 (0)