Skip to content

Commit c32e36b

Browse files
authored
Merge pull request #56145 from lpettyjo/OSDOCS-5234
OSDOCS-5234:vSphere topology in installer
2 parents d4fae84 + bac3cca commit c32e36b

6 files changed

+210
-96
lines changed
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
// Module included in the following assemblies:
2+
//
3+
// storage/container_storage_interface/persistent-storage-csi-vsphere.adoc
4+
//
5+
6+
:content-type: PROCEDURE
7+
[id="persistent-storage-csi-vsphere-top-aware-during-install_{context}"]
8+
= Creating vSphere storage topology during installation
9+
10+
== Procedure
11+
12+
* Specify the topology during installation. See the _Configuring regions and zones for a VMware vCenter_ section.
13+
14+
No additional action is necessary and the default storage class that is created by {product-title}
15+
is topology aware and should allow provisioning of volumes in different failure domains.
Lines changed: 34 additions & 89 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,22 @@
11
// Module included in the following assemblies:
22
//
3-
// persistent-storage-csi-vsphere.adoc
3+
// storage/container_storage_interface/persistent-storage-csi-vsphere.adoc
44
//
55

66
:content-type: PROCEDURE
7-
[id="persistent-storage-csi-vsphere-top-aware_{context}"]
8-
= Configuring vSphere CSI Topology
9-
10-
{product-title} provides the ability to deploy {product-title} for vSphere on different zones and regions, which allows you to deploy over multiple compute clusters, thus helping to avoid a single point of failure.
7+
[id="persistent-storage-csi-vsphere-top-aware-infra-top_{context}"]
8+
= Creating vSphere storage topology without an infra topology
119

1210
[NOTE]
1311
====
14-
{product-title} on vSphere does not support multiple Datacenters.
12+
{product-title} recommends using the infrastructure object for specifying failure domains in a topology aware setup. Specifying failure domains in the infrastructure object and specify topology-categories in the `ClusterCSIDriver` object at the same time is an unsupported operation.
1513
====
1614

17-
This is accomplished by defining zone and region categories in vCenter, and then assigning these categories to different failure domains, such as a compute cluster, by creating tags for these zone and region categories. After you have created the appropriate categories, and assigned tags to vCenter objects, you can create additional machinesets that create virtual machines (VMs) that are responsible for scheduling pods in those failure domains.
18-
19-
.Procedure
15+
== Procedure
2016
. In the VMware vCenter vSphere client GUI, define appropriate zone and region catagories and tags.
2117
+
2218
While vSphere allows you to create categories with any arbitrary name, {product-title} strongly recommends use of `openshift-region` and `openshift-zone` names for defining topology.
2319
+
24-
The following example defines two failure domains with one region and two zones:
25-
+
26-
.vSphere topology with one region and two zones
27-
|===
28-
|Compute cluster | Failure domain |Description
29-
30-
|Compute cluster: ocp1,
31-
Datacenter: Atlanta
32-
|openshift-region: us-east-1 (tag), openshift-zone: us-east-1a (tag)
33-
|This defines a failure domain in region us-east-1 with zone us-east-1a.
34-
35-
|Computer cluster: ocp2,
36-
Datacenter: Atlanta
37-
|openshift-region: us-east-1 (tag), openshift-zone: us-east-1b (tag)
38-
|This defines a different failure domain within the same region called us-east-1b.
39-
|===
40-
+
4120
For more information about vSphere categories and tags, see the VMware vSphere documentation.
4221

4322
. To allow the container storage interface (CSI) driver to detect this topology, edit the `clusterCSIDriver` object YAML file `driverConfig` section:
@@ -56,19 +35,19 @@ For more information about vSphere categories and tags, see the VMware vSphere d
5635
apiVersion: operator.openshift.io/v1
5736
kind: ClusterCSIDriver
5837
metadata:
59-
name: csi.vsphere.vmware.com
38+
name: csi.vsphere.vmware.com
6039
spec:
61-
logLevel: Normal
62-
managementState: Managed
63-
observedConfig: null
64-
operatorLogLevel: Normal
65-
unsupportedConfigOverrides: null
66-
driverConfig:
67-
driverType: vSphere <1>
68-
vSphere:
69-
topologyCategories: <2>
70-
- openshift-zone
71-
- openshift-region
40+
logLevel: Normal
41+
managementState: Managed
42+
observedConfig: null
43+
operatorLogLevel: Normal
44+
unsupportedConfigOverrides: null
45+
driverConfig:
46+
driverType: vSphere <1>
47+
vSphere:
48+
topologyCategories: <2>
49+
- openshift-zone
50+
- openshift-region
7251
----
7352
<1> Ensure that `driverType` is set to `vSphere`.
7453
<2> `openshift-zone` and `openshift-region` categories created earlier in vCenter.
@@ -84,14 +63,14 @@ spec:
8463
+
8564
[source, terminal]
8665
----
87-
NAME DRIVERS AGE
88-
co8-4s88d-infra-2m5vd 1 27m
89-
co8-4s88d-master-0 1 70m
90-
co8-4s88d-master-1 1 70m
91-
co8-4s88d-master-2 1 70m
92-
co8-4s88d-worker-j2hmg 1 47m
93-
co8-4s88d-worker-mbb46 1 47m
94-
co8-4s88d-worker-zlk7d 1 47m
66+
NAME DRIVERS AGE
67+
co8-4s88d-infra-2m5vd 1 27m
68+
co8-4s88d-master-0 1 70m
69+
co8-4s88d-master-1 1 70m
70+
co8-4s88d-master-2 1 70m
71+
co8-4s88d-worker-j2hmg 1 47m
72+
co8-4s88d-worker-mbb46 1 47m
73+
co8-4s88d-worker-zlk7d 1 47m
9574
----
9675
+
9776
[source, terminal]
@@ -105,14 +84,14 @@ co8-4s88d-worker-zlk7d 1 47m
10584
----
10685
...
10786
spec:
108-
drivers:
109-
- allocatable:
110-
count: 59
111-
name: csi-vsphere.vmware.com
112-
nodeID: co8-4s88d-worker-j2hmg
113-
topologyKeys: <1>
114-
- topology.csi.vmware.com/openshift-zone
115-
- topology.csi.vmware.com/openshift-region
87+
drivers:
88+
- allocatable:
89+
count: 59
90+
name: csi-vsphere.vmware.com
91+
nodeID: co8-4s88d-worker-j2hmg
92+
topologyKeys: <1>
93+
- topology.csi.vmware.com/openshift-zone
94+
- topology.csi.vmware.com/openshift-region
11695
----
11796
<1> Topology keys from vSphere `openshift-zone` and `openshift-region` catagories.
11897
+
@@ -172,38 +151,4 @@ volumeBindingMode: WaitForFirstConsumer
172151
[NOTE]
173152
====
174153
You can also create the storage class by editing the preceding YAML file and running the command `oc create -f $FILE`.
175-
====
176-
177-
.Results
178-
Creating persistent volume claims (PVCs) and PVs from the topology aware storage class are truly zonal, and should use the datastore in their respective zone depending on how pods are scheduled:
179-
180-
[source, terminal]
181-
----
182-
~ $ oc get pv <pv-name> -o yaml
183-
----
184-
185-
.Example output
186-
187-
[source, terminal]
188-
----
189-
...
190-
nodeAffinity:
191-
required:
192-
nodeSelectorTerms:
193-
- matchExpressions:
194-
- key: topology.csi.vmware.com/openshift-zone <1>
195-
operator: In
196-
values:
197-
- <openshift-zone>
198-
-key: topology.csi.vmware.com/openshift-region <1>
199-
operator: In
200-
values:
201-
- <openshift-region>
202-
...
203-
peristentVolumeclaimPolicy: Delete
204-
storageClassName: <zoned-storage-class-name> <2>
205-
volumeMode: Filesystem
206-
...
207-
----
208-
<1> PV has zoned keys.
209-
<2> PV is using the zoned storage class.
154+
====
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
// Module included in the following assemblies:
2+
//
3+
// storage/container_storage_interface/persistent-storage-csi-vsphere.adoc
4+
//
5+
6+
:content-type: CONCEPT
7+
[id="persistent-storage-csi-vsphere-top-aware-overview_{context}"]
8+
= vSphere CSI topology overview
9+
10+
{product-title} provides the ability to deploy {product-title} for vSphere on different zones and regions, which allows you to deploy over multiple compute clusters and datacenters, thus helping to avoid a single point of failure.
11+
12+
This is accomplished by defining zone and region categories in vCenter, and then assigning these categories to different failure domains, such as a compute cluster, by creating tags for these zone and region categories. After you have created the appropriate categories, and assigned tags to vCenter objects, you can create additional machinesets that create virtual machines (VMs) that are responsible for scheduling pods in those failure domains.
13+
14+
The following example defines two failure domains with one region and two zones:
15+
16+
.vSphere storage topology with one region and two zones
17+
|===
18+
|Compute cluster | Failure domain |Description
19+
20+
|Compute cluster: ocp1,
21+
Datacenter: Atlanta
22+
|openshift-region: us-east-1 (tag), openshift-zone: us-east-1a (tag)
23+
|This defines a failure domain in region us-east-1 with zone us-east-1a.
24+
25+
|Computer cluster: ocp2,
26+
Datacenter: Atlanta
27+
|openshift-region: us-east-1 (tag), openshift-zone: us-east-1b (tag)
28+
|This defines a different failure domain within the same region called us-east-1b.
29+
|===
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
// Module included in the following assemblies:
2+
//
3+
// storage/container_storage_interface/persistent-storage-csi-vsphere.adoc
4+
//
5+
6+
:content-type: PROCEDURE
7+
[id="persistent-storage-csi-vsphere-top-aware-post-install_{context}"]
8+
= Creating vSphere storage topology post-installation
9+
10+
== Procedure
11+
. In the VMware vCenter vSphere client GUI, define appropriate zone and region catagories and tags.
12+
+
13+
While vSphere allows you to create categories with any arbitrary name, {product-title} strongly recommends use of `openshift-region` and `openshift-zone` names for defining topology categories.
14+
+
15+
For more information about vSphere categories and tags, see the VMware vSphere documentation.
16+
17+
. In {product-title}, create failure domains. See the _Specifying multiple regions and zones for your cluster on vSphere_ section.
18+
19+
. Create a tag to assign to datastores across failure domains:
20+
+
21+
When an {product-title} spans more than one failure domain, the datastore might not be shared across those failure domains, which is where topology-aware provisioning of persistent volumes (PVs) is useful.
22+
+
23+
.. In vCenter, create a category for tagging the datastores. For example, `openshift-zonal-datastore-cat`. You can use any other category name, provided the category uniquely is used for tagging datastores participating in {product-title} cluster. Also, ensure that `StoragePod`, `Datastore`, and `Folder` are selected as Associable Entities for the created category.
24+
.. In vCenter, create a tag that uses the previously created category. This example uses the tag name `openshift-zonal-datastore`.
25+
.. Assign the previously created tag (in this example `openshift-zonal-datastore`) to each datastore in a failure domain that would be considered for dynamic provisioning.
26+
+
27+
[NOTE]
28+
====
29+
You can use any names you like for datastore categories and tags. The names used in this example are provided as recommendations. Ensure that the tags and categories that you define uniquely identify only datastores that are shared with all hosts in the {product-title} cluster.
30+
====
31+
32+
. As needed, create a storage policy that targets the tag-based datastores in each failure domain:
33+
.. In vCenter, from the main menu, click *Policies and Profiles*.
34+
.. On the *Policies and Profiles* page, in the navigation pane, click *VM Storage Policies*.
35+
.. Click *CREATE*.
36+
.. Type a name for the storage policy.
37+
.. For the rules, choose Tag Placement rules and select the tag and category that targets the desired datastores (in this example, the `openshift-zonal-datastore` tag).
38+
+
39+
The datastores are listed in the storage compatibility table.
40+
41+
. Create a new storage class that uses the new zoned storage policy:
42+
.. Click *Storage* > *StorageClasses*.
43+
.. On the *StorageClasses* page, click *Create StorageClass*.
44+
.. Type a name for the new storage class in *Name*.
45+
.. Under *Provisioner*, select *csi.vsphere.vmware.com*.
46+
.. Under *Additional parameters*, for the StoragePolicyName parameter, set *Value* to the name of the new zoned storage policy that you created earlier.
47+
.. Click *Create*.
48+
+
49+
.Example output
50+
+
51+
[source, yaml]
52+
----
53+
kind: StorageClass
54+
apiVersion: storage.k8s.io/v1
55+
metadata:
56+
name: zoned-sc <1>
57+
provisioner: csi.vsphere.vmware.com
58+
parameters:
59+
StoragePolicyName: zoned-storage-policy <2>
60+
reclaimPolicy: Delete
61+
allowVolumeExpansion: true
62+
volumeBindingMode: WaitForFirstConsumer
63+
----
64+
<1> New topology aware storage class name.
65+
<2> Specify zoned storage policy.
66+
+
67+
[NOTE]
68+
====
69+
You can also create the storage class by editing the preceding YAML file and running the command `oc create -f $FILE`.
70+
====
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
// Module included in the following assemblies:
2+
//
3+
// storage/container_storage_interface/persistent-storage-csi-vsphere.adoc
4+
//
5+
6+
:content-type: PROCEDURE
7+
[id="persistent-storage-csi-vsphere-top-aware-results_{context}"]
8+
= Results
9+
10+
Creating persistent volume claims (PVCs) and PVs from the topology aware storage class are truly zonal, and should use the datastore in their respective zone depending on how pods are scheduled:
11+
12+
[source, terminal]
13+
----
14+
~ $ oc get pv <pv-name> -o yaml
15+
----
16+
17+
.Example output
18+
19+
[source, terminal]
20+
----
21+
...
22+
nodeAffinity:
23+
required:
24+
nodeSelectorTerms:
25+
- matchExpressions:
26+
- key: topology.csi.vmware.com/openshift-zone <1>
27+
operator: In
28+
values:
29+
- <openshift-zone>
30+
-key: topology.csi.vmware.com/openshift-region <1>
31+
operator: In
32+
values:
33+
- <openshift-region>
34+
...
35+
peristentVolumeclaimPolicy: Delete
36+
storageClassName: <zoned-storage-class-name> <2>
37+
volumeMode: Filesystem
38+
...
39+
----
40+
<1> PV has zoned keys.
41+
<2> PV is using the zoned storage class.

storage/container_storage_interface/persistent-storage-csi-vsphere.adoc

Lines changed: 21 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -48,11 +48,6 @@ To remove a third-party CSI driver, see xref:../../storage/container_storage_int
4848

4949
include::modules/persistent-storage-csi-vsphere-install-issues.adoc[leveloffset=+1]
5050

51-
include::modules/persistent-storage-csi-vsphere-top-aware.adoc[leveloffset=+1]
52-
[role="_additional-resources"]
53-
.Additional resources
54-
* https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vcenter-esxi-management/GUID-16422FF7-235B-4A44-92E2-532F6AED0923.html?hWord=N4IghgNiBcIC5gOYgL5A[VMware vSphere tag documentation]
55-
5651
[id="vsphere-pv-encryption"]
5752
== vSphere persistent disks encryption
5853

@@ -83,6 +78,25 @@ include::modules/persistent-storage-csi-vsphere-encryption-datastore-url.adoc[le
8378

8479
include::modules/persistent-storage-csi-vsphere-encryption-tag-based.adoc[leveloffset=+2]
8580

86-
[id="additional-resources"]
81+
include::modules/persistent-storage-csi-vsphere-top-aware-overview.adoc[leveloffset=+1]
82+
83+
include::modules/persistent-storage-csi-vsphere-top-aware-during-install.adoc[leveloffset=+2]
84+
[role="_additional-resources"]
85+
.Additional resources
86+
* xref:../../installing/installing_vsphere/installing-vsphere-installer-provisioned-network-customizations.adoc#configuring-vsphere-regions-zones_installing-vsphere-installer-provisioned-network-customizations[Configuring regions and zones for a VMware vCenter]
87+
88+
include::modules/persistent-storage-csi-vsphere-top-aware-post-install.adoc[leveloffset=+2]
89+
[role="_additional-resources"]
90+
.Additional resources
91+
* xref:../../post_installation_configuration/post-install-vsphere-zones-regions-configuration.adoc#specifying-regions-zones-infrastructure-vsphere_post-install-vsphere-zones-regions-configuration[Specifying multiple regions and zones for your cluster on vSphere]
92+
* https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vcenter-esxi-management/GUID-16422FF7-235B-4A44-92E2-532F6AED0923.html?hWord=N4IghgNiBcIC5gOYgL5A[VMware vSphere tag documenation]
93+
94+
include::modules/persistent-storage-csi-vsphere-top-aware-infra-top.adoc[leveloffset=+2]
95+
[role="_additional-resources"]
96+
.Additional resources
97+
* https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vcenter-esxi-management/GUID-16422FF7-235B-4A44-92E2-532F6AED0923.html?hWord=N4IghgNiBcIC5gOYgL5A[VMware vSphere tag documenation]
98+
99+
include::modules/persistent-storage-csi-vsphere-top-aware-results.adoc[leveloffset=+2]
100+
87101
== Additional resources
88-
* xref:../../storage/container_storage_interface/persistent-storage-csi.adoc#persistent-storage-csi[Configuring CSI volumes]
102+
* xref:../../storage/container_storage_interface/persistent-storage-csi.adoc#persistent-storage-csi[Configuring CSI volumes]

0 commit comments

Comments
 (0)