Skip to content

Commit 24262de

Browse files
lpettyjolpettyjo
authored
Merge pull request #52116 from lpettyjo/OSDOCS-3898
OSDOCS-3898: vSphere Topology Awareness
2 parents 03c241f + d16648a commit 24262de

File tree

2 files changed

+214
-0
lines changed

2 files changed

+214
-0
lines changed

‎modules/persistent-storage-csi-vsphere-top-aware.adoc

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
// Module included in the following assemblies:
2+
//
3+
// persistent-storage-csi-vsphere.adoc
4+
//
5+
6+
:content-type: PROCEDURE
7+
[id="persistent-storage-csi-vsphere-top-aware_{context}"]
8+
= Configuring vSphere CSI Topology
9+
10+
{product-title} provides the ability to deploy {product-title} for vSphere on different zones and regions, which allows you to deploy over multiple compute clusters, thus helping to avoid a single point of failure.
11+
12+
[NOTE]
13+
====
14+
{product-title} on vSphere does not support multiple Datacenters.
15+
====
16+
17+
This is accomplished by defining zone and region categories in vCenter, and then assigning these categories to different failure domains, such as a compute cluster, by creating tags for these zone and region categories. After you have created the appropriate categories, and assigned tags to vCenter objects, you can create additional machinesets that create virtual machines (VMs) that are responsible for scheduling pods in those failure domains.
18+
19+
.Procedure
20+
. In the VMware vCenter vSphere client GUI, define appropriate zone and region catagories and tags.
21+
+
22+
While vSphere allows you to create categories with any arbitrary name, {product-title} strongly recommends use of `openshift-region` and `openshift-zone` names for defining topology.
23+
+
24+
The following example defines two failure domains with one region and two zones:
25+
+
26+
.vSphere topology with one region and two zones
27+
|===
28+
|Compute cluster | Failure domain |Description
29+
30+
|Compute cluster: ocp1,
31+
Datacenter: Atlanta
32+
|openshift-region: us-east-1 (tag), openshift-zone: us-east-1a (tag)
33+
|This defines a failure domain in region us-east-1 with zone us-east-1a.
34+
35+
|Computer cluster: ocp2,
36+
Datacenter: Atlanta
37+
|openshift-region: us-east-1 (tag), openshift-zone: us-east-1b (tag)
38+
|This defines a different failure domain within the same region called us-east-1b.
39+
|===
40+
+
41+
For more information about vSphere categories and tags, see the VMware vSphere documentation.
42+
43+
. To allow the container storage interface (CSI) driver to detect this topology, edit the `clusterCSIDriver` object YAML file `driverConfig` section:
44+
* Specify the `openshift-zone` and `openshift-region` categories that you created earlier.
45+
* Set `driverType` to `vSphere`.
46+
+
47+
[source, terminal]
48+
----
49+
~ $ oc edit clustercsidriver csi.vsphere.vmware.com -o yaml
50+
----
51+
+
52+
.Example output
53+
+
54+
[source, terminal]
55+
----
56+
apiVersion: operator.openshift.io/v1
57+
kind: ClusterCSIDriver
58+
metadata:
59+
name: csi.vsphere.vmware.com
60+
spec:
61+
logLevel: Normal
62+
managementState: Managed
63+
observedConfig: null
64+
operatorLogLevel: Normal
65+
unsupportedConfigOverrides: null
66+
driverConfig:
67+
driverType: vSphere <1>
68+
vSphere:
69+
topologyCategories: <2>
70+
- openshift-zone
71+
- openshift-region
72+
----
73+
<1> Ensure that `driverType` is set to `vSphere`.
74+
<2> `openshift-zone` and `openshift-region` categories created earlier in vCenter.
75+
76+
. Verify that `CSINode` object has topology keys by running the following commands:
77+
+
78+
[source, terminal]
79+
----
80+
~ $ oc get csinode
81+
----
82+
+
83+
.Example output
84+
+
85+
[source, terminal]
86+
----
87+
NAME DRIVERS AGE
88+
co8-4s88d-infra-2m5vd 1 27m
89+
co8-4s88d-master-0 1 70m
90+
co8-4s88d-master-1 1 70m
91+
co8-4s88d-master-2 1 70m
92+
co8-4s88d-worker-j2hmg 1 47m
93+
co8-4s88d-worker-mbb46 1 47m
94+
co8-4s88d-worker-zlk7d 1 47m
95+
----
96+
+
97+
[source, terminal]
98+
----
99+
~ $ oc get csinode co8-4s88d-worker-j2hmg -o yaml
100+
----
101+
+
102+
.Example output
103+
+
104+
[source, terminal]
105+
----
106+
...
107+
spec:
108+
drivers:
109+
- allocatable:
110+
count: 59
111+
name: csi-vsphere.vmware.com
112+
nodeID: co8-4s88d-worker-j2hmg
113+
topologyKeys: <1>
114+
- topology.csi.vmware.com/openshift-zone
115+
- topology.csi.vmware.com/openshift-region
116+
----
117+
<1> Topology keys from vSphere `openshift-zone` and `openshift-region` catagories.
118+
+
119+
[NOTE]
120+
=====
121+
`CSINode` objects might take some time to receive updated topology information. After the driver is updated, `CSINode` objects should have topology keys in them.
122+
=====
123+
124+
. Create a tag to assign to datastores across failure domains:
125+
+
126+
When an {product-title} spans more than one failure domain, the datastore might not be shared across those failure domains, which is where topology-aware provisioning of persistent volumes (PVs) is useful.
127+
+
128+
.. In vCenter, create a category for tagging the datastores. For example, `openshift-zonal-datastore-cat`. You can use any other category name, provided the category uniquely is used for tagging datastores participating in {product-title} cluster. Also, ensure that `StoragePod`, `Datastore`, and `Folder` are selected as Associable Entities for the created category.
129+
.. In vCenter, create a tag that uses the previously created category. This example uses the tag name `openshift-zonal-datastore`.
130+
.. Assign the previously created tag (in this example `openshift-zonal-datastore`) to each datastore in a failure domain that would be considered for dynamic provisioning.
131+
+
132+
[NOTE]
133+
====
134+
You can use any names you like for categories and tags. The names used in this example are provided as recommendations. Ensure that the tags and categories that you define uniquely identify only datastores that are shared with all hosts in the {product-title} cluster.
135+
====
136+
137+
. Create a storage policy that targets the tag-based datastores in each failure domain:
138+
.. In vCenter, from the main menu, click *Policies and Profiles*.
139+
.. On the *Policies and Profiles* page, in the navigation pane, click *VM Storage Policies*.
140+
.. Click *CREATE*.
141+
.. Type a name for the storage policy.
142+
.. For the rules, choose Tag Placement rules and select the tag and category that targets the desired datastores (in this example, the `openshift-zonal-datastore` tag).
143+
+
144+
The datastores are listed in the storage compatibility table.
145+
146+
. Create a new storage class that uses the new zoned storage policy:
147+
.. Click *Storage* > *StorageClasses*.
148+
.. On the *StorageClasses* page, click *Create StorageClass*.
149+
.. Type a name for the new storage class in *Name*.
150+
.. Under *Provisioner*, select *csi.vsphere.vmware.com*.
151+
.. Under *Additional parameters*, for the StoragePolicyName parameter, set *Value* to the name of the new zoned storage policy that you created earlier.
152+
.. Click *Create*.
153+
+
154+
.Example output
155+
+
156+
[source, yaml]
157+
----
158+
kind: StorageClass
159+
apiVersion: storage.k8s.io/v1
160+
metadata:
161+
name: zoned-sc <1>
162+
provisioner: csi.vsphere.vmware.com
163+
parameters:
164+
StoragePolicyName: zoned-storage-policy <2>
165+
reclaimPolicy: Delete
166+
allowVolumeExpansion: true
167+
volumeBindingMode: WaitForFirstConsumer
168+
----
169+
<1> New topology aware storage class name.
170+
<2> Specify zoned storage policy.
171+
+
172+
[NOTE]
173+
====
174+
You can also create the storage class by editing the preceding YAML file and running the command `oc create -f $FILE`.
175+
====
176+
177+
.Results
178+
Creating persistent volume claims (PVCs) and PVs from the topology aware storage class are truly zonal, and should use the datastore in their respective zone depending on how pods are scheduled:
179+
180+
[source, terminal]
181+
----
182+
~ $ oc get pv <pv-name> -o yaml
183+
----
184+
185+
.Example output
186+
187+
[source, terminal]
188+
----
189+
...
190+
nodeAffinity:
191+
required:
192+
nodeSelectorTerms:
193+
- matchExpressions:
194+
- key: topology.csi.vmware.com/openshift-zone <1>
195+
operator: In
196+
values:
197+
- <openshift-zone>
198+
-key: topology.csi.vmware.com/openshift-region <1>
199+
operator: In
200+
values:
201+
- <openshift-region>
202+
...
203+
peristentVolumeclaimPolicy: Delete
204+
storageClassName: <zoned-storage-class-name> <2>
205+
volumeMode: Filesystem
206+
...
207+
----
208+
<1> PV has zoned keys.
209+
<2> PV is using the zoned storage class.

‎storage/container_storage_interface/persistent-storage-csi-vsphere.adoc

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,5 +48,10 @@ To remove a third-party CSI driver, see xref:../../storage/container_storage_int
4848

4949
include::modules/persistent-storage-csi-vsphere-install-issues.adoc[leveloffset=+1]
5050

51+
include::modules/persistent-storage-csi-vsphere-top-aware.adoc[leveloffset=+1]
52+
[role="_additional-resources"]
53+
.Additional resources
54+
* https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vcenter-esxi-management/GUID-16422FF7-235B-4A44-92E2-532F6AED0923.html?hWord=N4IghgNiBcIC5gOYgL5A[VMware vSphere tag documenation]
55+
5156
== Additional resources
5257
* xref:../../storage/container_storage_interface/persistent-storage-csi.adoc#persistent-storage-csi[Configuring CSI volumes]

0 commit comments

Comments
 (0)