Skip to content

Commit aa07b7a

Browse files
authored
Merge pull request #46201 from windsonsea/storme
Make the markdown prettier: storage-version-migration.md
2 parents f83793a + 0611310 commit aa07b7a

File tree

1 file changed

+83
-37
lines changed

1 file changed

+83
-37
lines changed

content/en/docs/tasks/manage-kubernetes-objects/storage-version-migration.md

Lines changed: 83 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,22 @@
11
---
22
title: Migrate Kubernetes Objects Using Storage Version Migration
3-
43
reviewers:
5-
- deads2k
6-
- jpbetz
7-
- enj
8-
- nilekhc
9-
4+
- deads2k
5+
- jpbetz
6+
- enj
7+
- nilekhc
108
content_type: task
119
min-kubernetes-server-version: v1.30
1210
weight: 60
1311
---
1412

1513
<!-- overview -->
14+
1615
{{< feature-state feature_gate_name="StorageVersionMigrator" >}}
1716

18-
Kubernetes relies on API data being actively re-written, to support some
19-
maintenance activities related to at rest storage. Two prominent examples are
20-
the versioned schema of stored resources (that is, the preferred storage schema
17+
Kubernetes relies on API data being actively re-written, to support some
18+
maintenance activities related to at rest storage. Two prominent examples are
19+
the versioned schema of stored resources (that is, the preferred storage schema
2120
changing from v1 to v2 for a given resource) and encryption at rest
2221
(that is, rewriting stale data based on a change in how the data should be encrypted).
2322

@@ -27,12 +26,13 @@ Install [`kubectl`](/docs/tasks/tools/#kubectl).
2726

2827
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
2928

30-
3129
<!-- steps -->
3230

3331
## Re-encrypt Kubernetes secrets using storage version migration
34-
- To begin with, [configure KMS provider](/docs/tasks/administer-cluster/kms-provider/)
32+
33+
- To begin with, [configure KMS provider](/docs/tasks/administer-cluster/kms-provider/)
3534
to encrypt data at rest in etcd using following encryption configuration.
35+
3636
```yaml
3737
kind: EncryptionConfiguration
3838
apiVersion: apiserver.config.k8s.io/v1
@@ -45,15 +45,21 @@ Install [`kubectl`](/docs/tasks/tools/#kubectl).
4545
- name: key1
4646
secret: c2VjcmV0IGlzIHNlY3VyZQ==
4747
```
48+
4849
Make sure to enable automatic reload of encryption
49-
configuration file by setting `--encryption-provider-config-automatic-reload` to true.
50+
configuration file by setting `--encryption-provider-config-automatic-reload` to true.
51+
5052
- Create a Secret using kubectl.
53+
5154
```shell
5255
kubectl create secret generic my-secret --from-literal=key1=supersecret
5356
```
57+
5458
- [Verify](/docs/tasks/administer-cluster/kms-provider/#verifying-that-the-data-is-encrypted)
5559
the serialized data for that Secret object is prefixed with `k8s:enc:aescbc:v1:key1`.
60+
5661
- Update the encryption configuration file as follows to rotate the encryption key.
62+
5763
```yaml
5864
kind: EncryptionConfiguration
5965
apiVersion: apiserver.config.k8s.io/v1
@@ -70,9 +76,12 @@ configuration file by setting `--encryption-provider-config-automatic-reload` to
7076
- name: key1
7177
secret: c2VjcmV0IGlzIHNlY3VyZQ==
7278
```
79+
7380
- To ensure that previously created secret `my-secert` is re-encrypted
74-
with new key `key2`, you will use _Storage Version Migration_.
81+
with new key `key2`, you will use _Storage Version Migration_.
82+
7583
- Create a StorageVersionMigration manifest named `migrate-secret.yaml` as follows:
84+
7685
```yaml
7786
kind: StorageVersionMigration
7887
apiVersion: storagemigration.k8s.io/v1alpha1
@@ -84,27 +93,31 @@ with new key `key2`, you will use _Storage Version Migration_.
8493
version: v1
8594
resource: secrets
8695
```
96+
8797
Create the object using _kubectl_ as follows:
98+
8899
```shell
89100
kubectl apply -f migrate-secret.yaml
90101
```
102+
91103
- Monitor migration of Secrets by checking the `.status` of the StorageVersionMigration.
92104
A successful migration should have its
93-
`Succeeded` condition set to true. Get the StorageVersionMigration object
94-
as follows:
105+
`Succeeded` condition set to true. Get the StorageVersionMigration object as follows:
106+
95107
```shell
96108
kubectl get storageversionmigration.storagemigration.k8s.io/secrets-migration -o yaml
97109
```
98110

99111
The output is similar to:
112+
100113
```yaml
101114
kind: StorageVersionMigration
102115
apiVersion: storagemigration.k8s.io/v1alpha1
103116
metadata:
104117
name: secrets-migration
105118
uid: 628f6922-a9cb-4514-b076-12d3c178967c
106-
resourceVersion: '90'
107-
creationTimestamp: '2024-03-12T20:29:45Z'
119+
resourceVersion: "90"
120+
creationTimestamp: "2024-03-12T20:29:45Z"
108121
spec:
109122
resource:
110123
group: ""
@@ -113,19 +126,21 @@ as follows:
113126
status:
114127
conditions:
115128
- type: Running
116-
status: 'False'
117-
lastUpdateTime: '2024-03-12T20:29:46Z'
129+
status: "False"
130+
lastUpdateTime: "2024-03-12T20:29:46Z"
118131
reason: StorageVersionMigrationInProgress
119132
- type: Succeeded
120-
status: 'True'
121-
lastUpdateTime: '2024-03-12T20:29:46Z'
133+
status: "True"
134+
lastUpdateTime: "2024-03-12T20:29:46Z"
122135
reason: StorageVersionMigrationSucceeded
123-
resourceVersion: '84'
136+
resourceVersion: "84"
124137
```
138+
125139
- [Verify](/docs/tasks/administer-cluster/kms-provider/#verifying-that-the-data-is-encrypted)
126140
the stored secret is now prefixed with `k8s:enc:aescbc:v1:key2`.
127141

128142
## Update the preferred storage schema of a CRD
143+
129144
Consider a scenario where a {{< glossary_tooltip term_id="CustomResourceDefinition" text="CustomResourceDefinition" >}}
130145
(CRD) is created to serve custom resources (CRs) and is set as the preferred storage schema. When it's time
131146
to introduce v2 of the CRD, it can be added for serving only with a conversion
@@ -136,6 +151,7 @@ version, it's important to ensure that all existing CRs stored as v1 are migrate
136151
This migration can be achieved through _Storage Version Migration_ to migrate all CRs from v1 to v2.
137152

138153
- Create a manifest for the CRD, named `test-crd.yaml`, as follows:
154+
139155
```yaml
140156
apiVersion: apiextensions.k8s.io/v1
141157
kind: CustomResourceDefinition
@@ -163,35 +179,46 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
163179
strategy: Webhook
164180
webhook:
165181
clientConfig:
166-
url: https://127.0.0.1:9443/crdconvert
182+
url: "https://127.0.0.1:9443/crdconvert"
167183
caBundle: <CABundle info>
168184
conversionReviewVersions:
169185
- v1
170186
- v2
171187
```
172-
Create CRD using kubectl
188+
189+
Create CRD using kubectl:
190+
173191
```shell
174192
kubectl apply -f test-crd.yaml
175193
```
194+
176195
- Create a manifest for an example testcrd. Name the manifest `cr1.yaml` and use these contents:
196+
177197
```yaml
178198
apiVersion: stable.example.com/v1
179199
kind: SelfieRequest
180200
metadata:
181201
name: cr1
182202
namespace: default
183203
```
184-
Create CR using kubectl
204+
205+
Create CR using kubectl:
206+
185207
```shell
186208
kubectl apply -f cr1.yaml
187209
```
210+
188211
- Verify that CR is written and stored as v1 by getting the object from etcd.
212+
189213
```shell
190214
ETCDCTL_API=3 etcdctl get /kubernetes.io/stable.example.com/testcrds/default/cr1 [...] | hexdump -C
191215
```
216+
192217
where `[...]` contains the additional arguments for connecting to the etcd server.
218+
193219
- Update the CRD `test-crd.yaml` to include v2 version for serving and storage
194-
and v1 as serving only, as follows:
220+
and v1 as serving only, as follows:
221+
195222
```yaml
196223
apiVersion: apiextensions.k8s.io/v1
197224
kind: CustomResourceDefinition
@@ -230,34 +257,45 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
230257
strategy: Webhook
231258
webhook:
232259
clientConfig:
233-
url: 'https://127.0.0.1:9443/crdconvert'
260+
url: "https://127.0.0.1:9443/crdconvert"
234261
caBundle: <CABundle info>
235262
conversionReviewVersions:
236263
- v1
237264
- v2
238265
```
239-
Update CRD using kubectl
266+
267+
Update CRD using kubectl:
268+
240269
```shell
241270
kubectl apply -f test-crd.yaml
242271
```
272+
243273
- Create CR resource file with name `cr2.yaml` as follows:
274+
244275
```yaml
245276
apiVersion: stable.example.com/v2
246277
kind: SelfieRequest
247278
metadata:
248279
name: cr2
249280
namespace: default
250281
```
251-
- Create CR using kubectl
282+
283+
- Create CR using kubectl:
284+
252285
```shell
253286
kubectl apply -f cr2.yaml
254287
```
288+
255289
- Verify that CR is written and stored as v2 by getting the object from etcd.
290+
256291
```shell
257292
ETCDCTL_API=3 etcdctl get /kubernetes.io/stable.example.com/testcrds/default/cr2 [...] | hexdump -C
258293
```
294+
259295
where `[...]` contains the additional arguments for connecting to the etcd server.
296+
260297
- Create a StorageVersionMigration manifest named `migrate-crd.yaml`, with the contents as follows:
298+
261299
```yaml
262300
kind: StorageVersionMigration
263301
apiVersion: storagemigration.k8s.io/v1alpha1
@@ -269,26 +307,31 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
269307
version: v1
270308
resource: SelfieRequest
271309
```
310+
272311
Create the object using _kubectl_ as follows:
312+
273313
```shell
274314
kubectl apply -f migrate-crd.yaml
275315
```
316+
276317
- Monitor migration of secrets using status. Successful migration should have
277318
`Succeeded` condition set to "True" in the status field. Get the migration resource
278319
as follows:
320+
279321
```shell
280322
kubectl get storageversionmigration.storagemigration.k8s.io/crdsvm -o yaml
281323
```
282-
324+
283325
The output is similar to:
326+
284327
```yaml
285328
kind: StorageVersionMigration
286329
apiVersion: storagemigration.k8s.io/v1alpha1
287330
metadata:
288331
name: crdsvm
289332
uid: 13062fe4-32d7-47cc-9528-5067fa0c6ac8
290-
resourceVersion: '111'
291-
creationTimestamp: '2024-03-12T22:40:01Z'
333+
resourceVersion: "111"
334+
creationTimestamp: "2024-03-12T22:40:01Z"
292335
spec:
293336
resource:
294337
group: stable.example.com
@@ -297,17 +340,20 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
297340
status:
298341
conditions:
299342
- type: Running
300-
status: 'False'
301-
lastUpdateTime: '2024-03-12T22:40:03Z'
343+
status: "False"
344+
lastUpdateTime: "2024-03-12T22:40:03Z"
302345
reason: StorageVersionMigrationInProgress
303346
- type: Succeeded
304-
status: 'True'
305-
lastUpdateTime: '2024-03-12T22:40:03Z'
347+
status: "True"
348+
lastUpdateTime: "2024-03-12T22:40:03Z"
306349
reason: StorageVersionMigrationSucceeded
307-
resourceVersion: '106'
350+
resourceVersion: "106"
308351
```
352+
309353
- Verify that previously created cr1 is now written and stored as v2 by getting the object from etcd.
354+
310355
```shell
311356
ETCDCTL_API=3 etcdctl get /kubernetes.io/stable.example.com/testcrds/default/cr1 [...] | hexdump -C
312357
```
358+
313359
where `[...]` contains the additional arguments for connecting to the etcd server.

0 commit comments

Comments
 (0)