Skip to content

Commit 5dc51db

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #41480 from my-git9/blog-09-29
[zh-cn] sync blog: 2023-04-28-statefulset-migration.md
2 parents 6d1cc3d + bb7e71d commit 5dc51db

File tree

1 file changed

+374
-0
lines changed

1 file changed

+374
-0
lines changed

‎content/zh-cn/blog/_posts/2023-04-28-statefulset-migration.md

Lines changed: 374 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,374 @@
1+
---
2+
layout: blog
3+
title: "Kubernetes 1.27: StatefulSet 启动序号简化了迁移"
4+
date: 2023-04-28
5+
slug: statefulset-start-ordinal
6+
---
7+
8+
<!--
9+
layout: blog
10+
title: "Kubernetes 1.27: StatefulSet Start Ordinal Simplifies Migration"
11+
date: 2023-04-28
12+
slug: statefulset-start-ordinal
13+
-->
14+
15+
<!--
16+
**Author**: Peter Schuurman (Google)
17+
-->
18+
**作者：** Peter Schuurman (Google)
19+
20+
**译者：** Xin Li (DaoCloud)
21+
22+
<!--
23+
Kubernetes v1.26 introduced a new, alpha-level feature for
24+
[StatefulSets](/docs/concepts/workloads/controllers/statefulset/) that controls
25+
the ordinal numbering of Pod replicas. As of Kubernetes v1.27, this feature is
26+
now beta. Ordinals can start from arbitrary
27+
non-negative numbers. This blog post will discuss how this feature can be
28+
used.
29+
-->
30+
Kubernetes v1.26 为 [StatefulSet](/zh-cn/docs/concepts/workloads/controllers/statefulset/)
31+
引入了一个新的 Alpha 级别特性，可以控制 Pod 副本的序号。
32+
从 Kubernetes v1.27 开始，此特性进级到 Beta 阶段。序数可以从任意非负数开始，
33+
这篇博文将讨论如何使用此功能。
34+
35+
<!--
36+
## Background
37+
38+
StatefulSets ordinals provide sequential identities for pod replicas. When using
39+
[`OrderedReady` Pod management](/docs/tutorials/stateful-application/basic-stateful-set/#orderedready-pod-management)
40+
Pods are created from ordinal index `0` up to `N-1`.
41+
-->
42+
## 背景
43+
44+
StatefulSet 序号为 Pod 副本提供顺序标识。当使用
45+
[`OrderedReady` Pod 管理策略](/docs/tutorials/stateful-application/basic-stateful-set/#orderedready-pod-management)时，
46+
Pod 是从序号索引 `0` 到 `N-1` 顺序创建的。
47+
48+
<!--
49+
With Kubernetes today, orchestrating a StatefulSet migration across clusters is
50+
challenging. Backup and restore solutions exist, but these require the
51+
application to be scaled down to zero replicas prior to migration. In today's
52+
fully connected world, even planned application downtime may not allow you to
53+
meet your business goals. You could use
54+
[Cascading Delete](/docs/tutorials/stateful-application/basic-stateful-set/#cascading-delete)
55+
or
56+
[On Delete](/docs/tutorials/stateful-application/basic-stateful-set/#on-delete)
57+
to migrate individual pods, however this is error prone and tedious to manage.
58+
You lose the self-healing benefit of the StatefulSet controller when your Pods
59+
fail or are evicted.
60+
-->
61+
如今使用 Kubernetes 跨集群编排 StatefulSet 迁移具有挑战性。
62+
虽然存在备份和恢复解决方案，但这些解决方案需要在迁移之前将应用程序的副本数缩为 0。
63+
在当今这个完全互联的世界中，即使是计划内的应用停机可能也无法实现你的业务目标。
64+
65+
你可以使用[级联删除](/zh-cn/docs/tutorials/stateful-application/basic-stateful-set/#cascading-delete)或
66+
[OnDelete 策略](/zh-cn/docs/tutorials/stateful-application/basic-stateful-set/#on-delete)来迁移单个 Pod，
67+
但是这很容易出错并且管理起来很乏味。
68+
当你的 Pod 出现故障或被逐出时，你将失去 StatefulSet 控制器的自我修复优势。
69+
70+
<!--
71+
Kubernetes v1.26 enables a StatefulSet to be responsible for a range of ordinals
72+
within a range {0..N-1} (the ordinals 0, 1, ... up to N-1).
73+
With it, you can scale down a range
74+
{0..k-1} in a source cluster, and scale up the complementary range {k..N-1}
75+
in a destination cluster, while maintaining application availability. This
76+
enables you to retain *at most one* semantics (meaning there is at most one Pod
77+
with a given identity running in a StatefulSet) and
78+
[Rolling Update](/docs/tutorials/stateful-application/basic-stateful-set/#rolling-update)
79+
behavior when orchestrating a migration across clusters.
80+
-->
81+
Kubernetes v1.26 使 StatefulSet 能够负责 {0..N-1} 范围内的一系列序数（序数 0、1、... 直到 N-1）。
82+
有了它，你可以缩小源集群中的范围 {0..k-1}，并扩大目标集群中的互补范围 {k..N-1}，同时保证应用程序可用性。
83+
这使你在编排跨集群迁移时保留**至多一个**语义（意味着最多有一个具有给定身份的
84+
Pod 在 StatefulSet 中运行）和[滚动更新](/zh-cn/docs/tutorials/stateful-application/basic-stateful-set/#rolling-update）行为。
85+
86+
<!--
87+
## Why would I want to use this feature?
88+
89+
Say you're running your StatefulSet in one cluster, and need to migrate it out
90+
to a different cluster. There are many reasons why you would need to do this:
91+
* **Scalability**: Your StatefulSet has scaled too large for your cluster, and
92+
has started to disrupt the quality of service for other workloads in your
93+
cluster.
94+
* **Isolation**: You're running a StatefulSet in a cluster that is accessed
95+
by multiple users, and namespace isolation isn't sufficient.
96+
* **Cluster Configuration**: You want to move your StatefulSet to a different
97+
cluster to use some environment that is not available on your current
98+
cluster.
99+
* **Control Plane Upgrades**: You want to move your StatefulSet to a cluster
100+
running an upgraded control plane, and can't handle the risk or downtime of
101+
in-place control plane upgrades.
102+
-->
103+
## 我为什么要使用此功能？
104+
105+
假设你在一个集群中运行 StatefulSet，并且需要将其迁移到另一个集群。你需要这样做的原因有很多：
106+
107+
* **可扩展性**：你的 StatefulSet 对于你的集群而言规模过大，并且已经开始破坏集群中其他工作负载的服务质量。
108+
* **隔离性**：你在一个供多个用户访问的集群中运行 StatefulSet，而命名空间隔离是不够的。
109+
* **集群配置**：你想将 StatefulSet 迁移到另一个集群，以使用在当前集群上不存在的某些环境。
110+
* **控制平面升级**：你想将 StatefulSet 迁移到运行着较高版本控制平面，
111+
并且无法处承担就地升级控制平面所产生的风险或预留停机时间。
112+
113+
<!--
114+
## How do I use it?
115+
116+
Enable the `StatefulSetStartOrdinal` feature gate on a cluster, and create a
117+
StatefulSet with a customized `.spec.ordinals.start`.
118+
-->
119+
## 我该如何使用它？
120+
121+
在集群上启用 `StatefulSetStartOrdinal` 特性门控，并使用自定义的
122+
`.spec.ordinals.start` 创建一个 StatefulSet。
123+
124+
<!--
125+
## Try it out
126+
127+
In this demo, I'll use the new mechanism to migrate a
128+
StatefulSet from one Kubernetes cluster to another. The
129+
[redis-cluster](https://github.com/bitnami/charts/tree/main/bitnami/redis-cluster)
130+
Bitnami Helm chart will be used to install Redis.
131+
-->
132+
## 试试看吧
133+
134+
在此演示中，我将使用新机制将 StatefulSet 从一个 Kubernetes 集群迁移到另一个。
135+
[redis-cluster](https://github.com/bitnami/charts/tree/main/bitnami/redis-cluster)
136+
Bitnami Helm chart 将用于安装 Redis。
137+
138+
<!--
139+
Tools Required:
140+
* [yq](https://github.com/mikefarah/yq)
141+
* [helm](https://helm.sh/docs/helm/helm_install/)
142+
-->
143+
所需工具：
144+
145+
* [yq](https://github.com/mikefarah/yq)
146+
* [helm](https://helm.sh/docs/helm/helm_install/)
147+
148+
<!--
149+
### Pre-requisites {#demo-pre-requisites}
150+
151+
To do this, I need two Kubernetes clusters that can both access common
152+
networking and storage; I've named my clusters `source` and `destination`.
153+
Specifically, I need:
154+
-->
155+
### 先决条件 {#demo-pre-requisites}
156+
157+
为此，我需要两个可以访问公共网络和存储的 Kubernetes 集群；
158+
我已将集群命名为 `source` 和 `destination`。具体来说，我需要：
159+
160+
<!--
161+
* The `StatefulSetStartOrdinal` feature gate enabled on both clusters.
162+
* Client configuration for `kubectl` that lets me access both clusters as an
163+
administrator.
164+
* The same `StorageClass` installed on both clusters, and set as the default
165+
StorageClass for both clusters. This `StorageClass` should provision
166+
underlying storage that is accessible from either or both clusters.
167+
* A flat network topology that allows for pods to send and receive packets to
168+
and from Pods in either clusters. If you are creating clusters on a cloud
169+
provider, this configuration may be called private cloud or private network.
170+
-->
171+
* 在两个集群上都启用 `StatefulSetStartOrdinal` 特性门控。
172+
* `kubectl` 的客户端配置允许我以管理员身份访问这两个集群。
173+
* 两个集群上都安装了相同的 `StorageClass`，并设置为两个集群的默认 `StorageClass`。
174+
这个 `StorageClass` 应该提供可从一个或两个集群访问的底层存储。
175+
* 一种扁平的网络拓扑，允许 Pod 向任一集群中的 Pod 发送数据包和从中接收数据包。
176+
如果你在云提供商上创建集群，则此配置可能被称为私有云或私有网络。
177+
178+
<!--
179+
1. Create a demo namespace on both clusters:
180+
-->
181+
1. 在两个集群上创建一个用于演示的命名空间：
182+
183+
```
184+
kubectl create ns kep-3335
185+
```
186+
187+
<!--
188+
2. Deploy a Redis cluster with six replicas in the source cluster:
189+
-->
190+
2. 在 `source` 集群中部署一个有六个副本的 Redis 集群：
191+
192+
```
193+
helm repo add bitnami https://charts.bitnami.com/bitnami
194+
helm install redis --namespace kep-3335 \
195+
bitnami/redis-cluster \
196+
--set persistence.size=1Gi \
197+
--set cluster.nodes=6
198+
```
199+
200+
<!--
201+
3. Check the replication status in the source cluster:
202+
-->
203+
3. 检查 `source` 集群中的副本状态：
204+
205+
```
206+
kubectl exec -it redis-redis-cluster-0 -- /bin/bash -c \
207+
"redis-cli -c -h redis-redis-cluster -a $(kubectl get secret redis-redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d) CLUSTER NODES;"
208+
```
209+
210+
```
211+
2ce30362c188aabc06f3eee5d92892d95b1da5c3 10.104.0.14:6379@16379 myself,master - 0 1669764411000 3 connected 10923-16383
212+
7743661f60b6b17b5c71d083260419588b4f2451 10.104.0.16:6379@16379 slave 2ce30362c188aabc06f3eee5d92892d95b1da5c3 0 1669764410000 3 connected
213+
961f35e37c4eea507cfe12f96e3bfd694b9c21d4 10.104.0.18:6379@16379 slave a8765caed08f3e185cef22bd09edf409dc2bcc61 0 1669764411000 1 connected
214+
7136e37d8864db983f334b85d2b094be47c830e5 10.104.0.15:6379@16379 slave 2cff613d763b22c180cd40668da8e452edef3fc8 0 1669764412595 2 connected
215+
a8765caed08f3e185cef22bd09edf409dc2bcc61 10.104.0.19:6379@16379 master - 0 1669764411592 1 connected 0-5460
216+
2cff613d763b22c180cd40668da8e452edef3fc8 10.104.0.17:6379@16379 master - 0 1669764410000 2 connected 5461-10922
217+
```
218+
219+
<!--
220+
4. Deploy a Redis cluster with zero replicas in the destination cluster:
221+
-->
222+
4. 在 `destination` 集群中部署一个零副本的 Redis 集群：
223+
224+
```
225+
helm install redis --namespace kep-3335 \
226+
bitnami/redis-cluster \
227+
--set persistence.size=1Gi \
228+
--set cluster.nodes=0 \
229+
--set redis.extraEnvVars\[0\].name=REDIS_NODES,redis.extraEnvVars\[0\].value="redis-redis-cluster-headless.kep-3335.svc.cluster.local" \
230+
--set existingSecret=redis-redis-cluster
231+
```
232+
233+
<!--
234+
5. Scale down the `redis-redis-cluster` StatefulSet in the source cluster by 1,
235+
to remove the replica `redis-redis-cluster-5`:
236+
-->
237+
5. 将源集群中的 `redis-redis-cluster` StatefulSet 副本数缩小 1，
238+
以删除副本 `redis-redis-cluster-5`：
239+
240+
```
241+
kubectl patch sts redis-redis-cluster -p '{"spec": {"replicas": 5}}'
242+
```
243+
244+
<!--
245+
6. Migrate dependencies from the source cluster to the destination cluster:
246+
247+
The following commands copy resources from `source` to `destionation`. Details
248+
that are not relevant in `destination` cluster are removed (eg: `uid`,
249+
`resourceVersion`, `status`).
250+
251+
**Steps for the source cluster**
252+
-->
253+
6. 将依赖从 `source` 集群迁移到 `destionation` 集群：
254+
以下命令将依赖资源从 `source` 复制到 `destionation`，其中与 `destionation`
255+
集群无关的详细信息已被删除（例如：`uid`、`resourceVersion`、`status`）。
256+
257+
<!--
258+
Note: If using a `StorageClass` with `reclaimPolicy: Delete` configured, you
259+
should patch the PVs in `source` with `reclaimPolicy: Retain` prior to
260+
deletion to retain the underlying storage used in `destination`. See
261+
[Change the Reclaim Policy of a PersistentVolume](/docs/tasks/administer-cluster/change-pv-reclaim-policy/)
262+
for more details.
263+
-->
264+
265+
说明：如果使用配置了 `reclaimPolicy: Delete` 的 `StorageClass`，
266+
你应该在删除之前使用 `reclaimPolicy: Retain` 修补 `source` 中的 PV，
267+
以保留 `destination` 中使用的底层存储。
268+
有关详细信息，请参阅[更改 PersistentVolume](/zh-cn/docs/tasks/administer-cluster/change-pv-reclaim-policy/)
269+
的回收策略。
270+
271+
```
272+
kubectl get pvc redis-data-redis-redis-cluster-5 -o yaml | yq 'del(.metadata.uid, .metadata.resourceVersion, .metadata.annotations, .metadata.finalizers, .status)' > /tmp/pvc-redis-data-redis-redis-cluster-5.yaml
273+
kubectl get pv $(yq '.spec.volumeName' /tmp/pvc-redis-data-redis-redis-cluster-5.yaml) -o yaml | yq 'del(.metadata.uid, .metadata.resourceVersion, .metadata.annotations, .metadata.finalizers, .spec.claimRef, .status)' > /tmp/pv-redis-data-redis-redis-cluster-5.yaml
274+
kubectl get secret redis-redis-cluster -o yaml | yq 'del(.metadata.uid, .metadata.resourceVersion)' > /tmp/secret-redis-redis-cluster.yaml
275+
```
276+
277+
<!--
278+
**Steps for the destination cluster**
279+
280+
Note: For the PV/PVC, this procedure only works if the underlying storage system
281+
that your PVs use can support being copied into `destination`. Storage
282+
that is associated with a specific node or topology may not be supported.
283+
Additionally, some storage systems may store addtional metadata about
284+
volumes outside of a PV object, and may require a more specialized
285+
sequence to import a volume.
286+
-->
287+
288+
**`destination` 集群中的步骤**
289+
290+
说明：对于 PV/PVC，此过程仅在你的 PV 使用的底层存储系统支持复制到 `destination`
291+
集群时才有效。可能不支持与特定节点或拓扑关联的存储。此外，某些存储系统可能会在 PV
292+
对象之外存储有关卷的附加元数据，并且可能需要更专门的序列来导入卷。
293+
294+
```
295+
kubectl create -f /tmp/pv-redis-data-redis-redis-cluster-5.yaml
296+
kubectl create -f /tmp/pvc-redis-data-redis-redis-cluster-5.yaml
297+
kubectl create -f /tmp/secret-redis-redis-cluster.yaml
298+
```
299+
300+
<!--
301+
7. Scale up the `redis-redis-cluster` StatefulSet in the destination cluster by
302+
1, with a start ordinal of 5:
303+
-->
304+
7. 将 `destination` 集群中的 `redis-redis-cluster` StatefulSet 扩容 1，起始序号为 5：
305+
306+
```
307+
kubectl patch sts redis-redis-cluster -p '{"spec": {"ordinals": {"start": 5}, "replicas": 1}}'
308+
```
309+
310+
<!--
311+
8. Check the replication status in the destination cluster:
312+
-->
313+
8. 检查 `destination` 集群中的副本状态：
314+
315+
```
316+
kubectl exec -it redis-redis-cluster-5 -- /bin/bash -c \
317+
"redis-cli -c -h redis-redis-cluster -a $(kubectl get secret redis-redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d) CLUSTER NODES;"
318+
```
319+
320+
<!--
321+
I should see that the new replica (labeled `myself`) has joined the Redis
322+
cluster (the IP address belongs to a different CIDR block than the
323+
replicas in the source cluster).
324+
-->
325+
我应该看到新副本（标记为 `myself`）已加入 Redis 集群（IP
326+
地址与 `source` 集群中的副本归属于不同的 CIDR 块）。
327+
328+
```
329+
2cff613d763b22c180cd40668da8e452edef3fc8 10.104.0.17:6379@16379 master - 0 1669766684000 2 connected 5461-10922
330+
7136e37d8864db983f334b85d2b094be47c830e5 10.108.0.22:6379@16379 myself,slave 2cff613d763b22c180cd40668da8e452edef3fc8 0 1669766685609 2 connected
331+
2ce30362c188aabc06f3eee5d92892d95b1da5c3 10.104.0.14:6379@16379 master - 0 1669766684000 3 connected 10923-16383
332+
961f35e37c4eea507cfe12f96e3bfd694b9c21d4 10.104.0.18:6379@16379 slave a8765caed08f3e185cef22bd09edf409dc2bcc61 0 1669766683600 1 connected
333+
a8765caed08f3e185cef22bd09edf409dc2bcc61 10.104.0.19:6379@16379 master - 0 1669766685000 1 connected 0-5460
334+
7743661f60b6b17b5c71d083260419588b4f2451 10.104.0.16:6379@16379 slave 2ce30362c188aabc06f3eee5d92892d95b1da5c3 0 1669766686613 3 connected
335+
```
336+
337+
<!--
338+
9. Repeat steps #5 to #7 for the remainder of the replicas, until the
339+
Redis StatefulSet in the source cluster is scaled to 0, and the Redis
340+
StatefulSet in the destination cluster is healthy with 6 total replicas.
341+
-->
342+
9. 对剩余的副本重复 #5 到 #7 的步骤，直到 `source` 集群中的 Redis StatefulSet 副本缩放为 0，
343+
并且 `destination` 集群中的 Redis StatefulSet 健康，总共有 6 个副本。
344+
345+
<!--
346+
## What's Next?
347+
348+
This feature provides a building block for a StatefulSet to be split up across
349+
clusters, but does not prescribe the mechanism as to how the StatefulSet should
350+
be migrated. Migration requires coordination of StatefulSet replicas, along with
351+
orchestration of the storage and network layer. This is dependent on the storage
352+
and connectivity requirements of the application installed by the StatefulSet.
353+
Additionally, many StatefulSets are managed by
354+
[operators](/docs/concepts/extend-kubernetes/operator/), which adds another
355+
layer of complexity to migration.
356+
-->
357+
## 接下来？
358+
359+
此特性为跨集群拆分 StatefulSet 提供了一项基本支撑技术，但没有规定 StatefulSet 的迁移机制。
360+
迁移需要对 StatefulSet 副本的协调，以及对存储和网络层的编排。这取决于使用 StatefulSet
361+
安装的应用程序的存储和网络连接要求。此外，许多 StatefulSet 由
362+
[operator](/zh-cn/docs/concepts/extend-kubernetes/operator/) 管理，这也增加了额外的迁移复杂性。
363+
364+
<!--
365+
If you're interested in building enhancements to make these processes easier,
366+
get involved with
367+
[SIG Multicluster](https://github.com/kubernetes/community/blob/master/sig-multicluster)
368+
to contribute!
369+
-->
370+
如果你有兴趣构建增强功能以简化这些过程，请参与
371+
[SIG Multicluster](https://github.com/kubernetes/community/blob/master/sig-multicluster)
372+
做出贡献！
373+
374+

0 commit comments

Comments
 (0)