Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
3d02897
dumpling-lightning-job
yiduoyunQ Apr 11, 2025
5498425
dumpling-lightning-job
yiduoyunQ Apr 11, 2025
f3bd026
dumpling-lightning-job
yiduoyunQ Apr 11, 2025
3c507cd
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 14, 2025
f456260
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 14, 2025
d5fed6d
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 14, 2025
761eaae
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 14, 2025
d4e9539
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 14, 2025
bb31eb2
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 14, 2025
b11dba8
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 14, 2025
01c8428
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 14, 2025
6005953
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 15, 2025
530bfe2
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 15, 2025
382c330
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 15, 2025
9c29328
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 15, 2025
495bcb3
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 15, 2025
76c7579
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 15, 2025
4233e18
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 15, 2025
d21e183
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 15, 2025
ff59359
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 15, 2025
3a21b0d
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 15, 2025
f626353
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 15, 2025
06465c7
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 15, 2025
5d183c2
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 15, 2025
a05864c
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 15, 2025
a587b1a
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 15, 2025
f96ee79
Update zh/backup-to-gcs-using-job.md
yiduoyunQ Apr 15, 2025
ca0de1a
Update zh/backup-to-gcs-using-job.md
yiduoyunQ Apr 15, 2025
afd3e60
Update zh/backup-to-gcs-using-job.md
yiduoyunQ Apr 15, 2025
efa8952
Update zh/backup-to-gcs-using-job.md
yiduoyunQ Apr 15, 2025
5175a0a
Update zh/backup-to-gcs-using-job.md
yiduoyunQ Apr 15, 2025
bce7121
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 15, 2025
22659d1
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 15, 2025
c9f7d66
Update zh/backup-to-gcs-using-job.md
yiduoyunQ Apr 15, 2025
94011c0
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 15, 2025
0309ec7
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 15, 2025
b319cad
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 15, 2025
996d4d3
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 15, 2025
649d413
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 15, 2025
8dd0d52
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 15, 2025
07b4646
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 15, 2025
5de789b
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 15, 2025
0629499
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 15, 2025
e7450be
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 15, 2025
b4dfd91
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 15, 2025
87f9f30
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 15, 2025
3a3fbb9
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 15, 2025
79b52ca
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 15, 2025
455c503
refine external storage names
hfxsd Apr 16, 2025
81eec79
refine structures and wording
hfxsd Apr 16, 2025
fe1ec3d
Apply suggestions from code review
hfxsd Apr 16, 2025
d12cbfd
Update zh/backup-to-gcs.md
yiduoyunQ Apr 22, 2025
fca6219
Update zh/restore-from-s3.md
yiduoyunQ Apr 22, 2025
7d14c44
Update zh/TOC.md
yiduoyunQ Apr 22, 2025
c695f3f
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 22, 2025
1dd9ac7
Update zh/backup-to-gcs-using-job.md
yiduoyunQ Apr 22, 2025
eba038f
Update zh/backup-to-s3-using-job.md
yiduoyunQ Apr 22, 2025
6b10105
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 22, 2025
2676fa7
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 22, 2025
f4449b9
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 22, 2025
8a5019a
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 22, 2025
0193b18
Update zh/restore-from-s3-using-job.md
yiduoyunQ Apr 22, 2025
bb21f58
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 22, 2025
5642d22
Update zh/backup-to-gcs-using-job.md
yiduoyunQ Apr 22, 2025
0e3acd7
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 22, 2025
93a5a91
Update zh/restore-from-blob-using-job.md
yiduoyunQ Apr 22, 2025
ced8917
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 22, 2025
14a0940
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 22, 2025
5d69624
Update zh/restore-from-gcs-using-job.md
yiduoyunQ Apr 22, 2025
1af9314
Update zh/backup-to-s3.md
yiduoyunQ Apr 22, 2025
efb2510
Update zh/backup-to-s3.md
yiduoyunQ Apr 22, 2025
859f9a9
Update zh/restore-from-gcs.md
yiduoyunQ Apr 22, 2025
1b0d998
Update zh/restore-from-gcs.md
yiduoyunQ Apr 22, 2025
2204f4e
Update zh/TOC.md
yiduoyunQ Apr 22, 2025
4762dda
Update zh/backup-to-blob-using-job.md
yiduoyunQ Apr 22, 2025
0154b8a
Update zh/restore-from-s3.md
yiduoyunQ Apr 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 14 additions & 8 deletions zh/TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,18 +61,24 @@
- [备份与恢复 CR 介绍](backup-restore-cr.md)
- [远程存储访问授权](grant-permissions-to-remote-storage.md)
- 使用 Amazon S3 兼容的存储
- [使用 BR 备份 TiDB 集群数据到兼容 S3 的存储](backup-to-aws-s3-using-br.md)
- [使用 BR 恢复 S3 兼容存储上的备份数据](restore-from-aws-s3-using-br.md)
- [使用 Dumpling 备份 TiDB 集群数据到兼容 S3 的存储](backup-to-s3.md)
- [使用 TiDB Lightning 恢复 S3 兼容存储上的备份数据](restore-from-s3.md)
- [使用 BR 备份 TiDB 集群数据到兼容 Amazon S3 的存储](backup-to-aws-s3-using-br.md)
- [使用 BR 恢复 Amazon S3 兼容存储上的备份数据](restore-from-aws-s3-using-br.md)
- [使用 Dumpling 备份 TiDB 集群数据到兼容 Amazon S3 的存储](backup-to-s3-using-job.md)
- [使用 TiDB Lightning 恢复 Amazon S3 兼容存储上的备份数据](restore-from-s3-using-job.md)
- [使用 Dumpling 备份 TiDB 集群数据到兼容 Amazon S3 的存储(已弃用)](backup-to-s3.md)
- [使用 TiDB Lightning 恢复 Amazon S3 兼容存储上的备份数据(已弃用)](restore-from-s3.md)
- 使用 Google Cloud Storage
- [使用 BR 备份 TiDB 集群数据到 GCS](backup-to-gcs-using-br.md)
- [使用 BR 恢复 GCS 上的备份数据](restore-from-gcs-using-br.md)
- [使用 Dumpling 备份 TiDB 集群数据到 GCS](backup-to-gcs.md)
- [使用 TiDB Lightning 恢复 GCS 上的备份数据](restore-from-gcs.md)
- [使用 Dumpling 备份 TiDB 集群数据到 GCS](backup-to-gcs-using-job.md)
- [使用 TiDB Lightning 恢复 GCS 上的备份数据](restore-from-gcs-using-job.md)
- [使用 Dumpling 备份 TiDB 集群数据到 GCS(已弃用)](backup-to-gcs.md)
- [使用 TiDB Lightning 恢复 GCS 上的备份数据(已弃用)](restore-from-gcs.md)
- 使用 Azure Blob Storage
- [使用 BR 备份 TiDB 集群数据到 Azblob](backup-to-azblob-using-br.md)
- [使用 BR 恢复 Azblob 上的备份数据](restore-from-azblob-using-br.md)
- [使用 BR 备份 TiDB 集群数据到 Azure Blob Storage](backup-to-azblob-using-br.md)
- [使用 BR 恢复 Azure Blob Storage 上的备份数据](restore-from-azblob-using-br.md)
- [使用 Dumpling 备份 TiDB 集群数据到 Azure Blob Storage](backup-to-blob-using-job.md)
- [使用 TiDB Lightning 恢复 Azure Blob Storage 的备份数据](restore-from-blob-using-job.md)
- 使用持久卷
- [使用 BR 备份 TiDB 集群数据到持久卷](backup-to-pv-using-br.md)
- [使用 BR 恢复持久卷上的备份数据](restore-from-pv-using-br.md)
Expand Down
115 changes: 115 additions & 0 deletions zh/backup-to-blob-using-job.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
---
title: 使用 Dumpling 备份 TiDB 数据到 Azure Blob Storage
summary: 本文介绍如何使用 Dumpling 将 TiDB 集群数据备份到 Azure Blob Storage。
---

# 使用 Dumpling 备份 TiDB 数据到 Azure Blob Storage

本文档介绍如何使用 [Dumpling](https://docs.pingcap.com/zh/tidb/stable/dumpling-overview/) 将部署在 Azure AKS 上的 TiDB 集群数据备份到 Azure Blob Storage。Dumpling 是一款数据导出工具,可将 TiDB 或 MySQL 中的数据导出为 SQL 或 CSV 格式,用于全量数据备份或导出。

## 准备 Dumpling 节点池

你可以在现有节点池中运行 Dumpling,也可以创建一个专用节点池。以下命令示例展示了如何创建一个新的节点池。使用前,请根据实际情况替换以下变量:

- `${clusterName}`:AKS 集群名称
- `${resourceGroup}`:资源组名称

```shell
az aks nodepool add --name dumpling \
--cluster-name ${clusterName} \
--resource-group ${resourceGroup} \
--zones 1 2 3 \
--node-count 1 \
--labels dedicated=dumpling
```

## 部署 Dumpling Job

本章节介绍如何配置、部署以及监控 Dumpling Job。

### 配置 Dumpling Job

Dumpling Job 的配置文件 (`dumpling_job.yaml`) 示例如下。使用前,请替换以下变量:

- `${name}`:Job 名称
- `${namespace}`:Kubernetes 命名空间
- `${version}`:Dumpling 镜像版本
- Dumpling 的相关参数,请参考 [Dumpling 主要选项表](https://docs.pingcap.com/zh/tidb/stable/dumpling-overview/#dumpling-主要选项表)。

```yaml
# dumpling_job.yaml
---
apiVersion: batch/v1
kind: Job
metadata:
name: ${name}
namespace: ${namespace}
labels:
app.kubernetes.io/component: dumpling
spec:
template:
spec:
nodeSelector:
dedicated: dumpling
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- dumpling
topologyKey: kubernetes.io/hostname
containers:
- name: ${name}
image: pingcap/dumpling:${version}
command:
- /bin/sh
- -c
- |
/dumpling \
--host=basic-tidb \
--port=4000 \
--user=root \
--password='' \
--s3.region=us-west-2 \
--threads=16 \
--rows=20000 \
--filesize=256MiB \
--database=test \
--filetype=csv \
--output=azure://external/testfolder?account-name=${accountname}&account-key=${accountkey}
restartPolicy: Never
backoffLimit: 0
```

### 创建 Dumpling Job

执行以下命令创建 Dumpling Job:

```shell
export name=dumpling
export version=v8.5.1
export namespace=tidb-cluster
export accountname=<your-account-name>
export accountkey=<your-account-key>

envsubst < dumpling_job.yaml | kubectl apply -f -
```

### 查看 Dumpling Job 状态

运行以下命令查看 Dumpling Job 的 Pod 状态:

```shell
kubectl -n ${namespace} get pod ${name}
```

### 查看 Dumpling Job 日志

运行以下命令查看 Dumpling Job 的日志输出:

```shell
kubectl -n ${namespace} logs pod ${name}
```
123 changes: 123 additions & 0 deletions zh/backup-to-gcs-using-job.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
title: 使用 Dumpling 备份 TiDB 数据到 Google Cloud Storage (GCS)
summary: 本文介绍如何使用 Dumpling 将 TiDB 集群数据备份到 Google Cloud Storage (GCS)。
---

# 使用 Dumpling 备份 TiDB 数据到 Google Cloud Storage (GCS)

本文档介绍如何使用 [Dumpling](https://docs.pingcap.com/zh/tidb/stable/dumpling-overview/) 将部署在 Google GKE 上的 TiDB 集群数据备份到 [Google Cloud Storage (GCS)](https://cloud.google.com/storage/docs)。Dumpling 是一款数据导出工具,可将 TiDB 或 MySQL 中的数据导出为 SQL 或 CSV 格式,用于全量数据备份或导出。

## 准备 Dumpling 节点池

你可以在现有节点池中运行 Dumpling,也可以创建一个专用节点池。以下命令示例展示了如何创建一个新的节点池。请根据实际情况替换以下变量:

- `${clusterName}`:GKE 集群名称

```shell
gcloud container node-pools create dumpling \
--cluster ${clusterName} \
--machine-type n2-standard-4 \
--num-nodes=1 \
--node-labels=dedicated=dumpling
```

## 部署 Dumpling Job

### 创建凭证 ConfigMap

将从 Google Cloud Console 下载的 `service account key` 文件保存为 `google-credentials.json`,然后使用以下命令创建 ConfigMap:

```shell
kubectl -n ${namespace} create configmap google-credentials --from-file=google-credentials.json
```

### 配置 Dumpling Job

Dumpling Job 的配置文件 (`dumpling_job.yaml`) 示例如下。使用前,请替换以下变量:

- `${name}`:Job 名称
- `${namespace}`:Kubernetes 命名空间
- `${version}`:Dumpling 镜像版本
- Dumpling 的相关参数,请参考 [Dumpling 主要选项表](https://docs.pingcap.com/zh/tidb/stable/dumpling-overview/#dumpling-主要选项表)。

```yaml
# dumpling_job.yaml
---
apiVersion: batch/v1
kind: Job
metadata:
name: ${name}
namespace: ${namespace}
labels:
app.kubernetes.io/component: dumpling
spec:
template:
spec:
nodeSelector:
dedicated: dumpling
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- dumpling
topologyKey: kubernetes.io/hostname
containers:
- name: ${name}
image: pingcap/dumpling:${version}
command:
- /bin/sh
- -c
- |
/dumpling \
--host=basic-tidb \
--port=4000 \
--user=root \
--password='' \
--threads=16 \
--rows=20000 \
--filesize=256MiB \
--database=test \
--filetype=csv \
--output=gcs://external/testfolder?credentials-file=/etc/config/google-credentials.json
volumeMounts:
- name: google-credentials
mountPath: /etc/config
volumes:
- name: google-credentials
configMap:
name: google-credentials
restartPolicy: Never
backoffLimit: 0
```

### 创建 Dumpling Job

执行以下命令创建 Dumpling Job:

```shell
export name=dumpling
export version=v8.5.1
export namespace=tidb-cluster

envsubst < dumpling_job.yaml | kubectl apply -f -
```

### 查看 Dumpling Job 状态

运行以下命令查看 Dumpling Job 的 Pod 状态:

```shell
kubectl -n ${namespace} get pod ${name}
```

### 查看 Dumpling Job 日志

运行以下命令查看 Dumpling Job 的日志输出:

```shell
kubectl -n ${namespace} logs pod ${name}
```
10 changes: 7 additions & 3 deletions zh/backup-to-gcs.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
---
title: 使用 Dumpling 备份 TiDB 集群数据到 GCS
summary: 介绍如何使用 Dumpling 将 TiDB 集群数据备份到 Google Cloud Storage (GCS)。
title: 使用 Dumpling 备份 TiDB 集群数据到 GCS (Helm)
summary: 介绍如何使用 Helm 部署 Dumpling 将 TiDB 集群数据备份到 Google Cloud Storage (GCS)。
aliases: ['/docs-cn/tidb-in-kubernetes/dev/backup-to-gcs/']
---

# 使用 Dumpling 备份 TiDB 集群数据到 GCS
# 使用 Dumpling 备份 TiDB 集群数据到 GCS (Helm)

> **警告:**
>
> 本文介绍的 Helm 部署方式已弃用,建议使用 [Job 方式](backup-to-gcs-using-job.md)进行备份操作。

本文档介绍如何将 Kubernetes 上 TiDB 集群的数据备份到 [Google Cloud Storage (GCS)](https://cloud.google.com/storage/docs/) 上。本文档中的“备份”,均是指全量备份(即 Ad-hoc 全量备份和定时全量备份)。

Expand Down
Loading