commit doc

docs/Cloudberry System Tools:pg_verifybackup Overview.md

@@ -0,0 +1,83 @@
+# Cloudberry System Tools:pg_verifybackup Feature Overview
+
+pg_verifybackup is able to verify the integrity of a base backup of a PostgreSQL cluster.This document gives a general introduction to the features of Cloudberry System Tools:pg_verifybackup.
+
+
+## Feature Description
+
+pg_verifybackup is used to check the integrity of a database cluster backup taken using 'pg_basebackup' against a 'backup_manifest' generated by the server at the time of the backup. The backup must be stored in the "plain" format; a "tar" format backup can be checked after extracting it.
+
+
+## Points For Attention
+
+It is important to note that the validation which is performed by pg_verifybackup does not and cannot include every check which will be performed by a running server when attempting to make use of the backup. Even if you use this tool, you should still perform test restores and verify that the resulting databases work as expected and that they appear to contain the correct data. However, pg_verifybackup can detect many problems that commonly occur due to storage problems or user errors.
+
+- If extra WAL files which are not required to recover the backup are present, they will not be checked by this tool, although a separate invocation of 'pg_waldump' could be used for that purpose. 
+
+
+- That WAL verification is version-specific: you must use the version of 'pg_verifybackup', and thus of 'pg_waldump', which pertains to the backup being checked.
+ In contrast, the data file integrity checks should work with any version of the server that generates a 'backup_manifest' file.
+
+## Synopsis
+
+pg_verifybackup [option...]
+
+
+## Operation Steps 
+
+Backup verification proceeds in four stages.
+
+
+### Step 1: Read the backup_manifest file
+
+If that file does not exist, cannot be read, is malformed, or fails verification against its own internal checksum, 'pg_verifybackup' will terminate with a fatal error.
+
+### Step 2: Verify the Data File
+
+'pg_verifybackup' will attempt to verify that the data files currently stored on disk are exactly the same as the data files which the server intended to send, with some exceptions that are described below. Extra and missing files will be detected, with a few exceptions. This step will ignore the presence or absence of, or any modifications to, 'postgresql.auto.conf', 'standby.signal', and 'recovery.signal', because it is expected that these files may have been created or modified as part of the process of taking the backup. It also won't complain about a 'backup_manifest' file in the target directory or about anything inside 'pg_wal', even though these files won't be listed in the backup manifest. Only files are checked; the presence or absence of directories is not verified, except indirectly: if a directory is missing, any files it should have contained will necessarily also be missing.
+
+### Step 3: Document Inspection 
+
+'pg_verifybackup' will checksum all the files, compare the checksums against the values in the manifest, and emit errors for any files for which the computed checksum does not match the checksum stored in the manifest. This step is not performed for any files which produced errors in the previous step, since they are already known to have problems. Files which were ignored in the previous step are also ignored in this step.
+
+### Step 4: Write-Ahead Logging Record verification
+
+'pg_verifybackup' will use the manifest to verify that the write-ahead log records which will be needed to recover the backup are present and that they can be read and parsed. The 'backup_manifest' contains information about which write-ahead log records will be needed, and 'pg_verifybackup' will use that information to invoke 'pg_waldump' to parse those write-ahead log records. The '--quiet' flag will be used, so that 'pg_waldump' will only report errors, without producing any other output. While this level of verification is sufficient to detect obvious problems such as a missing file or one whose internal checksums do not match, they aren't extensive enough to detect every possible problem that might occur when attempting to recover. For instance, a server bug that produces write-ahead log records that have the correct checksums but specify nonsensical actions can't be detected by this method.
+
+## Options
+
+pg_verifybackup accepts the following command-line arguments:
+
+| Command | Option Parameters    | Functional Discription                                       |
+| ------- | -------------------- | ------------------------------------------------------------ |
+| -e      | --exit-on-error      | Exit as soon as a problem with the backup is detected. If this option is not specified, 'pg_verifybackup' will continue checking the backup even after a problem has been detected, and will report all problems detected as errors. |
+| -i path | --ignore=path        | Ignore the specified file or directory, which should be expressed as a relative path name, when comparing the list of data files actually present in the backup to those listed in the 'backup_manifest' file. If a directory is specified, this option affects the entire subtree rooted at that location. Complaints about extra files, missing files, file size differences, or checksum mismatches will be suppressed if the relative path name matches the specified path name. This option can be specified multiple times. |
+| -m path | --manifest-path=path | Use the manifest file at the specified path, rather than one located in the root of the backup directory. |
+| -n      | --no-parse-wal       | Don't attempt to parse write-ahead log data that will be needed to recover from this backup. |
+| -q      | --quiet              | Don't print anything when a backup is successfully verified. |
+| -s      | --skip-checksums     | Do not verify data file checksums. The presence or absence of files and the sizes of those files will still be checked. This is much faster, because the files themselves do not need to be read. |
+| -w path | --wal-directory=path | Try to parse WAL files stored in the specified directory, rather than in pg_wal. This may be useful if the backup is stored in a separate location from the WAL archive. |
+|         |                      | Other options are also available                             |
+| -V      | --version            | Print the pg_verifybackup version and exit.                  |
+| -?      | --help               | Show help about pg_verifybackup command line arguments, and exit. |
+
+
+## Examples
+
+To create a base backup of the server at 'mydbserver' and verify the integrity of the backup:
+
+```
+$ pg_basebackup -h mydbserver -D /usr/local/pgsql/data$ pg_verifybackup /usr/local/pgsql/data
+```
+
+To create a base backup of the server at 'mydbserver', move the manifest somewhere outside the backup directory, and verify the backup:
+
+```
+$ pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234$ mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234$ pg_verifybackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234
+```
+
+To verify a backup while ignoring a file that was added manually to the backup directory, and also skipping checksum verification:
+
+```
+$ pg_basebackup -h mydbserver -D /usr/local/pgsql/data$ edit /usr/local/pgsql/data/note.to.self$ pg_verifybackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data                                                                                                                                                                                                                                                                                                                                
+```

i18n/zh/docusaurus-plugin-content-docs/current/Cloudberry 系统工具：pg_verifybackup.md

@@ -0,0 +1,94 @@
+# Cloudberry 系统工具：pg_verifybackup
+
+pg_verifybackup 能够验证 PostgreSQL 集群基础备份的完整性。本文档介绍 Cloudberry 系统工具：pg_verifybackup 的功能。
+
+## 功能描述
+
+pg_verifybackup 用于对照备份时服务器生成的 backup_manifest 文件，检查使用 pg_basebackup 工具所做的数据库集群备份的完整性。备份必须以 “普通” 格式存储；“tar” 格式的备份在解压后可以进行检查。
+
+## 注意事项
+
+pg_verifybackup 执行的验证工作并不会也不能涵盖运行中的服务器在尝试使用该备份时所执行的每一项检查。即便使用了此工具，仍应执行测试恢复操作，并验证恢复后得到的数据库能否按预期工作以及其中的数据是否正确。不过，pg_verifybackup 能够检测出许多因存储问题或用户操作失误而经常出现的问题。
+
+- 如果存在恢复备份时不需要的额外的预写日志（WAL）文件，此工具不会对其进行检查，不过可以为此单独调用 'pg_waldump'。
+
+- WAL 验证是特定于版本的：必须使用与被检查备份相对应版本的 'pg_verifybackup'（以及相应版本的 pg_waldump）。
+
+相比之下，数据文件完整性检查应当适用于生成 'backup_manifest' 文件的服务器的任何版本。
+
+## 语法格式
+
+pg_verifybackup \[选项……\]
+
+## 操作步骤
+
+备份验证分四个阶段进行。具体的操作步骤如下：
+
+### 步骤 1：读取 backup_manifest 文件
+
+如果该文件不存在、无法读取、格式有误或者针对其自身内部校验和的验证失败，'pg_verifybackup' 将会以严重错误终止运行。
+
+### 步骤 2：验证数据文件
+
+'pg_verifybackup' 会尝试验证当前存储在磁盘上的数据文件与服务器原本打算发送的数据文件是否完全一致，但存在下文所述的一些例外情况。它能够检测出多余和缺失的文件（也有少数例外情况）。具体说明如下：
+
+- 此步骤会忽略 'postgresql.auto.conf'、'standby.signal' 和 'recovery.signal' 文件的存在与否以及任何修改情况，因为预计这些文件可能在备份过程中被创建或修改。
+
+- 同时不会对目标目录中的 'backup_manifest' 文件或者 'pg_wal' 目录内的任何内容提出异议，即便这些文件不会列在备份清单中。
+
+- 只对文件进行检查，不会直接验证目录的存在与否，不过存在间接验证的情况。如果某个目录缺失，那么该目录原本应包含的任何文件必然也会缺失。
+
+### 步骤 3：文件检查
+
+'pg_verifybackup' 会对所有文件计算校验和，将校验和与清单中的值进行对比，并针对计算出的校验和与清单中存储的校验和不匹配的任何文件报错。
+
+'''warning
+对于在上一步骤中出现错误的文件，不会执行此步骤，因为已知这些文件存在问题。在上一步骤中被忽略的文件在本步骤中同样会被忽略。 
+'''
+
+### 步骤 4：预写日志记录验证
+
+'pg_verifybackup' 会使用清单来验证恢复备份所需的预写日志记录是否存在，以及能否对其进行读取和解析。'backup_manifest' 文件包含了关于需要哪些预写日志记录的信息，'pg_verifybackup' 会利用这些信息调用 'pg_waldump' 来解析这些预写日志记录。会使用 '--quiet' 标志，这样 'pg_waldump' 就只会报告错误，而不会产生其他任何输出。尽管这种验证级别足以检测出例如文件缺失或者文件内部校验和不匹配之类的明显问题，但还不足以检测出在尝试恢复时可能出现的所有问题。例如，服务器存在漏洞导致生成的预写日志记录校验和正确但指定了无意义的操作，这种情况就无法通过这种方法检测出来。
+
+## 命令参数
+
+pg_verifybackup 接受以下命令行参数：
+
+|  |  |  |
+|:---|:---|:---|
+| **命令** | **选项参数** | **功能描述** |
+| -e | --exit-on-error | 一旦检测到备份存在问题就立即退出。如果未指定此选项，'pg_verifybackup' 在检测到问题后仍会继续检查备份，并将检测到的所有问题作为错误进行报告。 |
+| -i ***path*** | --ignore=***path*** | 在将备份中实际存在的数据文件列表与 'backup_manifest' 文件中列出的文件进行对比时，忽略指定的文件或目录（应当以相对路径名表示）。如果指定的是目录，此选项会影响以该位置为根的整个子树。如果相对路径名与指定的路径名匹配，那么关于多余文件、缺失文件、文件大小差异或者校验和不匹配的相关报错将会被抑制。此选项可以多次指定。 |
+| -m ***path*** | --manifest-path=***path*** | 使用指定路径下的清单文件，而非位于备份目录根目录下的清单文件。 |
+| -n | --no-parse-wal | 不要尝试解析从该备份恢复时所需的预写日志数据。 |
+| -q | --quiet | 在备份验证成功时不输出任何内容。 |
+| -s | --skip-checksums | 不验证数据文件的校验和。但仍会检查文件的存在与否以及文件大小。这样速度会快很多，因为无需读取文件本身。 |
+| -w ***path*** | --wal-directory=***path*** | 尝试解析存储在指定目录（而非 pg_wal 目录）中的预写日志（WAL）文件。如果备份存储的位置与 WAL 归档所在位置不同，此操作可能会很有用。 |
+| 其他选项也可使用 |  |  |
+| -V | --version | 打印 'pg_verifybackup' 的版本信息并退出。 |
+| -? | --help | 显示有关 'pg_verifybackup' 命令行参数的帮助信息并退出。 |
+
+## 示例
+
+对 'mydbserver' 的服务器创建基础备份并验证备份的完整性：
+
+'''
+$ pg_basebackup -h mydbserver -D /usr/local/pgsql/data
+$ pg_verifybackup /usr/local/pgsql/data
+'''
+
+对 'mydbserver' 的服务器创建基础备份，将清单文件移到备份目录之外的某个位置，然后验证备份：
+
+```
+$ pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234
+$ mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234
+$ pg_verifybackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234
+```
+
+要在验证备份时忽略手动添加到备份目录中的某个文件，并且跳过校验和验证：
+
+```
+$ pg_basebackup -h mydbserver -D /usr/local/pgsql/data
+$ edit /usr/local/pgsql/data/note.to.self
+$ pg_verifybackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data
+```