Merge pull request #177314 from v-amallick/Oct-24-2021-PGSQL-command

PGSQL Automation docs update

Author: ttorble

articles/backup/backup-azure-data-protection-use-rest-api-backup-postgresql.md

@@ -0,0 +1,231 @@
+---
+title: Restore Azure PostgreSQL databases via Azure CLI
+description: Learn how to restore Azure PostgreSQL databases using Azure CLI.
+ms.topic: conceptual
+ms.date: 10/25/2021
+author: v-amallick
+ms.service: backup
+ms.author: v-amallick
+---
+
+# Restore Azure PostgreSQL databases using Azure CLI
+
+This article explains how to restore [Azure PostgreSQL databases](/azure/postgresql/overview#azure-database-for-postgresql---single-server) to an Azure PostgreSQL server backed-up by Azure Backup.
+
+Being a PaaS database, the Original-Location Recovery (OLR) option to restore by replacing the existing database (from where the backups were taken) isn't supported. You can restore from a recovery point to create a new database in the same Azure PostgreSQL server or in any other PostgreSQL server. This is called Alternate-Location Recovery (ALR) that helps to keep both - the source database and the restored (new) database.
+
+In this article, you'll learn how to:
+
+- Restore to create a new PostgreSQL database
+
+- Track the restore operation status
+
+We'll refer to an existing Backup vault _TestBkpVault_, under the resource group _testBkpVaultRG_ in the examples.
+
+## Restore a backed-up PostgreSQL database
+
+### Set up permissions
+
+Backup vault uses Managed Identity to access other Azure resources. To restore from backup, Backup vault’s managed identity requires a set of permissions on the Azure PostgreSQL server to which the database should be restored.
+
+To assign the relevant permissions for vault's system-assigned managed identity on the target PostgreSQL server, see the [set of permissions needed to backup Azure PostgreSQL database](/azure/backup/backup-azure-database-postgresql-overview#set-of-permissions-needed-for-azure-postgresql-database-restore).
+
+To restore the recovery point as files to a storage account, the [Backup vault's system-assigned managed identity needs access on the target storage account](/azure/backup/restore-azure-database-postgresql#restore-permissions-on-the-target-storage-account).
+
+### Fetch the relevant recovery point
+
+To list all backup instances within a vault, use [az dataprotection backup-instance list](/cli/azure/dataprotection/backup-instance?view=azure-cli-latest&preserve-view=true#az_dataprotection_backup_instance_list) command, and then fetch the relevant instance using the [az dataprotection backup-instance show](/cli/azure/dataprotection/backup-instance?view=azure-cli-latest&preserve-view=true#az_dataprotection_backup_instance_show) command. Alternatively, for _at-scale_ scenarios, you can list backup instances across vaults and subscriptions using the [az dataprotection backup-instance list-from-resourcegraph](/cli/azure/dataprotection/backup-instance?view=azure-cli-latest&preserve-view=true#az_dataprotection_backup_instance_list_from_resourcegraph) command.
+
+```azurecli
+az dataprotection backup-instance list-from-resourcegraph --datasource-type AzureDatabaseForPostgreSQL -subscriptions "xxxxxxxx-xxxx-xxxx-xxxx"
+
+  {
+    "datasourceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/ossdemoRG/providers/Microsoft.DBforPostgreSQL/servers/testpostgresql/databases/empdb11",
+    "extendedLocation": null,
+    "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/testBkpVaultRG/providers/Microsoft.DataProtection/backupVaults/testBkpVault/backupInstances/testpostgresql-empdb11-957d23b1-c679-4c94-ade6-c4d34635e149",
+    "identity": null,
+    "kind": "",
+    "location": "",
+    "managedBy": "",
+    "name": "testpostgresql-empdb11-957d23b1-c679-4c94-ade6-c4d34635e149",
+    "plan": null,
+    "properties": {
+      "currentProtectionState": "ProtectionConfigured",
+      "dataSourceInfo": {
+        "baseUri": null,
+        "datasourceType": "Microsoft.DBforPostgreSQL/servers/databases",
+        "objectType": "Datasource",
+        "resourceID": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/ossdemoRG/providers/Microsoft.DBforPostgreSQL/servers/testpostgresql/databases/empdb11",
+        "resourceLocation": "westus",
+        "resourceName": "postgres",
+        "resourceProperties": null,
+        "resourceType": "Microsoft.DBforPostgreSQL/servers/databases",
+        "resourceUri": ""
+      },
+      "dataSourceProperties": null,
+      "dataSourceSetInfo": {
+        "baseUri": null,
+        "datasourceType": "Microsoft.DBforPostgreSQL/servers/databases",
+        "objectType": "DatasourceSet",
+        "resourceID": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/ossdemoRG/providers/Microsoft.DBforPostgreSQL/servers/testpostgresql",
+        "resourceLocation": "westus",
+        "resourceName": "testpostgresql",
+        "resourceProperties": null,
+        "resourceType": "Microsoft.DBforPostgreSQL/servers",
+        "resourceUri": ""
+      },
+      "datasourceAuthCredentials": {
+        "objectType": "SecretStoreBasedAuthCredentials",
+        "secretStoreResource": {
+          "secretStoreType": "AzureKeyVault",
+          "uri": "https://vikottur-test.vault.azure.net/secrets/dbauth3",
+          "value": null
+        }
+      },
+      "friendlyName": "testpostgresql\\empdb11",
+      "objectType": "BackupInstance",
+      "policyInfo": {
+        "policyId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/testBkpVaultRG/providers/Microsoft.DataProtection/backupVaults/testBkpVault/backupPolicies/osspol3",
+        "policyParameters": null,
+        "policyVersion": ""
+      },
+      "protectionErrorDetails": null,
+      "protectionStatus": {
+        "errorDetails": null,
+        "status": "ProtectionConfigured"
+      },
+      "provisioningState": "Succeeded",
+      "validationType": null
+    },
+    "protectionState": "ProtectionConfigured",
+    "resourceGroup": "testBkpVaultRG",
+    "sku": null,
+    "subscriptionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+    "tags": null,
+    "tenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+    "type": "microsoft.dataprotection/backupvaults/backupinstances",
+    "vaultName": "testBkpVault",
+    "zones": null
+  }
+.
+.
+.
+.
+.
+```
+
+Once the instance is identified, fetch the relevant recovery point using the [az dataprotection recovery-point list](/cli/azure/dataprotection/recovery-point?view=azure-cli-latest&preserve-view=true#az_dataprotection_recovery_point_list) command.
+
+```azurecli
+az dataprotection recovery-point list --backup-instance-name testpostgresql-empdb11-957d23b1-c679-4c94-ade6-c4d34635e149 -g testBkpVaultRG --vault-name TestBkpVault
+
+{
+  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/testBkpVaultRG/providers/Microsoft.DataProtection/backupVaults/testBkpVault/backupInstances/testpostgresql-empdb11-957d23b1-c679-4c94-ade6-c4d34635e149/recoveryPoints/9da55e757af94261afa009b43cd3222a",
+  "name": "9da55e757af94261afa009b43cd3222a",
+  "properties": {
+    "friendlyName": "2031fdb43a914114b6ce644eb6fcb5ce",
+    "objectType": "AzureBackupDiscreteRecoveryPoint",
+    "policyName": "oss-clitest-policy",
+    "policyVersion": null,
+    "recoveryPointDataStoresDetails": [
+      {
+        "creationTime": "2021-09-13T15:17:41.209845+00:00",
+        "expiryTime": null,
+        "id": "beddea84-7b30-42a5-a752-7c75baf96a52",
+        "metaData": "{\"objectType\":\"PostgresBackupMetadata\",\"version\":\"1.0\",\"postgresVersion\":\"11\",\"dbName\":\"postgres\",\"serverName\":\"testpostgresql\",\"serverFQDN\":\"testpostgresql.postgres.database.azure.com\",\"usernameUsed\":\"backupadmin@testpostgresql\",\"backupToolPath\":\"postgresql-11.6-1\\\\bin\\\\pg_dump.exe\",\"backupType\":\"Full\",\"backupDumpFormat\":\"CUSTOM\",\"backupToolArgsFormat\":\"--no-acl --no-owner --serializable-deferrable --no-tablespaces --quote-all-identifiers -Fc -d postgres://{0}:{1}@{2}:5432/{3}?sslmode=verify-full&sslrootcert=E:\\\\approot\\\\Plugins\\\\Postgres\\\\..\\\\..\\\\postgres-root.crt\",\"storageUnits\":{\"1\":\"DbBackupDumpData\"},\"streamNamesInFirstStorageUnit\":[\"dbbkpdmpdatastream-1631546260050\"],\"pitId\":\"2031fdb43a914114b6ce644eb6fcb5ce\",\"bytesTransferred\":2063,\"dataSourceSize\":8442527,\"backupToolVersion\":\"11\"}",
+        "rehydrationExpiryTime": null,
+        "rehydrationStatus": null,
+        "state": "COMMITTED",
+        "type": "VaultStore",
+        "visible": true
+      }
+    ],
+    "recoveryPointId": "9da55e757af94261afa009b43cd3222a",
+    "recoveryPointTime": "2021-09-13T15:17:41.209845+00:00",
+    "recoveryPointType": "Full",
+    "retentionTagName": "default",
+    "retentionTagVersion": "637671427933449525"
+  },
+  "resourceGroup": "testBkpVaultRG",
+  "systemData": null,
+  "type": "Microsoft.DataProtection/backupVaults/backupInstances/recoveryPoints"
+}
+```
+
+If you need to fetch the recovery point from archive tier, then the _type_ variable in _recoveryPointDataStoreDetails_ will be _ArchiveStore_.
+
+### Prepare the restore request
+
+There are various restore options for a PostgreSQL database. You can restore the recovery point as another database or restore as files. The recovery point can be on archive tier as well.
+
+#### Restore as database
+
+Construct the Azure Resource Manager ID (ARM ID) of the new PostgreSQL database to be created (with the target PostgreSQL server to which permissions were assigned as detailed [above](#set-up-permissions)) and the required PostgreSQL database name. For example, a PostgreSQL database can be named **emprestored21** under a target PostgreSQL server **targetossserver** in resource group **targetrg** with a different subscription.
+
+```azurecli
+$targetOssId = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx/resourceGroups/targetrg/providers/providers/Microsoft.DBforPostgreSQL/servers/targetossserver/databases/emprestored21"
+```
+
+Use the [az dataprotection backup-instance restore initialize-for-data-recovery](/cli/azure/dataprotection/backup-instance/restore?view=azure-cli-latest&preserve-view=true#az_dataprotection_backup_instance_restore_initialize_for_data_recovery) command to prepare the restore request with all relevant details.
+
+```azurecli
+az dataprotection backup-instance restore initialize-for-data-recovery --datasource-type AzureDatabaseForPostgreSQL  --restore-location {location} --source-datastore VaultStore --target-resource-id $targetOssId --recovery-point-id 9da55e757af94261afa009b43cd3222a --secret-store-type AzureKeyVault --secret-store-uri "https://restoreoss-test.vault.azure.net/secrets/dbauth3" > OssRestoreReq.JSON
+```
+
+For an archive-based recovery point, you need to:
+
+1. Rehydrate from archive datastore to vault store
+1. Modify the source datastore.
+1. Add other parameters to specify the rehydration priority.
+1. Specify the duration for which the rehydrated recovery point should be retained in the vault data store.
+1. Restore as a database from this recovery point.
+
+Use the following command to prepare the request for all the above mentioned operations, at once.
+
+```azurecli
+az dataprotection backup-instance restore initialize-for-data-recovery --datasource-type AzureDatabaseForPostgreSQL  --restore-location {location} --source-datastore ArchiveStore --target-resource-id $targetOssId --recovery-point-id 9da55e757af94261afa009b43cd3222a --secret-store-type AzureKeyVault --secret-store-uri "https://restoreoss-test.vault.azure.net/secrets/dbauth3" --rehydration-priority Standard --rehydration-duration 12 > OssRestoreFromArchiveReq.JSON
+```
+
+#### Restore as files
+
+Fetch the URI of the container, within the storage account to which permissions were assigned as detailed [above](#set-up-permissions). For example, a container named **testcontainerrestore** under a storage account **testossstorageaccount** with a different subscription.
+
+```azurecli
+$contURI = "https://testossstorageaccount.blob.core.windows.net/testcontainerrestore"
+```
+
+Use the [az dataprotection backup-instance restore initialize-for-data-recovery-as-files](/cli/azure/dataprotection/backup-instance/restore?view=azure-cli-latest&preserve-view=true#az_dataprotection_backup_instance_restore_initialize_for_data_recovery_as_files) command to prepare the restore request with all relevant details.
+
+```azurecli
+az dataprotection backup-instance restore initialize-for-data-recovery-as-files --datasource-type AzureDatabaseForPostgreSQL  --restore-location {location} --source-datastore VaultStore -target-blob-container-url $contURI --target-file-name "empdb11_postgresql-westus_1628853549768" --recovery-point-id 9da55e757af94261afa009b43cd3222a > OssRestoreAsFilesReq.JSON
+```
+
+For archive-based recovery point, modify the source datastore, and add the rehydration priority and the retention duration, in days, of the rehydrated recovery point as mentioned below:
+
+```azurecli
+az dataprotection backup-instance restore initialize-for-data-recovery-as-files --datasource-type AzureDatabaseForPostgreSQL  --restore-location {location} --source-datastore ArchiveStore -target-blob-container-url $contURI --target-file-name "empdb11_postgresql-westus_1628853549768" --recovery-point-id 9da55e757af94261afa009b43cd3222a --rehydration-priority Standard --rehydration-duration 12 > OssRestoreAsFilesReq.JSON
+```
+
+You can also validate if the JSON file will succeed to create new resources using the [az dataprotection backup-instance validate-for-restore](/cli/azure/dataprotection/backup-instance?view=azure-cli-latest&preserve-view=true#az_dataprotection_backup_instance_validate_for_restore) command.
+
+### Trigger the restore
+
+Use the [az dataprotection backup-instance restore trigger](/cli/azure/dataprotection/backup-instance/restore?view=azure-cli-latest&preserve-view=true#az_dataprotection_backup_instance_restore_trigger) command to trigger the restore operation with the request prepared above.
+
+```azurecli-interactive
+az dataprotection backup-instance restore trigger -g testBkpVaultRG --vault-name TestBkpVault --backup-instance-name testpostgresql-empdb11-957d23b1-c679-4c94-ade6-c4d34635e149 --parameters OssRestoreReq.JSON
+```
+
+## Tracking job
+
+Track all jobs using the [az dataprotection job list](/cli/azure/dataprotection/job?view=azure-cli-latest&preserve-view=true#az_dataprotection_job_list) command. You can list all jobs and fetch a particular job detail.
+
+You can also use _Az.ResourceGraph_ to track all jobs across all Backup vaults. Use the [az dataprotection job list-from-resourcegraph](/cli/azure/dataprotection/job?view=azure-cli-latest&preserve-view=true#az_dataprotection_job_list_from_resourcegraph) command to get the relevant job that is across all Backup vaults.
+
+```azurecli
+az dataprotection job list-from-resourcegraph --datasource-type AzureDatabaseForPostgreSQL --operation Restore
+```
+
+## Next steps
+
+- [Azure PostgreSQL Backup overview](backup-azure-database-postgresql-overview.md)