Skip to content

Backups page update #4215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
label: 'Export Backups to your Own Cloud Account'
collapsible: true
collapsed: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,361 @@
---
sidebar_label: 'Backup or restore from UI'
slug: /cloud/manage/backups/backup-restore-via-ui
title: 'Take a backup or restore a backup from the UI'
description: 'Page describing how to take a backup or restore a backup from the UI with your own bucket'
sidebar_position: 2
---

import Image from '@theme/IdealImage'
import arn from '@site/static/images/cloud/manage/backups/arn.png'
import change_external_backup from '@site/static/images/cloud/manage/backups/change_external_backup.png'
import configure_arn_s3_details from '@site/static/images/cloud/manage/backups/configure_arn_s3_details.png'
import view_backups from '@site/static/images/cloud/manage/backups/view_backups.png'
import backup_command from '@site/static/images/cloud/manage/backups/backup_command.png'
import gcp_configure from '@site/static/images/cloud/manage/backups/gcp_configure.png'
import gcp_stored_backups from '@site/static/images/cloud/manage/backups/gcp_stored_backups.png'
import gcp_restore_command from '@site/static/images/cloud/manage/backups/gcp_restore_command.png'
import azure_connection_details from '@site/static/images/cloud/manage/backups/azure_connection_details.png'
import view_backups_azure from '@site/static/images/cloud/manage/backups/view_backups_azure.png'

# Take a backup or restore a backup from the UI {#ui-experience}

## AWS {#AWS}

### Taking backups to AWS {#taking-backups-to-aws}

#### Steps to follow in AWS {#aws-steps}

Follow the steps below on your AWS account:

<VerticalStepper headerLevel="h5">

##### Create an S3 bucket {#create-s3-bucket}

Create an AWS S3 bucket in your account where you want to export backups.

##### Create an IAM role {#create-iam-role}

Create an IAM role that ClickHouse Cloud service will be able to assume into,
to write to this bucket - for role-based authentication:

* a. For this role, you will need to update the IAM role trust policy to include
the following statement:

```json
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::463754717262:role/CH-S3-bordeaux-ar-90-ue2-29-Role"
},
"Action": "sts:AssumeRole"
},
```
* b. The ARN in this case is obtained from the ClickHouse Cloud service settings
page which looks similar to this:

<Image img={arn} alt="AWS S3 ARN" size="lg" />

##### Update permissions for role {#update-permissions-for-role}

You will also need to update the permissions for this role so this ClickHouse
Cloud service can write to the S3 bucket. This is done by including a JSON similar
to this one in the role permissions:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::testchbackups"
],
"Effect": "Allow"
},
{
"Action": [
"s3:Get*",
"s3:List*",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::testchbackups/*"
],
"Effect": "Allow"
}
]
}
```

</VerticalStepper>

#### Steps to follow in ClickHouse Cloud {#cloud-steps}

Follow the steps below in the ClickHouse Cloud console to configure the external bucket:

<VerticalStepper headerLevel="h5">

##### Change external backup {#configure-external-bucket}

On the “Settings” page, click on `Change external backup`

<Image img={change_external_backup} alt="Change external backup" size="lg" />

##### Configure AWS IAM Role ARN and S3 bucket details {#configure-aws-iam-role-arn-and-s3-bucket-details}

On the next screen provide the AWS IAM Role ARN and S3 bucket details:

<Image img={configure_arn_s3_details} alt="Configure AWS IAM Role ARN and S3 bucket details" size="lg" />

##### Save changes {#save-changes}

Click on “Save External Bucket” to save the settings

##### Changing the backup schedule from the default schedule {#changing-the-backup-schedule}

External Backups will now happen in your bucket on the default schedule.
Alternatively, you can configure the backup schedule from the “Settings” page.
If configured differently, the custom schedule is used to write backups to your
bucket and the default schedule (backups every 24 hours) is used for backups in
the ClickHouse cloud owned bucket.

##### View backups stored in your bucket {#view-backups-stored-in-your-bucket}

The Backups page should display these backups in your bucket in a separate table
as shown below:

<Image img={view_backups} alt="View backups stored in your bucket" size="lg" />

</VerticalStepper>

### Restoring backups from AWS {#restoring-backups-from-aws}

Follow the steps below to restore backups from AWS:

<VerticalStepper headerLevel="h5">

##### Create a new service to restore to {#create-new-service-to-restore-to}

Create a new service to restore the backup to. Currently, we only support
restoring a backup into a new service.

##### Add service ARN {#add-service-arn}

Add the newly created service’s ARN (from the service settings page in Clickhouse
Cloud console) to the trust policy for the IAM role. This is the same as the
[second step](#create-iam-role) in the AWS Steps section above. This is required
so the new service can access the S3 bucket.

##### Get SQL command used to restore backup {#obtain-sql-command-to-restore-backup}

Click on the “access or restore a backup” link above the list of backups in the
UI to get the SQL command to restore the backup. The command should look like
the one shown below, and you can pick the appropriate backup from the dropdown
to get the restore command for that specific backup:

<Image img={backup_command} alt="Get SQL command used to restore backup" size="lg" />

##### Run the restore command {#run-the-restore-command}

Run the restore command from the SQL console in the newly created service to
restore the backup.

</VerticalStepper>

## GCP {#gcp}

### Taking backups to GCP {#taking-backups-to-gcp}

Follow the steps below to take backups to GCP:

#### Steps to follow in GCP {#gcp-steps-to-follow}

<VerticalStepper headerLevel="h5">

##### Create a GCP storage bucket {#create-a-gcp-storage-bucket}

Create a storage bucket in your GCP account to export backups.

##### Generate an HMAC Key and Secret {#generate-an-hmac-key-and-secret}

Generate an HMAC Key and Secret, which is required for password-based authentication. Follow the steps below to generate the keys:

* a. Create a service account
* I. Navigate to the IAM & Admin section in the Google Cloud Console and select Service Accounts.
* II. Click Create Service Account, provide a name and ID, and click Create and Continue.
* III. Grant the necessary roles for Cloud Storage access (e.g., Storage Object Admin or more granular roles like
Storage Object Creator and Storage Object Viewer) to this service account.
* IV. Click Done to finalize the service account creation.

* b. Generate the HMAC key
* I. Go to Cloud Storage in the Google Cloud Console, and select `Settings`
* II Go to the Interoperability tab.
* III. In the `Service account HMAC` section, click Create a key for a service account.
* IV. Choose the service account you created in the previous step from the dropdown menu.
* V. Click Create key.

* c. Securely store the credentials:
* I. The system will display the Access ID (your HMAC key) and the Secret (your HMAC secret). Save these values, as
the secret will not be displayed again after you close this window.

</VerticalStepper>

#### Steps to follow in ClickHouse Cloud {#gcp-cloud-steps}

Follow the steps below in the ClickHouse Cloud console to configure the external bucket:

<VerticalStepper headerLevel="h5">

##### Change external backup {#gcp-configure-external-bucket}

On the “Settings” page, click on “Change external backup”

<Image img={change_external_backup} alt="Change external backup" size="lg" />

##### Configure GCP HMAC Key and Secret {#gcp-configure-gcp-hmac-key-and-secret}

On the next screen provide the GCP bucket path, HMAC key and Secret
created in the previous section.

<Image img={gcp_configure} alt="Configure GCP HMAC Key and Secret" size="lg" />

##### Save external bucket {#gcp-save-external-bucket}

Click on `Save External Bucket` to save the settings.

##### Changing the backup schedule from the default schedule {#gcp-changing-the-backup-schedule}

External Backups will now happen in your bucket on the default schedule.
Alternatively, you can configure the backup schedule from the `Settings` page.
If configured differently, the custom schedule is used to write backups to your
bucket and the default schedule (backups every 24 hours) is used for backups in
ClickHouse cloud owned bucket.

##### View backups stored in your bucket {#gcp-view-backups-stored-in-your-bucket}

The Backups page should display these backups in your bucket in a separate table as shown below

<Image img={gcp_stored_backups} alt="View backups stored in your bucket" size="lg" />

</VerticalStepper>

### Restoring backups from GCP {#gcp-restoring-backups-from-gcp}

Follow the steps below to restore backups from GCP:

<VerticalStepper headerLevel="h5">

##### Create a new service to restore to {#gcp-create-new-service-to-restore-to}

Create a new service to restore the backup to. Currently, we only support restoring a backup into a new service.

##### Get SQL command used to restore backup {#gcp-obtain-sql-command-to-restore-backup}

Click on the `access or restore a backup` link above the list of backups in the
UI to get the SQL command to restore the backup. The command should look like this,
and you can pick the appropriate backup from the dropdown to get the restore
command for that specific backup. You will need to add your secret access key
to the command.

<Image img={gcp_restore_command} alt="Get SQL command used to restore backup" size="lg" />

##### Run SQL command to restore backup {#gcp-run-sql-command-to-restore-backup}

Run the restore command from the SQL console in the newly created service to
restore the backup.

</VerticalStepper>

## Azure {#azure}

### Taking backups to Azure {#taking-backups-to-azure}

Follow the steps below to take backups to Azure:

#### Steps to follow in Azure {#steps-to-follow-in-azure}

<VerticalStepper headerLevel="h5">

##### Create a storage account {#azure-create-a-storage-account}

Create a storage account or select an existing storage account in the Azure
portal where you want to store your backups.

##### Get connection string {#azure-get-connection-string}

* a. In your storage account overview, look for the section called `Security + networking` and click on `Access keys`.
* b. Here, you will see `key1` and `key2`. Under each key, you’ll find a `Connection string` field.
* c. Click `Show` to reveal the connection string. Copy the connection string to set up on ClickHouse Cloud.

</VerticalStepper>

#### Steps to follow in ClickHouse Cloud {#azure-cloud-steps}

Follow the steps below in the ClickHouse Cloud console to configure the external bucket:

<VerticalStepper headerLevel="h5">

##### Change external backup {#azure-configure-external-bucket}

On the `Settings` page, click on `Change external backup`

<Image img={change_external_backup} alt="Change external backup" size="lg" />

##### Provide connection string and container name for your Azure storage account {#azure-provide-connection-string-and-container-name-azure}

On the next screen provide the Connection String and Container Name for your
Azure storage account created in the previous section:

<Image img={azure_connection_details} alt="Provide connection string and container name for your Azure storage account" size="lg" />

##### Save external bucket {#azure-save-external-bucket}

Click on `Save External Bucket` to save the settings

##### Changing the backup schedule from the default schedule {#azure-changing-the-backup-schedule}

External Backups will now happen in your bucket on the default schedule. Alternatively,
you can configure the backup schedule from the “Settings” page. If configured differently,
the custom schedule is used to write backups to your bucket and the default schedule
(backups every 24 hours) is used for backups in ClickHouse cloud owned bucket.

##### View backups stored in your bucket {#azure-view-backups-stored-in-your-bucket}

The Backups page should display these backups in your bucket in a separate table
as shown below:

<Image img={view_backups_azure} alt="View backups stored in your bucket" size="lg" />

</VerticalStepper>

### Restoring backups from Azure {#azure-restore-steps}

To restore backups from Azure, follow the steps below:

<VerticalStepper headerLevel="h5">

##### Create a new service to restore to {#azure-create-new-service-to-restore-to}

Create a new service to restore the backup to. Currently, we only support
restoring a backup into a new service.

##### Get SQL command used to restore backup {#azure-obtain-sql-command-to-restore-backup}

Click on the `access or restore a backup` link above the list of backups in the
UI to obtain the SQL command to restore the backup. The command should look like
this, and you can pick the appropriate backup from the dropdown to get the
restore command for that specific backup. You will need to add your Azure
storage account connection string to the command.

##### Run SQL command to restore backup {#azure-run-sql-command-to-restore-backup}

Run the restore command from the SQL console in the newly created service to
restore the backup.

</VerticalStepper>
Loading