Skip to content

Commit 76a8a76

Browse files
v-regandownerv-regandowner
authored
Merge pull request #198447 from seesharprun/may16-cosmos-container-copy
Cosmos DB | Intra-account container copy
2 parents ada898c + b354b07 commit 76a8a76

File tree

4 files changed

+204
-0
lines changed

4 files changed

+204
-0
lines changed

‎articles/cosmos-db/TOC.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,8 @@
160160
href: data-residency.md
161161
- name: Automatic recommendations
162162
href: automated-recommendations.md
163+
- name: Move data within Azure Cosmos DB
164+
href: intra-account-container-copy.md
163165
- name: SQL/Core API
164166
expanded: true
165167
items:
@@ -1597,6 +1599,9 @@
15971599
- name: Limit total account throughput
15981600
displayName: cost, RUs, RU
15991601
href: limit-total-account-throughput.md
1602+
- name: Move data within Cosmos DB
1603+
displayName: Move data within Cosmos DB
1604+
href: how-to-container-copy.md
16001605
- name: Access preview features
16011606
href: access-previews.md
16021607
- name: Manage Azure Cosmos DB resources

‎articles/cosmos-db/how-to-container-copy.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
title: Create and manage intra-account container copy jobs in Azure Cosmos DB
3+
description: Learn how to create, monitor, and manage container copy jobs within an Azure Cosmos DB account using CLI commands.
4+
author: nayakshweta
5+
ms.service: cosmos-db
6+
ms.topic: how-to
7+
ms.date: 04/18/2022
8+
ms.author: shwetn
9+
---
10+
11+
# Create and manage intra-account container copy jobs in Azure Cosmos DB (Preview)
12+
[!INCLUDE[appliesto-sql-cassandra-api](includes/appliesto-sql-cassandra-api.md)]
13+
14+
[Container copy jobs](intra-account-container-copy.md) creates offline copies of collections within an Azure Cosmos DB account.
15+
16+
This article describes how to create, monitor, and manage intra-account container copy jobs using Azure CLI commands.
17+
18+
## Set shell variables
19+
20+
First, set all of the variables that each individual script will use.
21+
22+
```azurecli-interactive
23+
$accountName = "<cosmos-account-name>"
24+
$resourceGroup = "<resource-group-name>"
25+
$jobName = ""
26+
$sourceDatabase = ""
27+
$sourceContainer = ""
28+
$destinationDatabase = ""
29+
$destinationContainer = ""
30+
```
31+
32+
## Create an intra-account container copy job for SQL API account
33+
34+
Create a job to copy a container within an Azure Cosmos DB SQL API account:
35+
36+
```azurecli-interactive
37+
az cosmosdb dts copy \
38+
--resource-group $resourceGroup \
39+
--job-name $jobName \
40+
--account-name $accountName \
41+
--source-sql-container database=$sourceDatabase container=$sourceContainer \
42+
--dest-sql-container database=$destinationDatabase container=$destinationContainer
43+
```
44+
45+
## Create intra-account container copy job for Cassandra API account
46+
47+
Create a job to copy a container within an Azure Cosmos DB Cassandra API account:
48+
49+
```azurecli-interactive
50+
az cosmosdb dts copy \
51+
--resource-group $resourceGroup \
52+
--job-name $jobName \
53+
--account-name $accountName \
54+
--source-cassandra-table keyspace=$sourceKeySpace table=$sourceTable \
55+
--dest-cassandra-table keyspace=$destinationKeySpace table=$destinationTable
56+
```
57+
58+
## Monitor the progress of a container copy job
59+
60+
View the progress and status of a copy job:
61+
62+
```azurecli-interactive
63+
az cosmosdb dts show \
64+
--account-name $accountName \
65+
--resource-group $resourceGroup \
66+
--job-name $jobName
67+
```
68+
69+
## List all the container copy jobs created in an account
70+
71+
To list all the container copy jobs created in an account:
72+
73+
```azurecli-interactive
74+
az cosmosdb dts list \
75+
--account-name $accountName \
76+
--resource-group $resourceGroup
77+
```
78+
79+
## Pause a container copy job
80+
81+
In order to pause an ongoing container copy job, you may use the command:
82+
83+
```azurecli-interactive
84+
az cosmosdb dts pause \
85+
--account-name $accountName \
86+
--resource-group $resourceGroup \
87+
--job-name $jobName
88+
```
89+
90+
## Resume a container copy job
91+
92+
In order to resume an ongoing container copy job, you may use the command:
93+
94+
```azurecli-interactive
95+
az cosmosdb dts resume \
96+
--account-name $accountName \
97+
--resource-group $resourceGroup \
98+
--job-name $jobName
99+
```
100+
101+
## Next steps
102+
103+
- For more information about intra-account container copy jobs, see [Container copy jobs](intra-account-container-copy.md).

‎articles/cosmos-db/includes/appliesto-sql-cassandra-api.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
APPLIES TO: :::image type="icon" source="../media/applies-to/yes.png" border="false":::SQL API :::image type="icon" source="../media/applies-to/yes.png" border="false":::Cassandra API

‎articles/cosmos-db/intra-account-container-copy.md

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
---
2+
title: Intra-account container copy jobs in Azure Cosmos DB
3+
description: Learn about container data copy capability within an Azure Cosmos DB account.
4+
author: nayakshweta
5+
ms.service: cosmos-db
6+
ms.topic: conceptual
7+
ms.date: 04/18/2022
8+
ms.author: shwetn
9+
---
10+
11+
# Intra-account container copy jobs in Azure Cosmos DB (Preview)
12+
[!INCLUDE[appliesto-sql-cassandra-api](includes/appliesto-sql-cassandra-api.md)]
13+
14+
You can perform offline container copy within an Azure Cosmos DB account using container copy jobs.
15+
16+
You may need to copy data within your Azure Cosmos DB account if you want to achieve any of these scenarios:
17+
18+
* Copy all items from one container to another.
19+
* Change the [granularity at which throughput is provisioned - from database to container](set-throughput.md) and vice-versa.
20+
* Change the [partition key](partitioning-overview.md#choose-partitionkey) of a container.
21+
* Update the [unique keys](unique-keys.md) for a container.
22+
* Rename a container/database.
23+
* Adopt new features that are only supported on new containers.
24+
25+
Intra-account container copy jobs can be currently [created and managed using CLI commands](how-to-container-copy.md).
26+
27+
## Get started
28+
29+
To get started using container copy jobs, enroll in the preview by filing a support ticket in the [Azure portal](https://portal.azure.com).
30+
31+
## How does intra-account container copy work?
32+
33+
Intra-account container copy jobs perform offline data copy using the source container's incremental change feed log.
34+
35+
* Within the platform, we allocate two 4-vCPU 16-GB memory server-side compute instances per Azure Cosmos DB account by default.
36+
* The instances are allocated when one or more container copy jobs are created within the account.
37+
* The container copy jobs run on these instances.
38+
* The instances are shared by all the container copy jobs running within the same account.
39+
* The platform may de-allocate the instances if they're idle for >15 mins.
40+
41+
> [!NOTE]
42+
> We currently only support offline container copy. So, we strongly recommend to stop performing any operations on the source container prior to beginning the container copy.
43+
> Item deletions and updates done on the source container after beginning the copy job may not be captured. Hence, continuing to perform operations on the source container while the container job is in progress may result in data missing on the target container.
44+
45+
## Overview of steps needed to do a container copy
46+
47+
1. Stop the operations on the source container by pausing the application instances or any clients connecting to it.
48+
2. [Create the container copy job](how-to-container-copy.md).
49+
3. [Monitor the progress of the container copy job](how-to-container-copy.md#monitor-the-progress-of-a-container-copy-job) and wait until it's completed.
50+
4. Resume the operations by appropriately pointing the application or client to the source or target container copy as intended.
51+
52+
## Factors affecting the rate of container copy job
53+
54+
The rate of container copy job progress is determined by these factors:
55+
56+
* Source container/database throughput setting.
57+
58+
* Target container/database throughput setting.
59+
60+
* Server-side compute instances allocated to the Azure Cosmos DB account for the performing the data transfer.
61+
62+
> [!IMPORTANT]
63+
> The default SKU offers two 4-vCPU 16-GB server-side instances per account. You may opt to sign up for [larger SKUs](#large-skus-preview) in preview.
64+
65+
## FAQs
66+
67+
### Is there an SLA for the container copy jobs?
68+
69+
Container copy jobs are currently supported on best-effort basis. We don't provide any SLA guarantees for the time taken to complete these jobs.
70+
71+
### Can I create multiple container copy jobs within an account?
72+
73+
Yes, you can create multiple jobs within the same account. The jobs will run consecutively. You can [list all the jobs](how-to-container-copy.md#list-all-the-container-copy-jobs-created-in-an-account) created within an account and monitor their progress.
74+
75+
### Can I copy an entire database within the Azure Cosmos DB account?
76+
77+
You'll have to create a job for each collection in the database.
78+
79+
### I have an Azure Cosmos DB account with multiple regions. In which region will the container copy job run?
80+
81+
The container copy job will run in the write region. If there are accounts configured with multi-region writes, the job will run in one of the regions from the list.
82+
83+
### What happens to the container copy jobs when the account's write region changes?
84+
85+
The account's write region may change in the rare scenario of a region outage or due to manual failover. In such scenario, incomplete container copy jobs created within the account would fail. You would need to recreate such jobs. Recreated jobs would then run against the new (current) write region.
86+
87+
## Large SKUs preview
88+
89+
If you want to run the container copy jobs faster, you may do so by adjusting one of the [factors that affect the rate of the copy job](#factors-affecting-the-rate-of-container-copy-job). In order to adjust the configuration of the server-side compute instances, you may sign up for "Large SKU support for container copy" preview.
90+
91+
This preview will allow you to choose larger a SKU size for the server-side instances. Large SKU sizes are billable at a higher rate. You can also choose a node count of up to 5 of these instances.
92+
93+
## Next Steps
94+
95+
- You can learn about [how to create, monitor and manage container copy jobs within Azure Cosmos DB account using CLI commands](how-to-container-copy.md).

0 commit comments

Comments
 (0)