Skip to content

Commit 91784ac

Browse files
denreadenrea
authored
Merge pull request #236181 from pauljewellmsft/pauljewell-copy-python
Update copy articles for Python dev guide
2 parents 4cd7356 + e3ef6eb commit 91784ac

File tree

4 files changed

+227
-71
lines changed

4 files changed

+227
-71
lines changed

‎articles/storage/blobs/TOC.yml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -872,7 +872,13 @@ items:
872872
- name: Download blobs
873873
href: storage-blob-download-python.md
874874
- name: Copy blobs
875-
href: storage-blob-copy-python.md
875+
items:
876+
- name: Overview of copy operations
877+
href: storage-blob-copy-python.md
878+
- name: Copy from a source object URL
879+
href: storage-blob-copy-url-python.md
880+
- name: Copy with asynchronous scheduling
881+
href: storage-blob-copy-async-python.md
876882
- name: List blobs
877883
href: storage-blobs-list-python.md
878884
- name: Delete and restore blobs

‎articles/storage/blobs/storage-blob-copy-async-python.md

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
---
2+
title: Copy a blob with asynchronous scheduling using Python
3+
titleSuffix: Azure Storage
4+
description: Learn how to copy a blob with asynchronous scheduling in Azure Storage by using the Python client library.
5+
author: pauljewellmsft
6+
7+
ms.author: pauljewell
8+
ms.date: 04/28/2023
9+
ms.service: storage
10+
ms.subservice: blobs
11+
ms.topic: how-to
12+
ms.devlang: python
13+
ms.custom: devx-track-python, devguide-python
14+
---
15+
16+
# Copy a blob with asynchronous scheduling using Python
17+
18+
This article shows how to copy a blob with asynchronous scheduling using the [Azure Storage client library for Python](/python/api/overview/azure/storage). You can copy a blob from a source within the same storage account, from a source in a different storage account, or from any accessible object retrieved via HTTP GET request on a given URL. You can also abort a pending copy operation.
19+
20+
The client library methods covered in this article use the [Copy Blob](/rest/api/storageservices/copy-blob) REST API operation, and can be used when you want to perform a copy with asynchronous scheduling. For most copy scenarios where you want to move data into a storage account and have a URL for the source object, see [Copy a blob from a source object URL with Python](storage-blob-copy-url-python.md).
21+
22+
## Prerequisites
23+
24+
To work with the code examples in this article, make sure you have:
25+
26+
- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).
27+
- Permissions to perform a copy operation. To learn more, see the authorization guidance for the following REST API operations:
28+
- [Copy Blob](/rest/api/storageservices/copy-blob#authorization)
29+
- [Abort Copy Blob](/rest/api/storageservices/abort-copy-blob#authorization)
30+
- Packages installed to your project directory. These examples use **azure-storage-blob**. If you're using `DefaultAzureCredential` for authorization, you also need **azure-identity**. To learn more about setting up your project, see [Get Started with Azure Storage and Python](storage-blob-python-get-started.md#set-up-your-project). To see the necessary `using` directives, see [Code samples](#code-samples).
31+
32+
## About copying blobs with asynchronous scheduling
33+
34+
The `Copy Blob` operation can finish asynchronously and is performed on a best-effort basis, which means that the operation isn't guaranteed to start immediately or complete within a specified time frame. The copy operation is scheduled in the background and performed as the server has available resources. The operation can complete synchronously if the copy occurs within the same storage account.
35+
36+
A `Copy Blob` operation can perform any of the following actions:
37+
38+
- Copy a source blob to a destination blob with a different name. The destination blob can be an existing blob of the same blob type (block, append, or page), or it can be a new blob created by the copy operation.
39+
- Copy a source blob to a destination blob with the same name, which replaces the destination blob. This type of copy operation removes any uncommitted blocks and overwrites the destination blob's metadata.
40+
- Copy a source file in the Azure File service to a destination blob. The destination blob can be an existing block blob, or can be a new block blob created by the copy operation. Copying from files to page blobs or append blobs isn't supported.
41+
- Copy a snapshot over its base blob. By promoting a snapshot to the position of the base blob, you can restore an earlier version of a blob.
42+
- Copy a snapshot to a destination blob with a different name. The resulting destination blob is a writeable blob and not a snapshot.
43+
44+
The source blob for a copy operation may be one of the following types: block blob, append blob, page blob, blob snapshot, or blob version. The copy operation always copies the entire source blob or file. Copying a range of bytes or set of blocks isn't supported.
45+
46+
If the destination blob already exists, it must be of the same blob type as the source blob, and the existing destination blob is overwritten. The destination blob can't be modified while a copy operation is in progress, and a destination blob can only have one outstanding copy operation.
47+
48+
To learn more about the `Copy Blob` operation, including information about properties, index tags, metadata, and billing, see [Copy Blob remarks](/rest/api/storageservices/copy-blob#remarks).
49+
50+
## Copy a blob with asynchronous scheduling
51+
52+
This section gives an overview of methods provided by the Azure Storage client library for Python to perform a copy operation with asynchronous scheduling.
53+
54+
The following methods wrap the [Copy Blob](/rest/api/storageservices/copy-blob) REST API operation, and begin an asynchronous copy of data from the source blob:
55+
56+
- [start_copy_from_url](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-start-copy-from-url)
57+
58+
The `start_copy_from_url` returns a dictionary containing *copy_status* and *copy_id*. The *copy_status* property is **success** if the copy completed synchronously or **pending** if the copy has been started asynchronously.
59+
60+
## Copy a blob from a source within Azure
61+
62+
If you're copying a blob within the same storage account, the operation can complete synchronously. Access to the source blob can be authorized via Azure Active Directory (Azure AD), a shared access signature (SAS), or an account key. For an alterative synchronous copy operation, see [Copy a blob from a source object URL with Python](storage-blob-copy-url-python.md).
63+
64+
If the copy source is a blob in a different storage account, the operation can complete asynchronously. The source blob must either be public or authorized via SAS token. The SAS token needs to include the **Read ('r')** permission. To learn more about SAS tokens, see [Delegate access with shared access signatures](../common/storage-sas-overview.md).
65+
66+
The following example shows a scenario for copying a source blob from a different storage account with asynchronous scheduling. In this example, we create a source blob URL with an appended user delegation SAS token. The example shows how to generate the SAS token using the client library, but you can also provide your own. The example also shows how to lease the source blob during the copy operation to prevent changes to the blob from a different client. The `Copy Blob` operation saves the `ETag` value of the source blob when the copy operation starts. If the `ETag` value is changed before the copy operation finishes, the operation fails.
67+
68+
:::code language="python" source="~/azure-storage-snippets/blobs/howto/python/blob-devguide-py/blob-devguide-copy-blob.py" id="Snippet_copy_blob_from_azure_async":::
69+
70+
> [!NOTE]
71+
> User delegation SAS tokens offer greater security, as they're signed with Azure AD credentials instead of an account key. To create a user delegation SAS token, the Azure AD security principal needs appropriate permissions. For authorization requirements, see [Get User Delegation Key](/rest/api/storageservices/get-user-delegation-key#authorization).
72+
73+
## Copy a blob from a source outside of Azure
74+
75+
You can perform a copy operation on any source object that can be retrieved via HTTP GET request on a given URL, including accessible objects outside of Azure. The following example shows a scenario for copying a blob from an accessible source object URL.
76+
77+
:::code language="python" source="~/azure-storage-snippets/blobs/howto/python/blob-devguide-py/blob-devguide-copy-blob.py" id="Snippet_copy_blob_external_source_async":::
78+
79+
## Check the status of a copy operation
80+
81+
To check the status of an asynchronous `Copy Blob` operation, you can poll the [get_blob_properties](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-get-blob-properties) method and check the copy status.
82+
83+
The following code example shows how to check the status of a pending copy operation:
84+
85+
:::code language="python" source="~/azure-storage-snippets/blobs/howto/python/blob-devguide-py/blob-devguide-copy-blob.py" id="Snippet_check_copy_status":::
86+
87+
## Abort a copy operation
88+
89+
Aborting a pending `Copy Blob` operation results in a destination blob of zero length. However, the metadata for the destination blob has the new values copied from the source blob or set explicitly during the copy operation. To keep the original metadata from before the copy, make a snapshot of the destination blob before calling one of the copy methods.
90+
91+
To abort a pending copy operation, call the following operation:
92+
93+
- [abort_copy](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-abort-copy)
94+
95+
This method wraps the [Abort Copy Blob](/rest/api/storageservices/abort-copy-blob) REST API operation, which cancels a pending `Copy Blob` operation. The following code example shows how to abort a pending `Copy Blob` operation:
96+
97+
:::code language="python" source="~/azure-storage-snippets/blobs/howto/python/blob-devguide-py/blob-devguide-copy-blob.py" id="Snippet_abort_copy":::
98+
99+
## Resources
100+
101+
To learn more about copying blobs with asynchronous scheduling using the Azure Blob Storage client library for Python, see the following resources.
102+
103+
### REST API operations
104+
105+
The Azure SDK for Python contains libraries that build on top of the Azure REST API, allowing you to interact with REST API operations through familiar Python paradigms. The client library methods covered in this article use the following REST API operations:
106+
107+
- [Copy Blob](/rest/api/storageservices/copy-blob) (REST API)
108+
- [Abort Copy Blob](/rest/api/storageservices/abort-copy-blob) (REST API)
109+
110+
### Code samples
111+
112+
- [View code samples from this article (GitHub)](https://github.com/Azure-Samples/AzureStorageSnippets/blob/master/blobs/howto/python/blob-devguide-py/blob-devguide-copy-blob.py)
113+
114+
[!INCLUDE [storage-dev-guide-resources-python](../../../includes/storage-dev-guides/storage-dev-guide-resources-python.md)]

‎articles/storage/blobs/storage-blob-copy-python.md

Lines changed: 17 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ services: storage
66
author: pauljewellmsft
77

88
ms.author: pauljewell
9-
ms.date: 01/25/2023
9+
ms.date: 04/28/2023
1010
ms.service: storage
1111
ms.subservice: blobs
1212
ms.topic: how-to
@@ -16,81 +16,28 @@ ms.custom: devx-track-python, devguide-python
1616

1717
# Copy a blob with Python
1818

19-
This article shows how to copy a blob in a storage account using the [Azure Storage client library for Python](/python/api/overview/azure/storage). It also shows how to abort a pending copy operation.
19+
This article provides an overview of copy operations using the [Azure Storage client library for Python](/python/api/overview/azure/storage).
2020

21-
## About copying blobs
21+
## About copy operations
2222

23-
A copy operation can perform any of the following actions:
23+
Copy operations can be used to move data within a storage account, between storage accounts, or into a storage account from a source outside of Azure. When using the Blob Storage client libraries to copy data resources, it's important to understand the REST API operations behind the client library methods. The following table lists REST API operations that can be used to copy data resources to a storage account. The table also includes links to detailed guidance about how to perform these operations using the [Azure Storage client library for Python](/python/api/overview/azure/storage).
2424

25-
- Copy a source blob to a destination blob with a different name. The destination blob can be an existing blob of the same blob type (block, append, or page), or can be a new blob created by the copy operation.
26-
- Copy a source blob to a destination blob with the same name, effectively replacing the destination blob. Such a copy operation removes any uncommitted blocks and overwrites the destination blob's metadata.
27-
- Copy a source file in the Azure File service to a destination blob. The destination blob can be an existing block blob, or can be a new block blob created by the copy operation. Copying from files to page blobs or append blobs isn't supported.
28-
- Copy a snapshot over its base blob. By promoting a snapshot to the position of the base blob, you can restore an earlier version of a blob.
29-
- Copy a snapshot to a destination blob with a different name. The resulting destination blob is a writeable blob and not a snapshot.
25+
| REST API operation | When to use | Client library methods | Guidance |
26+
| --- | --- | --- | --- |
27+
| [Put Blob From URL](/rest/api/storageservices/put-blob-from-url) | This operation is preferred for scenarios where you want to move data into a storage account and have a URL for the source object. This operation completes synchronously. | [upload_blob_from_url](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-upload-blob-from-url) | [Copy a blob from a source object URL with Python](storage-blob-copy-url-python.md) |
28+
| [Put Block From URL](/rest/api/storageservices/put-block-from-url) | For large objects, you can use [Put Block From URL](/rest/api/storageservices/put-block-from-url) to write individual blocks to Blob Storage, and then call [Put Block List](/rest/api/storageservices/put-block-list) to commit those blocks to a block blob. This operation completes synchronously. | [stage_block_from_url](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-stage-block-from-url) | [Copy a blob from a source object URL with Python](storage-blob-copy-url-python.md) |
29+
| [Copy Blob](/rest/api/storageservices/copy-blob) | This operation can be used when you want asynchronous scheduling for a copy operation. | [start_copy_from_url](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-start-copy-from-url) | [Copy a blob with asynchronous scheduling using Python](storage-blob-copy-async-python.md) |
3030

31-
The source blob for a copy operation may be one of the following types:
32-
- Block blob
33-
- Append blob
34-
- Page blob
35-
- Blob snapshot
36-
- Blob version
31+
For append blobs, you can use the [Append Block From URL](/rest/api/storageservices/append-block-from-url) operation to commit a new block of data to the end of an existing append blob. The following client library method wraps this operation:
3732

38-
If the destination blob already exists, it must be of the same blob type as the source blob. An existing destination blob will be overwritten.
33+
- [append_block_from_url](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-append-block-from-url)
3934

40-
The destination blob can't be modified while a copy operation is in progress. A destination blob can only have one outstanding copy operation. One way to enforce this requirement is to use a blob lease, as shown in the code example. For more information on blob leases, see [Create and manage blob leases with Python](storage-blob-lease-python.md).
35+
For page blobs, you can use the [Put Page From URL](/rest/api/storageservices/put-page-from-url) operation to write a range of pages to a page blob where the contents are read from a URL. The following client library method wraps this operation:
4136

42-
The entire source blob or file is always copied. Copying a range of bytes or set of blocks isn't supported. When a blob is copied, its system properties are copied to the destination blob with the same values.
37+
- [upload_pages_from_url](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-upload-pages-from-url)
4338

44-
## Copy a blob
39+
## Client library resources
4540

46-
To copy a blob, use the following method:
47-
48-
- [BlobClient.start_copy_from_url](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-start-copy-from-url)
49-
50-
This method returns a dictionary containing *copy_status* and *copy_id*, which can be used to check the status of the copy operation. The *copy_status* property will be 'success' if the copy completed synchronously or 'pending' if the copy has been started asynchronously. For asynchronous copies, the status can be checked by polling the [get_blob_properties](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-get-blob-properties) method and checking *copy_status*. To force the copy operation to be synchronous, set *requires_sync* to `True`. To learn more about the underlying operation, see [REST API operations](#rest-api-operations).
51-
52-
The following code example gets a `BlobClient` object representing an existing blob and copies it to a new blob in a different container within the same storage account. This example also gets a lease on the source blob before copying so that no other client can modify the blob until the copy is complete and the lease is broken.
53-
54-
:::code language="python" source="~/azure-storage-snippets/blobs/howto/python/blob-devguide-py/blob-devguide-blobs.py" id="Snippet_copy_blob":::
55-
56-
Sample output is similar to:
57-
58-
```console
59-
Source blob lease state: leased
60-
Copy status: success
61-
Copy progress: 5/5
62-
Copy completion time: 2022-11-14T16:53:54Z
63-
Total bytes copied: 5
64-
Source blob lease state: broken
65-
```
66-
67-
## Abort a copy operation
68-
69-
If you have a pending copy operation and need to cancel it, you can abort the operation. Aborting a copy operation results in a destination blob of zero length and full metadata. To learn more about the underlying operation, see [REST API operations](#rest-api-operations).
70-
71-
The metadata for the destination blob will have the new values copied from the source blob or set explicitly during the copy operation. To keep the original metadata from before the copy, make a snapshot of the destination blob before calling one of the copy methods. The final blob will be committed when the copy completes.
72-
73-
To abort a copy operation, use the following method:
74-
75-
- [BlobClient.abort_copy](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-abort-copy)
76-
77-
The following example stops a pending copy and leaves a destination blob with zero length and full metadata:
78-
79-
:::code language="python" source="~/azure-storage-snippets/blobs/howto/python/blob-devguide-py/blob-devguide-blobs.py" id="Snippet_abort_copy":::
80-
81-
## Resources
82-
83-
To learn more about copying blobs using the Azure Blob Storage client library for Python, see the following resources.
84-
85-
### REST API operations
86-
87-
The Azure SDK for Python contains libraries that build on top of the Azure REST API, allowing you to interact with REST API operations through familiar Python paradigms. The client library methods covered in this article use the following REST API operations:
88-
89-
- [Copy Blob](/rest/api/storageservices/copy-blob) (REST API)
90-
- [Abort Copy Blob](/rest/api/storageservices/abort-copy-blob) (REST API)
91-
92-
### Code samples
93-
94-
- [View code samples from this article (GitHub)](https://github.com/Azure-Samples/AzureStorageSnippets/blob/master/blobs/howto/python/blob-devguide-py/blob-devguide-blobs.py)
95-
96-
[!INCLUDE [storage-dev-guide-resources-python](../../../includes/storage-dev-guides/storage-dev-guide-resources-python.md)]
41+
- [Client library reference documentation](/python/api/azure-storage-blob)
42+
- [Client library source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/storage/azure-storage-blob)
43+
- [Package (PyPi)](https://pypi.org/project/azure-storage-blob/)

0 commit comments

Comments
 (0)