Skip to content

Commit 57c2a4a

Browse files
Merge pull request #34160 from MicrosoftDocs/NEW-copy-blobs-using-dotnet
New copy blobs using dotnet
2 parents cb36f67 + 4dd9638 commit 57c2a4a

File tree

9 files changed

+619
-0
lines changed

9 files changed

+619
-0
lines changed
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
### YamlMime:ModuleUnit
2+
uid: learn.azure.copy-blobs-using-dotnet.introduction
3+
title: Introduction
4+
metadata:
5+
title: Introduction
6+
description: Introduction
7+
ms.date: 04/20/2023
8+
author: normesta
9+
ms.author: normesta
10+
ms.topic: interactive-tutorial
11+
ms.prod: learning-azure
12+
durationInMinutes: 2
13+
content: |
14+
[!include[](includes/1-introduction.md)]
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
### YamlMime:ModuleUnit
2+
uid: learn.azure.copy-blobs-using-dotnet.move-blobs-using-dotnet
3+
title: Move Azure Storage blobs using the .NET Storage Client
4+
metadata:
5+
title: Move Azure Storage blob using the .NET Storage Client
6+
description: Move Azure Storage blobs using the .NET Storage Client
7+
ms.date: 04/20/2023
8+
author: normesta
9+
ms.author: normesta
10+
ms.topic: interactive-tutorial
11+
ms.prod: learning-azure
12+
durationInMinutes: 7
13+
content: |
14+
[!include[](includes/2-move-blobs-using-dotnet.md)]
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
### YamlMime:ModuleUnit
2+
uid: learn.azure.copy-blobs-using-dotnet.exercise-move-blobs-using-dotnet
3+
title: Exercise - Move Azure Storage blobs using the .NET Storage Client
4+
metadata:
5+
title: Exercise - Move Azure Storage blob using the .NET Storage Client
6+
description: Exercise - Move Azure Storage blobs using the .NET Storage Client
7+
ms.date: 04/20/2023
8+
author: normesta
9+
ms.author: normesta
10+
ms.topic: interactive-tutorial
11+
ms.prod: learning-azure
12+
durationInMinutes: 6
13+
interactive: bash
14+
azureSandbox: true
15+
content: |
16+
[!include[](includes/3-exercise-move-blobs-using-dotnet.md)]
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
### YamlMime:ModuleUnit
2+
uid: learn.azure.copy-blobs-using-dotnet.summary
3+
title: Summary
4+
metadata:
5+
title: Summary
6+
description: Summary
7+
ms.date: 04/20/2023
8+
author: normesta
9+
ms.author: normesta
10+
ms.topic: interactive-tutorial
11+
ms.prod: learning-azure
12+
durationInMinutes: 2
13+
content: |
14+
[!include[](includes/4-summary.md)]
15+
quiz:
16+
title: Check your knowledge
17+
questions:
18+
- content: "The .NET Storage Client library ships only as a single package which you can use to interact with all storage services?"
19+
choices:
20+
- content: "True"
21+
isCorrect: false
22+
explanation: "No. The .NET Storage Client library is available as individual packages which target specific storage services such as Blob Storage or Queue Storage."
23+
- content: "False"
24+
isCorrect: true
25+
explanation: "Correct. The .NET Storage Client library is available as individual packages which target specific storage services such as Blob Storage or Queue Storage."
26+
- content: "Which object would you create to authorize access, and then perform the greatest number of Blob Storage operations without having to authorize again?"
27+
choices:
28+
- content: "BlobClient"
29+
isCorrect: false
30+
explanation: "No. While the BlobClient class can accept a connection string, this class is scoped to a specific set of Blob Storage operations."
31+
- content: "BlobServiceClient"
32+
isCorrect: true
33+
explanation: "Correct. Use the BlobServiceClient class to authorize the use of all public classes and methods that operate on blobs in the account."
34+
- content: "BlobContainerClient"
35+
isCorrect: false
36+
explanation: "No. While the BlobContainerClient class can accept a connection string, this class is scoped to a specific set of Blob Storage operations."
37+
- content: "How can you modify your call to the GetBlobAsync method so that you retrieve only blobs which start with a specific string?"
38+
choices:
39+
- content: "Add query syntax as a parameter to the GetBlobAsync method."
40+
isCorrect: false
41+
explanation: "No. There is no such parameter. Instead, use the prefix parameter to return only blobs whose name begins with a specific string."
42+
- content: "Use parameters which filter the types of blobs returned."
43+
isCorrect: false
44+
explanation: "No. While there are various parameters which can reduce the number of blobs returned, you can limit based on blob name only by using the prefix parameter."
45+
- content: "Specify a string prefix."
46+
isCorrect: true
47+
explanation: "Correct. Use the prefix parameter to return only blobs whose name begins with a specific string."
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
Azure Blob Storage provides a convenient space for storing large quantities of data in a safe, reliable manner. You can use blobs to hold all types of data, ranging from individual files and graphics to an entire database. Blob Storage is useful as a store for active data, but it's also ideally suited for acting as low-cost archive storage.
2+
3+
Imagine you're the DevOps administrator for a prototyping and engineering company. The company maintains an Azure Storage account that serves as a repository for client product designs, specifications, and supporting documentation. Your company has decided to move any inactive documents older than six months to a separate storage location. You'll use a new Azure Storage account for archiving.
4+
5+
The .NET Storage Client library is a collection of objects and methods you can use to build custom applications that manipulate items held in Azure Storage. You can construct your own applications to upload, download, and migrate blobs between storage accounts. You can incorporate your code into existing applications, and you can deploy your code to a variety of environments, such as Azure Functions.
6+
7+
The .NET Storage Client library provides full access to the metadata for a blob, and you can access any properties of a blob. This feature allows you, for example, to select a blob based on its last modified time, creation time, or any other available attribute.
8+
9+
The .NET Storage Client library implements asynchronous operations, letting you take full programmatic advantage of the multitasking capabilities of the .NET Framework.
10+
11+
The .NET Storage Client library requires a development investment, and may not be suitable for quick, one-off jobs. However, if you have a complex task that is frequently repeated, the investment could be worthwhile.
12+
13+
In this module, you'll learn how to copy and move blobs between Azure Storage accounts using the .NET Client Library.
14+
15+
## Learning objectives
16+
17+
In this module, you will:
18+
19+
- Identify when to use synchronous and asynchronous methods for transferring blobs
20+
- Copy and move blobs between Azure storage accounts using the .NET Storage Client library
21+
22+
## Prerequisites
23+
24+
- Beginner-level knowledge of Azure Storage and blobs
Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
The .NET Storage Client library provides programmatic access to Azure Storage. Using this library, you can build custom apps that manipulate items in Azure Storage. This library provides the most flexible approach to uploading, downloading, and transferring blobs between storage accounts.
2+
3+
## Overview of the .NET Storage Client library
4+
5+
The .NET Storage Client library provides a series of objects and methods that you can use to build apps that access Azure storage. The library is available for several languages. This unit shows examples using C#.
6+
7+
The library is available as individual packages for the different storage products, for example Table, Blob, and so on. As this library supports cross-platform operation, you can run this library on your preferred operating system.
8+
9+
## Connect to Azure Blob Storage
10+
11+
The first task is to create a *service client* object for Azure Blob Storage by instantiating the `BlobServiceClient` class, which takes a connection string as a parameter. You can find the connection string for the Storage Account from the Azure portal.
12+
13+
The following code example shows how to perform this task. The relevant types are defined in several namespaces including `Azure`, `Azure.Storage.Blobs`, `Azure.Storage.Blobs.Models`, and `Azure.Storage.Sas`.
14+
15+
```C#
16+
...
17+
18+
using Azure;
19+
using Azure.Storage.Blobs;
20+
using Azure.Storage.Blobs.Models;
21+
using Azure.Storage.Sas;
22+
23+
...
24+
25+
// The variable sourceConnection is a string holding the connection string for the storage account
26+
BlobServiceClient sourceClient = new BlobServiceClient(sourceConnection);
27+
28+
...
29+
```
30+
31+
## Download a blob
32+
33+
To download a blob, you'll obtain a *blob container client* that has a reference to the container holding the blob using the `BlobContainerClient` object. Through this container, you'll obtain a reference to the blob itself, called `BlobClient`. You can then invoke the `DownloadToAsync` method to fetch the contents of the blob to a local file.
34+
35+
The blob reference also provides access to the properties of the blob, such as its last modified date and creation date. The following code shows an example that downloads a blob named *MyBlob.doc* to a local file named *MyFile.doc*. The code also displays the last modified date for the blob.
36+
37+
```C#
38+
...
39+
40+
// The variable sourceContainer is a string holding the container name
41+
BlobContainerClient sourceBlobContainer = sourceClient.GetBlobContainerClient(sourceContainer);
42+
sourceBlobContainer.CreateIfNotExists();
43+
44+
BlobClient sourceBlob = blobContainer.GetBlobClient("MyBlob.doc");
45+
await sourceBlob.DownloadToAsync("MyFile.doc");
46+
47+
BlobProperties properties = (await sourceBlob.GetPropertiesAsync()).Value;
48+
Console.WriteLine($"Last modified: {properties.LastModified}");
49+
50+
...
51+
```
52+
53+
You can stream data from a large blob with the `DownloadStreamingAsync` method of your source blob object.
54+
55+
## Upload a blob
56+
57+
To create a new blob using a local file, the process is similar to downloading. Obtain a reference to the container that will hold the blob and create a reference for your new blob. Specify the name for your new blob. You can then use the `UploadAsync` method to read data from a local file and copy it to the new blob. The following code shows an example of this approach.
58+
59+
```C#
60+
...
61+
62+
// The variable destClient is a blob service client for the destination storage account
63+
// The variable destContainer is a string holding the container name
64+
BlobContainerClient destBlobContainer = destClient.GetBlobContainerClient(destContainer);
65+
destBlobContainer.CreateIfNotExists();
66+
67+
BlobClient destBlob = destBlobContainer.GetBlobClient("NewBlob.doc");
68+
await destBlob.UploadAsync("MyFile.doc");
69+
70+
...
71+
```
72+
73+
You can also use the same method, `UploadAsync`, for large blob data streaming. In this case, use the stream object instead.
74+
75+
## Copy blobs between storage accounts
76+
77+
You can transfer blobs between storage accounts using a combination of the download and upload techniques previously illustrated. However, a more efficient approach is to make use of the blob copying facilities provided by the Azure Storage service. Copying a blob in this way transfers it directly from one container to another without requiring that you download it to an intermediate storage location.
78+
79+
The .NET Client library provides the `StartCopyFromUriAsync` method of a blob object to initiate copying the data in this blob to another blob in a destination container. You'll specify the destination blob using its URI. Additionally, because the data is transferred directly, you need to ensure that the Azure Storage service is provided with the appropriate access rights to the source blob. One way to supply this access is with a Shared Access Signature (SAS) token, as described previously in this module. You'll append the SAS token to the URI of the source blob.
80+
81+
The following example code shows how to use the `StartCopyFromUriAsync` method. The *sourceBlob* variable is a reference to a blob being copied. The code creates a reference to a new blob (*destBlob*) using the same name as the source blob. The `StartCopyFromUriAsync` method takes the URI of the blob to be copied and starts to transfer the data to the new destination blob. The `GetSharedAccessUri` method creates a URI containing a read-only SAS token for the source blob that is valid for 60 minutes.
82+
83+
```C#
84+
...
85+
86+
// The variable sourceBlob is a blob client representing the source blob
87+
BlobClient destBlob = destContainer.GetBlobClient(sourceBlob.Name);
88+
CopyFromUriOperation ops = await destBlob.StartCopyFromUriAsync(GetSharedAccessUri(sourceBlob.Name, sourceContainer));
89+
90+
...
91+
92+
// Create a SAS token for the source blob, to enable it to be read by the StartCopyFromUriAsync method
93+
private static Uri GetSharedAccessUri(string blobName, BlobContainerClient container)
94+
{
95+
DateTimeOffset expiredOn = DateTimeOffset.UtcNow.AddMinutes(60);
96+
97+
BlobClient blob = container.GetBlobClient(blobName);
98+
Uri sasUri = blob.GenerateSasUri(BlobSasPermissions.Read, expiredOn);
99+
100+
return sasUri;
101+
}
102+
```
103+
104+
The `StartCopyFromUriAsync` method initiates the blob copy operation, and the process runs in the background. As the method returns the `CopyFromUriOperation` object, you can check on the progress of the operation by retrieving a reference to the destination blob and querying its `HasCompleted` property. The property has a value of `false` while the copy is in progress. Additionally, you can find out how many bytes have been copied using the `WaitForCompletionAsync` method of the `CopyFromUriOperation` object.
105+
106+
The following code sample retrieves a reference to the destination blob and monitors the progress as it's copied. The `WaitForCompletionAsync` method updates the *copied* variable with the number of bytes that have been copied.
107+
108+
```C#
109+
...
110+
111+
// Display the status of the blob as it is copied
112+
while(ops.HasCompleted == false)
113+
{
114+
long copied = await ops.WaitForCompletionAsync();
115+
116+
Console.WriteLine($"Blob: {destBlob.Name}, Copied: {copied} of {properties.ContentLength}");
117+
await Task.Delay(500);
118+
}
119+
120+
Console.WriteLine($"Blob: {destBlob.Name} Complete");
121+
122+
...
123+
```
124+
125+
## Delete a blob
126+
127+
To remove a blob from a container, use one of the *Delete* methods of the blob object. There are two methods available, `DeleteAsync` and `DeleteIfExistsAsync`. Both methods delete the specified blob, but `DeleteIfExistsAsync` returns a boolean value indicating whether the blob actually existed before or not.
128+
129+
The following code uses `DeleteIfExistsAsync` to remove a blob, and detects whether the blob existed beforehand.
130+
131+
```C#
132+
bool blobExisted = await sourceBlob.DeleteIfExistsAsync();
133+
```
134+
135+
## Iterate blobs in a container
136+
137+
You can iterate through the blobs in a container with the `GetBlobsAsync` method of a blob container object. This method fetches the details of blobs as a collection.
138+
139+
The following code shows an example of this method:
140+
141+
```C#
142+
...
143+
144+
// Iterate through the blobs in a container
145+
List<BlobItem> segment = await blobContainer.GetBlobsAsync(prefix: "").ToListAsync();
146+
foreach (BlobItem blobItem in segment)
147+
{
148+
BlobClient blob = blobContainer.GetBlobClient(blobItem.Name);
149+
// Check the source file's metadata
150+
Response<BlobProperties> propertiesResponse = await blob.GetPropertiesAsync();
151+
BlobProperties properties = propertiesResponse.Value;
152+
153+
// Check the last modified date and time
154+
// Add the blob to the list if has been modified since the specified date and time
155+
if (DateTimeOffset.Compare(properties.LastModified.ToUniversalTime(), transferBlobsModifiedSince.ToUniversalTime()) > 0)
156+
{
157+
blobList.Add(blob);
158+
}
159+
}
160+
...
161+
```
162+
163+
The *prefix* parameter to `GetBlobsAsync` lets you find all blobs with names that have the specified prefix.

0 commit comments

Comments
 (0)