Skip to content

Commit 8f17a51

Browse files
authored
[Storage] [DataMovement] Update README and minor sample updates Part 1 (Azure#47727)
* Update readme/samples * Removed base samples, rely on blob and share samples * Update blob readme * Fix sample snippet references to use blobs; added extension samples to base blob README * Mispellings * Fix links * Fix links
1 parent 42ad564 commit 8f17a51

File tree

12 files changed

+365
-586
lines changed

12 files changed

+365
-586
lines changed

sdk/storage/Azure.Storage.Blobs/README.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,76 @@ Get started with our [Blob samples][samples]:
188188
1. [Hello World](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/samples/Sample01a_HelloWorld.cs): Upload, download, and list blobs (or [asynchronously](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/samples/Sample01b_HelloWorldAsync.cs))
189189
2. [Auth](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/samples/Sample02_Auth.cs): Authenticate with connection strings, public access, shared keys, shared access signatures, and Azure Active Directory.
190190

191+
## Advanced Scenarios using Azure.DataMovement.Blobs
192+
193+
For more advanced scenarios like transferring blob virtual directories, we recommend looking into our [Azure.Storage.DataMovement](https://www.nuget.org/packages/Azure.Storage.DataMovement) and [Azure.Storage.DataMovement.Blob](https://www.nuget.org/packages/Azure.Storage.DataMovement.Blobs) packages. Get started with our [DataMovement Blob Samples](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.DataMovement.Blobs/samples/Sample01b_HelloWorldAsync.cs).
194+
195+
Upload a local directory to the root of the `BlobContainerClient`.
196+
```C# Snippet:ExtensionMethodSimpleUploadToRoot
197+
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath);
198+
199+
await transfer.WaitForCompletionAsync();
200+
```
201+
202+
Upload a local directory to a virtual blob directory in the `BlobContainerClient` by specifying a directory prefix
203+
```C# Snippet:ExtensionMethodSimpleUploadToDirectoryPrefix
204+
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, blobDirectoryPrefix);
205+
206+
await transfer.WaitForCompletionAsync();
207+
```
208+
209+
Upload a local directory to a virtual blob directory in the `BlobContainerClient` specifying more advanced options
210+
```C# Snippet:ExtensionMethodSimpleUploadWithOptions
211+
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
212+
{
213+
BlobContainerOptions = new BlobStorageResourceContainerOptions
214+
{
215+
BlobDirectoryPrefix = blobDirectoryPrefix
216+
},
217+
TransferOptions = new DataTransferOptions()
218+
{
219+
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
220+
}
221+
};
222+
223+
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, options);
224+
225+
await transfer.WaitForCompletionAsync();
226+
```
227+
228+
Download the entire `BlobContainerClient` to a local directory
229+
```C# Snippet:ExtensionMethodSimpleDownloadContainer
230+
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath);
231+
232+
await transfer.WaitForCompletionAsync();
233+
```
234+
235+
Download a virtual blob directory in the `BlobContainerClient` by specifying a directory prefix
236+
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectory
237+
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, blobDirectoryPrefix);
238+
239+
await transfer.WaitForCompletionAsync();
240+
```
241+
242+
Download from the `BlobContainerClient` specifying more advanced options
243+
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectoryWithOptions
244+
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
245+
{
246+
BlobContainerOptions = new BlobStorageResourceContainerOptions
247+
{
248+
BlobDirectoryPrefix = blobDirectoryPrefix
249+
},
250+
TransferOptions = new DataTransferOptions()
251+
{
252+
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
253+
}
254+
};
255+
256+
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, options);
257+
258+
await transfer.WaitForCompletionAsync();
259+
```
260+
191261
## Contributing
192262

193263
See the [Storage CONTRIBUTING.md][storage_contrib] for details on building,

sdk/storage/Azure.Storage.DataMovement.Blobs/README.md

Lines changed: 88 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,16 @@
11
# Azure Storage Data Movement Blobs client library for .NET
22

3-
> Server Version: 2020-04-08, 2020-02-10, 2019-12-12, 2019-07-07, and 2020-02-02
4-
53
## Project Status: Beta
64

75
This product is in beta. Some features will be missing or have significant bugs. Please see [Known Issues](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.DataMovement/KnownIssues.md) for detailed information.
86

97
---
108

119
Azure Storage is a Microsoft-managed service providing cloud storage that is
12-
highly available, secure, durable, scalable, and redundant. Azure Storage
13-
includes Azure Blobs (objects), Azure Data Lake Storage Gen2, Azure Files,
14-
and Azure Queues.
10+
highly available, secure, durable, scalable, and redundant.
1511

16-
The Azure Storage Data Movement library is optimized for uploading, downloading and
17-
copying customer data.
12+
The Azure Storage Data Movement Blobs library is optimized for uploading, downloading and
13+
copying blobs.
1814

1915
The Azure.Storage.DataMovement.Blobs library provides infrastructure shared by the other
2016
Azure Storage client libraries.
@@ -72,82 +68,10 @@ We guarantee that all client instance methods are thread-safe and independent of
7268

7369
This section demonstrates usage of Data Movement for interacting with blob storage.
7470

75-
### Extensions on `BlobContainerClient`
76-
77-
For applications with preexisting code using Azure.Storage.Blobs, this package provides extension methods for `BlobContainerClient` to get some of the benefits of the `TransferManager` with minimal extra code.
78-
79-
Instantiate the BlobContainerClient
80-
```C# Snippet:ExtensionMethodCreateContainerClient
81-
BlobServiceClient service = new BlobServiceClient(serviceUri, credential);
82-
83-
BlobContainerClient container = service.GetBlobContainerClient(containerName);
84-
```
85-
86-
Upload a local directory to the root of the container
87-
```C# Snippet:ExtensionMethodSimpleUploadToRoot
88-
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath);
89-
90-
await transfer.WaitForCompletionAsync();
91-
```
92-
93-
Upload a local directory to a virtual directory in the container by specifying a directory prefix
94-
```C# Snippet:ExtensionMethodSimpleUploadToDirectoryPrefix
95-
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, blobDirectoryPrefix);
96-
97-
await transfer.WaitForCompletionAsync();
98-
```
99-
100-
Upload a local directory to a virtual directory in the container specifying more advanced options
101-
```C# Snippet:ExtensionMethodSimpleUploadWithOptions
102-
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
103-
{
104-
BlobContainerOptions = new BlobStorageResourceContainerOptions
105-
{
106-
BlobDirectoryPrefix = blobDirectoryPrefix
107-
},
108-
TransferOptions = new DataTransferOptions()
109-
{
110-
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
111-
}
112-
};
113-
114-
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, options);
71+
### Using the TransferManager for Blob Transfers
11572

116-
await transfer.WaitForCompletionAsync();
117-
```
73+
The `TransferManager` is the primary class for managing data transfers between storage resources. See [Setup the TransferManager sample](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement#setup-the-transfermanager).
11874

119-
Download the entire container to a local directory
120-
```C# Snippet:ExtensionMethodSimpleDownloadContainer
121-
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath);
122-
123-
await transfer.WaitForCompletionAsync();
124-
```
125-
126-
Download a directory in the container by specifying a directory prefix
127-
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectory
128-
DataTransfer tranfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, blobDirectoryPrefix);
129-
130-
await tranfer.WaitForCompletionAsync();
131-
```
132-
133-
Download from the container specifying more advanced options
134-
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectoryWithOptions
135-
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
136-
{
137-
BlobContainerOptions = new BlobStorageResourceContainerOptions
138-
{
139-
BlobDirectoryPrefix = blobDirectoryPrefix
140-
},
141-
TransferOptions = new DataTransferOptions()
142-
{
143-
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
144-
}
145-
};
146-
147-
DataTransfer tranfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, options);
148-
149-
await tranfer.WaitForCompletionAsync();
150-
```
15175

15276
### Initializing Blob Storage `StorageResource`
15377

@@ -302,13 +226,92 @@ destinationResource: blobs.FromContainer(
302226
await dataTransfer.WaitForCompletionAsync();
303227
```
304228

229+
### Extensions on `BlobContainerClient`
230+
231+
For applications with preexisting code using Azure.Storage.Blobs, this package provides extension methods for `BlobContainerClient` to get some of the benefits of the `TransferManager` with minimal extra code.
232+
233+
Instantiate the BlobContainerClient
234+
```C# Snippet:ExtensionMethodCreateContainerClient
235+
BlobServiceClient service = new BlobServiceClient(serviceUri, credential);
236+
237+
BlobContainerClient container = service.GetBlobContainerClient(containerName);
238+
```
239+
240+
Upload a local directory to the root of the container
241+
```C# Snippet:ExtensionMethodSimpleUploadToRoot
242+
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath);
243+
244+
await transfer.WaitForCompletionAsync();
245+
```
246+
247+
Upload a local directory to a virtual directory in the container by specifying a directory prefix
248+
```C# Snippet:ExtensionMethodSimpleUploadToDirectoryPrefix
249+
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, blobDirectoryPrefix);
250+
251+
await transfer.WaitForCompletionAsync();
252+
```
253+
254+
Upload a local directory to a virtual directory in the container specifying more advanced options
255+
```C# Snippet:ExtensionMethodSimpleUploadWithOptions
256+
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
257+
{
258+
BlobContainerOptions = new BlobStorageResourceContainerOptions
259+
{
260+
BlobDirectoryPrefix = blobDirectoryPrefix
261+
},
262+
TransferOptions = new DataTransferOptions()
263+
{
264+
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
265+
}
266+
};
267+
268+
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, options);
269+
270+
await transfer.WaitForCompletionAsync();
271+
```
272+
273+
Download the entire container to a local directory
274+
```C# Snippet:ExtensionMethodSimpleDownloadContainer
275+
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath);
276+
277+
await transfer.WaitForCompletionAsync();
278+
```
279+
280+
Download a directory in the container by specifying a directory prefix
281+
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectory
282+
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, blobDirectoryPrefix);
283+
284+
await transfer.WaitForCompletionAsync();
285+
```
286+
287+
Download from the container specifying more advanced options
288+
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectoryWithOptions
289+
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
290+
{
291+
BlobContainerOptions = new BlobStorageResourceContainerOptions
292+
{
293+
BlobDirectoryPrefix = blobDirectoryPrefix
294+
},
295+
TransferOptions = new DataTransferOptions()
296+
{
297+
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
298+
}
299+
};
300+
301+
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, options);
302+
303+
await transfer.WaitForCompletionAsync();
304+
```
305+
305306
## Troubleshooting
306307

307-
***TODO***
308+
See [Handling Failed Transfers](#handling-failed-transfers) and [Enabling Logging](https://learn.microsoft.com/dotnet/azure/sdk/logging) to assist with any troubleshooting.
308309

309310
## Next steps
310311

311-
***TODO***
312+
Get started with our [Share Files Samples][share_samples].
313+
314+
For more base Transfer Manager scenarios see [DataMovement samples][datamovement_base].
312315

313316
## Contributing
314317

@@ -342,7 +345,8 @@ additional questions or comments.
342345
[azure_sub]: https://azure.microsoft.com/free/dotnet/
343346
[RequestFailedException]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/core/Azure.Core/src/RequestFailedException.cs
344347
[error_codes]: https://learn.microsoft.com/rest/api/storageservices/common-rest-api-error-codes
345-
[samples]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement.Blobs/samples
348+
[datamovement_base]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement
349+
[share_samples]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement.Files.Shares/samples
346350
[storage_contrib]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/CONTRIBUTING.md
347351
[cla]: https://cla.microsoft.com
348352
[coc]: https://opensource.microsoft.com/codeofconduct/

0 commit comments

Comments
 (0)