Skip to content

Commit 5c54016

Browse files
Data Movement Migration: Azure.Storage.DataMovement.Samples.Tests (Azure#48028)
* datamovement.samples.tests first sample test written * upload directory * download blob * download directory * blob2blob * blob2file * provider API changes from rebase * updatesnippets
1 parent e025370 commit 5c54016

File tree

4 files changed

+556
-65
lines changed

4 files changed

+556
-65
lines changed

sdk/storage/Azure.Storage.DataMovement.sln

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Azure.Storage.DataMovement.
3939
EndProject
4040
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Azure.Storage.DataMovement.Files.Shares.Samples.Tests", "Azure.Storage.DataMovement.Files.Shares\samples\Azure.Storage.DataMovement.Files.Shares.Samples.Tests.csproj", "{76FB3889-81CD-4AD2-8372-13E2CA98B226}"
4141
EndProject
42+
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Azure.Storage.DataMovement.Samples.Tests", "Azure.Storage.DataMovement\samples\Azure.Storage.DataMovement.Samples.Tests.csproj", "{180F8DFB-74DA-4753-8674-A35252688732}"
43+
EndProject
4244
Global
4345
GlobalSection(SolutionConfigurationPlatforms) = preSolution
4446
Debug|Any CPU = Debug|Any CPU
@@ -117,6 +119,10 @@ Global
117119
{76FB3889-81CD-4AD2-8372-13E2CA98B226}.Debug|Any CPU.Build.0 = Debug|Any CPU
118120
{76FB3889-81CD-4AD2-8372-13E2CA98B226}.Release|Any CPU.ActiveCfg = Release|Any CPU
119121
{76FB3889-81CD-4AD2-8372-13E2CA98B226}.Release|Any CPU.Build.0 = Release|Any CPU
122+
{180F8DFB-74DA-4753-8674-A35252688732}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
123+
{180F8DFB-74DA-4753-8674-A35252688732}.Debug|Any CPU.Build.0 = Debug|Any CPU
124+
{180F8DFB-74DA-4753-8674-A35252688732}.Release|Any CPU.ActiveCfg = Release|Any CPU
125+
{180F8DFB-74DA-4753-8674-A35252688732}.Release|Any CPU.Build.0 = Release|Any CPU
120126
EndGlobalSection
121127
GlobalSection(SolutionProperties) = preSolution
122128
HideSolutionNode = FALSE

sdk/storage/Azure.Storage.DataMovement/MigrationGuide.md

Lines changed: 67 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ The `TransferManager` is still the core type that handles transfers in Data Move
6969
It is no longer a static type, though **we recommend maintaining a singleton instance** for optimal usage.
7070

7171
All transfer methods for upload, download, and copy, for both files and directories, have been replaced with a single instance method:
72-
```csharp
72+
```C#
7373
Task<TransferOperation> StartTransferAsync(
7474
StorageResource sourceResource,
7575
StorageResource destinationResource,
@@ -101,7 +101,7 @@ Here is an example using providers to create an upload to an Azure blob.
101101
Note that local files use a static provider, while blobs (and every other provider in our packages) must be instantiated.
102102
Further examples can be found in our [migration samples](#migration-samples).
103103

104-
```csharp
104+
```C#
105105
BlobsStorageResourceProvider blobs = new(myTokenCredential);
106106
TransferManager transferManager = new TransferManager();
107107

@@ -169,62 +169,62 @@ These samples are not meant to be exhaustive, but demonstrate a wide variety of
169169
#### Upload single file to blob storage
170170

171171
**Legacy:**
172-
```csharp
172+
```C#
173173
// these values provided by your code
174174
string filePath, containerName, blobName;
175175
CloudBlobClient client;
176176
```
177-
```csharp
177+
```C#
178178
// upload blob
179179
await TransferManager.UploadAsync(
180180
filePath,
181181
client.GetContainerReference(containerName).GetBlockBlobReference(blobName));
182182
```
183183
**Modern:**
184-
```csharp
184+
```C# Snippet:DataMovementMigration_UploadSingleFile_VarDeclaration
185185
// these values provided by your code
186-
string filePath, blobUri;
186+
string filePath;
187+
Uri blobUri;
187188
BlobsStorageResourceProvider blobs;
188189
TransferManager transferManager;
189190
```
190-
191-
```csharp
191+
```C# Snippet:DataMovementMigration_UploadSingleFile
192192
// upload blob
193-
TranferOperation operation = await transferManager.StartTransferAsync(
193+
TransferOperation operation = await transferManager.StartTransferAsync(
194194
LocalFilesStorageResourceProvider.FromFile(filePath),
195-
blobs.FromBlob(blobUri));
195+
await blobs.FromBlobAsync(blobUri));
196196
await operation.WaitForCompletionAsync();
197197
```
198198

199199
#### Upload directory to blob storage
200200

201201
**Legacy:**
202-
```csharp
202+
```C#
203203
// these values provided by your code
204204
string directoryPath, containerName, blobDirectoryPath;
205205
CloudBlobClient client;
206206
```
207-
```csharp
207+
```C#
208208
// upload blob
209209
await TransferManager.UploadDirectoryAsync(
210210
directoryPath,
211211
client.GetContainerReference(containerName).GetDirectoryReference(blobDirectoryPath));
212212
```
213213
**Modern:**
214-
```csharp
214+
```C# Snippet:DataMovementMigration_UploadBlobDirectory_VarDeclaration
215215
// these values provided by your code
216-
string directoryPath, containerUri, blobDirectoryPath;
216+
string directoryPath, blobDirectoryPath;
217+
Uri containerUri;
217218
BlobsStorageResourceProvider blobs;
218219
TransferManager transferManager;
219220
```
220-
221-
```csharp
221+
```C# Snippet:DataMovementMigration_UploadBlobDirectory
222222
// upload blobs
223-
TranferOperation operation = await transferManager.StartTransferAsync(
223+
TransferOperation operation = await transferManager.StartTransferAsync(
224224
LocalFilesStorageResourceProvider.FromDirectory(directoryPath),
225-
blobs.FromContainer(containerUri, new BlobStorageResourceContainerOptions()
225+
await blobs.FromContainerAsync(containerUri, new BlobStorageResourceContainerOptions()
226226
{
227-
BlobDirectoryPrefix = blobDirectoryPath,
227+
BlobPrefix = blobDirectoryPath,
228228
}));
229229
await operation.WaitForCompletionAsync();
230230
```
@@ -234,41 +234,42 @@ await operation.WaitForCompletionAsync();
234234
#### Download single blob
235235

236236
**Legacy:**
237-
```csharp
237+
```C#
238238
// these values provided by your code
239239
string filePath, containerName, blobName;
240240
CloudBlobClient client;
241241
```
242-
```csharp
242+
```C#
243243
// download blob
244244
await TransferManager.DownloadAsync(
245245
client.GetContainerReference(containerName).GetBlockBlobReference(blobName),
246246
filePath);
247247
```
248248
**Modern:**
249-
```csharp
249+
```C# Snippet:DataMovementMigration_DownloadBlob_VarDeclaration
250250
// these values provided by your code
251-
string filePath, blobUri;
251+
string filePath;
252+
Uri blobUri;
252253
BlobsStorageResourceProvider blobs;
253254
TransferManager transferManager;
254255
```
255-
```csharp
256+
```C# Snippet:DataMovementMigration_DownloadBlob
256257
// download blob
257-
TranferOperation operation = await transferManager.StartTransferAsync(
258-
blobs.FromBlob(blobUri),
258+
TransferOperation operation = await transferManager.StartTransferAsync(
259+
await blobs.FromBlobAsync(blobUri),
259260
LocalFilesStorageResourceProvider.FromFile(filePath));
260261
await operation.WaitForCompletionAsync();
261262
```
262263

263264
#### Download blob directory
264265

265266
**Legacy:**
266-
```csharp
267+
```C#
267268
// these values provided by your code
268269
string directoryPath, containerName, blobDirectoryPath;
269270
CloudBlobClient client;
270271
```
271-
```csharp
272+
```C#
272273
// download blob directory
273274
await TransferManager.DownloadDirectoryAsync(
274275
client.GetContainerReference(containerName).GetDirectoryReference(blobDirectoryPath),
@@ -277,18 +278,19 @@ await TransferManager.DownloadDirectoryAsync(
277278
context: null);
278279
```
279280
**Modern:**
280-
```csharp
281+
```C# Snippet:DataMovementMigration_DownloadBlobDirectory_VarDeclaration
281282
// these values provided by your code
282-
string directoryPath, containerUri, blobDirectoryPath;
283+
string directoryPath, blobDirectoryPath;
284+
Uri containerUri;
283285
BlobsStorageResourceProvider blobs;
284286
TransferManager transferManager;
285287
```
286-
```csharp
288+
```C# Snippet:DataMovementMigration_DownloadBlobDirectory
287289
// download blob directory
288-
TranferOperation operation = await transferManager.StartTransferAsync(
289-
blobs.FromContainer(containerUri, new BlobStorageResourceContainerOptions()
290+
TransferOperation operation = await transferManager.StartTransferAsync(
291+
await blobs.FromContainerAsync(containerUri, new BlobStorageResourceContainerOptions()
290292
{
291-
BlobDirectoryPrefix = blobDirectoryPath,
293+
BlobPrefix = blobDirectoryPath,
292294
}),
293295
LocalFilesStorageResourceProvider.FromDirectory(directoryPath));
294296
await operation.WaitForCompletionAsync();
@@ -301,30 +303,30 @@ await operation.WaitForCompletionAsync();
301303
Note: The modern data movement library only supports service side sync copy.
302304

303305
**Legacy:**
304-
```csharp
306+
```C#
305307
// these values provided by your code
306308
string srcContainerName, srcBlobName, dstContainerName, dstBlobName;
307309
CloudBlobClient client;
308310
```
309-
```csharp
311+
```C#
310312
// copy blob
311313
await TransferManager.DownloadAsync(
312314
client.GetContainerReference(srcContainerName).GetBlockBlobReference(srcBlobName),
313315
client.GetContainerReference(dstContainerName).GetBlockBlobReference(dstBlobName),
314316
CopyMethod.ServiceSideSyncCopy);
315317
```
316318
**Modern:**
317-
```csharp
319+
```C# Snippet:DataMovementMigration_CopyBlobToBlob_VarDeclaration
318320
// these values provided by your code
319-
string srcBlobUri, dstBlobUri;
321+
Uri srcBlobUri, dstBlobUri;
320322
BlobsStorageResourceProvider blobs;
321323
TransferManager transferManager;
322324
```
323-
```csharp
324-
// copy blob
325-
TranferOperation operation = await transferManager.StartTransferAsync(
326-
blobs.FromBlob(srcBlobUri),
327-
blobs.FromBlob(dstBlobUri));
325+
```C# Snippet:DataMovementMigration_CopyBlobToBlob
326+
// upload blob
327+
TransferOperation operation = await transferManager.StartTransferAsync(
328+
await blobs.FromBlobAsync(srcBlobUri),
329+
await blobs.FromBlobAsync(dstBlobUri));
328330
await operation.WaitForCompletionAsync();
329331
```
330332

@@ -333,32 +335,32 @@ await operation.WaitForCompletionAsync();
333335
Note: File shares requires the Azure.Storage.DataMovement.Files.Shares package.
334336

335337
**Legacy:**
336-
```csharp
338+
```C#
337339
// these values provided by your code
338340
string containerName, blobName, shareName, filePath;
339341
CloudBlobClient blobClient;
340342
CloudFileClient fileClient;
341343
```
342-
```csharp
344+
```C#
343345
// copy file
344346
await TransferManager.DownloadAsync(
345347
blobClient.GetContainerReference(srcContainerName).GetBlockBlobReference(srcBlobName),
346348
fileClient.GetShareReference(dstContainerName).GetRootDirectoryReference().GetBlockBlobReference(dstBlobName),
347349
CopyMethod.ServiceSideSyncCopy);
348350
```
349351
**Modern:**
350-
```csharp
352+
```C# Snippet:DataMovementMigration_CopyBlobToShareFile_VarDeclaration
351353
// these values provided by your code
352-
string blobUri, fileUri;
354+
Uri blobUri, fileUri;
353355
BlobsStorageResourceProvider blobs;
354356
ShareFilesStorageResourceProvider files;
355357
TransferManager transferManager;
356358
```
357-
```csharp
358-
// copy file
359-
TranferOperation operation = await transferManager.StartTransferAsync(
360-
blobs.FromBlob(blobUri),
361-
files.FromFile(fileUri));
359+
```C# Snippet:DataMovementMigration_CopyBlobToShareFile
360+
// upload blob
361+
TransferOperation operation = await transferManager.StartTransferAsync(
362+
await blobs.FromBlobAsync(blobUri),
363+
await files.FromFileAsync(fileUri));
362364
await operation.WaitForCompletionAsync();
363365
```
364366

@@ -369,7 +371,7 @@ await operation.WaitForCompletionAsync();
369371
In the legacy data movement library, it was possible to check a report of how many files were or were not transferred in a directory transfer, as shown below.
370372
With the modern library, applications must instead enable progress reporting or listen to transfer events, as detailed in the following sections.
371373

372-
```csharp
374+
```C#
373375
TransferStatus status = await TransferManager.UploadDirectoryAsync(
374376
directoryPath,
375377
client.GetContainerReference(containerName).GetDirectoryReference(blobDirectoryPath));
@@ -383,11 +385,11 @@ TransferStatus status = await TransferManager.UploadDirectoryAsync(
383385
#### IProgress
384386

385387
**Legacy:**
386-
```csharp
388+
```C#
387389
// progress handler provided by your code
388390
IProgress<TransferStatus> progress;
389391
```
390-
```csharp
392+
```C#
391393
await TransferManager.UploadDirectoryAsync(
392394
directoryPath,
393395
client.GetContainerReference(containerName).GetDirectoryReference(blobDirectoryPath),
@@ -398,14 +400,14 @@ await TransferManager.UploadDirectoryAsync(
398400
});
399401
```
400402
**Modern:**
401-
```csharp
403+
```C#
402404
// progress handler provided by your code
403405
IProgress<TransferProgress> progress;
404406
// if TransferProgress report is desired for
405407
// each transfer of bytes, set to true.
406408
bool reportBytes;
407409
```
408-
```csharp
410+
```C#
409411
// upload blobs
410412
TranferOperation operation = await transferManager.StartTransferAsync(
411413
LocalFilesStorageResourceProvider.FromDirectory(directoryPath),
@@ -427,13 +429,13 @@ await operation.WaitForCompletionAsync();
427429
#### Eventing (new to modern library)
428430

429431
**Modern:**
430-
```csharp
432+
```C#
431433
// callback provided by your code
432434
Func<TransferItemCompletedEventArgs, Task> onItemCompleted;
433435
Func<TransferItemSkippedEventArgs, Task> onItemSkipped;
434436
Func<TransferItemFailedEventArgs, Task> onItemFailed;
435437
```
436-
```csharp
438+
```C#
437439
TransferOptions options = new TransferOptions();
438440
options.ItemTransferCompleted += onItemCompleted;
439441
options.ItemTransferSkipped += onItemSkipped;
@@ -456,13 +458,13 @@ In the legacy library, transfers were paused by invoking a cancellation token.
456458
In the modern library, transfers are paused by calling the pause method on a given `TransferOperation`.
457459

458460
**Legacy:**
459-
```csharp
461+
```C#
460462
CancellationTokenSource cts = new();
461463
Task uploadTask = TransferManager.UploadAsync(source, destination, cts.Token);
462464
cts.Cancel();
463465
```
464466
**Modern:**
465-
```csharp
467+
```C#
466468
TransferOperation transfer = await transferManager.StartTransferAsync(source, destination);
467469
await transfer.PauseAsync();
468470
```
@@ -481,7 +483,7 @@ No checkpointer is needed to be supplied; it is enabled by default to write to a
481483
Checkpointer options may be supplied to the transfer manager to configure the directory or to disable checkpointing entirely.
482484

483485
**Legacy:**
484-
```csharp
486+
```C#
485487
SingleTransferContext context = new(journalStream);
486488
Task uploadTask = TransferManager.UploadAsync(
487489
source,
@@ -490,7 +492,7 @@ Task uploadTask = TransferManager.UploadAsync(
490492
context);
491493
```
492494
**Modern:**
493-
```csharp
495+
```C#
494496
TransferManager transferManager = new(new TransferManagerOptions()
495497
{
496498
// enable to resume transfers involving blob storage
@@ -501,7 +503,7 @@ TransferOperation transfer = await transferManager.ResumeTransferAsync(transferI
501503

502504
The modern `TransferManager` also has methods to enumerate resumable transfers.
503505

504-
```csharp
506+
```C#
505507
List<TranferOperation> operations = new();
506508
await foreach (TransferProperties transferProperties in transferManager.GetResumableTransfersAsync())
507509
{
@@ -511,7 +513,7 @@ await foreach (TransferProperties transferProperties in transferManager.GetResum
511513

512514
The above sample is a simplified version of `TransferManager.ResumeAllTransfersAsync()` and can be simplified to the following.
513515

514-
```csharp
516+
```C#
515517
List<TranferOperation> operations = await transferManager.ResumeAllTransfersAsync();
516518
```
517519

0 commit comments

Comments
 (0)