Skip to content

Commit 3733e05

Browse files
ttorblettorble
authored
Merge pull request #196007 from mikewill4/patch-19
[SFTP] Update blob storage event support
2 parents 083b1f4 + ca75ff2 commit 3733e05

File tree

2 files changed

+231
-2
lines changed

2 files changed

+231
-2
lines changed

‎articles/event-grid/event-schema-blob-storage.md

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,19 @@ These events are triggered if you enable a hierarchical namespace on the storage
4545
> [!NOTE]
4646
> For **Azure Data Lake Storage Gen2**, if you want to ensure that the **Microsoft.Storage.BlobCreated** event is triggered only when a Block Blob is completely committed, filter the event for the `FlushWithClose` REST API call. This API call triggers the **Microsoft.Storage.BlobCreated** event only after data is fully committed to a Block Blob. To learn how to create a filter, see [Filter events for Event Grid](./how-to-filter-events.md).
4747
48+
### List of the events for SFTP APIs
49+
50+
These events are triggered if you enable a hierarchical namespace on the storage account, and clients use SFTP APIs. For more information about SFTP support for Azure Blob Storage, see [SSH File Transfer Protocol (SFTP) in Azure Blob Storage](../storage/blobs/secure-file-transfer-protocol-support.md).
51+
52+
|Event name|Description|
53+
|----------|-----------|
54+
|**Microsoft.Storage.BlobCreated** |Triggered when a blob is created or overwritten. <br>Specifically, this event is triggered when clients use the `put` operation, which corresponds to the `SftpCreate` and `SftpCommit` APIs. An empty blob is created when the file is opened and the uploaded contents are committed when the file is closed.|
55+
|**Microsoft.Storage.BlobDeleted** |Triggered when a blob is deleted. <br>Specifically, this event is also triggered when clients call the `rm` operation, which corresponds to the `SftpRemove` API.|
56+
|**Microsoft.Storage.BlobRenamed**|Triggered when a blob is renamed. <br>Specifically, this event is triggered when clients use the `rename` operation on files, which corresponds to the `SftpRename` API.|
57+
|**Microsoft.Storage.DirectoryCreated**|Triggered when a directory is created. <br>Specifically, this event is triggered when clients use the `mkdir` operation, which corresponds to the `SftpMakeDir` API.|
58+
|**Microsoft.Storage.DirectoryRenamed**|Triggered when a directory is renamed. <br>Specifically, this event is triggered when clients use the `rename` operation on a directory, which corresponds to the `SftpRename` API.|
59+
|**Microsoft.Storage.DirectoryDeleted**|Triggered when a directory is deleted. <br>Specifically, this event is triggered when clients use the `rmdir` operation, which corresponds to the `SftpRemoveDir` API.|
60+
4861
### List of policy-related events
4962

5063
These events are triggered when the actions defined by a policy are performed.
@@ -127,6 +140,52 @@ If the blob storage account has a hierarchical namespace, the data looks similar
127140
}]
128141
```
129142

143+
### Microsoft.Storage.BlobCreated event (SFTP)
144+
145+
If the blob storage account uses SFTP to create or overwrite a blob, then the data looks similar to the previous example with an exception of these changes:
146+
147+
* The `dataVersion` key is set to a value of `3`.
148+
149+
* The `data.api` key is set to the string `SftpCreate` or `SftpCommit`.
150+
151+
* The `clientRequestId` key is not included.
152+
153+
* The `contentType` key is set to `application/octet-stream`.
154+
155+
* The `contentOffset` key is included in the data set.
156+
157+
* The `identity` key is included in the data set. This corresponds to the local user used for SFTP authentication.
158+
159+
> [!NOTE]
160+
> SFTP uploads will generate 2 events. One `SftpCreate` for an initial empty blob created when opening the file and one `SftpCommit` when the file contents are written.
161+
162+
```json
163+
[{
164+
"topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/my-storage-account",
165+
"subject": "/blobServices/default/containers/testcontainer/blobs/new-file.txt",
166+
"eventType": "Microsoft.Storage.BlobCreated",
167+
"eventTime": "2022-04-25T19:13:00.1522383Z",
168+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
169+
"data": {
170+
"api": "SftpCommit",
171+
"requestId": "831e1650-001e-001b-66ab-eeb76e000000",
172+
"eTag": "\"0x8D4BCC2E4835CD0\"",
173+
"contentType": "application/octet-stream",
174+
"contentLength": 0,
175+
"contentOffset": 0,
176+
"blobType": "BlockBlob",
177+
"url": "https://my-storage-account.blob.core.windows.net/testcontainer/new-file.txt",
178+
"sequencer": "00000000000004420000000000028963",
179+
"identity":"localuser",
180+
"storageDiagnostics": {
181+
"batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
182+
}
183+
},
184+
"dataVersion": "3",
185+
"metadataVersion": "1"
186+
}]
187+
```
188+
130189
### Microsoft.Storage.BlobDeleted event
131190

132191
```json
@@ -189,6 +248,44 @@ If the blob storage account has a hierarchical namespace, the data looks similar
189248
}]
190249
```
191250

251+
### Microsoft.Storage.BlobDeleted event (SFTP)
252+
253+
If the blob storage account uses SFTP to delete a blob, then the data looks similar to the previous example with an exception of these changes:
254+
255+
* The `dataVersion` key is set to a value of `2`.
256+
257+
* The `data.api` key is set to the string `SftpRemove`.
258+
259+
* The `clientRequestId` key is not included.
260+
261+
* The `contentType` key is set to `application/octet-stream`.
262+
263+
* The `identity` key is included in the data set. This corresponds to the local user used for SFTP authentication.
264+
265+
```json
266+
[{
267+
"topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/my-storage-account",
268+
"subject": "/blobServices/default/containers/testcontainer/blobs/new-file.txt",
269+
"eventType": "Microsoft.Storage.BlobDeleted",
270+
"eventTime": "2022-04-25T19:13:00.1522383Z",
271+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
272+
"data": {
273+
"api": "SftpRemove",
274+
"requestId": "831e1650-001e-001b-66ab-eeb76e000000",
275+
"contentType": "text/plain",
276+
"blobType": "BlockBlob",
277+
"url": "https://my-storage-account.blob.core.windows.net/testcontainer/new-file.txt",
278+
"sequencer": "00000000000004420000000000028963",
279+
"identity":"localuser",
280+
"storageDiagnostics": {
281+
"batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
282+
}
283+
},
284+
"dataVersion": "2",
285+
"metadataVersion": "1"
286+
}]
287+
```
288+
192289
### Microsoft.Storage.BlobTierChanged event
193290

194291
```json
@@ -269,6 +366,39 @@ If the blob storage account has a hierarchical namespace, the data looks similar
269366
}]
270367
```
271368

369+
### Microsoft.Storage.BlobRenamed event (SFTP)
370+
371+
If the blob storage account uses SFTP to rename a blob, then the data looks similar to the previous example with an exception of these changes:
372+
373+
* The `data.api` key is set to the string `SftpRename`.
374+
375+
* The `clientRequestId` key is not included.
376+
377+
* The `identity` key is included in the data set. This corresponds to the local user used for SFTP authentication.
378+
379+
```json
380+
[{
381+
"topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/my-storage-account",
382+
"subject": "/blobServices/default/containers/testcontainer/blobs/my-renamed-file.txt",
383+
"eventType": "Microsoft.Storage.BlobRenamed",
384+
"eventTime": "2022-04-25T19:13:00.1522383Z",
385+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
386+
"data": {
387+
"api": "SftpRename",
388+
"requestId": "831e1650-001e-001b-66ab-eeb76e000000",
389+
"destinationUrl": "https://my-storage-account.blob.core.windows.net/testcontainer/my-renamed-file.txt",
390+
"sourceUrl": "https://my-storage-account.blob.core.windows.net/testcontainer/my-original-file.txt",
391+
"sequencer": "00000000000004420000000000028963",
392+
"identity":"localuser",
393+
"storageDiagnostics": {
394+
"batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
395+
}
396+
},
397+
"dataVersion": "1",
398+
"metadataVersion": "1"
399+
}]
400+
```
401+
272402
### Microsoft.Storage.DirectoryCreated event
273403

274404
```json
@@ -293,6 +423,40 @@ If the blob storage account has a hierarchical namespace, the data looks similar
293423
}]
294424
```
295425

426+
### Microsoft.Storage.DirectoryCreated event (SFTP)
427+
428+
If the blob storage account uses SFTP to create a directory, then the data looks similar to the previous example with an exception of these changes:
429+
430+
* The `dataVersion` key is set to a value of `2`.
431+
432+
* The `data.api` key is set to the string `SftpMakeDir`.
433+
434+
* The `clientRequestId` key is not included.
435+
436+
* The `identity` key is included in the data set. This corresponds to the local user used for SFTP authentication.
437+
438+
```json
439+
[{
440+
"topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/my-storage-account",
441+
"subject": "/blobServices/default/containers/testcontainer/blobs/my-new-directory",
442+
"eventType": "Microsoft.Storage.DirectoryCreated",
443+
"eventTime": "2022-04-25T19:13:00.1522383Z",
444+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
445+
"data": {
446+
"api": "SftpMakeDir",
447+
"requestId": "831e1650-001e-001b-66ab-eeb76e000000",
448+
"url": "https://my-storage-account.blob.core.windows.net/testcontainer/my-new-directory",
449+
"sequencer": "00000000000004420000000000028963",
450+
"identity":"localuser",
451+
"storageDiagnostics": {
452+
"batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
453+
}
454+
},
455+
"dataVersion": "2",
456+
"metadataVersion": "1"
457+
}]
458+
```
459+
296460
### Microsoft.Storage.DirectoryRenamed event
297461

298462
```json
@@ -318,6 +482,39 @@ If the blob storage account has a hierarchical namespace, the data looks similar
318482
}]
319483
```
320484

485+
### Microsoft.Storage.DirectoryRenamed event (SFTP)
486+
487+
If the blob storage account uses SFTP to rename a directory, then the data looks similar to the previous example with an exception of these changes:
488+
489+
* The `data.api` key is set to the string `SftpRename`.
490+
491+
* The `clientRequestId` key is not included.
492+
493+
* The `identity` key is included in the data set. This corresponds to the local user used for SFTP authentication.
494+
495+
```json
496+
[{
497+
"topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/my-storage-account",
498+
"subject": "/blobServices/default/containers/testcontainer/blobs/my-renamed-directory",
499+
"eventType": "Microsoft.Storage.DirectoryRenamed",
500+
"eventTime": "2022-04-25T19:13:00.1522383Z",
501+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
502+
"data": {
503+
"api": "SftpRename",
504+
"requestId": "831e1650-001e-001b-66ab-eeb76e000000",
505+
"destinationUrl": "https://my-storage-account.blob.core.windows.net/testcontainer/my-renamed-directory",
506+
"sourceUrl": "https://my-storage-account.blob.core.windows.net/testcontainer/my-original-directory",
507+
"sequencer": "00000000000004420000000000028963",
508+
"identity":"localuser",
509+
"storageDiagnostics": {
510+
"batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
511+
}
512+
},
513+
"dataVersion": "1",
514+
"metadataVersion": "1"
515+
}]
516+
```
517+
321518
### Microsoft.Storage.DirectoryDeleted event
322519

323520
```json
@@ -343,6 +540,39 @@ If the blob storage account has a hierarchical namespace, the data looks similar
343540
}]
344541
```
345542

543+
### Microsoft.Storage.DirectoryDeleted event (SFTP)
544+
545+
If the blob storage account uses SFTP to delete a directory, then the data looks similar to the previous example with an exception of these changes:
546+
547+
* The `data.api` key is set to the string `SftpRemoveDir`.
548+
549+
* The `clientRequestId` key is not included.
550+
551+
* The `identity` key is included in the data set. This corresponds to the local user used for SFTP authentication.
552+
553+
```json
554+
[{
555+
"topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/my-storage-account",
556+
"subject": "/blobServices/default/containers/testcontainer/blobs/directory-to-delete",
557+
"eventType": "Microsoft.Storage.DirectoryDeleted",
558+
"eventTime": "2022-04-25T19:13:00.1522383Z",
559+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
560+
"data": {
561+
"api": "SftpRemoveDir",
562+
"requestId": "831e1650-001e-001b-66ab-eeb76e000000",
563+
"url": "https://my-storage-account.blob.core.windows.net/testcontainer/directory-to-delete",
564+
"recursive": "false",
565+
"sequencer": "00000000000004420000000000028963",
566+
"identity":"localuser",
567+
"storageDiagnostics": {
568+
"batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
569+
}
570+
},
571+
"dataVersion": "1",
572+
"metadataVersion": "1"
573+
}]
574+
```
575+
346576
### Microsoft.Storage.BlobInventoryPolicyCompleted event
347577

348578
```json
@@ -668,6 +898,7 @@ The data object has the following properties:
668898
| `url` | string | The path to the blob. <br>If the client uses a Blob REST API, then the url has this structure: `<storage-account-name>.blob.core.windows.net\<container-name>\<file-name>`. <br>If the client uses a Data Lake Storage REST API, then the url has this structure: `<storage-account-name>.dfs.core.windows.net/<file-system-name>/<file-name>`. |
669899
| `recursive` | string | `True` to run the operation on all child directories; otherwise `False`. <br>Appears only for events triggered on blob storage accounts that have a hierarchical namespace. |
670900
| `sequencer` | string | An opaque string value representing the logical sequence of events for any particular blob name. Users can use standard string comparison to understand the relative sequence of two events on the same blob name. |
901+
| `identity` | string | A string value representing the identity associated with the event. For SFTP, this is the local user name.|
671902
| `storageDiagnostics` | object | Diagnostic data occasionally included by the Azure Storage service. When present, should be ignored by event consumers. |
672903

673904
## Tutorials and how-tos

‎articles/storage/blobs/storage-blob-event-overview.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,8 +108,6 @@ This table shows how this feature is supported in your account and the impact on
108108
| Standard general-purpose v2 | ![Yes](../media/icons/yes-icon.png) |![Yes](../media/icons/yes-icon.png) <sup>2</sup> | ![No](../media/icons/no-icon.png) | ![Yes](../media/icons/yes-icon.png) |
109109
| Premium block blobs | ![Yes](../media/icons/yes-icon.png) |![Yes](../media/icons/yes-icon.png) <sup>2</sup> | ![No](../media/icons/no-icon.png) | ![Yes](../media/icons/yes-icon.png) |
110110

111-
<sup>1</sup> Data Lake Storage Gen2 and the Network File System (NFS) 3.0 protocol both require a storage account with a hierarchical namespace enabled.
112-
113111
<sup>1</sup> Data Lake Storage Gen2, Network File System (NFS) 3.0 protocol, and SSH File Transfer Protocol (SFTP) support all require a storage account with a hierarchical namespace enabled.
114112

115113
## Next steps

0 commit comments

Comments
 (0)