Skip to content

Commit a40b266

Browse files
wsugarmanwsugarman
committed
Update template
1 parent ba83a2e commit a40b266

File tree

1 file changed

+259
-50
lines changed

1 file changed

+259
-50
lines changed

‎articles/healthcare-apis/dicom/dicom-storage-indexing.md

Lines changed: 259 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.date: 05/31/2025
99
ms.author: wisuga
1010
---
1111

12-
# Azure Data Lake Storage Indexing
12+
# Azure Data Lake Storage Indexing (Preview)
1313

1414
The [DICOM® service](overview.md) automatically uploads DICOM files to Azure Data Lake Storage when using STOW-RS. This way, users can query their data either using DICOMweb™ APIs, like WADO-RS, or Azure Blob/Data Lake APIs. However, with storage indexing DICOM files will be automatically indexed by the DICOM service after uploading the data directly to the ADLS Gen 2 file system. Whether the files were uploaded using STOW-RS, an Azure Blob SDK, or even `AzCopy`, they can be accessed using either DICOMweb™ APIs or directly through the ADLS Gen 2 file system.
1515

@@ -26,10 +26,6 @@ The DICOM service indexes an ADLS Gen 2 file system by reacting to Blob or Data
2626

2727
First, create an [Azure Storage Queue](../../storage/queues/storage-queues-introduction.md) in the same Azure Storage Account connected to the DICOM service The DICOM service will also need access to the queue; it will need to be able to dequeue messages as well as enqueue its own messages for errors and complex tasks. So, make sure the same managed identity used by the DICOM service, either user-assigned or system-assigned, has the [**Storage Queue Data Contributor**](../../storage/queues/assign-azure-role-data-access.md) role assigned.
2828

29-
#### [Azure Portal](#tab/queue-portal)
30-
31-
#### [ARM](#tab/queue-arm)
32-
3329
### Publish Storage Events to the Queue
3430

3531
With the Storage Queue in place, events must be published from the Storage Account to an Azure Event Grid System Topic and routed to queue using an Azure Event Grid Subscription. Before creating the event subscription, be sure to grant the role **Storage Queue Data Message Sender** to the event subscription so that it is authorized to send messages. The event subscription can either use a user-assigned or system-assigned managed identity from the system topic to authenticate its operations.
@@ -38,7 +34,7 @@ By default, event subscriptions send all of the subscribed event types to their
3834
- The message must be a Base64 `CloudEvent`
3935
- The event type must be `Microsoft.Storage.BlobCreated` or `Microsoft.Storage.BlobDeleted`
4036
- The file system must be the same one configured for the DICOM service
41-
- The file path must be within `AHDS/<workspace-name>/dicom/<dicom-service-name>`
37+
- The file path must be within `AHDS/<workspace-name>/dicom/<dicom-service-name>[/<partition-name>]`
4238
- The file must be a DICOM file as defined in Part 10 of the DICOM standard
4339
- The operation must not have been performed by the DICOM service itself
4440

@@ -47,61 +43,274 @@ Thankfully, the event subscription can be configured to filter out irrelevant da
4743
- Optionally, the subject ends with `.dcm`
4844
- Under advanced filters, the key `data.clientRequestId` does not begin with `tag:<workspace-name>-<dicom-service-name>.dicom.azurehealthcareapis.com,`
4945

50-
#### [Azure Portal](#tab/events-portal)
51-
52-
#### [ARM](#tab/events-arm)
53-
54-
1. Use the Azure CLI command [`az deployment group create`](../../azure-resource-manager/bicep/deploy-cli.md) to deploy a system topic and event subscription like below:
55-
5646
### Enable Storage Indexing
5747

58-
Once the event grid subscription has been successfully configured, it's time to let the DICOM service know from where to read the storage events.
48+
Once the event grid subscription has been successfully configured, it's time to let the DICOM service know from where to read the storage events. Storage indexing is available starting in version `2025-04-01-preview` for Azure Resource Manager (ARM) which introduced a new property `storageConfiguration.storageIndexingConfiguration.storageEventQueueName`. It is currently unavailable to configure via the Azure Portal.
5949

60-
#### [ARM](#tab/dicom-arm)
61-
62-
Storage indexing is available starting in the preview ARM version `2025-04-01-preview` which introduced a new property within `storageConfiguration` called `storageIndexingConfiguration.storageEventQueueName`. Deploy, or redeploy, a DICOM service using this new property with the Azure CLI command [`az deployment group create`](../../azure-resource-manager/bicep/deploy-cli.md):
50+
The example ARM template below can be deployed using the Azure CLI command [`az deployment group create`](../../azure-resource-manager/bicep/deploy-cli.md). It includes all of the necessary resources for a DICOM service:
6351

6452
```json
6553
{
66-
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
67-
"contentVersion": "1.0.0.0",
68-
"parameters": {
69-
"workspaceName": {
70-
"type": "String"
71-
},
72-
"dicomServiceName": {
73-
"type": "String"
74-
},
75-
"storageAccountResourceId": {
76-
"type": "String"
77-
},
78-
"fileSystemName": {
79-
"type": "String"
80-
},
81-
"queueName": {
82-
"type": "String"
54+
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
55+
"contentVersion": "1.0.0.0",
56+
"parameters": {
57+
"workspaceName": {
58+
"type": "String"
59+
},
60+
"dicomServiceName": {
61+
"type": "String"
62+
},
63+
"enableDataPartitions": {
64+
"defaultValue": false,
65+
"type": "bool"
66+
},
67+
"storageAccountName": {
68+
"type": "String"
69+
},
70+
"storageAccountSku": {
71+
"defaultValue": "Standard_LRS",
72+
"type": "String"
73+
},
74+
"fileSystemName": {
75+
"type": "String"
76+
},
77+
"storageEventQueueName": {
78+
"defaultValue": "storage-events",
79+
"type": "String"
80+
},
81+
"systemTopicName": {
82+
"type": "String"
83+
},
84+
"eventSubscriptionName": {
85+
"defaultValue": "dicom-storage-events",
86+
"type": "String"
87+
}
88+
},
89+
"variables": {
90+
"storageBlobDataContributor": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe')]",
91+
"storageQueueDataContributor": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', '974c5e8b-45b9-4653-ba55-5f855dd0fb88')]",
92+
"storageQueueDataMessageSender": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'c6a89b2d-59bc-44d0-9896-0f6e12d7b80a')]",
93+
"dicomIdentityName": "[concat(parameters('storageAccountName'), '-', parameters('storageEventQueueName'))]"
94+
},
95+
"resources": [
96+
{
97+
"type": "Microsoft.ManagedIdentity/userAssignedIdentities",
98+
"apiVersion": "2023-01-31",
99+
"name": "[variables('dicomIdentityName')]",
100+
"location": "[resourceGroup().location]"
101+
},
102+
{
103+
"type": "Microsoft.ManagedIdentity/userAssignedIdentities",
104+
"apiVersion": "2023-01-31",
105+
"name": "[parameters('systemTopicName')]",
106+
"location": "[resourceGroup().location]"
107+
},
108+
{
109+
"type": "Microsoft.Storage/storageAccounts",
110+
"apiVersion": "2022-05-01",
111+
"name": "[parameters('storageAccountName')]",
112+
"location": "[resourceGroup().location]",
113+
"sku": {
114+
"name": "[parameters('storageAccountSku')]"
115+
},
116+
"kind": "StorageV2",
117+
"properties": {
118+
"isHnsEnabled": true,
119+
"accessTier": "Hot",
120+
"supportsHttpsTrafficOnly": true,
121+
"minimumTlsVersion": "TLS1_2",
122+
"defaultToOAuthAuthentication": true,
123+
"allowBlobPublicAccess": false,
124+
"allowSharedKeyAccess": false,
125+
"encryption": {
126+
"keySource": "Microsoft.Storage",
127+
"requireInfrastructureEncryption": true,
128+
"services": {
129+
"blob": {
130+
"enabled": true
131+
},
132+
"queue": {
133+
"enabled": true
134+
}
135+
}
136+
}
137+
}
138+
},
139+
{
140+
"type": "Microsoft.Storage/storageAccounts/blobServices/containers",
141+
"apiVersion": "2022-05-01",
142+
"name": "[format('{0}/default/{1}', parameters('storageAccountName'), parameters('fileSystemName'))]",
143+
"dependsOn": [
144+
"[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
145+
]
146+
},
147+
{
148+
"type": "Microsoft.Storage/storageAccounts/queueServices/queues",
149+
"apiVersion": "2024-01-01",
150+
"name": "[format('{0}/default/{1}', parameters('storageAccountName'), parameters('storageEventQueueName'))]",
151+
"dependsOn": [
152+
"[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
153+
]
154+
},
155+
{
156+
"type": "Microsoft.Storage/storageAccounts/queueServices/queues",
157+
"apiVersion": "2024-01-01",
158+
"name": "[format('{0}/default/{1}-poison', parameters('storageAccountName'), parameters('storageEventQueueName'))]",
159+
"dependsOn": [
160+
"[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
161+
]
162+
},
163+
{
164+
"type": "Microsoft.Authorization/roleAssignments",
165+
"apiVersion": "2021-04-01-preview",
166+
"name": "[guid(resourceGroup().id, parameters('workspaceName'), parameters('dicomServiceName'))]",
167+
"location": "[resourceGroup().location]",
168+
"dependsOn": [
169+
"[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]",
170+
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('dicomIdentityName'))]"
171+
],
172+
"properties": {
173+
"roleDefinitionId": "[variables('storageBlobDataContributor')]",
174+
"principalId": "[reference(resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('dicomIdentityName'))).principalId]",
175+
"principalType": "ServicePrincipal"
176+
},
177+
"scope": "[concat('Microsoft.Storage/storageAccounts', '/', parameters('storageAccountName'))]"
178+
},
179+
{
180+
"type": "Microsoft.Authorization/roleAssignments",
181+
"apiVersion": "2021-04-01-preview",
182+
"name": "[guid(resourceGroup().id, parameters('workspaceName'), parameters('dicomServiceName'), parameters('storageEventQueueName'))]",
183+
"location": "[resourceGroup().location]",
184+
"dependsOn": [
185+
"[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]",
186+
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('dicomIdentityName'))]"
187+
],
188+
"properties": {
189+
"roleDefinitionId": "[variables('storageQueueDataContributor')]",
190+
"principalId": "[reference(resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('dicomIdentityName'))).principalId]",
191+
"principalType": "ServicePrincipal"
192+
},
193+
"scope": "[concat('Microsoft.Storage/storageAccounts', '/', parameters('storageAccountName'))]"
194+
},
195+
{
196+
"type": "Microsoft.Authorization/roleAssignments",
197+
"apiVersion": "2021-04-01-preview",
198+
"name": "[guid(resourceGroup().id, parameters('systemTopicName'), parameters('storageEventQueueName'))]",
199+
"location": "[resourceGroup().location]",
200+
"dependsOn": [
201+
"[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]",
202+
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('systemTopicName'))]"
203+
],
204+
"properties": {
205+
"roleDefinitionId": "[variables('storageQueueDataMessageSender')]",
206+
"principalId": "[reference(resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('systemTopicName'))).principalId]",
207+
"principalType": "ServicePrincipal"
208+
},
209+
"scope": "[concat('Microsoft.Storage/storageAccounts', '/', parameters('storageAccountName'))]"
210+
},
211+
{
212+
"type": "Microsoft.EventGrid/systemTopics",
213+
"apiVersion": "2025-02-15",
214+
"name": "[parameters('systemTopicName')]",
215+
"location": "[resourceGroup().location]",
216+
"identity": {
217+
"type": "userAssigned",
218+
"userAssignedIdentities": {
219+
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities/', parameters('systemTopicName'))]": {}
83220
}
221+
},
222+
"dependsOn": [
223+
"[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]",
224+
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('systemTopicName'))]"
225+
],
226+
"properties": {
227+
"source": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]",
228+
"topicType": "Microsoft.Storage.StorageAccounts"
229+
}
84230
},
85-
"resources": [
86-
{
87-
"type": "Microsoft.HealthcareApis/workspaces/dicomservices",
88-
"apiVersion": "2024-03-31",
89-
"name": "[concat(parameters('workspaceName'), '/', parameters('dicomServiceName'))]",
90-
"location": "[resourceGroup().location]",
91-
"identity": {
92-
"type": "SystemAssigned"
231+
{
232+
"type": "Microsoft.EventGrid/systemTopics/eventSubscriptions",
233+
"apiVersion": "2025-02-15",
234+
"name": "[concat(parameters('systemTopicName'), '/', parameters('eventSubscriptionName'))]",
235+
"dependsOn": [
236+
"[resourceId('Microsoft.EventGrid/systemTopics', parameters('systemTopicName'))]"
237+
],
238+
"properties": {
239+
"deliveryWithResourceIdentity": {
240+
"identity": {
241+
"type": "UserAssigned",
242+
"userAssignedIdentity": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('systemTopicName'))]"
243+
},
244+
"destination": {
245+
"properties": {
246+
"resourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]",
247+
"queueName": "[parameters('storageEventQueueName')]",
248+
"queueMessageTimeToLiveInSeconds": 604800
249+
},
250+
"endpointType": "StorageQueue"
251+
}
93252
},
94-
"properties": {
95-
"storageConfiguration": {
96-
"fileSystemName": "[parameters('fileSystemName')]",
97-
"storageResourceId": "[parameters('storageAccountResourceId')]",
98-
"storageIndexingConfiguration": {
99-
"storageEventQueueName": "[parameters('queueName')]"
253+
"filter": {
254+
"subjectBeginsWith": "[format('/blobServices/default/containers/{0}/blobs/AHDS/{1}/dicom/{2}/', parameters('fileSystemName'), parameters('workspaceName'), parameters('dicomServiceName'))]",
255+
"subjectEndsWith": ".dcm",
256+
"includedEventTypes": [
257+
"Microsoft.Storage.BlobCreated",
258+
"Microsoft.Storage.BlobDeleted"
259+
],
260+
"isSubjectCaseSensitive": true,
261+
"enableAdvancedFilteringOnArrays": true,
262+
"advancedFilters": [
263+
{
264+
"values": [
265+
"[format('tag:{0}-{1}.dicom.azurehealthcareapis.com,', parameters('workspaceName'), parameters('dicomServiceName'))]"
266+
],
267+
"operatorType": "StringNotBeginsWith",
268+
"key": "data.clientRequestId"
100269
}
101-
}
270+
]
271+
},
272+
"labels": [],
273+
"eventDeliverySchema": "CloudEventSchemaV1_0",
274+
"retryPolicy": {
275+
"maxDeliveryAttempts": 30,
276+
"eventTimeToLiveInMinutes": 1440
102277
}
103278
}
104-
]
279+
},
280+
{
281+
"type": "Microsoft.HealthcareApis/workspaces",
282+
"name": "[parameters('workspaceName')]",
283+
"apiVersion": "2024-03-31",
284+
"location": "[resourceGroup().location]"
285+
},
286+
{
287+
"type": "Microsoft.HealthcareApis/workspaces/dicomservices",
288+
"apiVersion": "2024-03-31",
289+
"name": "[concat(parameters('workspaceName'), '/', parameters('dicomServiceName'))]",
290+
"location": "[resourceGroup().location]",
291+
"dependsOn": [
292+
"[resourceId('Microsoft.HealthcareApis/workspaces', parameters('workspaceName'))]",
293+
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('dicomIdentityName'))]",
294+
"[resourceId('Microsoft.EventGrid/systemTopics/eventSubscriptions', parameters('systemTopicName'), parameters('eventSubscriptionName'))]"
295+
],
296+
"identity": {
297+
"type": "userAssigned",
298+
"userAssignedIdentities": {
299+
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities/', variables('dicomIdentityName'))]": {}
300+
}
301+
},
302+
"properties": {
303+
"storageConfiguration": {
304+
"storageResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]",
305+
"fileSystemName": "[parameters('fileSystemName')]",
306+
"storageIndexingConfiguration": {
307+
"storageEventQueueName": "[parameters('storageEventQueueName')]"
308+
}
309+
},
310+
"enableDataPartitions": "[parameters('enableDataPartitions')]"
311+
}
312+
}
313+
]
105314
}
106315
```
107316

@@ -111,7 +320,7 @@ Storage indexing is available starting in the preview ARM version `2025-04-01-pr
111320

112321
If there is an error when processing an event, the problematic event will be enqueued in a "poison queue" called `<queue-name>-poison` in the same Storage Account. Details about every processed event can be found in the `AHDSDicomAuditLogs` and `AHDSDicomDiagnosticLogs` tables by filtering for all logs where `OperationName = 'index-storage'`. The audit logs will only record when the operation started and completed whereas the diagnostic table will provide details about each operation including any errors, if any. Operations may be correlated across the tables using `CorrelationId`.
113322

114-
Failures are divided into two types: `User` and `Server`. User errors include
323+
Failures are divided into two types: `User` and `Server`. User errors include any problem connecting to the storage account or with the DICOM file itself, while server errors include any unexpected error that prevents processing. Unlike server errors, user errors are not retried by the DICOM service.
115324

116325
## Next Steps
117326
* [Interact with data using DICOMweb&trade;](dicomweb-standard-apis-with-dicom-services.md)

0 commit comments

Comments
 (0)