You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/howto-configure-adlsv2-endpoint.md
+57-1Lines changed: 57 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ ms.author: patricka
6
6
ms.service: azure-iot-operations
7
7
ms.subservice: azure-data-flows
8
8
ms.topic: how-to
9
-
ms.date: 10/16/2024
9
+
ms.date: 10/27/2024
10
10
ai-usage: ai-assisted
11
11
12
12
#CustomerIntent: As an operator, I want to understand how to configure dataflow endpoints for Azure Data Lake Storage Gen2 in Azure IoT Operations so that I can send data to Azure Data Lake Storage Gen2.
@@ -37,6 +37,26 @@ Then, assign a role to the managed identity that grants permission to write to t
37
37
38
38
Finally, create the *DataflowEndpoint* resource and specify the managed identity authentication method. Replace the placeholder values like `<ENDPOINT_NAME>` with your own.
39
39
40
+
# [Portal](#tab/portal)
41
+
42
+
1. In the IoT Operations portal, select the **Dataflow endpoints** tab.
43
+
1. Under **Create new dataflow endpoint**, select **Azure Data Lake Storage (2nd generation)** > **New**.
44
+
45
+
:::image type="content" source="media/howto-configure-adlsv2-endpoint/create-adls-endpoint.png" alt-text="Screenshot using operations experience to create a new ADLS V2 dataflow endpoint.":::
| Host | The hostname of the Azure Data Lake Storage Gen2 endpoint in the format `<account>.blob.core.windows.net`. Replace the account placeholder with the endpoint account name. |
53
+
| Authentication method | The method used for authentication. Choose *System assigned managed identity*, *User assigned managed identity*, or *Access token*. |
54
+
| Client ID | The client ID of the user-assigned managed identity. Required if using *User assigned managed identity*. |
55
+
| Tenant ID | The tenant ID of the user-assigned managed identity. Required if using *User assigned managed identity*. |
56
+
| Access token secret name | The name of the Kubernetes secret containing the SAS token. Required if using *Access token*. |
57
+
58
+
1. Select **Apply** to provision the endpoint.
59
+
40
60
# [Bicep](#tab/bicep)
41
61
42
62
Create a Bicep `.bicep` file with the following content.
@@ -114,6 +134,14 @@ Follow the steps in the [access token](#access-token) section to get a SAS token
114
134
115
135
Then, create the *DataflowEndpoint* resource and specify the access token authentication method. Here, replace `<SAS_SECRET_NAME>` with name of the secret containing the SAS token as well as other placeholder values.
116
136
137
+
# [Portal](#tab/portal)
138
+
139
+
1. In the Azure IoT Operations Preview portal, create a new dataflow or edit an existing dataflow by selecting the **Dataflows** tab on the left. If creating a new dataflow, select a source for the dataflow.
140
+
1. In the editor, select the destination dataflow endpoint.
141
+
1. Choose the Azure Data Lake Storage Gen2 endpoint that you created previously.
142
+
143
+
:::image type="content" source="media/howto-configure-adlsv2-endpoint/dataflow-mq-adls.png" alt-text="Screenshot using operations experience to create a dataflow with an MQTT source and ADLS V2 destination.":::
144
+
117
145
# [Bicep](#tab/bicep)
118
146
119
147
Create a Bicep `.bicep` file with the following content.
@@ -199,6 +227,12 @@ Before creating the dataflow endpoint, assign a role to the managed identity tha
199
227
200
228
To use system-assigned managed identity, specify the managed identity authentication method in the *DataflowEndpoint* resource. In most cases, you don't need to specify other settings. Not specifying an audience creates a managed identity with the default audience scoped to your storage account.
201
229
230
+
# [Portal](#tab/portal)
231
+
232
+
In the operations experience dataflow endpoint settings page, select the **Basic** tab then choose **Authentication method** > **System assigned managed identity**.
233
+
234
+
In most cases, you don't need to specify a service audience. Not specifying an audience creates a managed identity with the default audience scoped to your storage account.
235
+
202
236
# [Bicep](#tab/bicep)
203
237
204
238
```bicep
@@ -223,6 +257,10 @@ dataLakeStorageSettings:
223
257
224
258
If you need to override the system-assigned managed identity audience, you can specify the `audience` setting.
225
259
260
+
# [Portal](#tab/portal)
261
+
262
+
In most cases, you don't need to specify a service audience. Not specifying an audience creates a managed identity with the default audience scoped to your storage account.
263
+
226
264
# [Bicep](#tab/bicep)
227
265
228
266
```bicep
@@ -274,6 +312,12 @@ You can also use the IoT Operations portal to create and manage the secret. To l
274
312
275
313
Finally, create the *DataflowEndpoint* resource with the secret reference.
276
314
315
+
# [Portal](#tab/portal)
316
+
317
+
In the operations experience dataflow endpoint settings page, select the **Basic** tab then choose **Authentication method** > **Access token**.
318
+
319
+
Enter the access token secret name you created in **Access token secret name**.
320
+
277
321
# [Bicep](#tab/bicep)
278
322
279
323
```bicep
@@ -303,6 +347,12 @@ dataLakeStorageSettings:
303
347
304
348
To use a user-assigned managed identity, specify the `UserAssignedManagedIdentity` authentication method and provide the `clientId` and `tenantId` of the managed identity.
305
349
350
+
# [Portal](#tab/portal)
351
+
352
+
In the operations experience dataflow endpoint settings page, select the **Basic** tab then choose **Authentication method** > **User assigned managed identity**.
353
+
354
+
Enter the user assigned managed identity client ID and tenant ID in the appropriate fields.
355
+
306
356
# [Bicep](#tab/bicep)
307
357
308
358
```bicep
@@ -343,6 +393,12 @@ Use the `batching` settings to configure the maximum number of messages and the
343
393
344
394
For example, to configure the maximum number of messages to 1000 and the maximum latency to 100 seconds, use the following settings:
345
395
396
+
# [Portal](#tab/portal)
397
+
398
+
In the operations experience, select the **Advanced** tab for the dataflow endpoint.
399
+
400
+
:::image type="content" source="media/howto-configure-adlsv2-endpoint/adls-advanced.png" alt-text="Screenshot using operations experience to set ADLS V2 advanced settings.":::
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/howto-configure-adx-endpoint.md
+38Lines changed: 38 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -59,6 +59,23 @@ To send data to Azure Data Explorer in Azure IoT Operations Preview, you can con
59
59
60
60
Create the dataflow endpoint resource with your cluster and database information. We suggest using the managed identity of the Azure Arc-enabled Kubernetes cluster. This approach is secure and eliminates the need for secret management. Replace the placeholder values like `<ENDPOINT_NAME>` with your own.
61
61
62
+
# [Portal](#tab/portal)
63
+
64
+
1. In the operations experience, select the **Dataflow endpoints** tab.
65
+
1. Under **Create new dataflow endpoint**, select **Azure Data Explorer** > **New**.
66
+
67
+
:::image type="content" source="media/howto-configure-adx-endpoint/create-adx-endpoint.png" alt-text="Screenshot using operations experience to create an Azure Data Explorer dataflow endpoint.":::
| Host | The hostname of the Azure Data Explorer endpoint in the format `<cluster>.<region>.kusto.windows.net`. |
75
+
| Authentication method | The method used for authentication. Choose *System assigned managed identity* or *User assigned managed identity* |
76
+
| Client ID | The client ID of the user-assigned managed identity. Required if using *User assigned managed identity*. |
77
+
| Tenant ID | The tenant ID of the user-assigned managed identity. Required if using *User assigned managed identity*. |
78
+
62
79
# [Bicep](#tab/bicep)
63
80
64
81
Create a Bicep `.bicep` file with the following content.
@@ -143,6 +160,10 @@ Before you create the dataflow endpoint, assign a role to the managed identity t
143
160
144
161
In the *DataflowEndpoint* resource, specify the managed identity authentication method. In most cases, you don't need to specify other settings. This configuration creates a managed identity with the default audience `https://api.kusto.windows.net`.
145
162
163
+
# [Portal](#tab/portal)
164
+
165
+
In the operations experience dataflow endpoint settings page, select the **Basic** tab then choose **Authentication method** > **System assigned managed identity**.
166
+
146
167
# [Bicep](#tab/bicep)
147
168
148
169
```bicep
@@ -167,6 +188,11 @@ dataExplorerSettings:
167
188
168
189
If you need to override the system-assigned managed identity audience, you can specify the `audience` setting.
169
190
191
+
192
+
# [Portal](#tab/portal)
193
+
194
+
In most cases, you don't need to specify a service audience. Not specifying an audience creates a managed identity with the default audience scoped to your storage account.
195
+
170
196
# [Bicep](#tab/bicep)
171
197
172
198
```bicep
@@ -196,6 +222,12 @@ dataExplorerSettings:
196
222
197
223
To use a user-assigned managed identity, specify the `UserAssignedManagedIdentity` authentication method and provide the `clientId` and `tenantId` of the managed identity.
198
224
225
+
# [Portal](#tab/portal)
226
+
227
+
In the operations experience dataflow endpoint settings page, select the **Basic** tab then choose **Authentication method** > **User assigned managed identity**.
228
+
229
+
Enter the user assigned managed identity client ID and tenant ID in the appropriate fields.
230
+
199
231
# [Bicep](#tab/bicep)
200
232
201
233
```bicep
@@ -236,6 +268,12 @@ Use the `batching` settings to configure the maximum number of messages and the
236
268
237
269
For example, to configure the maximum number of messages to 1000 and the maximum latency to 100 seconds, use the following settings:
238
270
271
+
# [Portal](#tab/portal)
272
+
273
+
In the operations experience, select the **Advanced** tab for the dataflow endpoint.
274
+
275
+
:::image type="content" source="media/howto-configure-adx-endpoint/adx-advanced.png" alt-text="Screenshot using operations experience to set Azure Data Explorer advanced settings.":::
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/howto-configure-fabric-endpoint.md
+46-3Lines changed: 46 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,11 +32,30 @@ To send data to Microsoft Fabric OneLake in Azure IoT Operations Preview, you ca
32
32
33
33
To configure a dataflow endpoint for Microsoft Fabric OneLake, we suggest using the managed identity of the Azure Arc-enabled Kubernetes cluster. This approach is secure and eliminates the need for secret management.
34
34
35
-
First, in Azure portal, go to the Arc-connected Kubernetes cluster and select **Settings** > **Extensions**. In the extension list, find the name of your Azure IoT Operations extension. Copy the name of the extension.
35
+
1. In Azure portal, go to the Arc-connected Kubernetes cluster and select **Settings** > **Extensions**. In the extension list, find the name of your Azure IoT Operations extension. Copy the name of the extension.
36
+
1. In the Microsoft Fabric workspace you created, select **Manage access** > **+ Add people or groups**. Search for the Azure IoT Operations Preview Arc extension by its name and select it. Select **Contributor** as the role, then select **Add**.
37
+
1. reate the *DataflowEndpoint* resource and specify the managed identity authentication method. Replace the placeholder values like `<ENDPOINT_NAME>` with your own.
36
38
37
-
Then, in the Microsoft Fabric workspace you created, select **Manage access** > **+ Add people or groups**. Search for the Azure IoT Operations Preview Arc extension by its name and select it. Select **Contributor** as the role, then select **Add**.
39
+
# [Portal](#tab/portal)
38
40
39
-
Finally, create the *DataflowEndpoint* resource and specify the managed identity authentication method. Replace the placeholder values like `<ENDPOINT_NAME>` with your own.
41
+
1. In the operations experience, select the **Dataflow endpoints** tab.
42
+
1. Under **Create new dataflow endpoint**, select **Microsoft Fabric OneLake** > **New**.
43
+
44
+
:::image type="content" source="media/howto-configure-fabric-endpoint/create-fabric-endpoint.png" alt-text="Screenshot using operation experience to create a Microsoft Fabric OneLake dataflow endpoint.":::
| Host | The hostname of the Microsoft Fabric OneLake endpoint in the format `onelake.dfs.fabric.microsoft.com`. |
51
+
| Lakehouse name | The name of the lakehouse where the data should be stored. |
52
+
| Workspace name | The name of the workspace associated with the lakehouse. |
53
+
| OneLake path type | The type of path used in OneLake. Select *Files* or *Tables*. |
54
+
| Authentication method | The method used for authentication. Choose *System assigned managed identity* or *User assigned managed identity*|
55
+
| Client ID | The client ID of the user-assigned managed identity. Required if using *User assigned managed identity*. |
56
+
| Tenant ID | The tenant ID of the user-assigned managed identity. Required if using *User assigned managed identity*. |
57
+
58
+
1. Select **Apply** to provision the endpoint.
40
59
41
60
# [Bicep](#tab/bicep)
42
61
@@ -135,6 +154,10 @@ Using the system-assigned managed identity is the recommended authentication met
135
154
136
155
In the *DataflowEndpoint* resource, specify the managed identity authentication method. In most cases, you don't need to specify other settings. This configuration creates a managed identity with the default audience.
137
156
157
+
# [Portal](#tab/portal)
158
+
159
+
In the operations experience dataflow endpoint settings page, select the **Basic** tab then choose **Authentication method** > **System assigned managed identity**.
160
+
138
161
# [Bicep](#tab/bicep)
139
162
140
163
```bicep
@@ -160,6 +183,10 @@ fabricOneLakeSettings:
160
183
161
184
If you need to override the system-assigned managed identity audience, you can specify the `audience` setting.
162
185
186
+
# [Portal](#tab/portal)
187
+
188
+
In most cases, you don't need to specify a service audience. Not specifying an audience creates a managed identity with the default audience scoped to your storage account.
189
+
163
190
# [Bicep](#tab/bicep)
164
191
165
192
```bicep
@@ -187,6 +214,12 @@ fabricOneLakeSettings:
187
214
188
215
#### User-assigned managed identity
189
216
217
+
# [Portal](#tab/portal)
218
+
219
+
In the operations experience dataflow endpoint settings page, select the **Basic** tab then choose **Authentication method** > **User assigned managed identity**.
220
+
221
+
Enter the user assigned managed identity client ID and tenant ID in the appropriate fields.
222
+
190
223
# [Bicep](#tab/bicep)
191
224
192
225
```bicep
@@ -224,6 +257,10 @@ You can set advanced settings for the Fabric OneLake endpoint, such as the batch
224
257
225
258
The `oneLakePathType` setting determines the type of path to use in the OneLake path. The default value is `Tables`, which is the recommended path type for the most common use cases. The `Tables` path type is a table in the OneLake lakehouse that is used to store the data. It can also be set as `Files`, which is a file in the OneLake lakehouse that is used to store the data. The `Files` path type is useful when you want to store the data in a file format that is not supported by the `Tables` path type.
226
259
260
+
# [Portal](#tab/portal)
261
+
262
+
The OneLake path type is set in the **Basic** tab for the dataflow endpoint.
263
+
227
264
# [Bicep](#tab/bicep)
228
265
229
266
```bicep
@@ -252,6 +289,12 @@ Use the `batching` settings to configure the maximum number of messages and the
252
289
253
290
For example, to configure the maximum number of messages to 1000 and the maximum latency to 100 seconds, use the following settings:
254
291
292
+
# [Portal](#tab/portal)
293
+
294
+
In the operations experience, select the **Advanced** tab for the dataflow endpoint.
295
+
296
+
:::image type="content" source="media/howto-configure-fabric-endpoint/fabric-advanced.png" alt-text="Screenshot using operations experience to set Microsoft Fabric advanced settings.":::
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/howto-configure-local-storage-endpoint.md
+16Lines changed: 16 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,6 +28,22 @@ To send data to local storage in Azure IoT Operations Preview, you can configure
28
28
29
29
Use the local storage option to send data to a locally available persistent volume, through which you can upload data via Azure Container Storage enabled by Azure Arc edge volumes.
30
30
31
+
# [Portal](#tab/portal)
32
+
33
+
1. In the operations experience, select the **Dataflow endpoints** tab.
34
+
1. Under **Create new dataflow endpoint**, select **Local Storage** > **New**.
35
+
36
+
:::image type="content" source="media/howto-configure-local-storage-endpoint/create-local-storage-endpoint.png" alt-text="Screenshot using operations experience to create a Local Storage dataflow endpoint.":::
0 commit comments