Skip to content

Commit 83f30fe

Browse files
Added some details on how to download the dataflow
1 parent 30fb9d5 commit 83f30fe

File tree

1 file changed

+29
-8
lines changed

1 file changed

+29
-8
lines changed

articles/iot-operations/connect-to-cloud/howto-create-dataflow.md

Lines changed: 29 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,15 @@ To create a dataflow in the operations experience portal, select **Dataflow** >
6060

6161
# [Bicep](#tab/bicep)
6262

63+
This Bicep template file from [Bicep File to create Dataflow](https://github.com/Azure-Samples/explore-iot-operations/blob/main/samples/quickstarts/dataflow.bicep) deploys the necessary resources for dataflows.
64+
65+
1. Download the file to your local, and make sure to replace the values for `customLocationName`, `aioInstanceName`, `schemaRegistryName`, `opcuaSchemaName`, and `persistentVCName`.
66+
67+
1. Next, deploy the resources using the [az stack group](/azure/azure-resource-manager/bicep/deployment-stacks?tabs=azure-powershell) command in your terminal:
68+
69+
```azurecli
70+
az stack group create --name MyDeploymentStack --resource-group $RESOURCE_GROUP --template-file /workspaces/explore-iot-operations/<filename>.bicep --action-on-unmanage 'deleteResources' --deny-settings-mode 'none' --yes
71+
```
6372
The overall structure of a dataflow configuration for Bicep is as follows:
6473

6574
```bicep
@@ -171,10 +180,10 @@ Configuring an asset as a source is only available in the operations experience
171180

172181
# [Bicep](#tab/bicep)
173182

174-
To configure a source using an MQTT endpoint, use the following configuration:
183+
The MQTT endpoint is configured as a source in the Bicep template file, using the following configuration
175184

176185
```bicep
177-
{
186+
{
178187
operationType: 'Source'
179188
sourceSettings: {
180189
endpointRef: defaultDataflowEndpoint.name
@@ -187,11 +196,12 @@ To configure a source using an MQTT endpoint, use the following configuration:
187196

188197
Datasources allow you to specify multiple MQTT or Kafka topics without needing to modify the endpoint configuration. This means the same endpoint can be reused across multiple dataflows, even if the topics vary. To learn more, see [Reuse dataflow endpoints](./howto-configure-dataflow-endpoint.md#reuse-endpoints).
189198

190-
#### Specify schema to deserialize data
199+
<!-- TODO: Put the right article link here -->
200+
For more information about creating an MQTT endpoint as a dataflow source, see [MQTT Endpoint](concept-schema-registry.md).
191201

192-
If the source data has optional fields or fields with different types, specify a deserialization schema to ensure consistency. For example, the data might have fields that aren't present in all messages. Without the schema, the transformation can't handle these fields as they would have empty values. With the schema, you can specify default values or ignore the fields.
202+
#### Specify schema to deserialize data
193203

194-
The following configuration demonstrates how to define a schema in your Bicep file. This schema will ensure proper deserialization of asset data. In this example, the schema defines fields such as `asset_id`, `asset_name`, `location`, `temperature`, `manufacturer`, `production_date`, and `serial_number`. Each field is assigned a specific data type (e.g., `string`) and marked as non-nullable. This ensures all incoming messages contain these fields with valid data. Such structure maintains consistency and enables the system to handle structured input more reliably.
204+
Schemas are documents that describe the format of a message and its contents to enable processing and contextualization. You can upload schemas using the ARM/Bicep templates. The following configuration demonstrates how to define a schema in your Bicep file. In this example, the schema defines fields such as `asset_id`, `asset_name`, `location`, `temperature`, `manufacturer`, `production_date`, and `serial_number`. Each field is assigned a specific data type (e.g., `string`) and marked as non-nullable. This ensures all incoming messages contain these fields with valid data.
195205

196206
```bicep
197207
var assetDeltaSchema = '''
@@ -254,9 +264,6 @@ Once the schema is registered, it can be referenced in transformations to ensure
254264
}
255265
```
256266

257-
> [!NOTE]
258-
> The only supported serialization format is Delta or Parquet. The schema is optional.
259-
260267
For more information about schema registry, see [Understand message schemas](concept-schema-registry.md).
261268

262269
# [Kubernetes](#tab/kubernetes)
@@ -341,6 +348,20 @@ In the operations experience portal, select **Dataflow** > **Add transform (opti
341348

342349
# [Bicep](#tab/bicep)
343350

351+
```bicep
352+
{
353+
operationType: 'BuiltInTransformation'
354+
builtInTransformationSettings: {
355+
map: [
356+
// ...
357+
]
358+
filter: [
359+
// ...
360+
]
361+
}
362+
}
363+
```
364+
344365
# [Kubernetes](#tab/kubernetes)
345366

346367
```yaml

0 commit comments

Comments
 (0)