You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/howto-dataflow-graph-webassembly.md
+30-30Lines changed: 30 additions & 30 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
-
title: Use WebAssembly (WASM) with dataflow graphs (Preview)
3
-
description: Learn how to deploy and use WebAssembly modules with dataflow graphs in Azure IoT Operations to process data at the edge.
2
+
title: Use WebAssembly (WASM) with data flow graphs (Preview)
3
+
description: Learn how to deploy and use WebAssembly modules with data flow graphs in Azure IoT Operations to process data at the edge.
4
4
author: PatAltimore
5
5
ms.author: patricka
6
6
ms.service: azure-iot-operations
@@ -18,7 +18,7 @@ ai-usage: ai-assisted
18
18
>
19
19
> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.
20
20
21
-
Azure IoT Operations dataflow graphs support WebAssembly (WASM) modules for custom data processing at the edge. This capability allows you to deploy custom business logic and data transformations as part of your dataflow pipelines.
21
+
Azure IoT Operations data flow graphs support WebAssembly (WASM) modules for custom data processing at the edge. This capability allows you to deploy custom business logic and data transformations as part of your data flow pipelines.
22
22
23
23
## Prerequisites
24
24
@@ -28,11 +28,11 @@ Azure IoT Operations dataflow graphs support WebAssembly (WASM) modules for cust
28
28
29
29
## Overview
30
30
31
-
WebAssembly (WASM) modules in Azure IoT Operations dataflow graphs enable custom data processing at the edge with high performance and security. WASM provides a sandboxed execution environment that supports multiple programming languages including Rust, C++, and AssemblyScript.
31
+
WebAssembly (WASM) modules in Azure IoT Operations data flow graphs enable custom data processing at the edge with high performance and security. WASM provides a sandboxed execution environment that supports multiple programming languages including Rust, C++, and AssemblyScript.
32
32
33
-
### How WASM dataflow graphs work
33
+
### How WASM data flow graphs work
34
34
35
-
The WASM dataflow implementation follows this workflow:
35
+
The WASM data flow implementation follows this workflow:
36
36
37
37
1.**Develop WASM modules**: Write custom processing logic in supported languages and compile to WebAssembly Component Model format
38
38
1.**Develop graph definition**: Define how data flows through the modules using YAML configuration files
@@ -51,7 +51,7 @@ The following examples demonstrate how to set up and deploy WASM data flow graph
51
51
52
52
Azure IoT Operations requires access to a container registry to pull WASM modules and graph definitions. You can use either Azure Container Registry (ACR) or another OCI-compatible registry.
53
53
54
-
To create and configure an Azure Container Registry, see [Deploy Azure Container Registry](). <TODO>
54
+
To create and configure an Azure Container Registry, see [Deploy Azure Container Registry](). <!--TODO-->
55
55
56
56
### Install ORAS CLI
57
57
@@ -212,7 +212,7 @@ For detailed instructions, see [Assign Azure roles using the Azure portal](/azur
212
212
213
213
## Example 1: Basic deployment with one WASM module
214
214
215
-
This scenario demonstrates a simple data flow that uses a WASM module to convert temperature data from Fahrenheit to Celsius. The source code for this module is available [here](). <PLACEHOLDER> Instead of building the module yourself, we use the precompiled version that has already been pushed to the ACR as `graph-simple:1.0.0` in the earlier steps.
215
+
This scenario demonstrates a simple data flow that uses a WASM module to convert temperature data from Fahrenheit to Celsius. The source code for this module is available [here](). <!--PLACEHOLDER--> Instead of building the module yourself, we use the precompiled version that has already been pushed to the ACR as `graph-simple:1.0.0` in the earlier steps.
216
216
217
217
<!-- TODO: Add simple graph YAML definition and explanation -->
To test the dataflow, you need to send MQTT messages from within the cluster. First, deploy the MQTT client pod by following the instructions in [Test connectivity to MQTT broker with MQTT clients](../manage-mqtt-broker/howto-test-connection.md). The MQTT client provides the necessary authentication tokens and certificates to connect to the broker.
367
+
To test the data flow, you need to send MQTT messages from within the cluster. First, deploy the MQTT client pod by following the instructions in [Test connectivity to MQTT broker with MQTT clients](../manage-mqtt-broker/howto-test-connection.md). The MQTT client provides the necessary authentication tokens and certificates to connect to the broker.
368
368
369
369
After deploying the MQTT client pod, open two terminal sessions and connect to the pod:
370
370
@@ -584,7 +584,7 @@ spec:
584
584
585
585
---
586
586
587
-
### Test the complex dataflow
587
+
### Test the complex data flow
588
588
589
589
Use the same MQTT client pod setup from the previous example. Connect to the pod and run the following commands from within it.
590
590
@@ -666,21 +666,21 @@ done
666
666
667
667
## Configuration reference
668
668
669
-
This section provides detailed information about configuring dataflow graphs with WASM modules. It covers all configuration options, dataflow endpoints, and advanced settings.
669
+
This section provides detailed information about configuring data flow graphs with WASM modules. It covers all configuration options, data flow endpoints, and advanced settings.
670
670
671
-
### Dataflow graph overview
671
+
### Data flow graph overview
672
672
673
-
A dataflow graph defines how data flows through WebAssembly modules for processing. Each graph consists of:
673
+
A data flow graph defines how data flows through WebAssembly modules for processing. Each graph consists of:
674
674
675
675
- **Mode**: Controls whether the graph is enabled or disabled
676
-
- **Profile reference**: Links to a dataflow profile that defines scaling and resource settings
676
+
- **Profile reference**: Links to a data flow profile that defines scaling and resource settings
677
677
- **Disk persistence**: Optionally enables persistent storage for graph state
678
678
- **Nodes**: Define the source, processing, and destination components
679
679
- **Node connections**: Specify how data flows between nodes
680
680
681
681
### Mode configuration
682
682
683
-
The mode property determines whether the dataflow graph is actively processing data. You can set the mode to `Enabled` or `Disabled` (case-insensitive). When disabled, the graph stops processing data but retains its configuration.
683
+
The mode property determines whether the data flow graph is actively processing data. You can set the mode to `Enabled` or `Disabled` (case-insensitive). When disabled, the graph stops processing data but retains its configuration.
684
684
685
685
# [Bicep](#tab/bicep)
686
686
@@ -711,11 +711,11 @@ spec:
711
711
712
712
### Profile reference
713
713
714
-
The profile reference connects your dataflow graph to a dataflow profile, which defines scaling settings, instance counts, and resource limits. If you don't specify a profile reference, you must use a Kubernetes owner reference instead. Most scenarios use the default profile provided by Azure IoT Operations.
714
+
The profile reference connects your data flow graph to a data flow profile, which defines scaling settings, instance counts, and resource limits. If you don't specify a profile reference, you must use a Kubernetes owner reference instead. Most scenarios use the default profile provided by Azure IoT Operations.
715
715
716
716
# [Bicep](#tab/bicep)
717
717
718
-
In Bicep, you specify the profile by creating the dataflow graph as a child resource of the profile:
718
+
In Bicep, you specify the profile by creating the data flow graph as a child resource of the profile:
Disk persistence allows dataflow graphs to maintain state across restarts. When enabled, the graph can recover processing state if pods are restarted. This is useful for stateful processing scenarios where losing intermediate data would be problematic.
749
+
Disk persistence allows data flow graphs to maintain state across restarts. When enabled, the graph can recover processing state if pods are restarted. This is useful for stateful processing scenarios where losing intermediate data would be problematic.
750
750
751
751
The setting accepts `Enabled` or `Disabled` (case-insensitive), with `Disabled` as the default.
752
752
@@ -779,17 +779,17 @@ spec:
779
779
780
780
### Node configuration
781
781
782
-
Nodes are the building blocks of a dataflow graph. Each node has a unique name within the graph and performs a specific function. There are three types of nodes:
782
+
Nodes are the building blocks of a data flow graph. Each node has a unique name within the graph and performs a specific function. There are three types of nodes:
783
783
784
784
#### Source nodes
785
785
786
-
Source nodes define where data enters the graph. They connect to dataflow endpoints that receive data from MQTT brokers, Kafka topics, or other messaging systems. Each source node must specify:
786
+
Source nodes define where data enters the graph. They connect to data flow endpoints that receive data from MQTT brokers, Kafka topics, or other messaging systems. Each source node must specify:
787
787
788
-
- **Endpoint reference**: Points to a configured dataflow endpoint
788
+
- **Endpoint reference**: Points to a configured data flow endpoint
789
789
- **Data sources**: List of MQTT topics or Kafka topics to subscribe to
790
790
- **Asset reference** (optional): Links to an Azure Device Registry asset for schema inference
791
791
792
-
The data sources array allows you to subscribe to multiple topics without modifying the endpoint configuration. This flexibility enables endpoint reuse across different dataflows.
792
+
The data sources array allows you to subscribe to multiple topics without modifying the endpoint configuration. This flexibility enables endpoint reuse across different data flows.
793
793
794
794
# [Bicep](#tab/bicep)
795
795
@@ -894,9 +894,9 @@ The configuration key-value pairs are passed to the WASM module at runtime. The
894
894
895
895
#### Destination nodes
896
896
897
-
Destination nodes define where processed data is sent. They connect to dataflow endpoints that send data to MQTT brokers, cloud storage, or other systems. Each destination node specifies:
897
+
Destination nodes define where processed data is sent. They connect to data flow endpoints that send data to MQTT brokers, cloud storage, or other systems. Each destination node specifies:
898
898
899
-
- **Endpoint reference**: Points to a configured dataflow endpoint
899
+
- **Endpoint reference**: Points to a configured data flow endpoint
900
900
- **Data destination**: The specific topic, path, or location for output data
901
901
- **Output schema settings** (optional): Defines serialization format and schema validation
902
902
@@ -986,9 +986,9 @@ nodeConnections:
986
986
987
987
---
988
988
989
-
### Dataflow endpoints
989
+
### Data flow endpoints
990
990
991
-
Dataflow graphs connect to external systems through dataflow endpoints. The type of endpoint determines whether it can be used as a source, destination, or both:
991
+
Data flow graphs connect to external systems through data flow endpoints. The type of endpoint determines whether it can be used as a source, destination, or both:
992
992
993
993
#### MQTT endpoints
994
994
@@ -998,7 +998,7 @@ MQTT endpoints can serve as both sources and destinations. They connect to MQTT
998
998
- **Azure Event Grid MQTT**
999
999
- **Custom MQTT brokers**
1000
1000
1001
-
For detailed configuration information, see [Configure MQTT dataflow endpoints](howto-configure-mqtt-endpoint.md).
1001
+
For detailed configuration information, see [Configure MQTT data flow endpoints](howto-configure-mqtt-endpoint.md).
1002
1002
1003
1003
#### Kafka endpoints
1004
1004
@@ -1008,7 +1008,7 @@ Kafka endpoints can serve as both sources and destinations. They connect to Kafk
1008
1008
- **Apache Kafka clusters**
1009
1009
- **Confluent Cloud**
1010
1010
1011
-
For detailed configuration information, see [Configure Azure Event Hubs and Kafka dataflow endpoints](howto-configure-kafka-endpoint.md).
1011
+
For detailed configuration information, see [Configure Azure Event Hubs and Kafka data flow endpoints](howto-configure-kafka-endpoint.md).
1012
1012
1013
1013
#### Storage endpoints
1014
1014
@@ -1022,7 +1022,7 @@ Storage endpoints typically require output schema settings to define data serial
1022
1022
1023
1023
#### Registry endpoints
1024
1024
1025
-
Registry endpoints provide access to container registries for pulling WASM modules and graph definitions. They're not used directly in dataflow but are referenced by graph processing nodes.
1025
+
Registry endpoints provide access to container registries for pulling WASM modules and graph definitions. They're not used directly in data flow but are referenced by graph processing nodes.
1026
1026
1027
1027
For detailed configuration information, see [Configure registry endpoints](howto-configure-registry-endpoint.md).
0 commit comments