You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add comprehensive WASM development guide and enhance data flow graph documentation
- Add new howto-develop-wasm-modules.md with complete development guide
- Cover both WASM module and graph definition development
- Include tabbed sections for Rust and Python development paths
- Add detailed explanations of timely dataflow architecture
- Provide comprehensive SDK reference and build instructions
- Explain graph definition structure and registry deployment
- Update howto-dataflow-graph-wasm.md with better architectural explanation
- Follow Microsoft Writing Style Guide with appropriate bullet point usage
@@ -25,10 +25,11 @@ Azure IoT Operations data flow graphs support WebAssembly (WASM) modules for cus
25
25
- Deploy an Azure IoT Operations instance on an Arc-enabled Kubernetes cluster. For more information, see [Deploy Azure IoT Operations](../deploy-iot-ops/howto-deploy-iot-operations.md).
26
26
- Use Azure Container Registry (ACR) to store WASM modules and graphs.
27
27
- Install the ORAS CLI to push WASM modules to the registry.
28
+
- To develop custom WASM modules, see [Develop WebAssembly modules for data flow graphs](howto-develop-wasm-modules.md).
28
29
29
30
## Overview
30
31
31
-
WebAssembly (WASM) modules in Azure IoT Operations data flow graphs let you process data at the edge with high performance and security. WASM runs in a sandboxed environment and supports programming languages such as Rust, C++, and AssemblyScript.
32
+
WebAssembly (WASM) modules in Azure IoT Operations data flow graphs let you process data at the edge with high performance and security. WASM runs in a sandboxed environment and supports Rustand Python.
32
33
33
34
### How WASM data flow graphs work
34
35
@@ -51,7 +52,7 @@ These examples show how to set up and deploy WASM data flow graphs for common sc
51
52
52
53
Azure IoT Operations needs a container registry to pull WASM modules and graph definitions. You can use Azure Container Registry (ACR) or another OCI-compatible registry.
53
54
54
-
To create and configure an Azure Container Registry, see [Deploy Azure Container Registry](). <!-- TODO -->
55
+
To create and configure an Azure Container Registry, see [Deploy Azure Container Registry](/azure/container-registry/container-registry-get-started-portal).
55
56
56
57
### Install ORAS CLI
57
58
@@ -63,40 +64,37 @@ For this preview, use prebuilt sample modules:
> Update the ACR references in your data flow deployments if you use a different registry from the sample modules.
99
-
100
98
### Create a registry endpoint
101
99
102
100
A registry endpoint defines the connection to your container registry. Data flow graphs use registry endpoints to pull WASM modules and graph definitions from container registries. For detailed information about configuring registry endpoints with different authentication methods and registry types, see [Configure registry endpoints](howto-configure-registry-endpoint.md).
@@ -212,12 +210,58 @@ For detailed instructions, see [Assign Azure roles using the Azure portal](/azur
212
210
213
211
## Example 1: Basic deployment with one WASM module
214
212
215
-
This scenario shows a simple data flow that uses a WASM module to convert temperature data from Fahrenheit to Celsius. The source code for this module is available [here](). <!--PLACEHOLDER--> Instead of building the module yourself, use the precompiled version that's already pushed to the ACR as `graph-simple:1.0.0` in earlier steps.
213
+
This example converts temperature data from Fahrenheit to Celsius using a WASM module. The [temperature module source code](https://github.com/Azure-Samples/explore-iot-operations/tree/wasm/samples/wasm/operators/temperature) is available on GitHub. Use the precompiled version `graph-simple:1.0.0` that you pushed to your container registry.
214
+
215
+
### How it works
216
+
217
+
The [graph definition](https://github.com/Azure-Samples/explore-iot-operations/blob/wasm/samples/wasm/graph-simple.yaml) creates a simple three-stage pipeline:
218
+
219
+
1.**Source**: Receives temperature data from MQTT
220
+
2.**Map**: Processes data with the temperature WASM module
221
+
3.**Sink**: Sends converted data back to MQTT
216
222
217
-
<!-- TODO: Add simple graph YAML definition and explanation -->
223
+
The graph references the temperature module:
224
+
225
+
```yaml
226
+
operations:
227
+
- operationType: "map"
228
+
name: "module-temperature/map"
229
+
module: "temperature:1.0.0"
230
+
```
231
+
232
+
The [temperature module](https://github.com/Azure-Samples/explore-iot-operations/blob/wasm/samples/wasm/operators/temperature/src/lib.rs) converts Fahrenheit to Celsius using the standard formula `(F - 32) × 5/9 = C`:
233
+
234
+
```rust
235
+
if measurement.unit == MeasurementTemperatureUnit::Fahrenheit {
The following configuration creates a data flow graph that uses this temperature conversion pipeline. The graph references the `graph-simple:1.0.0` artifact, which contains the YAML definition and pulls the temperature module from your container registry.
218
252
219
253
### Configure the data flow graph
220
254
255
+
This configuration defines three nodes that implement the temperature conversion workflow: a source node that subscribes to incoming temperature data, a graph processing node that runs the WASM module, and a destination node that publishes the converted results.
256
+
257
+
The data flow graph resource "wraps" the graph definition artifact and connects its abstract source/sink operations to concrete endpoints:
258
+
259
+
- The graph definition's `source` operation connects to the data flow's source node (MQTT topic)
260
+
- The graph definition's `sink` operation connects to the data flow's destination node (MQTT topic)
261
+
- The graph definition's processing operations run within the graph processing node
262
+
263
+
This separation allows the same graph definition to be deployed with different endpoints across environments while keeping the processing logic unchanged.
This scenario shows a more complex data flow graph that processes multiple data sources and includes advanced processing modules.
479
+
This example demonstrates a sophisticated data processing workflow that handles multiple data types including temperature, humidity, and image data. The [complex graph definition](https://github.com/Azure-Samples/explore-iot-operations/blob/wasm/samples/wasm/graph-complex.yaml) orchestrates multiple WASM modules to perform advanced analytics and object detection.
480
+
481
+
### How it works
482
+
483
+
The complex graph processes three data streams and combines them into enriched sensor analytics:
484
+
485
+
1. Temperature processing: Converts Fahrenheit to Celsius, filters invalid readings, and calculates statistics
486
+
2. Humidity processing: Accumulates humidity measurements over time intervals
487
+
3. Image processing: Performs object detection on camera snapshots and formats results
488
+
489
+
The graph uses specialized modules from the [operators directory](https://github.com/Azure-Samples/explore-iot-operations/tree/wasm/samples/wasm/operators):
490
+
491
+
- Window module: Delays data for time-based processing
492
+
- Temperature modules: Handle conversion, filtering, and statistical analysis
493
+
- Humidity module: Processes environmental data
494
+
- Snapshot modules: Manage image data routing and object detection
495
+
- Format module: Prepares images for processing
496
+
- Collection module: Aggregates multi-sensor data
497
+
- Enrichment module: Adds metadata and overtemperature alerts
498
+
499
+
The following diagram shows the data flow through the various processing modules:
436
500
437
501
<!-- TODO: Add complex graph YAML definition and explanation -->
As shown in the diagram, data flows from a single source through multiple processing stages:
540
+
541
+
- The window module delays incoming data for time-based processing
542
+
- Branch operations route data based on content type (sensor data vs. snapshots)
543
+
- Temperature data undergoes conversion, filtering, and statistical accumulation
544
+
- Humidity data gets accumulated with statistical analysis
545
+
- Image data gets formatted and processed through object detection
546
+
- All processed data streams merge at the concatenate operation
547
+
- The collection module aggregates the multi-sensor results
548
+
- The enrichment module adds final metadata and alerts before output
549
+
550
+
Branch operations enable parallel processing of different sensor inputs, allowing the graph to handle multiple data types efficiently within a single workflow.
551
+
439
552
### Configure the complex data flow graph
440
553
554
+
This configuration implements the multi-sensor processing workflow using the `graph-complex:1.0.0` artifact. Notice how the data flow graph deployment is similar to Example 1 - both use the same three-node pattern (source, graph processor, destination) even though the processing logic is completely different.
555
+
556
+
This similarity occurs because the data flow graph resource acts as a host environment that loads and executes graph definitions. The actual processing logic resides in the graph definition artifact (`graph-simple:1.0.0` vs `graph-complex:1.0.0`), which contains the YAML specification of operations and connections between WASM modules. The data flow graph resource provides the runtime infrastructure to pull the artifact, instantiate the modules, and route data through the defined workflow.
> The VS Code extension for WASM module development is currently in private preview. Contact your Microsoft representative for access.
765
+
To create custom data processing logic for your data flow graphs, develop WebAssembly modules in Rust or Python. Custom modules enable you to implement specialized business logic, data transformations, and analytics that aren't available in the built-in operators.
766
+
767
+
For comprehensive development guidance including:
768
+
- Setting up your development environment
769
+
- Creating operators in Rust and Python
770
+
- Understanding the data model and interfaces
771
+
- Building and testing your modules
772
+
- Configuring graph definitions with parameters
656
773
657
-
<!-- TODO: Add WASM module development overview -- ### Use WASM operators -->
0 commit comments