Skip to content

Commit fe72f7a

Browse files
jlianjlian
authored
Sample code for WASM modules
1 parent becc548 commit fe72f7a

File tree

34 files changed

+4921
-0
lines changed

34 files changed

+4921
-0
lines changed

‎samples/wasm/graph-complex.yaml‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
moduleRequirements:
2+
apiVersion: "0.2.0"
3+
hostlibVersion: "0.2.0"
4+
5+
moduleConfigurations:
6+
- name: module-temperature/map
7+
parameters:
8+
key1:
9+
name: key2
10+
description: key2
11+
- name: module-snapshot/branch
12+
parameters:
13+
snapshot_topic:
14+
name: snapshot_topic
15+
description: Transform app snapshot_topic in snapshot branch's init routine
16+
operations:
17+
- operationType: "source"
18+
name: "source"
19+
20+
- operationType: delay
21+
name: module-window/delay
22+
module: window:1.0.0
23+
- operationType: "map"
24+
name: "module-format/map"
25+
module: "format:1.0.0"
26+
- operationType: map
27+
name: module-snapshot/map
28+
module: snapshot:1.0.0
29+
- operationType: branch
30+
name: module-snapshot/branch
31+
module: snapshot:1.0.0
32+
- operationType: accumulate
33+
name: module-snapshot/accumulate
34+
module: snapshot:1.0.0
35+
- operationType: map
36+
name: module-temperature/map
37+
module: temperature:1.0.0
38+
- operationType: branch
39+
name: module-temperature/branch
40+
module: temperature:1.0.0
41+
- operationType: filter
42+
name: module-temperature/filter
43+
module: temperature:1.0.0
44+
- operationType: accumulate
45+
name: module-temperature/accumulate
46+
module: temperature:1.0.0
47+
- operationType: accumulate
48+
name: module-humidity/accumulate
49+
module: humidity:1.0.0
50+
- operationType: concatenate
51+
name: concatenate1
52+
module:
53+
- operationType: accumulate
54+
name: module-collection/accumulate
55+
module: collection:1.0.0
56+
- operationType: map
57+
name: module-enrichment/map
58+
module: enrichment:1.0.0
59+
60+
- operationType: "sink"
61+
name: "sink"
62+
63+
connections:
64+
- from:
65+
name: source
66+
to:
67+
name: module-window/delay
68+
69+
- from:
70+
name: module-window/delay
71+
to:
72+
name: module-snapshot/branch
73+
74+
- from:
75+
name: module-snapshot/branch
76+
arm: "False"
77+
to:
78+
name: module-temperature/branch
79+
80+
- from:
81+
name: module-snapshot/branch
82+
arm: "True"
83+
to:
84+
name: module-format/map
85+
86+
- from:
87+
name: module-format/map
88+
to:
89+
name: module-snapshot/map
90+
91+
- from:
92+
name: module-snapshot/map
93+
to:
94+
name: module-snapshot/accumulate
95+
96+
- from:
97+
name: module-snapshot/accumulate
98+
to:
99+
name: concatenate1
100+
101+
- from:
102+
name: module-temperature/branch
103+
arm: "True"
104+
to:
105+
name: module-temperature/map
106+
107+
- from:
108+
name: module-temperature/branch
109+
arm: "False"
110+
to:
111+
name: module-humidity/accumulate
112+
113+
- from:
114+
name: module-humidity/accumulate
115+
to:
116+
name: concatenate1
117+
118+
- from:
119+
name: module-temperature/map
120+
to:
121+
name: module-temperature/filter
122+
123+
- from:
124+
name: module-temperature/filter
125+
to:
126+
name: module-temperature/accumulate
127+
128+
- from:
129+
name: module-temperature/accumulate
130+
to:
131+
name: concatenate1
132+
133+
- from:
134+
name: concatenate1
135+
to:
136+
name: module-collection/accumulate
137+
138+
- from:
139+
name: module-collection/accumulate
140+
to:
141+
name: module-enrichment/map
142+
143+
- from:
144+
name: module-enrichment/map
145+
to:
146+
name: sink

‎samples/wasm/graph-simple.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
moduleRequirements:
2+
apiVersion: "0.2.0"
3+
hostlibVersion: "0.2.0"
4+
5+
moduleConfigurations:
6+
- name: module-temperature/map
7+
parameters:
8+
key1:
9+
name: key2
10+
description: key2
11+
operations:
12+
- operationType: "source"
13+
name: "source"
14+
15+
- operationType: "map"
16+
name: "module-temperature/map"
17+
module: "temperature:1.0.0"
18+
19+
- operationType: "sink"
20+
name: "sink"
21+
22+
connections:
23+
- from:
24+
name: "source"
25+
to:
26+
name: "module-temperature/map"
27+
28+
- from:
29+
name: "module-temperature/map"
30+
to:
31+
name: "sink"

‎samples/wasm/operators/README.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Sample WASM modules
2+
3+
WebAssembly example test modules.
4+
5+
Following are the example modules that can be used in the graph. See the [example graph](../docs/example-graph.md) for a complete example.
6+
7+
## collection
8+
9+
1. **collection/accumulate**: accumulate all sensor data, including temperature, humidity and object detection result.
10+
11+
## enrichment
12+
13+
1. **enrichment/map**: enrich existing temperature by checking and adding/removing over temperature flag status.
14+
15+
## humidity
16+
17+
1. **humidity/accumulate**: accumulate and process all incoming humidity data within a specific time interval.
18+
19+
## snapshot
20+
21+
1. **snapshot/branch**: check and determine current data to be processed through snapshot or non-snapshot routine.
22+
2. **snapshot/map**: convert a single snapshot to a detected object with the highest probability.\
23+
3. **snapshot/accumulate**: accumulate and process all incoming snapshot data within a specific time interval.
24+
25+
## temperature
26+
27+
1. **temperature/map**: convert temperature data value from Fahrenheit to Celsius if necessary.\
28+
2. **temperature/branch**: check and determine current data to be processed through temperature or humidity routine.
29+
3. **temperature/filter**: filter temperature data value which exceeds the pre-configured limitations.\
30+
4. **temperature/accumulate**: accumulate and process all incoming temperature data within a specific time interval.
31+
32+
## window
33+
34+
1. **window/delay**: delay all incoming data to a specific time interval.
35+
36+
## format
37+
38+
1. **format/map**: decode and rescale the snapshot image to the specific format required by snapshot object detection.

‎samples/wasm/operators/collection/Cargo.toml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
[package]
2+
name = "collection"
3+
version = "0.0.2"
4+
authors = ["TinyKube Devs"]
5+
license = "MIT"
6+
edition = "2021"
7+
8+
[dependencies]
9+
wit-bindgen = "0.22"
10+
tinykube_wasm_sdk = { version = "0.2.0", registry="azure-vscode-tinykube" }
11+
12+
serde = { version = "1", features = [
13+
"derive",
14+
"rc",
15+
] }
16+
serde_json = { version = "1.0", default-features = false, features = [
17+
"alloc",
18+
] }
19+
20+
[lib]
21+
crate-type = ["cdylib"]
22+
path = "src/lib.rs"

‎samples/wasm/operators/collection/Dockerfile‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
ARG IMAGE=tinykube-wasm-rust-build
2+
FROM $IMAGE AS operator-build
3+
4+
ARG APP_NAME
5+
ARG BUILD_MODE="release"
6+
7+
ENV CARGO_REGISTRIES_AZURE_VSCODE_TINYKUBE_INDEX="sparse+https://pkgs.dev.azure.com/azure-iot-sdks/iot-operations/_packaging/preview/Cargo/index/"
8+
ENV CARGO_NET_GIT_FETCH_WITH_CLI=true
9+
10+
WORKDIR /src
11+
COPY ./Cargo.toml ./Cargo.toml
12+
COPY ./src ./src
13+
14+
RUN if [ "${BUILD_MODE}" = "release" ]; then \
15+
cargo build --release --target wasm32-wasip2; \
16+
else \
17+
cargo build --target wasm32-wasip2; \
18+
fi
19+
20+
FROM scratch
21+
ARG BUILD_MODE
22+
ARG APP_NAME
23+
COPY --from=operator-build "/src/target/wasm32-wasip2/${BUILD_MODE}/${APP_NAME}.wasm" "${APP_NAME}.wasm"
24+
ENTRYPOINT [ "${APP_NAME}.wasm" ]

‎samples/wasm/operators/collection/README.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Operator Builder
2+
3+
This project provides a Docker-based environment for building WebAssembly Dataflow Operators.
4+
5+
## Prerequisites
6+
7+
- Docker must be installed on your machine.
8+
- GitHub token (for private repository access) — make sure to set up a GitHub token in your Docker build secrets.
9+
10+
## Files
11+
12+
- **Dockerfile**: Defines the Rust-based build process for Dataflow Operators. The Dockerfile is based off of an image that our extension creates titled `tinykube-wasm-rust-build`, which contains:
13+
- The official `rust` image based on Alpine Linux.
14+
- Dependencies like `clang`, `lld`, `musl-dev`, `git`, `perl`, `make`, and `cmake`.
15+
- Configured build environment by adding the necessary Rust, WASM, and architecture targets.
16+
17+
The dockerfile in your source folder is specific to each Dataflow Operator, and you can make customizations to this image to customize how each specific Operator is built.
18+
19+
20+
## Setup
21+
22+
### Configure your GitHub token
23+
24+
For accessing private GitHub repositories, you'll need to provide a GitHub token. **Read-only** permissions on a fine-grained access token should suffice.
25+
26+
Set the `GITHUB_TOKEN` environment variable in your shell or in the `.env` file (if you are using one).
27+
28+
To set the environment variable in your shell, you can use the following command:
29+
30+
```bash
31+
export GITHUB_TOKEN=your_personal_access_token_here
32+
```
33+
34+
### Output
35+
36+
The built `.wasm` files will be placed in the following directory structure:
37+
38+
`bin/<ARCH>/<BUILD_MODE>/`
39+
40+
41+
For example, if you use the default architecture (`x86_64`) and build mode (`release`), the `.wasm` file will be located in:
42+
43+
`bin/x86_64/release/`
44+
45+
## Troubleshooting
46+
47+
- Ensure Docker is installed and running on your machine.
48+
- If you encounter permission errors with the GitHub token, check the file path and permissions for the token.

0 commit comments

Comments
 (0)