Skip to content

Commit d2323fd

Browse files
authored
Fix Python Docker builder and reorganize README build sections
- Fixed Python Docker builder to use flexible filename matching local method - Changed from hardcoded ${APP_TYPE}.py to ${APP_NAME}.py pattern - Both Docker and local builds now expect same naming convention - Reorganized README to present local builds first, then Docker builds - Added clear guidance on when to use each build approach - Removed 'Recommended' labels for neutral presentation - Made both Rust and Python sections consistent in structure
1 parent 2e87dbe commit d2323fd

File tree

2 files changed

+99
-24
lines changed

2 files changed

+99
-24
lines changed

samples/wasm/README.md

Lines changed: 87 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,22 @@ This ensures:
108108
- Add WASM target: `rustup target add wasm32-wasip2`
109109
- **Build tools**: `cargo install wasm-tools --version '=1.201.0' --locked`
110110

111+
### Configure Azure IoT Operations Registry
112+
The WASM Rust SDK is available through a custom Azure DevOps registry. Configure access by setting these environment variables:
113+
114+
```bash
115+
export CARGO_REGISTRIES_AZURE_VSCODE_TINYKUBE_INDEX="sparse+https://pkgs.dev.azure.com/azure-iot-sdks/iot-operations/_packaging/preview/Cargo/index/"
116+
export CARGO_NET_GIT_FETCH_WITH_CLI=true
117+
```
118+
119+
Add these to your shell profile (`.bashrc`, `.zshrc`, etc.) for persistent access:
120+
121+
```bash
122+
echo 'export CARGO_REGISTRIES_AZURE_VSCODE_TINYKUBE_INDEX="sparse+https://pkgs.dev.azure.com/azure-iot-sdks/iot-operations/_packaging/preview/Cargo/index/"' >> ~/.bashrc
123+
echo 'export CARGO_NET_GIT_FETCH_WITH_CLI=true' >> ~/.bashrc
124+
source ~/.bashrc
125+
```
126+
111127
For comprehensive documentation of the WASM Rust SDK including APIs for state store, metrics, and logging, see the [WASM Rust SDK Reference](#wasm-rust-sdk-reference).
112128

113129
### Create your operator project
@@ -124,6 +140,7 @@ version = "0.1.0"
124140
edition = "2021"
125141

126142
[dependencies]
143+
wit-bindgen = "0.22"
127144
tinykube_wasm_sdk = { version = "0.2.0", registry = "azure-vscode-tinykube" }
128145
serde = { version = "1", default-features = false, features = ["derive"] }
129146
serde_json = { version = "1", default-features = false, features = ["alloc"] }
@@ -137,10 +154,12 @@ crate-type = ["cdylib"]
137154
// src/lib.rs
138155
use tinykube_wasm_sdk::logger::{self, Level};
139156
use tinykube_wasm_sdk::macros::map_operator;
140-
use tinykube_wasm_sdk::{DataModel, ModuleConfiguration};
141157
use serde_json::{json, Value};
142158

143-
fn temperature_converter_init(configuration: ModuleConfiguration) -> bool {
159+
// Import the generated types from wit-bindgen
160+
use crate::tinykube_graph::processor::types::{DataModel, ModuleConfiguration, BufferOrBytes};
161+
162+
fn temperature_converter_init(_configuration: ModuleConfiguration) -> bool {
144163
logger::log(Level::Info, "temperature-converter", "Init invoked");
145164
true
146165
}
@@ -162,7 +181,7 @@ fn temperature_converter(input: DataModel) -> DataModel {
162181
});
163182

164183
if let Ok(output_str) = serde_json::to_string(&data) {
165-
result.payload.write(output_str.as_bytes());
184+
result.payload = BufferOrBytes::Bytes(output_str.into_bytes());
166185
}
167186
}
168187
}
@@ -178,18 +197,33 @@ For comprehensive examples of map, filter, branch, accumulate, and delay operato
178197
- **Timely operators**: [Timely dataflow documentation](https://docs.rs/timely/latest/timely/dataflow/operators/index.html)
179198

180199
### Build your module
200+
201+
#### Local Build
202+
203+
Build directly on your development machine using the Rust toolchain:
204+
205+
**Prerequisites**: Ensure you have configured the Azure IoT Operations registry environment variables (see [Configure Azure IoT Operations Registry](#configure-azure-iot-operations-registry) above) before building locally.
206+
181207
```bash
182208
# Build WASM module
183209
cargo build --release --target wasm32-wasip2
184210

185-
# Find your module
211+
# Find your module
186212
ls target/wasm32-wasip2/release/*.wasm
187213
file target/wasm32-wasip2/release/temperature_converter.wasm
188214
```
189215

190-
#### Alternative: Docker Builder
216+
The wit-bindgen dependency will automatically generate the required type bindings during build.
191217

192-
For simplified development without local toolchain setup, use the streamlined Docker builder:
218+
**Use local builds when you:**
219+
- Want fastest iteration during development
220+
- Need full control over the build environment
221+
- Are debugging build issues or dependencies
222+
- Prefer working with familiar Rust tooling
223+
224+
#### Docker Build
225+
226+
Build using a containerized environment with all dependencies pre-installed:
193227

194228
```bash
195229
# Build release version (default)
@@ -201,6 +235,12 @@ docker run --rm -v "$(pwd):/workspace" ghcr.io/azure-samples/explore-iot-operati
201235

202236
Output will be placed in `bin/<ARCH>/<BUILD_MODE>/temperature-converter.wasm`. For complete Docker builder documentation, see the [Rust README](rust/README.md).
203237

238+
**Use Docker builds when you:**
239+
- Want consistent builds across different development environments
240+
- Don't want to install and configure the Rust toolchain locally
241+
- Are building in CI/CD pipelines
242+
- Need to ensure reproducible builds
243+
204244
## Python Development
205245

206246
### Prerequisites
@@ -221,9 +261,15 @@ class Map(exports.Map):
221261
return True
222262

223263
def process(self, message: types.DataModel) -> types.DataModel:
264+
# Ensure the input is of the expected type
265+
if not isinstance(message, types.DataModel_Message):
266+
imports.logger.log(imports.logger.Level.ERROR, "temperature-converter", "Unexpected input type")
267+
return message
268+
224269
# Extract and decode the payload
225270
buffer = message.value.payload.value
226-
data_str = buffer.decode('utf-8')
271+
payload = buffer.read()
272+
data_str = payload.decode('utf-8')
227273

228274
try:
229275
data = json.loads(data_str)
@@ -242,43 +288,64 @@ class Map(exports.Map):
242288
output_str = json.dumps(output)
243289
output_bytes = output_str.encode('utf-8')
244290

245-
return types.DataModel_Message(
246-
content_type=message.value.content_type,
247-
payload=types.DataModel_MessagePayload_Bytes(output_bytes)
248-
)
291+
# Update the message payload
292+
message.value.payload = types.BufferOrBytes_Bytes(value=output_bytes)
249293

250-
return message # Pass through if no temperature found
294+
return message # Return the modified message
251295

252296
except Exception as e:
253297
imports.logger.log(imports.logger.Level.ERROR, "temperature-converter", f"Error: {e}")
254298
return message
255299
```
256300

257301
### Build your module
302+
303+
#### Local Build
304+
305+
Build directly on your development machine using componentize-py:
306+
258307
```bash
259308
# Generate Python bindings from schema
260-
componentize-py -d ../../schema/ -w map-impl bindings ./
309+
componentize-py -d /path/to/schema/ -w map-impl bindings ./
261310

262311
# Build WASM module
263-
componentize-py -d ../../schema/ -w map-impl componentize temperature_converter -o temperature_converter.wasm
312+
componentize-py -d /path/to/schema/ -w map-impl componentize temperature_converter -o temperature_converter.wasm
264313

265314
# Verify build
266315
file temperature_converter.wasm # Should show: WebAssembly (wasm) binary module
267316
```
268317

269-
#### Alternative: Docker Builder
318+
**Note**: Replace `/path/to/schema/` with the actual path to the schema directory. For example:
319+
- If you're in the main `wasm/` directory: `samples/wasm/python/schema/`
320+
- If you're in `samples/wasm/python/examples/`: `../../schema/`
321+
322+
**Use local builds when you:**
323+
- Want fastest iteration during development
324+
- Need to debug Python code or binding generation
325+
- Prefer working with familiar Python tooling
326+
- Want to customize the build process
270327

271-
For simplified development without local toolchain setup, use the streamlined Docker builder:
328+
#### Docker Build
329+
330+
Build using a containerized environment with all dependencies and schema paths pre-configured:
272331

273332
```bash
274-
# Build release version (requires --app-type for Python)
275-
docker run --rm -v "$(pwd):/workspace" ghcr.io/azure-samples/explore-iot-operations/python-wasm-builder --app-name temperature-converter --app-type map
333+
# Build release version (app-name should match your Python filename without .py extension)
334+
docker run --rm -v "$(pwd):/workspace" ghcr.io/azure-samples/explore-iot-operations/python-wasm-builder --app-name temperature_converter --app-type map
276335

277336
# Build debug version with symbols
278-
docker run --rm -v "$(pwd):/workspace" ghcr.io/azure-samples/explore-iot-operations/python-wasm-builder --app-name temperature-converter --app-type map --build-mode debug
337+
docker run --rm -v "$(pwd):/workspace" ghcr.io/azure-samples/explore-iot-operations/python-wasm-builder --app-name temperature_converter --app-type map --build-mode debug
279338
```
280339

281-
Output will be placed in `bin/<ARCH>/<BUILD_MODE>/temperature-converter.wasm`. For complete Docker builder documentation, see the [Python README](python/README.md).
340+
Output will be placed in `bin/<ARCH>/<BUILD_MODE>/temperature_converter.wasm`. The Docker builder expects your Python file to be named `${APP_NAME}.py` (e.g., `temperature_converter.py` for `--app-name temperature_converter`), matching the local development pattern.
341+
342+
For complete Docker builder documentation, see the [Python README](python/README.md).
343+
344+
**Use Docker builds when you:**
345+
- Want consistent builds across different development environments
346+
- Don't want to install componentize-py and its dependencies locally
347+
- Need automatic schema path resolution
348+
- Are building in CI/CD pipelines or want reproducible builds
282349

283350
### Implementing operators
284351

samples/wasm/python/Dockerfile

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,15 @@ if [ -z "$APP_TYPE" ]; then
8585
exit 1
8686
fi
8787

88+
# Validate that the Python file exists (expects ${APP_NAME}.py)
89+
if [ ! -f "/workspace/${APP_NAME}.py" ]; then
90+
echo "Error: Python file '${APP_NAME}.py' not found in workspace"
91+
echo "Make sure the file exists and matches the app name"
92+
exit 1
93+
fi
94+
8895
echo "Building Python WASM module: $APP_NAME (type: $APP_TYPE, mode: $BUILD_MODE, arch: $ARCH)"
96+
echo "Using Python file: ${APP_NAME}.py"
8997

9098
# Clean up any existing bindings
9199
rm -rf "${APP_TYPE}_impl"
@@ -100,11 +108,11 @@ mkdir -p "/workspace/bin/$ARCH/$BUILD_MODE"
100108
# Build the WASM module
101109
echo "Building WASM component..."
102110
if [ "$BUILD_MODE" = "release" ]; then
103-
componentize-py -d /schema/ -w "${APP_TYPE}-impl" componentize "${APP_TYPE}" -o "/workspace/bin/$ARCH/$BUILD_MODE/${APP_NAME}.wasm"
111+
componentize-py -d /schema/ -w "${APP_TYPE}-impl" componentize "$APP_NAME" -o "/workspace/bin/$ARCH/$BUILD_MODE/${APP_NAME}.wasm"
104112
elif [ "$BUILD_MODE" = "debug" ]; then
105-
python /tools/inject_pdb.py "${APP_TYPE}.py"
106-
componentize-py -d /schema/ -w "${APP_TYPE}-impl" componentize "${APP_TYPE}_debug" -o "/workspace/bin/$ARCH/$BUILD_MODE/${APP_NAME}.wasm"
107-
cp "${APP_TYPE}_debug.py" "/workspace/bin/$ARCH/$BUILD_MODE/${APP_NAME}_debug.py"
113+
python /tools/inject_pdb.py "${APP_NAME}.py"
114+
componentize-py -d /schema/ -w "${APP_TYPE}-impl" componentize "${APP_NAME}_debug" -o "/workspace/bin/$ARCH/$BUILD_MODE/${APP_NAME}.wasm"
115+
cp "${APP_NAME}_debug.py" "/workspace/bin/$ARCH/$BUILD_MODE/${APP_NAME}_debug.py"
108116
else
109117
echo "Invalid BUILD_MODE: $BUILD_MODE. Use 'release' or 'debug'."
110118
exit 1

0 commit comments

Comments
 (0)