feat(instrument): Add --heap-trace flag for stable memory-free profiling (#107)

* feat(instrument): Add --heap-trace flag for stable memory-free profiling

This adds a new `--heap-trace` flag to the `instrument` command that stores
profiling traces in WASM linear memory instead of stable memory.

## Problem

The current `instrument` command assumes exclusive use of stable memory for
storing profiling logs. Canisters built with Emscripten, Idris2, AssemblyScript,
or other toolchains that use stable memory for their own purposes will trap
when calling `__get_profiling` because it attempts to read incompatible memory
structures.

## Solution

The `--heap-trace` flag enables profiling while avoiding stable memory entirely:

- Allocates a dedicated trace buffer at the end of WASM linear memory
- Uses `memory.grow` to extend the heap by `--heap-pages` (default: 64 pages = 4MB)
- Stores function entry/exit traces in the heap buffer
- `__get_profiling` reads from heap memory instead of stable memory
- Skips pre_upgrade/post_upgrade persistence (traces don't survive upgrades)

## Usage

```bash
ic-wasm input.wasm -o output.wasm instrument --heap-trace
ic-wasm input.wasm -o output.wasm instrument --heap-trace --heap-pages 128
```

## Trade-offs

- Traces do not persist across canister upgrades
- Buffer size is limited by available heap memory
- Detailed function traces are preserved (unlike --heap-only in PR #106)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(heap-trace): Increase memory maximum to allow memory.grow

When using --heap-trace, the WASM memory maximum must be increased
to accommodate the trace buffer. Without this fix, memory.grow fails
silently when the module's max equals its initial size.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: Update instrument documentation to reflect heap-trace and stub-wasi features

- Restructure Instrument section to show execution tracing with two modes
  (stable memory and heap memory) and WASI compatibility as separate capabilities
- Add dedicated "Heap-based tracing" subsection with usage and key differences
- Update usage examples to include --heap-trace and --heap-pages flags
- Consolidate CHANGELOG entries into single enhancement with sub-points

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Linwei Shang <linwei.shang@dfinity.org>

Author: 3 people

CHANGELOG.md

@@ -6,7 +6,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [unreleased]
 
-* Add `--stub-wasi` flag to `instrument` subcommand to replace WASI imports with stub functions.
+* Enhance `instrument` subcommand with new capabilities:
+  - Add `--stub-wasi` flag to replace WASI imports with stub functions, enabling modules compiled with Emscripten or wasi-sdk to run on the Internet Computer.
+  - Add `--heap-trace` flag to store execution traces in heap memory instead of stable memory, enabling profiling for canisters that use stable memory for their own purposes (e.g., Emscripten, Idris2, AssemblyScript-based canisters).
 
 ## [0.9.9] - 2025-11-18
 

README.md

@@ -127,10 +127,12 @@ Usage: `ic-wasm <input.wasm> check-endpoints [--candid <file>] [--hidden <file>]
 ### Instrument (experimental)
 
 Provides instrumentation capabilities for canister WebAssembly modules:
-- **Execution tracing**: Instrument canister methods to emit execution trace to stable memory for performance profiling
+- **Execution tracing**: Instrument canister methods to emit execution trace for performance profiling
+  - Stable memory mode (default): Stores traces in stable memory, persists across upgrades
+  - Heap memory mode (`--heap-trace`): Stores traces in heap memory, for canisters that use stable memory for their own purposes
 - **WASI compatibility**: Replace WASI imports with stub functions to enable modules compiled with Emscripten or wasi-sdk to run on the Internet Computer
 
-Usage: `ic-wasm <input.wasm> -o <output.wasm> instrument [--trace-only func1] [--start-page 16] [--page-limit 30] [--stub-wasi]`
+Usage: `ic-wasm <input.wasm> -o <output.wasm> instrument [--trace-only func1] [--start-page 16] [--page-limit 30] [--heap-trace] [--heap-pages 64] [--stub-wasi]`
 
 #### Execution tracing
 
@@ -207,6 +209,21 @@ fn post_upgrade() {
 * We cannot measure query calls.
 * No concurrent calls.
 
+#### Heap-based tracing
+
+The `--heap-trace` flag provides an alternative tracing mode that stores execution traces in heap memory instead of stable memory. This is useful for canisters that use stable memory for their own purposes (e.g., Emscripten, Idris2, or AssemblyScript-based canisters).
+
+**Usage**: `ic-wasm <input.wasm> -o <output.wasm> instrument --heap-trace [--heap-pages 64]`
+
+**Key differences from stable memory tracing**:
+- Traces are stored in WASM linear memory and **will not persist across upgrades**
+- No need to pre-allocate stable memory or specify `--start-page`
+- The `--heap-pages` flag controls buffer size (default: 64 pages = 4MB)
+- Memory maximum is automatically increased to accommodate the trace buffer
+- Cannot be used together with `--start-page` or `--page-limit`
+
+The same query endpoints (`__get_profiling`, `__get_cycles`, etc.) work with both stable and heap-based tracing.
+
 #### Stubbing WASI imports
 
 The `--stub-wasi` flag replaces WASI imports with local stub functions, enabling WASM modules compiled with Emscripten or wasi-sdk to run on the Internet Computer. Without this flag, such modules would fail at install time with errors like:

src/bin/main.rs

@@ -101,6 +101,16 @@ enum SubCommand {
         /// The number of pages of the preallocated stable memory
         #[clap(short, long, requires("start_page"))]
         page_limit: Option<i32>,
+        /// Store profiling traces in heap memory instead of stable memory.
+        /// Use this for canisters that use stable memory for their own purposes
+        /// (e.g., Emscripten, Idris2, AssemblyScript-based canisters).
+        /// Traces are stored in WASM linear memory and will not persist across upgrades.
+        #[clap(long, conflicts_with_all = &["start_page", "page_limit"])]
+        heap_trace: bool,
+        /// Number of WASM pages (64KB each) to allocate for heap trace buffer.
+        /// Only used with --heap-trace. Default: 64 (4MB total)
+        #[clap(long, default_value = "64", requires = "heap_trace")]
+        heap_pages: i32,
         /// Replace WASI imports with stub functions that return 0 (success)
         #[clap(long)]
         stub_wasi: bool,
@@ -167,13 +177,17 @@ fn main() -> anyhow::Result<()> {
             trace_only,
             start_page,
             page_limit,
+            heap_trace,
+            heap_pages,
             stub_wasi,
         } => {
             use ic_wasm::instrumentation::{instrument, Config};
             let config = Config {
                 trace_only_funcs: trace_only.clone().unwrap_or(vec![]),
                 start_address: start_page.map(|page| i64::from(page) * 65536),
                 page_limit: *page_limit,
+                heap_trace: *heap_trace,
+                heap_pages: *heap_pages,
                 stub_wasi: *stub_wasi,
             };
             instrument(&mut m, config).map_err(|e| anyhow::anyhow!("{e}"))?;