|
| 1 | +## jsonstat-wasm v0.3.0 |
| 2 | + |
| 3 | +A second performance pass that turns the two remaining losing phases — **string |
| 4 | +parsing** and **`Transform({type:'arrobj'})`** — into wins, and makes |
| 5 | +`ds.value` allocation amortize to zero on repeated reads. |
| 6 | + |
| 7 | +v0.2.0 closed the gap with the plain-JS `jsonstat-toolkit` on the value getter |
| 8 | +and `Data()` boundary. Profiling after v0.2.0 still showed two regressions: |
| 9 | +string-input parsing did a double traversal (`JSON.parse` *then* a |
| 10 | +`serde-wasm-bindgen` property walk), and `Transform` built one |
| 11 | +`serde_json::Value` object per cell plus a full boundary re-walk. v0.3.0 |
| 12 | +eliminates both. On large (~100k-cell) datasets versus `jsonstat-toolkit`, |
| 13 | +measured in Chrome 148 (`.idea/test/bench.html`) and Node 23 / macOS: |
| 14 | + |
| 15 | +| Phase | WASM vs JS toolkit | |
| 16 | +|---|---| |
| 17 | +| `JSONstat(string)` | **~1.6–2.1× faster** (single-pass Rust `serde_json`) | |
| 18 | +| `Transform({type:'arrobj'})` | **~7–15× faster** (columnar fast path) | |
| 19 | +| `JSONstat(obj)` | **~2.3–4× slower** (irreducible — see below) | |
| 20 | +| `ds.value`, `Data()` | tied (sub-millisecond) | |
| 21 | + |
| 22 | +> **Why `JSONstat(obj)` stays slower.** When handed an already-parsed JS |
| 23 | +> object, the JS toolkit walks V8's heap directly with no serialization, while |
| 24 | +> WASM pays a `Reflect::get` per property to cross the boundary. We verified |
| 25 | +> that re-stringifying + reparsing in Rust is dataset-dependent (it wins on |
| 26 | +> sparse, loses on dense) and does **not** close the gap on any dataset. This |
| 27 | +> path cannot be made competitive without abandoning the WASM boundary. The |
| 28 | +> remedy is to feed WASM the **text** instead: `JSONstat(await |
| 29 | +> response.text())` is a ~2× win over both the toolkit and the object path. |
| 30 | +
|
| 31 | +This release has **two minor behavior changes** (caching identity and string |
| 32 | +routing) — hence the minor-version bump. The toolkit-compatible API shape is |
| 33 | +unchanged. |
| 34 | + |
| 35 | +### ⚠️ Behavior change 1: `ds.value` is cached (`ds.value === ds.value`) |
| 36 | + |
| 37 | +The `value` getter now memoizes its result. The first read still does the bulk |
| 38 | +copy (one `Float64Array` for all-numeric datasets, one `Array` for mixed); |
| 39 | +every subsequent read returns the **same instance** in O(1). |
| 40 | + |
| 41 | +- **What's new:** `ds.value === ds.value` now holds. Repeated reads are free. |
| 42 | +- **What to watch:** the returned buffer is shared. Mutating it |
| 43 | + (`ds.value[0] = 999`) will affect subsequent reads. Treat the returned |
| 44 | + `Float64Array`/`Array` as **read-only** — copy it (`ds.value.slice()`) before |
| 45 | + mutating. |
| 46 | +- **The `Float64Array`-for-numeric return type itself is unchanged from |
| 47 | + v0.2.0** (see that release's notes). Only the caching is new. |
| 48 | + |
| 49 | +### ⚠️ Behavior change 2: `{`-leading strings are parsed as inline documents |
| 50 | + |
| 51 | +`JSONstat(string)` now peeks the first non-whitespace character: |
| 52 | + |
| 53 | +- **`{`** → treated as an inline JSON-stat document and parsed by the Rust |
| 54 | + constructor in a single `serde_json` pass. (Previously every string was |
| 55 | + `JSON.parse`d first, then handed to `fromObject` — two full traversals.) |
| 56 | +- **anything else** → fetched as a URL (unchanged). |
| 57 | + |
| 58 | +This matches the toolkit's convention that object-shaped strings are documents |
| 59 | +and other strings are URLs. A malformed object (e.g. `"{not json"`) now surfaces |
| 60 | +as a clean `serde_json` error instead of a silent URL fetch. Numbers, booleans, |
| 61 | +arrays, and bare strings are not valid JSON-stat documents and still fall |
| 62 | +through to the URL path. |
| 63 | + |
| 64 | +### Columnar `Transform` fast path |
| 65 | + |
| 66 | +For `Transform({type:'arrobj'})` **without** `by` or `meta` (the common case), |
| 67 | +the output is now built via a columnar pipeline: |
| 68 | + |
| 69 | +1. **Rust emits columns**, not rows. Each dimension column is |
| 70 | + `{kind:'enum', uniques:[...labels], indices:Uint32Array}` — a packed label |
| 71 | + table plus per-row indices. The value column is `{kind:'number', |
| 72 | + data:Float64Array}` (NaN encodes absent values for sparse datasets). Mixed/ |
| 73 | + `comma`/status columns fall back to `{kind:'cells', data:Array}`. |
| 74 | +2. **A JS assembler** pre-materializes each column as a dense array, then builds |
| 75 | + the row objects with a `new Function(src)()`-JIT'd object literal whose keys |
| 76 | + are the column names. No per-cell `serde_json` tree, no per-cell map |
| 77 | + allocation, no boundary re-walk. |
| 78 | + |
| 79 | +The result is byte-identical to the serde path (verified across 11 option |
| 80 | +combinations: default, status, content:id, field:label, vlabel, drop, |
| 81 | +multi-dim, 4D, comma, and mixed). Other transform types (`array`, `object`, |
| 82 | +`objarr`, and `arrobj`/`objarr` with `by` or `meta`) use the original serde |
| 83 | +fallback unchanged — only plain `arrobj` is columnar. |
| 84 | + |
| 85 | +### Internal: `DatasetValue` typed-storage rework |
| 86 | + |
| 87 | +The value model gained a dedicated `Numbers(Vec<f64>)` variant for all-numeric |
| 88 | +dense arrays, alongside the existing `Cells(Vec<Cell>)` (mixed dense) and |
| 89 | +`Sparse` (object-keyed) variants. A custom `Deserialize` picks `Numbers` at |
| 90 | +parse time when every element is a number, so the `Vec<f64>` is built directly |
| 91 | +with no per-cell `Cell` boxing. All `get_at` call sites (`dice`, `data`, |
| 92 | +`unflatten`, `transform`, `value`) branch on the variant and take the zero-copy |
| 93 | +`as_numbers()` slice on the fast path. `Serialize` round-trips each variant to |
| 94 | +its JSON-stat wire form (`Numbers`/`Cells` → array; `Sparse` → preserved sparse |
| 95 | +object). |
| 96 | + |
| 97 | +### What's unchanged |
| 98 | + |
| 99 | +- The toolkit-compatible API (`JSONstat()`, `Data()`, `Datum()`, `Dimension()`, |
| 100 | + `Item()`, `Unflatten()`, `Transform()`, `Dice()`, `ToJSON()`) is unchanged in |
| 101 | + shape and semantics. `Transform()` output is byte-identical for every option |
| 102 | + combination. |
| 103 | +- `Data()`, `Datum()`, `Dimension()`, `Dice()`, `Unflatten()` are unaffected by |
| 104 | + the columnar change. |
| 105 | +- The release build profile (speed-first: `opt-level = 3`, `lto = "fat"`, |
| 106 | + `codegen-units = 1`) is unchanged from v0.2.0. |
| 107 | + |
| 108 | +### Verification |
| 109 | + |
| 110 | +- **60 Rust host tests pass** (was 57; +3 for `DatasetValue` variant selection, |
| 111 | + mixed-`Cells` fallback, and per-variant `Serialize` round-trip). |
| 112 | +- **11 columnar-vs-serde `Transform` byte-equivalence checks pass** (Node, |
| 113 | + `verify-columnar.mjs`). |
| 114 | +- **25 transform-type checks pass** covering `array`, `object`, `objarr`, |
| 115 | + `arrobj` (plain / `by` / `meta` / `status`), and `objarr` with `by` |
| 116 | + (`verify-transform-types.mjs`). |
| 117 | +- The benchmark harness comparing WASM against `jsonstat-toolkit` lives at |
| 118 | + [`bench-raw.mjs`](../../bench-raw.mjs) (Node, uses the `nodejs`-target glue |
| 119 | + directly) and [`.idea/test/bench.html`](../../.idea/test/bench.html) (browser). |
| 120 | + |
| 121 | +### Upgrading from v0.2.x |
| 122 | + |
| 123 | +- If you mutate `ds.value` in place, copy it first: `const v = ds.value.slice()`. |
| 124 | +- If you passed object-shaped JSON strings expecting them to be fetched as URLs, |
| 125 | + switch to a non-`{`-leading URL (this was never a supported pattern — the |
| 126 | + toolkit treats `{`-strings as documents too). |
| 127 | + |
| 128 | +--- |
| 129 | + |
| 130 | +**Install:** `npm i jsonstat-wasm@0.3.0` · **CDN:** `https://cdn.jsdelivr.net/npm/jsonstat-wasm@0.3.0/jsonstat.js` |
0 commit comments