Merge pull request #25 from zeixcom/bugfix/list-lazy-activation

Add FLAG_RELINK and cascade cleanup for dynamic edges

Author: estherbrunner

ARCHITECTURE.md

@@ -77,7 +77,12 @@ Before a sink recomputes, the engine sets `activeSink = node`, ensuring all `.ge
 
 After a sink finishes recomputing, `trimSources()` removes any edges beyond `sourcesTail` — these are dependencies from the previous execution that were not accessed this time. This is how the graph adapts to conditional dependencies.
 
-`unlink()` removes an edge from the source's sink list. If the source's sink list becomes empty and the source has a `stop` callback, that callback is invoked — this is how lazy resources (Sensor, Collection, watched Store/List) are deallocated when no longer observed.
+`unlink()` removes an edge from the source's sink list. If the source's sink list becomes empty:
+
+1. **Watched cleanup**: If the source has a `stop` callback, it is invoked — this is how lazy resources (Sensor, Collection, watched Store/List) are deallocated when no longer observed.
+2. **Cascading cleanup**: If the source is also a sink (a MemoNode or TaskNode — identified by having a `sources` field), its own sources are trimmed via `trimSources()`. This recursively unlinks the node from its upstream dependencies, allowing their `stop` callbacks to fire if they also become unobserved.
+
+The cascade is critical for intermediate nodes like `deriveCollection`'s internal MemoNode: when the last effect unsubscribes from the derived collection, `unlink()` removes the effect→derived edge, which triggers cascade cleanup of the derived→List edge, which in turn fires the List's `stop` (the `watched` cleanup). Without the cascade, the List would retain a stale sink reference and never clean up its watcher. Recursion depth is bounded by graph depth since the graph is a DAG.
 
 ### Dependency Tracking Opt-Out: `untrack(fn)`
 
@@ -87,14 +92,19 @@ After a sink finishes recomputing, `trimSources()` removes any edges beyond `sou
 
 ### Flag-Based Dirty Tracking
 
-Each sink node has a `flags` field with four states:
+Each sink node has a `flags` field using a bitmap with five flags:
 
 | Flag | Value | Meaning |
 |------|-------|---------|
 | `FLAG_CLEAN` | 0 | Value is up to date |
 | `FLAG_CHECK` | 1 | A transitive dependency may have changed — verify before recomputing |
 | `FLAG_DIRTY` | 2 | A direct dependency changed — recomputation required |
 | `FLAG_RUNNING` | 4 | Currently executing (used for circular dependency detection and edge reuse) |
+| `FLAG_RELINK` | 8 | Structural change requires edge re-establishment on next read |
+
+The first four flags (`CLEAN`/`CHECK`/`DIRTY`/`RUNNING`) are used by the core graph engine in `propagate()` and `refresh()`. They are tested with bitmask operations that ignore higher bits, so `FLAG_RELINK` is invisible to the propagation and refresh machinery.
+
+`FLAG_RELINK` is used exclusively by composite signal types (Store, List, Collection, deriveCollection) that manage their own child signals. When a structural mutation adds or removes child signals, the node is flagged `FLAG_DIRTY | FLAG_RELINK`. On the next `get()`, the composite signal's fast path reads the flag: if `FLAG_RELINK` is set, it forces a tracked `refresh()` after rebuilding the value so that `recomputeMemo()` can call `link()` for new child signals and `trimSources()` for removed ones. This avoids the previous approach of nulling `node.sources`/`node.sourcesTail`, which orphaned edges in upstream sink lists. `FLAG_RELINK` is always cleared by `recomputeMemo()`, which assigns `node.flags = FLAG_RUNNING` (clearing all bits) at the start of recomputation.
 
 ### The `propagate(node)` Function
 
@@ -229,7 +239,7 @@ A reactive object where each property is its own signal. Properties are automati
 
 **Structural reactivity**: The internal `MemoNode` tracks edges from child signals to the store node. When consumers call `store.get()`, the node acts as both a source (to the consumer) and a sink (of its child signals). Changes to any child signal propagate through the store to its consumers.
 
-**Two-path access**: On first `get()`, `refresh()` executes `buildValue()` with `activeSink = storeNode`, establishing edges from each child signal to the store. Subsequent reads use a fast path: `untrack(buildValue)` rebuilds the value without re-establishing edges. Structural mutations (`add`/`remove`) call `invalidateEdges()` (nulling `node.sources`) to force re-establishment on the next read.
+**Two-path access with `FLAG_RELINK`**: On first `get()`, `refresh()` executes `buildValue()` with `activeSink = storeNode`, establishing edges from each child signal to the store. Subsequent reads use a fast path: `untrack(buildValue)` rebuilds the value without re-establishing edges. Structural mutations (`add`/`remove`/`set` with additions or removals) set `FLAG_DIRTY | FLAG_RELINK` on the node. The next `get()` detects `FLAG_RELINK` and forces a tracked `refresh()` after rebuilding the value, so `recomputeMemo()` links new child signals and trims removed ones without orphaning edges.
 
 **Diff-based updates**: `store.set(newObj)` diffs the new object against the current state, applying only the granular changes to child signals. This preserves identity of unchanged child signals and their downstream edges.
 
@@ -241,11 +251,11 @@ A reactive object where each property is its own signal. Properties are automati
 
 A reactive array with stable keys and per-item reactivity. Each item becomes a `State<T>` signal, keyed by a configurable key generation strategy (auto-increment, string prefix, or custom function).
 
-**Structural reactivity**: Uses the same `MemoNode` + `invalidateEdges` + two-path access pattern as Store. The `buildValue()` function reads all child signals in key order, establishing edges on the refresh path.
+**Structural reactivity**: Uses the same `MemoNode` + `FLAG_RELINK` + two-path access pattern as Store. The `buildValue()` function reads all child signals in key order, establishing edges on the refresh path.
 
 **Stable keys**: Keys survive sorting and reordering. `byKey(key)` returns a stable `State<T>` reference regardless of the item's current index. `sort()` reorders the keys array without destroying signals.
 
-**Diff-based updates**: `list.set(newArray)` uses `diffArrays()` to compute granular additions, changes, and removals. Changed items update their existing `State` signals; structural changes (add/remove) trigger `invalidateEdges()`.
+**Diff-based updates**: `list.set(newArray)` uses `diffArrays()` to compute granular additions, changes, and removals. Changed items update their existing `State` signals; structural changes (add/remove) set `FLAG_DIRTY | FLAG_RELINK`.
 
 ### Collection (`src/nodes/collection.ts`)
 
@@ -259,9 +269,9 @@ An externally-driven reactive collection with a watched lifecycle, mirroring `cr
 
 **Lazy lifecycle**: Like Sensor, the `watched` callback is invoked on first sink attachment. The returned cleanup is stored as `node.stop` and called when the last sink detaches (via `unlink()`). The `startWatching()` guard ensures `watched` fires before `link()` so synchronous mutations inside `watched` update `node.value` before the activating effect reads it.
 
-**External mutation via `applyChanges`**: Additions create new item signals (via configurable `createItem` factory, default `createState`). Changes update existing `State` signals. Removals delete signals and keys. Structural changes null out `node.sources` to force edge re-establishment. The node uses `equals: () => false` since structural changes are managed externally rather than detected by diffing.
+**External mutation via `applyChanges`**: Additions create new item signals (via configurable `createItem` factory, default `createState`). Changes update existing `State` signals. Removals delete signals and keys. Structural changes set `FLAG_DIRTY | FLAG_RELINK` to trigger edge re-establishment on the next read. The node uses `equals: () => false` since structural changes are managed externally rather than detected by diffing.
 
-**Two-path access**: Same pattern as Store/List — first `get()` uses `refresh()` to establish edges from child signals to the collection node; subsequent reads use `untrack(buildValue)` to avoid re-linking.
+**Two-path access with `FLAG_RELINK`**: Same pattern as Store/List — first `get()` uses `refresh()` to establish edges from child signals to the collection node; subsequent reads use `untrack(buildValue)` to avoid re-linking. When `FLAG_RELINK` is set, the next `get()` forces a tracked `refresh()` after rebuilding to link new child signals and trim removed ones.
 
 #### `deriveCollection(source, callback)` — internally derived
 
@@ -271,6 +281,18 @@ An internal factory (not exported from the public API) that creates a read-only
 
 **Consistent with Store/List/createCollection**: The `MemoNode.value` is a `T[]` (cached computed values), and keys are tracked in a separate local `string[]` variable. The `equals` function uses shallow reference equality on array elements to prevent unnecessary downstream propagation when re-evaluation produces the same item references. The node starts `FLAG_DIRTY` to ensure the first `refresh()` establishes edges.
 
-**Two-path access with structural fallback**: Same pattern as Store/List — first `get()` uses `refresh()` to establish edges; subsequent reads use `untrack(buildValue)`. A `syncKeys()` step inside `buildValue` syncs the signals map with `source.keys()`. If keys changed, `syncKeys` nulls `node.sources` to force edge re-establishment via `refresh()`, ensuring new child signals are properly linked.
+**Initialization**: Source keys are read via `untrack(() => source.keys())` to populate the signals map for direct access (`at()`, `byKey()`, `keyAt()`, `[Symbol.iterator]()`) without triggering premature `watched` activation on the upstream source. The node stays `FLAG_DIRTY` so the first `refresh()` with a real subscriber establishes proper graph edges.
+
+**Non-destructive `syncKeys()` with `FLAG_RELINK`**: Like Store/List/createCollection, `deriveCollection`'s `syncKeys()` sets `FLAG_RELINK` on the node when keys change, without touching the edge lists. This avoids orphaning edges in upstream sink lists, which would prevent the cascading cleanup in `unlink()` from reaching the source List's `watched` lifecycle. All four composite signal types now use the same `FLAG_RELINK` mechanism for structural edge invalidation.
+
+**Three-path `ensureFresh()`**: Access to the derived collection's value follows three distinct paths depending on the node's edge state:
+
+| Path | Condition | Behavior |
+|------|-----------|---------|
+| Fast path | `node.sources` exists | `untrack(buildValue)` rebuilds without re-linking. If `FLAG_RELINK` is set, forces a tracked `refresh()` to link new child signals and trim deleted ones. |
+| First subscriber | no `node.sources`, but `node.sinks` | `refresh()` via `recomputeMemo()` establishes all graph edges (source → derived, child signals → derived) in a single tracked pass. This is where `watched` activation propagates upstream. |
+| No subscriber | neither sources nor sinks | `untrack(buildValue)` computes the value without establishing edges. Keeps `FLAG_DIRTY` so the first real subscriber triggers the "first subscriber" path. Used during chained `deriveCollection` initialization. |
+
+The first-subscriber path is the key to `watched` lifecycle propagation: when an effect first reads a derived collection, `recomputeMemo()` sets `activeSink = derivedNode`, then `buildValue()` calls `source.keys()`, which triggers `subscribe()` on the upstream List with a non-null `activeSink`. This creates the List→derived edge and activates the List's `watched` callback. When the effect later disposes, the cascading cleanup in `unlink()` traverses effect→derived→List, firing the List's `stop` cleanup.
 
-**Chaining**: `.deriveCollection()` creates a new derived collection from an existing one, forming a pipeline. Each level in the chain has its own `MemoNode` for value caching and its own set of per-item derived signals.
+**Chaining**: `.deriveCollection()` creates a new derived collection from an existing one, forming a pipeline. Each level in the chain has its own `MemoNode` for value caching and its own set of per-item derived signals. The "no subscriber" path in `ensureFresh()` ensures intermediate levels don't prematurely activate upstream `watched` callbacks during construction — activation cascades through the entire chain only when the terminal effect subscribes.

CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.18.2
+
+### Fixed
+
+- **`watched` propagation through `deriveCollection()` chains**: When an effect reads a derived collection, the `watched` callback on the source List, Store, or Collection now activates correctly — even through multiple levels of `.deriveCollection()` chaining. Previously, `deriveCollection` did not propagate sink subscriptions back to the source's `watched` lifecycle.
+- **Stable `watched` lifecycle during mutations**: Adding, removing, or sorting items on a List (or Store/Collection) consumed through `deriveCollection()` no longer tears down and restarts the `watched` callback. The watcher remains active as long as at least one downstream effect is subscribed.
+- **Cleanup cascade on disposal**: When the last effect unsubscribes from a derived collection chain, cleanup now propagates upstream through all intermediate nodes to the source, correctly invoking the `watched` cleanup function.
+
+### Changed
+
+- **`FLAG_RELINK` replaces source-nulling in composite signals**: Store, List, Collection, and deriveCollection no longer null out `node.sources`/`node.sourcesTail` on structural mutations. Instead, a new `FLAG_RELINK` bitmap flag triggers a tracked `refresh()` on the next `.get()` call, re-establishing edges cleanly via `link()`/`trimSources()` without orphaning them.
+- **Cascading `trimSources()` in `unlink()`**: When a MemoNode loses all sinks, its own sources are now trimmed recursively, ensuring upstream `watched` cleanup propagates correctly through intermediate nodes.
+- **Three-path `ensureFresh()` in `deriveCollection`**: The internal freshness check now distinguishes between fast path (has sources, clean), first subscriber (has sinks but no sources yet), and no subscriber (untracked build). This prevents premature `watched` activation during initialization.
+
 ## 0.18.1
 
 ### Added

CLAUDE.md

@@ -168,6 +168,8 @@ const user = createStore({ name: 'Alice', email: 'alice@example.com' }, {
 2. Last effect stops watching → returned cleanup function executed
 3. New effect accesses signal → watched callback executed again
 
+**Watched propagation through `deriveCollection()`**: When an effect reads a derived collection, the `watched` callback on the source List, Store, or Collection activates automatically — even through multiple levels of `.deriveCollection()` chaining. The reactive graph establishes edges from effect → derived collection → source, and `watched` fires when the first edge reaches the source. Mutations (add, remove, sort) on the source do **not** tear down and restart `watched` — the watcher remains stable as long as at least one downstream effect is subscribed. When the last effect disposes, cleanup cascades upstream through all intermediate nodes.
+
 This pattern enables **lazy resource allocation** - resources are only consumed when actually needed and automatically freed when no longer used.
 
 ## Advanced Patterns and Best Practices
@@ -346,6 +348,28 @@ const display = createMemo(() => user.name.get() + user.email.get())
 4. **Async Race Conditions**: Trust automatic cancellation with AbortSignal
 5. **Circular Dependencies**: The graph detects and throws `CircularDependencyError`
 6. **Untracked `byKey()`/`at()` access**: On Store, List, and Collection, `byKey()`, `at()`, `keyAt()`, and `indexOfKey()` do **not** create graph edges. They are direct lookups that bypass structural tracking. An effect using only `collection.byKey('x')?.get()` will react to value changes of key `'x'`, but will **not** re-run if key `'x'` is added or removed. Use `get()`, `keys()`, or `length` to track structural changes.
+7. **Conditional reads delay `watched` activation**: Dependencies are tracked dynamically based on which `.get()` calls actually execute during each effect run. If a signal read is inside a branch that doesn't execute (e.g., inside the `ok` branch of `match()` while a Task is still pending), no edge is created and `watched` does not activate until that branch runs. **Fix:** read the signal eagerly before conditional logic:
+
+```typescript
+// Good: watched activates immediately, errors/nil in derived are also caught
+createEffect(() => {
+  match([task, derived], {
+    ok: ([result, values]) => renderList(values, result),
+    nil: () => showLoading(),
+  })
+})
+
+// Bad: watched only activates after task resolves
+createEffect(() => {
+  match([task], {
+    ok: ([result]) => {
+      const values = derived.get() // only tracked in this branch
+      renderList(values, result)
+    },
+    nil: () => showLoading(),
+  })
+})
+```
 
 ## Advanced Patterns
 

README.md

@@ -1,6 +1,6 @@
 # Cause & Effect
 
-Version 0.18.1
+Version 0.18.2
 
 **Cause & Effect** is a reactive state management primitives library for TypeScript. It provides the foundational building blocks for managing complex, dynamic, composite, and asynchronous state — correctly and performantly — in a unified signal graph.
 
@@ -517,6 +517,19 @@ const user = createStore({ name: 'Alice' }, {
 })
 ```
 
+**Watched propagation through `deriveCollection()`**: When an effect reads a derived collection, the `watched` callback on the source List, Store, or Collection activates automatically — even through multiple levels of chaining. Mutations on the source do not tear down the watcher. When the last effect disposes, cleanup cascades upstream through all intermediate nodes.
+
+**Tip — conditional reads delay activation**: Dependencies are tracked based on which `.get()` calls actually execute. If a signal read is inside a branch that doesn't run yet (e.g., inside `match()`'s `ok` branch while a Task is pending), `watched` won't activate until that branch executes. Read signals eagerly before conditional logic to ensure immediate activation:
+
+```js
+createEffect(() => {
+  match([task, derived], { // derived is always tracked
+    ok: ([result, values]) => renderList(values, result),
+    nil: () => showLoading(),
+  })
+})
+```
+
 Memo and Task signals also support a `watched` option, but their callback receives an `invalidate` function that marks the signal dirty and triggers recomputation:
 
 ```js

package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@zeix/cause-effect",
-	"version": "0.18.1",
+	"version": "0.18.2",
 	"author": "Esther Brunner",
 	"type": "module",
 	"main": "index.js",

src/graph.ts

@@ -161,6 +161,7 @@ const FLAG_CLEAN = 0
 const FLAG_CHECK = 1 << 0
 const FLAG_DIRTY = 1 << 1
 const FLAG_RUNNING = 1 << 2
+const FLAG_RELINK = 1 << 3
 
 /* === Module State === */
 
@@ -244,9 +245,19 @@ function unlink(edge: Edge): Edge | null {
 	if (prevSink) prevSink.nextSink = nextSink
 	else source.sinks = nextSink
 
-	if (!source.sinks && source.stop) {
-		source.stop()
-		source.stop = undefined
+	if (!source.sinks) {
+		if (source.stop) {
+			source.stop()
+			source.stop = undefined
+		}
+
+		// Cascade: if the source is also a sink (e.g. MemoNode, derived collection),
+		// trim its own sources so upstream watched callbacks can clean up
+		if ('sources' in source && source.sources) {
+			const sinkNode = source as SinkNode
+			sinkNode.sourcesTail = null
+			trimSources(sinkNode)
+		}
 	}
 
 	return nextSource
@@ -591,6 +602,7 @@ export {
 	SKIP_EQUALITY,
 	FLAG_CLEAN,
 	FLAG_DIRTY,
+	FLAG_RELINK,
 	flush,
 	link,
 	propagate,