|
| 1 | +# 4. Centralized Layout Management with CRDT |
| 2 | + |
| 3 | +Date: 2025-08-27 |
| 4 | + |
| 5 | +## Status |
| 6 | + |
| 7 | +Proposed |
| 8 | + |
| 9 | +## Context |
| 10 | + |
| 11 | +ComfyUI's node graph editor currently suffers from fundamental architectural limitations around spatial data management that prevent us from achieving key product goals. |
| 12 | + |
| 13 | +### Current Architecture Problems |
| 14 | + |
| 15 | +The existing system allows each node to directly mutate its position within LiteGraph's canvas renderer. This creates several critical issues: |
| 16 | + |
| 17 | +1. **Performance Bottlenecks**: UI updates require full graph traversals to detect position changes. Large workflows (100+ nodes) can create bottlenecks during interactions due to this O(n) polling approach. |
| 18 | + |
| 19 | +2. **Position Conflicts**: Multiple systems (LiteGraph canvas, DOMwidgets.ts overlays) currently compete to control node positions. Future Vue widget overlays will compound this maintenance burden. |
| 20 | + |
| 21 | +3. **No Collaboration Foundation**: Direct position mutations make concurrent editing impossible—there's no mechanism to merge conflicting position updates from multiple users. |
| 22 | + |
| 23 | +4. **Renderer Lock-in**: Spatial data is tightly coupled to LiteGraph's canvas implementation, preventing alternative rendering approaches (WebGL, DOM, other libraries, hybrid approaches). |
| 24 | + |
| 25 | +5. **Inefficient Change Detection**: While LiteGraph provides some events, many operations require polling via changeTracker.ts. The current undo/redo system performs expensive diffs on every interaction rather than using reactive push/pull signals, creating performance bottlenecks and blocking efficient animations and viewport culling. |
| 26 | + |
| 27 | + This represents a fundamental architectural limitation: diff-based systems scale O(n) with graph complexity (traverse entire structure to detect changes), while signal-based reactive systems scale O(1) with actual changes (data mutations automatically notify subscribers). Modern frameworks (Vue 3, Angular signals, SolidJS) have moved to reactive approaches for precisely this performance reason. |
| 28 | + |
| 29 | +### Business Context |
| 30 | + |
| 31 | +- Performance issues emerge with workflow complexity (100+ nodes) |
| 32 | +- The AI workflow community increasingly expects collaborative features (similar to Figma, Miro) |
| 33 | +- Accessibility requirements will necessitate DOM-based rendering options |
| 34 | +- Technical debt compounds with each new spatial feature |
| 35 | + |
| 36 | +This decision builds on [ADR-0001 (Merge LiteGraph)](0001-merge-litegraph-into-frontend.md), which enables the architectural restructuring proposed here. |
| 37 | + |
| 38 | +## Decision |
| 39 | + |
| 40 | +We will implement a centralized layout management system using CRDT (Conflict-free Replicated Data Types) with command pattern architecture to separate spatial data from rendering behavior. |
| 41 | + |
| 42 | +### Centralized State Management Foundation |
| 43 | + |
| 44 | +This solution applies proven centralized state management patterns: |
| 45 | + |
| 46 | +- **Centralized Store**: All spatial data (position, size, bounds, transform) managed in a single CRDT-backed store |
| 47 | +- **Command Interface**: All mutations flow through explicit commands rather than direct property access |
| 48 | +- **Observer Pattern**: Independent systems (rendering, interaction, layout) subscribe to state changes |
| 49 | +- **Domain Separation**: Layout logic completely separated from rendering and UI concerns |
| 50 | + |
| 51 | +This provides single source of truth, predictable state updates, and natural system decoupling—solving our core architectural problems. |
| 52 | + |
| 53 | +### Core Architecture |
| 54 | + |
| 55 | +1. **Centralized Layout Store**: A Yjs CRDT maintains all spatial data in a single authoritative store: |
| 56 | + ```typescript |
| 57 | + // Instead of: node.position = {x, y} |
| 58 | + layoutStore.moveNode(nodeId, {x, y}) |
| 59 | + ``` |
| 60 | + |
| 61 | +2. **Command Pattern**: All spatial mutations flow through explicit commands: |
| 62 | + ``` |
| 63 | + User Input → Commands → Layout Store → Observer Notifications → Renderers |
| 64 | + ``` |
| 65 | + |
| 66 | +3. **Observer-Based Systems**: Multiple independent systems subscribe to layout changes: |
| 67 | + - **Rendering Systems**: LiteGraph canvas, WebGL, DOM accessibility renderers |
| 68 | + - **Interaction Systems**: Drag handlers, selection, hover states |
| 69 | + - **Layout Systems**: Auto-layout, alignment, distribution |
| 70 | + - **Animation Systems**: Smooth transitions, physics simulations |
| 71 | + |
| 72 | +4. **Reactive Updates**: Store changes propagate through observers, eliminating polling and enabling efficient system coordination. |
| 73 | + |
| 74 | +### Implementation Strategy |
| 75 | + |
| 76 | +**Phase 1: Parallel System** |
| 77 | +- Build CRDT layout store alongside existing system |
| 78 | +- Layout store initially mirrors LiteGraph changes via observers |
| 79 | +- Gradually migrate user interactions to use command interface |
| 80 | +- Maintain full backward compatibility |
| 81 | + |
| 82 | +**Phase 2: Inversion of Control** |
| 83 | +- CRDT store becomes single source of truth |
| 84 | +- LiteGraph receives position updates via reactive subscriptions |
| 85 | +- Enable alternative renderers and advanced features |
| 86 | + |
| 87 | +### Why Centralized State + CRDT? |
| 88 | + |
| 89 | +This combination provides both architectural and technical benefits: |
| 90 | + |
| 91 | +**Centralized State Benefits:** |
| 92 | +- **Single Source of Truth**: All layout data managed in one place, eliminating conflicts |
| 93 | +- **System Decoupling**: Rendering, interaction, and layout systems operate independently |
| 94 | +- **Predictable Updates**: Clear data flow makes debugging and testing easier |
| 95 | +- **Extensibility**: Easy to add new layout behaviors without modifying existing systems |
| 96 | + |
| 97 | +**CRDT Benefits:** |
| 98 | +- **Conflict Resolution**: Automatic merging eliminates position conflicts between systems |
| 99 | +- **Collaboration-Ready**: Built-in support for multi-user editing |
| 100 | +- **Eventual Consistency**: Guaranteed convergence to same state across all clients |
| 101 | + |
| 102 | +**Yjs-Specific Benefits:** |
| 103 | +- **Event-Driven**: Native observer pattern removes need for polling |
| 104 | +- **Selective Updates**: Only changed nodes trigger system updates |
| 105 | +- **Fine-Grained Changes**: Efficient delta synchronization |
| 106 | + |
| 107 | +## Consequences |
| 108 | + |
| 109 | +### Positive |
| 110 | + |
| 111 | +- **Eliminates Polling**: Observer pattern removes O(n) graph traversals, improving performance |
| 112 | +- **System Modularity**: Independent systems can be developed, tested, and optimized separately |
| 113 | +- **Renderer Flexibility**: Easy to add WebGL, DOM accessibility, or hybrid rendering systems |
| 114 | +- **Rich Interactions**: Command pattern enables robust undo/redo, macros, and interaction history |
| 115 | +- **Collaboration-Ready**: CRDT foundation enables real-time multi-user editing |
| 116 | +- **Conflict Resolution**: Eliminates position "snap-back" behavior between competing systems |
| 117 | +- **Better Developer Experience**: Clear separation of concerns and predictable data flow patterns |
| 118 | + |
| 119 | +### Negative |
| 120 | + |
| 121 | +- **Learning Curve**: Team must understand CRDT concepts and centralized state management |
| 122 | +- **Migration Complexity**: Gradual migration of existing direct property access requires careful coordination |
| 123 | +- **Memory Overhead**: Yjs library (~30KB) plus operation history storage |
| 124 | +- **CRDT Performance**: CRDTs have computational overhead compared to direct property access |
| 125 | +- **Increased Abstraction**: Additional layer between user interactions and visual updates |
| 126 | + |
| 127 | +### Risk Mitigations |
| 128 | + |
| 129 | +- Provide comprehensive migration documentation and examples |
| 130 | +- Build compatibility layer for gradual, low-risk migration |
| 131 | +- Implement operation history pruning for long-running sessions |
| 132 | +- Phase implementation to validate approach before full migration |
| 133 | + |
| 134 | +## Notes |
| 135 | + |
| 136 | +This centralized state + CRDT architecture follows patterns from modern collaborative applications: |
| 137 | + |
| 138 | +**Centralized State Management**: Similar to Redux/Vuex patterns in complex web applications, but with CRDT backing for collaboration. This provides predictable state updates while enabling real-time multi-user features. |
| 139 | + |
| 140 | +**CRDT in Collaboration**: Tools like Figma, Linear, and Notion use similar approaches for real-time collaboration, demonstrating the effectiveness of separating authoritative data from presentation logic. |
| 141 | + |
| 142 | +**Future Capabilities**: This foundation enables advanced features that would be difficult with the current architecture: |
| 143 | +- Macro recording and workflow automation |
| 144 | +- Programmatic layout optimization and constraints |
| 145 | +- API-driven workflow construction |
| 146 | +- Multiple simultaneous renderers (canvas + accessibility DOM) |
| 147 | +- Real-time collaborative editing |
| 148 | +- Advanced spatial features (physics, animations, auto-layout) |
| 149 | + |
| 150 | +The architecture provides immediate single-user benefits while creating infrastructure for collaborative and advanced spatial features. |
| 151 | + |
| 152 | +## References |
| 153 | + |
| 154 | +- [Yjs Documentation](https://docs.yjs.dev/) |
| 155 | +- [CRDTs: The Hard Parts](https://martin.kleppmann.com/2020/07/06/crdt-hard-parts-hydra.html) by Martin Kleppmann |
| 156 | +- [Figma's Multiplayer Technology](https://www.figma.com/blog/how-figmas-multiplayer-technology-works/) |
0 commit comments