Unified Positronic Content Architecture - Cache-first State Management#255
Unified Positronic Content Architecture - Cache-first State Management#255
Conversation
Sets up Vite alongside existing esbuild for gradual migration: New files: - vite.config.ts - Vite configuration with path aliases - src/vite-entry.ts - Reactive primitives (signal, effect, computed) - index.html - Demo showcasing old vs new patterns New npm scripts: - npm run dev:vite - Vite dev server on port 9100 - npm run build:vite - Vite production build Reactive patterns demonstrated: - signal() - Observable values - effect() - Auto-run on signal changes - createWidgetStore() - Centralized widget state - bindText/bindClass() - Reactive DOM bindings This enables incremental widget migration: 1. Old widgets keep working with esbuild 2. New/migrated widgets use Vite patterns 3. Both run together during transition 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Git was storing .sh files as 100644 (non-executable) instead of 100755. This caused `chmod +x` to be needed after every merge/checkout. Fixed 30+ shell scripts in: - scripts/ - workers/ - backups/ - commands/ - system/genome/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Migrate 12 widget stylesheets from plain CSS to SCSS: - buttons, status-view (simple widgets) - room-list, user-list (sidebar widgets) - continuum-widget, main-panel, sidebar-panel (layout) - continuum-emoter, continuum-metrics, cognition-histogram (status) - theme-widget (controls) Benefits: - Use shared variables from widgets/shared/styles/variables - Nested selectors for cleaner organization - Generate both .css and .styles.ts via compile-sass.ts - Foundation for reactive Positron widget system Note: chat-widget.css kept as plain CSS (SCSS conversion caused layout issues - will revisit with more careful conversion) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Performance improvements: - Enable Vite minification (2.8MB → 2.6MB, 657KB gzipped) - Update HTML template to use ES module loading - Add waitForJtag() for module load synchronization - Export jtag to globalThis at module load time Code splitting was attempted but caused circular dependency issues (Cannot access 'Xe' before initialization). Keeping single bundle with minification as the safe optimization for now. Note: Future optimization could split widgets/commands into lazy chunks once circular deps are resolved. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Vector search now fails fast if Rust worker unavailable instead of falling back to slow TypeScript implementation. Performance improvement: - Before (TypeScript fallback): 12-25 seconds for 46K vectors - After (Rust only): 785ms for 46K vectors (~15-30x faster) Changes: - Remove try/catch fallback in vectorSearch() method - Remove TypeScript cosine similarity computation - Remove fetch-all-vectors + fetch-each-record pattern - Remove unused SimilarityMetrics import - Add clear error message when Rust worker unavailable The getAllVectors() method remains for backfillVectors() which needs to iterate all records for embedding generation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add esbuild CLI bundler (6.6MB bundle with minification) - Lazy-load proto in InferenceGrpcClient to enable bundling - Update ./jtag script to use bundle when available - Add build:cli npm script and include in postbuild - Fix CommandGenerator to not run CLI code when bundled The bundle reduces CLI startup from ~2.6s to ~0.6s CPU time. Network latency to server dominates total wall-clock time. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The connect() method was calling list() after initialize() had already discovered commands via discoverCommands(). This caused two round-trips to the server for command listing. Now connect() builds the listResult from the already-populated discoveredCommands map, eliminating the redundant network call. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Memory leak fix (event delegation): - Add MessageEventDelegator for single-listener pattern on chat container - Remove per-element addEventListener in ImageMessageAdapter, URLCardAdapter - Add static handler methods called via data-action attributes - Wire delegation into ChatWidget with proper cleanup in disconnectedCallback Avatar animation fix: - UserListWidget migration to ReactiveListWidget broke live AI status updates - EntityScroller caches rendered elements, so reactive state changes alone don't update existing items - Fix: updateAIStatus() now does direct DOM manipulation alongside reactive state updates for cached elements 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Group type filters (All|Human|Persona|Agent) separate from status (Online) - Add vertical divider between filter groups - Labels hidden by default, expand on hover with smooth animation - Active chips always show their label - Smaller, more compact chip design Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add requestUpdate() to ensure header re-renders on filter change - Increase active chip label opacity to 1 (was 0.8) - Active chips now have stronger glow and always show labels - Clicking type filter properly removes ALL and shows selected type label Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use scroller.clear() + scroller.load() instead of refresh() to ensure items are re-rendered with updated filter state. The getRenderFunction checks matchesFilters() for each item on load. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- WIDGET-TECHNICAL-DEBT.md: Full audit of innerHTML (20), timeouts (26), BaseWidget (9) - PR-DESCRIPTION-WIDGET-OVERHAUL.md: Ready for PR creation - Priority order for fixes documented - Testing checklist included Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add MemberChipData interface for type-safe member chip rendering - Replace header innerHTML with targeted DOM updates (updateHeaderElements) - Replace message innerHTML with DOM structure building - Add buildMemberChipData, createMemberChip, updateMemberChip methods - Only adapter content uses innerHTML (adapters return HTML strings) - Update technical debt docs to reflect ChatWidget progress Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Change connectedCallback from blocking async to non-blocking sync - Render "Loading..." state immediately on mount - Fire async initialization in background without blocking UI - All BaseWidget subclasses now show instant feedback This fixes 10-30 second UI freezes when switching tabs. The React pattern: 1. Render shell immediately (loading state) 2. Async work runs in background 3. UI updates when data arrives Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- UserListWidget: Open profile tab immediately, persist in background - Tab shows "Loading..." instantly via BaseWidget non-blocking pattern - Server content/open command runs in background (fire-and-forget) This completes the React pattern for tab opening: 1. Local state update → tab appears instantly 2. Widget renders loading state → user sees feedback 3. Server persistence → eventually consistent Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
RoomListWidget was missing contentState update - tab had to wait for server event. Now creates tab immediately via contentState.addItem(). Flow: 1. contentState.addItem() → tab appears instantly 2. pageState.setContent() → ChatWidget loads room 3. Commands.execute() → server persists (background) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sidebar widgets (RoomListWidget, UserListWidget) should not manage contentState directly - that creates race conditions with the server event system. Kept: - BaseWidget non-blocking render (Loading... immediately) - MainWidget tab switching (existing tabs flip instantly) Reverted: - RoomListWidget optimistic contentState updates - UserListWidget optimistic contentState updates Sidebar clicks now go through server properly: 1. Command fires (fire-and-forget) 2. Server creates content item 3. Server emits content:opened 4. PositronContentStateAdapter updates local state + triggers view switch Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The entityId from pageState could be UUID or uniqueId format, but currentRoomId comparison in renderItem uses room.id (UUID). Fix: Look up matching room by both UUID and uniqueId to find the correct room.id for highlighting. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The handleTabClick method was using entityId (UUID) for building URLs instead of uniqueId (human-readable like "general", "dev-updates"). Changed to look up contentItem.uniqueId and use it for URL paths, falling back to entityId if uniqueId is not available. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The actual tab clicks go through ContentTabsWidget.handleTabClick, not MainWidget.handleTabClick. Added URL updates to ContentTabsWidget: - Added uniqueId to TabInfo interface - Pass uniqueId when building tabs from contentState - Update URL in handleTabClick using uniqueId for human-readable paths - Update URL in handleTabClose when switching to new current tab Note: URL updates are now scattered across widgets - needs centralized UrlStateService in future refactor. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1. ContentService: New centralized service for all content operations - open(): handles tab creation, view switch, URL update, persist - switchTo(): handles tab switch, view, URL, persist - close(): handles tab close, switch to next, URL, persist - Single source of truth instead of scattered widget logic 2. EntityScroller: Fix event-added entities being wiped on refresh - Track entities added via real-time events in eventAddedIds Set - diffAndUpdateDOM now preserves event-added entities - Only removes entities from previous DB loads, not live events - Fixes messages disappearing when switching tabs 3. Widget updates: - ContentTabsWidget: uses ContentService.switchTo/close - RoomListWidget: uses ContentService.open Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 1-3 of Unified Positronic Architecture: - Create EntityCacheService with per-collection stores - eventAddedIds tracking (entities from events survive DB refresh) - Subscribe/notify pattern with microtask batching - Auto-subscribe to entity events when widgets subscribe - Integrate ChatWidget to populate cache when loading messages This lays the foundation for: - 60fps responsive UI (instant cache reads vs DB queries) - No message flickering on tab switch - Consistent state across all widgets Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ContentService (Phase 4): - Look up recipe ID for content type on open() - Store recipeId in content item metadata - Pre-populate EntityCacheService for content type's collection - Map content types to entity collections RoutingService (Phase 5): - Check EntityCacheService before DB queries (cache-first) - Populate EntityCacheService on DB resolution - Resolution order: local cache → EntityCache → DB - Instant resolution when entities already cached This completes the Unified Positronic Content Architecture: - EntityCacheService as single source of truth - 60fps-capable instant cache reads - Recipe-aware content management - Unified caching across all entity resolution Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR introduces the Unified Positronic Content Architecture, a cache-first state management system designed to achieve 60fps-capable UI responsiveness through event-driven entity caching and centralized content orchestration.
Changes:
- New EntityCacheService - Single source of truth for entity data with event subscription and microtask batching
- New ContentService - Centralized orchestrator for content operations with recipe awareness
- Enhanced RoutingService - Cache-first resolution pattern for instant navigation
- Widget migrations - Multiple widgets converted to ReactiveWidget/ReactiveListWidget patterns with Lit templates
- Event delegation - Memory-efficient message interaction handling via MessageEventDelegator
Reviewed changes
Copilot reviewed 178 out of 327 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| continuum-metrics.css/scss | Auto-generated SCSS compilation output with minified styles |
| continuum-emoter.css/scss | Auto-generated SCSS compilation output with minified styles |
| user-list.css/scss | Auto-generated SCSS compilation output with minified styles |
| room-list-widget.css/scss | Auto-generated SCSS compilation output with minified styles |
| chat-widget.scss | Auto-generated SCSS compilation output with minified styles |
| ContinuumMetricsWidget.ts | Complete rewrite using ReactiveWidget with Lit templates, added auto-detect time range |
| ContinuumEmoterWidget.ts | Added verbose logging helpers and event cleanup tracking |
| ContentTabsWidget.ts | Migrated to ReactiveWidget with contentState subscription pattern |
| UserListWidget.ts | Migrated to ReactiveListWidget with filter chips and targeted DOM updates |
| RoomListWidget.ts | Migrated to ReactiveListWidget with pageState subscription |
| ChatWidget.ts | Added signal-based state management, event delegation, optimistic updates |
| InfiniteScrollHelper.ts | Added verbose logging helpers throughout |
| ChatMessageRenderer.ts | Added verbose logging helper |
| ChatMessageLoader.ts | Added verbose logging helper |
| ChatMessageCache.ts | New file - localStorage-based message caching for instant tab switches |
| BaseMessageRowWidget.ts | Added verbose logging helper |
| URLCardAdapter.ts | Converted to static action handlers for event delegation |
| MessageEventDelegator.ts | New file - Single delegated event handler pattern for memory efficiency |
| universal-demo.html | Added ES module loading with async wait pattern |
Files not reviewed (1)
- src/debug/jtag/package-lock.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| protected override async onWidgetCleanup(): Promise<void> { | ||
| // Unsubscribe from ALL events to prevent memory leaks | ||
| for (const unsub of this._eventUnsubscribers) { | ||
| try { unsub(); } catch { /* ignore */ } |
There was a problem hiding this comment.
Silent catch block ignores all errors during unsubscribe. Consider logging errors in verbose mode to help debug cleanup issues: catch (e) { verbose() && console.warn('Unsubscribe error:', e); }
| try { unsub(); } catch { /* ignore */ } | |
| try { | |
| unsub(); | |
| } catch (e) { | |
| verbose() && console.warn('Unsubscribe error:', e); | |
| } |
| // CRITICAL: Also update DOM directly for cached elements | ||
| // EntityScroller doesn't re-render existing items when state changes | ||
| const userElement = this.shadowRoot?.querySelector(`[data-user-id="${personaId}"]`) as HTMLElement; |
There was a problem hiding this comment.
DOM selector uses string interpolation with unescaped personaId. If personaId contains special CSS selector characters (e.g., quotes, brackets), this query could fail or cause injection issues. Consider using CSS.escape() or a data-attribute approach: this.shadowRoot?.querySelector('[data-user-id]')?.filter(el => el.dataset.userId === personaId)
|
|
||
| const verbose = () => typeof window !== 'undefined' && (window as any).JTAG_VERBOSE === true; | ||
|
|
There was a problem hiding this comment.
Repeated verbose logging helper pattern across multiple files (appears in 10+ files). Consider extracting to a shared utility module to ensure consistency and reduce duplication: import { verbose } from '@system/utils/verbose';
| const verbose = () => typeof window !== 'undefined' && (window as any).JTAG_VERBOSE === true; | |
| import { verbose } from '@system/utils/verbose'; |
| base64Length: messageEntity.content.media[0].base64?.length ?? 0 | ||
| })); | ||
| // 🚀 OPTIMISTIC UPDATE: Show message immediately with "sending" state | ||
| const tempId = `temp-${Date.now()}-${Math.random().toString(36).substring(2, 8)}` as UUID; |
There was a problem hiding this comment.
Temporary ID generation uses timestamp + random string but is cast directly to UUID. This could cause type confusion since it doesn't match UUID format. Consider using a more explicit type like type TempMessageId = string & { __brand: 'temp' } or adding a 'temp-' prefix check in UUID validation.
| this.roomMembers.set(member.userId, userResult.data); | ||
| try { | ||
| const result = await Commands.execute<DataListParams, DataListResult<UserEntity>>(DATA_COMMANDS.LIST, { | ||
| collection: UserEntity.collection, |
There was a problem hiding this comment.
The $in operator usage in the filter suggests MongoDB-style queries, but this isn't documented. Add a comment explaining the query syntax expected by the data layer: // Uses MongoDB-style $in operator for batch ID lookup
| collection: UserEntity.collection, | |
| collection: UserEntity.collection, | |
| // Uses MongoDB-style $in operator for batch ID lookup |
| const actionElement = target.closest('[data-action]') as HTMLElement; | ||
| if (!actionElement) return; | ||
|
|
||
| const action = actionElement.dataset.action; |
There was a problem hiding this comment.
The .closest('[data-action]') selector will match ANY ancestor with data-action, potentially triggering unintended handlers if nested elements both have data-action attributes. Consider adding a max depth check or container boundary: target.closest('.message-row [data-action]') to scope the search within message boundaries.
| <script type="module" src="/dist/index.js"></script> | ||
| <script> | ||
| let jtagSystem = null; | ||
| let activeExample = null; | ||
| let exampleConfig = null; | ||
|
|
||
|
|
||
| // Wait for ES module to load and export jtag | ||
| async function waitForJtag(maxWait = 5000) { | ||
| const start = Date.now(); | ||
| while (typeof jtag === 'undefined' && Date.now() - start < maxWait) { | ||
| await new Promise(r => setTimeout(r, 50)); | ||
| } | ||
| if (typeof jtag === 'undefined') { | ||
| throw new Error('JTAG module failed to load'); | ||
| } | ||
| return jtag; | ||
| } | ||
|
|
||
| // Initialize JTAG system | ||
| async function initializeJTAG() { | ||
| try { | ||
| log('core', '🔌 Waiting for JTAG module...'); | ||
| await waitForJtag(); |
There was a problem hiding this comment.
Polling every 50ms for 5 seconds creates 100 timer callbacks. This is inefficient for module loading. Consider using a Promise-based approach with module import: const { jtag } = await import('/dist/index.js') or use a MutationObserver to detect when the global is set.
| <script type="module" src="/dist/index.js"></script> | |
| <script> | |
| let jtagSystem = null; | |
| let activeExample = null; | |
| let exampleConfig = null; | |
| // Wait for ES module to load and export jtag | |
| async function waitForJtag(maxWait = 5000) { | |
| const start = Date.now(); | |
| while (typeof jtag === 'undefined' && Date.now() - start < maxWait) { | |
| await new Promise(r => setTimeout(r, 50)); | |
| } | |
| if (typeof jtag === 'undefined') { | |
| throw new Error('JTAG module failed to load'); | |
| } | |
| return jtag; | |
| } | |
| // Initialize JTAG system | |
| async function initializeJTAG() { | |
| try { | |
| log('core', '🔌 Waiting for JTAG module...'); | |
| await waitForJtag(); | |
| <script type="module"> | |
| let jtagSystem = null; | |
| let activeExample = null; | |
| let exampleConfig = null; | |
| let jtag = null; | |
| // Initialize JTAG system | |
| async function initializeJTAG() { | |
| try { | |
| log('core', '🔌 Loading JTAG module...'); | |
| const module = await import('/dist/index.js'); | |
| jtag = module.jtag || module.default; | |
| if (!jtag) { | |
| throw new Error('JTAG export not found in module'); | |
| } |
- UserListWidget: Use ContentService.open() for instant profile tabs - DiagnosticsWidget: Use ContentService.open() for instant log tabs - UserProfileWidget: Use ContentService.open() for cognition/navigation - PersonaBrainWidget: Use ContentService.open() for log viewer tabs - PositronContentStateAdapter: Fix temp ID → real ID update race condition that caused tabs to disappear for 5 seconds then reappear The root cause was handleContentOpened() setting currentItemId to server's real ID before updating the existing item's temp ID, causing currentItem lookup to fail temporarily. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Circuit Breaker (CandleGrpcAdapter): - Track consecutive timeouts and fail fast after 2 failures - 60-second cooldown before retrying local inference - Prevents cascading failures when Candle is overloaded - Health check returns unhealthy when circuit is open Instant Hydration Pattern: - Widgets implement onActivate(entityId, metadata) for instant rendering - Pass full entity in ContentService.open() metadata - Clear/populate/query pattern: clear old state, populate with passed data, query only what's missing - Same entity = refresh deltas, different entity = full clear Tab Flickering Fix: - PositronContentStateAdapter checks ContentStateService singleton first - Updates temp ID to real ID before setting currentItemId - Prevents tabs from disappearing during ID synchronization Room Selection Fix: - RoomListWidget guard checks pageState instead of roomId - Prevents "click does nothing" when roomId matches default Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When returning to a previously viewed room, the ChatWidget was skipping the refresh entirely because roomId === currentRoomId. This caused stale message history since new messages weren't loaded. Fix: Call scroller.refresh() when switching to the same room instead of just returning early. This loads any new messages that arrived while viewing other content. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Unified data interface for all widgets using EntityCacheService:
- useCollection<T>(): Subscribe to entity collections with filtering, sorting, limit
- Automatically subscribes to cache for real-time updates
- Loads from DB if cache is empty
- Returns handle with refresh(), setFilter(), count()
- Auto-cleanup on widget disconnect
- useEntity<T>(): Subscribe to single entity by ID
- Real-time updates when entity changes
- Auto-cleanup on disconnect
This eliminates per-widget caching code and provides a consistent
React-like data layer across all widgets.
Example usage:
this.useCollection<ChatMessageEntity>({
collection: 'chat_messages',
filter: m => m.roomId === this.roomId,
onData: (messages) => this.renderMessages(messages)
});
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The loading state wasn't centered because the shadow root host element lacked width/height. Added :host styles to ensure the container fills its parent. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add 25-point humanMessage bonus to scoring weights - Add senderType param to detect human vs AI senders - Add isHumanSender() lookup when senderType not provided - Update browser/server commands with new scoreBreakdown fields Ensures AIs prioritize responding to human messages over AI-to-AI chatter. Human question now scores 40 (25 human + 10 question + 5 unanswered) vs 15 before. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Regenerated after humanMessage priority scoring addition. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
EntityCacheService is now the single source of truth for entity data. ChatMessageCache was implemented but never imported or used. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add verbose logging to catch block in ContinuumEmoterWidget - Use CSS.escape() for personaId in UserListWidget selectors - Add comment documenting MongoDB-style $in operator in ChatWidget - Scope .closest() selector in MessageEventDelegator to message-row - Replace polling with dynamic import in universal-demo.html Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Summary
This PR introduces the Unified Positronic Content Architecture - a cache-first, event-driven state management system that provides 60fps-capable UI responsiveness.
Key Architectural Changes
EntityCacheService (
system/state/EntityCacheService.ts) - NEWContentService (
system/state/ContentService.ts) - NEWRoutingService (
system/routing/RoutingService.ts) - ENHANCEDChatWidget & EntityScroller - ENHANCED
Architecture Flow
Benefits
Test plan
🤖 Generated with Claude Code