Odd Heisenbug: very high memory usage

[taoeffect]

Problem

Group Income is using more memory than both Slack and Discord combined! 😧

Even after refreshing the page it's still showing this much usage!

NB: I'm not seeing this in Firefox though! From about:memory:

Odd that suddenly, after disabling an extension from Kagi and restarting the browser, I briefly saw 1GB of RAM usage, but now Brave is showing only 74MB!

Solution

Find out why it's doing this and fix. This is definitely a bug of some sort, as I have a bunch of Discords I'm part of (more than GI groups), so this shouldn't be happening.

Here's a heap snapshot I made with Brave's dev tools:

The problem though is that the total heap size isn't that bad:

So what's the actual source of this memory usage?!

Could it have have something to do with uploading videos / file attachments? I'm not sure.

[corrideat]

I need to see what those numbers mean, but could it be that it's reporting memory that's not been reclaimed by the GC? Also, as you point out, it could have to do with blobs, to some extent.

[taoeffect]

I need to see what those numbers mean, but could it be that it's reporting memory that's not been reclaimed by the GC?

Dunno.

I noticed that after I made a snapshot a new entry for Main would appear here:

Also don't know what that means.

And in the heap snapshot it shows quite a large number of event listeners, so maybe we're not cleaning some up:

That is a common source of leaks in JS.

[taoeffect]

I asked Gemini Flash via crush to search the codebases for Group Income and libcheloniajs. Here's what it said:

Group Income

Round 1

Details

• I analyzed the codebase for the following event listener types and identified several memory leaks and areas of concern:
  • addEventListener
  • .on... property assignments
  • sbp('okTurtles.events/on', ...)
• Tooltip.vue: Identified major memory leaks where click, mouseenter, mouseleave, focus, and blur listeners on triggerDOM, and a touchstart listener on document.body (for iOS), were registered in mounted() but never removed in beforeDestroy().
• ChatAttachmentPreview.vue: Discovered that a click listener on an a tag used for downloads was added but never removed. Additionally, found that sbp('okTurtles.events/off', ...) in beforeDestroy() was conditional on this.hasMediaAttachments, which could change during the component's lifecycle and lead to listeners not being cleared.
• ChatMain.vue: Confirmed that a matchMedia listener cleanup was commented out, potentially causing leaks when switching chatrooms.
• Modal.vue: Found that multiple sbp('okTurtles.events/off', ...) calls in beforeDestroy() were missing the handler argument. While this might be intended to clear all listeners for those selectors, it is inconsistent with other parts of the codebase.

Round 2

Prompt:

search through the frontend to find potential memory leaks for event listeners that are never removed. remember to search for event listeners registered in different ways. via:

• addEventListener
• .onclick and related
• vue-style @submit=, @click=, this.$on(.. etc.
• 'okTurtles.events/on' selector

check all of the possible methods and see if any of them don't properly remove the event listeners.

for every possible lead you find, direct subagents to confirm your suspicions to conserve context.

Result:

Details

I've identified several confirmed memory leaks and redundant event listener registrations in the frontend.

Confirmed Memory Leaks

1. Emoticons Mixin Leak

• File: frontend/views/containers/chatroom/EmoticonsMixins.js
• Issue: Registers global sbp listeners for CLOSE_EMOTICON and SELECT_EMOTICON in openEmoticon.
• Leak: If SendArea.vue or MessageBase.vue are destroyed (e.g., during navigation or virtual list recycling) while the emoticon picker is open, these listeners remain attached to the global sbp bus, preventing the entire component instance from being garbage collected.
• Files affected: SendArea.vue, MessageBase.vue.

2. Service Worker Lifecycle Leaks

• File: frontend/controller/service-worker.js
• waitUntilSwReady: Adds a controllerchange listener to navigator.serviceWorker but fails to remove it if the returned Promise rejects (e.g., on timeout or message error).
• service-worker/setup: Calling this selector multiple times (e.g., during re-initialization) appends duplicate anonymous message listeners to the singleton navigator.serviceWorker. It also leaks a setInterval that pings the SW every 5 seconds.
• updatefound: Listeners are appended to swRegistration every time setup is called without any removal logic.

3. Native Permissions & Intervals

• File: frontend/model/notifications/nativeNotification.js
• Permission Listeners: setupNativeNotificationsListeners appends change event listeners to PermissionStatus objects every time it's called (triggered on every LOGIN_COMPLETE).
• Dangling Intervals: fallbackChangeListener starts a setInterval that is never cleared. Repeated calls spawn multiple parallel intervals checking the same Notification.permission value every second.

4. Chelonia Initialization DupLicates

• File: frontend/setupChelonia.js
• Issue: The PUBSUB_ERROR handler resets the module's internal singleton promise and re-invokes setupChelonia().
• Leak: setupChelonia contains five sbp('okTurtles.events/on', ...) registrations for events like LOGIN_COMPLETE and LOGOUT. Since SBP appends listeners, every pubsub reconnection adds another set of duplicate listeners, leading to exponential action execution over time.

Recommendation

• Implement isInitialized guards in global setup files.
• Add beforeDestroy hooks to components using EmoticonsMixins.js to call closeEmoticon().
• Ensure all addEventListener calls have a corresponding removeEventListener in error paths and cleanup routines.

Libcheloniajs

Round 1 (with crush)

Prompt:

search through the frontend to find potential memory leaks for event listeners that are never removed. remember to search for event listeners registered in different ways. via:

• addEventListener
• 'okTurtles.events/on' selector
• also pay special attention to how contract manifests & source is loaded and verify that each contract version is only loaded once and not every time that it's encountered

check all of the possible methods and see if any of them don't properly cleanup resources

for every possible lead you find, direct subagents to confirm your suspicions to conserve context.

Details

I investigated the event listeners and contract loading logic in @chelonia/lib. Here are the findings:

1. Memory Leaks in Event Listeners

• src/chelonia.ts:947: The CONTRACTS_MODIFIED listener is registered during initialization but never removed, even during chelonia/reset. Since it binds this.contractsModifiedListener, it will leak the CheloniaContext if multiple initializations occur.
• src/internals.ts:782: The EVENT_HANDLED listener is registered globally in internals.ts. It is never removed and is not cleaned up during a reset.
• src/pubsub/index.ts:
  • addEventListener (lines 766, 855) are generally paired with removeEventListener in the _cleanup function, but some paths (notably WebSocket re-connections) may leave old listeners if the socket is replaced without triggering the full cleanup.
  • okTurtles.events/on (line 1034) returns a cleanup function that is stored but not always called if the pubsub client is abandoned.

[!NOTE]
Greg asks it to investigate contract source loading in more detail

Contract source and manifests are not loaded anew on every event. They are cached in the manifestToContract object by their manifestHash, and this cache persists even through a chelonia/reset. Redundant loads are effectively prevented by a guard in processMessage and a secondary guard inside loadManifest.

However, I found significant memory leaks related to event listeners:

• publishEvent Leak: In src/internals.ts:782, a listener for EVENT_HANDLED is registered for every outgoing message. It is only removed if the specific message hash is successfully processed. If a message is dropped or fails, the listener (and its closure over entry and hooks) is leaked permanently.
• Initialization Leak: In src/chelonia.ts:947, a CONTRACTS_MODIFIED listener is registered during initialization but is never removed, even during a reset.
• Contract Eval Double-Load: While manifestToContract prevents redundant work, if two contracts using the same manifestHash trigger a sync simultaneously, both may enter the async loadManifest function before the cache is populated, leading to duplicate network requests and evaluation until the first one completes.

Contract source and manifests are stored as follows:

• this.manifestToContract: Stores parsed manifest and contract context.
• SBP Selectors: Evaluating the contract source registers selectors in the global SBP registry (e.g., ${manifestHash}/${action}/process). These are never unregistered, which is a potential buildup over many protocol versions.
• this.whitelistedActions: Grows with every unique action encountered across all contracts.

Round 2 (with opencode)

Details

I've analyzed the codebase for potential memory leaks and identified several areas of concern. My findings align with the initial exploration, specifically regarding global event listeners and SBP event persistence.

1. Global Event Listener Leaks (PubSubClient)
   In src/pubsub/index.ts, the PubSubClient registers online and offline event listeners on the global self object (e.g., window or Worker).

• The Issue: While destroy() clears entries from a globalEventMap, it never calls removeEventListener.
• The Bug: The implementation of globalEventMap only supports a single active client for these global events. If multiple PubSubClient instances are created, they overwrite each other in the map. When any instance is destroyed, it clears the handler for all instances, but the actual listener remains attached to self indefinitely.
• Location: src/pubsub/index.ts (lines 755-768 and 875-880).

2. SBP Event Listener Leaks
   Several long-lived SBP event listeners are registered without any mechanism for removal.

• External State Sync: In src/local-selectors/index.ts, chelonia/externalStateSetup registers listeners for EVENT_HANDLED and CONTRACTS_MODIFIED. There is no "teardown" or off call available. If a project re-initializes Chelonia multiple times, these listeners will accumulate.
• Chelonia Core: src/chelonia.ts registers a CONTRACTS_MODIFIED listener upon registration. While typically a singleton, it lacks a cleanup path which may cause leaks in testing environments where multiple contexts are created and destroyed.

3. Contract Loading & Resource Management
   The contract loading logic appears to correctly cache manifests and signing keys. However, there is a minor architectural risk:

• Scratchpad Pattern: CheloniaContext uses temporary properties (defContract, defContractSelectors) during contract loading. While currently safe due to synchronous execution after fetching, this pattern is brittle and could lead to race conditions if the loading process becomes more asynchronous in the future.

4. Monotonic State Growth
   In src/internals.ts, when a contract is removed, it is set to null in state.contracts rather than being deleted. This "tombstone" pattern ensures consistency but means the state.contracts object grows indefinitely for every unique contract ID ever encountered.

---

Proposed Plan

1. Fix PubSubClient Global Listeners:
   • Modify globalEventMap to support multiple handlers (using a Set).
   • Ensure removeEventListener is called when the last client is destroyed.
2. Implement Cleanup for Local Selectors:
   • Add a chelonia/externalStateTeardown selector to remove listeners in src/local-selectors/index.ts.
3. Audit CheloniaContext Lifecycle:
   • Ensure that when a context is destroyed or reset, all associated SBP listeners are removed.
4. Refactor Contract Loading:
   • Pass contract definitions through return values instead of using the context as a scratchpad.

[corrideat]

I wouldn't put too much weight to high reported memory use if it's not causing performance issues. That said, 900 MiB does sound a bit excessive.

See screenshot below showing 100+ MiB being quite typical (but then 1 GiB is still a lot), with 44 MiB for an empty tab:

Since you've reported the actual JS use using the memory tool, it'd look like whatever's spiking memory use has to do primarily with the UI (with the bulk of Chelonia running in the SW, which is about ~5 MiB).

My guess is that this partially (read: mostly) has to do with attachments / images, since I believe those are not stored in the JS heap (e.g., if one creates a blob or loads an asset using the DOM). I'd need to investigate, and also see how we can get these reported.

From the memory snapshot snippet you posted (did you save this? It could help see what's going on, although it probably contains information that you shouldn't share), two things that stand out are EventListener and MessagePort / MessageChannel. (I wouldn't worry too much about Vue, because listeners attached to a DOM element should be removed when the element is removed from the DOM, unless there are dangling references to it.)

These could be related, and likely are, to the serdes library that we use to communicate between the SW and the app. In total, they add up to about 100 MiB, which is 'small' compared to the 900 MiB usage, but the fact that it's anywhere near 100 MiB suggests a leak. If these relate to serdes, they should be short-lived objects.

[taoeffect]

Good thinking @corrideat ! It could be serdes!