lukas/websocket improvements #1807

LukasDeco · 2025-08-08T20:01:05Z

Overview:

This PR introduces a new version of the websocket account subscriber for the drift client. (WebSocketDriftClientAccountSubscriberV2)
This uses WebSocketProgramAccountSubscriberV2 under the hood which opens a single programAccountSubscribe WS connection for an account type.
We open a connection for PerpMarkets and a second connection for SpotMarkets, and if we detect that a certain market account isn't getting an update that it should, we refresh the connection. This is done through polling logic rather than assuming things are stale and refreshing the connection, which is more resource intensive than just doing a poll.
We should have a faster subscribe time on initial load, and less websocket connections, but still have the preferable benefit of real-time events that we don't get with polling.
These classes leverage the v2 of solana's web3.js known was kit or gill depending on which npm you install. We are using gill here. There are some inherit websocket optimizations for memory management and stability.

Notes:

There is some initial code added for WebSocketProgramUserAccountSubscriber but that is not going to be utilized immediately on the UI. But it is added and ready to be used when we want.
Please review the code for readability, maintainability and any performance optimizations that might be available to add.
This PR should NOT affect any current consumer of the SDK UNLESS they specifically opt-in to these new classes via the config.

Commits:

feat: initial implementation for users and markets WS improvements
lukas/gill websocket sub (lukas/gill websocket sub #1781)
feat: initial implementation for users and markets WS improvements
feat: polling check on websocket acct subscriber v2 + naming
fix: lint
fix: non-hanging WS subscription async loop handling
fix: bugs with program ws subs hanging on asynciter
fix: goofy self imports
feat: initial batch fetching temp
temp: sub second WS subscribe time
fix: ws program account subscriber v2 bugs and optimizations

ChesterSim · 2025-08-15T03:28:38Z

sdk/src/accounts/README_WebSocketAccountSubscriberV2.md

+
+**How it works:**
+1. Initially subscribes to WebSocket updates
+2. If no WebSocket data is received for `resubTimeoutMs` (30s), switches to polling mode


is this only true, if usePollingInsteadOfResub is true?

ChesterSim · 2025-08-15T07:03:05Z

sdk/src/accounts/webSocketAccountSubscriberV2.ts

-			// Start notification loop without awaiting
-			this.handleNotificationLoop(subscription);
+			// Start notification loop with the subscription promise
+			this.handleNotificationLoop(subscriptionPromise);


feels weird to be accepting a promise for a function

maybe could use .then instead and pass the resolved Promise to this.handleNotificationLoop?

ok it is totally weird but I did this for performance haha. I didn't want to wait for the WS subscription to succeed in the overall .subscribe() path because it slows down the app from rendering, while we can put this off to a separate thread because we do an initial fetch of the data anyway.

ChesterSim · 2025-08-15T07:22:17Z

sdk/src/accounts/webSocketDriftClientAccountSubscriberV2.ts

+		this.subscriptionPromiseResolver(true);
+
+		// delete initial data
+		this.removeInitialData();


should this be here? this.initialPerpMarketAccountData and this.initialSpotMarketAccountData are set just above, but we are deleting it here

idk tbh 😄 I was just following the original pattern from v1 which does this as well. Afraid to remove it and cause a regression. WDYT should we try removing it?

ChesterSim · 2025-08-15T07:27:13Z

sdk/src/accounts/webSocketDriftClientAccountSubscriberV2.ts

+	/**
+	 * This is a no-op method that always returns true.
+	 * Unlike the previous implementation, we don't need to manually subscribe to individual perp markets
+	 * because we automatically receive updates for all program account changes via a single websocket subscription.


something that im unsure - when you mention single websocket subscription for all program account changes, this means a single WS subscription for all perp market account changes right? both spot markets and perp markets use a different websocket?

we could technically use a single websocket for both perp and spot markets. But in this case basically yeah we use one WS connection for all perp market account changes. I think it might be fine to keep them separate. There are benefits to keep them separate like isolation if one of the connections has a problem, it doesn't affect state for the other states.

ChesterSim · 2025-08-15T07:44:05Z

sdk/src/accounts/webSocketDriftClientAccountSubscriberV2.ts

+
+		await this.unsubscribeFromMarketAccounts();
+		await this.unsubscribeFromSpotMarketAccounts();
+		await this.unsubscribeFromOracles();


can these use Promise.all()?

ChesterSim · 2025-08-15T09:50:07Z

sdk/src/accounts/webSocketProgramAccountSubscriberV2.ts

@@ -51,6 +71,16 @@ export class WebSocketProgramAccountSubscriberV2<T>
 	private accountsCurrentlyPolling: Set<string> = new Set(); // Track which accounts are being polled
 	private batchPollingTimeout?: ReturnType<typeof setTimeout>; // Single timeout for batch polling

+	// For batching initial account fetches
+	private accountsPendingInitialMonitorFetch: Set<string> = new Set(); // Track accounts waiting for initial monitor fetch


without much docs on this, it really confused me a lot, thinking why we are doing the fallback polling at the beginning

for sure let me add docs for this functionality

ChesterSim · 2025-08-15T10:00:56Z

what if resubOpts isn't passed in?

ChesterSim · 2025-08-15T10:05:16Z

i think the idea is there, but its been pretty hard to review ngl...

I don't think its a you problem btw, this whole SDK has been coded this way where each class contains too much logic. One suggestion is to ask Cursor to help abstract this class' logic into utility managers e.g. WebsocketSubscriberManager, FallbackPollingManager, InitialFallbackPollingManager etc.

* websockets gill temp * feat: feature parity between gill version ws acct sub and reg one + optional passing into driftClient * fix: post rebase bugs and cleanup * chore: websocket account subscriber export * feat: logging string update on ws acct v2 * rm: useless logging * chore: cleanup ws subscriber v2 docs * chore: specific name on custom ws acct sub param * fix: post rebase again cleanup * fix: prettier fixed

LukasDeco · 2025-08-15T21:40:40Z

i think the idea is there, but its been pretty hard to review ngl...

I don't think its a you problem btw, this whole SDK has been coded this way where each class contains too much logic. One suggestion is to ask Cursor to help abstract this class' logic into utility managers e.g. WebsocketSubscriberManager, FallbackPollingManager, InitialFallbackPollingManager etc.

I appreciate that and I think you are right. I've added some deeper docs on the WebsocketProgramAccountSubscriberV2 and WebsocketAccountSubscriberV2 which I think need it the most so the polling vs ws logic in there is understood.

Regarding adding utility manager classes, I definitely think some smaller classes/better code isolation and single responsibility could be very helpful. I am wondering if maybe we could start with adding real good docs on these classes, and then I could do a follow up to break down logic into shareable utility classes/functions. Since that would be a huge refactor and then potentially implicates adjusting other classes so I'd wanna make sure I did it real well. Although I could just give it a short in this PR. Up to you.

Thanks so much for the review!!!

LukasDeco marked this pull request as ready for review August 12, 2025 17:05

ChesterSim reviewed Aug 15, 2025

View reviewed changes

LukasDeco and others added 18 commits August 15, 2025 11:52

feat: initial implementation for users and markets WS improvements

c093ff9

feat: initial implementation for users and markets WS improvements

d50642a

feat: polling check on websocket acct subscriber v2 + naming

7217cc7

fix: lint

3b50c7a

fix: non-hanging WS subscription async loop handling

28ff6bf

fix: bugs with program ws subs hanging on asynciter

812ec08

fix: goofy self imports

5ee6f49

feat: initial batch fetching temp

1a0ecf2

temp: sub second WS subscribe time

da84940

fix: ws program account subscriber v2 bugs and optimizations

3843a20

feat: chunk stuff account requests

dc2064d

feat: more subscribe optimizations ws driftclient sub v2

9815c48

chore: cleanup ws sub v2 logs

bc1fcfa

feat: conditional check on using ws account subscriber + unused

3577582

fix: bad import

ef08f7e

chore: add export of WebSocketProgramAccountSubscriberV2

30862b8

fix: unneeded drift idl export messing up common build

7195770

LukasDeco force-pushed the lukas/websocket-improvements branch from 0388104 to 7195770 Compare August 15, 2025 18:17

LukasDeco added 2 commits August 15, 2025 15:03

fix: consolidate rpc ws subscriptions for oracles

6c18e01

feat: docs for ws v2 and cleanup

9351341

chore: more docs on ws acct susbcriber v2

f635fa5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

lukas/websocket improvements #1807

lukas/websocket improvements #1807

Uh oh!

LukasDeco commented Aug 8, 2025 •

edited

Loading

Uh oh!

ChesterSim Aug 15, 2025

Uh oh!

LukasDeco Aug 15, 2025

Uh oh!

ChesterSim Aug 15, 2025

Uh oh!

LukasDeco Aug 15, 2025

Uh oh!

ChesterSim Aug 15, 2025 •

edited

Loading

Uh oh!

LukasDeco Aug 15, 2025

Uh oh!

ChesterSim Aug 15, 2025

Uh oh!

LukasDeco Aug 15, 2025

Uh oh!

ChesterSim Aug 15, 2025

Uh oh!

LukasDeco Aug 15, 2025

Uh oh!

ChesterSim Aug 15, 2025

Uh oh!

LukasDeco Aug 15, 2025

Uh oh!

ChesterSim commented Aug 15, 2025

Uh oh!

ChesterSim commented Aug 15, 2025

Uh oh!

LukasDeco commented Aug 15, 2025

Uh oh!

Uh oh!

lukas/websocket improvements #1807

Are you sure you want to change the base?

lukas/websocket improvements #1807

Uh oh!

Conversation

LukasDeco commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChesterSim Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChesterSim commented Aug 15, 2025

Uh oh!

ChesterSim commented Aug 15, 2025

Uh oh!

LukasDeco commented Aug 15, 2025

Uh oh!

Uh oh!

LukasDeco commented Aug 8, 2025 •

edited

Loading

ChesterSim Aug 15, 2025 •

edited

Loading