feat: refactor Peer communication and schedule incoming messages on sync#2602
feat: refactor Peer communication and schedule incoming messages on sync#2602gdorsi merged 13 commits intofeat/storage-apifrom
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
1 Skipped Deployment
|
484f936 to
2699468
Compare
Jazz pre-releasePackages:{
"cojson": "https://pkg.pr.new/garden-co/jazz/cojson@6247fac6c532a6164566cddd30de596f9b8793cd",
"cojson-storage-indexeddb": "https://pkg.pr.new/garden-co/jazz/cojson-storage-indexeddb@6247fac6c532a6164566cddd30de596f9b8793cd",
"cojson-storage-sqlite": "https://pkg.pr.new/garden-co/jazz/cojson-storage-sqlite@6247fac6c532a6164566cddd30de596f9b8793cd",
"cojson-transport-ws": "https://pkg.pr.new/garden-co/jazz/cojson-transport-ws@6247fac6c532a6164566cddd30de596f9b8793cd",
"create-jazz-app": "https://pkg.pr.new/garden-co/jazz/create-jazz-app@6247fac6c532a6164566cddd30de596f9b8793cd",
"cursor-docs": "https://pkg.pr.new/garden-co/jazz/cursor-docs@6247fac6c532a6164566cddd30de596f9b8793cd",
"hash-slash": "https://pkg.pr.new/garden-co/jazz/hash-slash@6247fac6c532a6164566cddd30de596f9b8793cd",
"jazz-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-auth-betterauth@6247fac6c532a6164566cddd30de596f9b8793cd",
"jazz-betterauth-client-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-client-plugin@6247fac6c532a6164566cddd30de596f9b8793cd",
"jazz-betterauth-server-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-server-plugin@6247fac6c532a6164566cddd30de596f9b8793cd",
"jazz-react-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-react-auth-betterauth@6247fac6c532a6164566cddd30de596f9b8793cd",
"jazz-run": "https://pkg.pr.new/garden-co/jazz/jazz-run@6247fac6c532a6164566cddd30de596f9b8793cd",
"jazz-tools": "https://pkg.pr.new/garden-co/jazz/jazz-tools@6247fac6c532a6164566cddd30de596f9b8793cd"
} |
| export const MAX_OUTGOING_MESSAGES_CHUNK_BYTES = 25_000; | ||
|
|
||
| export class BatchedOutgoingMessages { | ||
| export class BatchedOutgoingMessages |
There was a problem hiding this comment.
Moved the PriorityBasedMessageQueue inside this class, so now outgoing messages prioritization is done here.
Now we can fully control the outgoing messages scheduling from this class, and I've changed to scheduling from debouncing to a simple delay of 10ms before start the messages processing.
This makes the batching and prioritization more effective as subsequent operations are more likely to be synced and reduces its overhead because we don't do anymore setTimeout/clearTimeout on each outgoing.push.
| .catch((e) => | ||
| logger.error("Error while pushing ping timeout", { err: e }), | ||
| ); | ||
| incoming.push("Disconnected"); |
There was a problem hiding this comment.
Removed the PingTimeout event, so now we only have "Disconnected" which is more generic and works for every situation.
This change simplified the Peer code in general, as we need to check only for a single exception.
| } | ||
| }, | ||
| }, | ||
| outgoing, |
There was a problem hiding this comment.
BatchedOutgoingMessages implements the OutgoingPeerChannel interface and can be used directly as outgoing channel.
| missingDependencies = new Set<RawCoID>(); | ||
|
|
||
| // Checks if the current CoValueCore is already a missing dependency of the given CoValueCore | ||
| checkCircularDependencies(dependency: CoValueCore) { |
There was a problem hiding this comment.
The changes I've done in the tests have highlighted that the previous fix on the circular deps didn't catch all the possible cases of missing circular deps.
So I've implemented a more strict algorithm to mark the missing dependencies, by checking before if the current CoValue is already marked as missing dependency of the dependency CoValue.
| correctionCallback: (data: CoValueKnownState) => void; | ||
| }; | ||
|
|
||
| export class StoreQueue { |
There was a problem hiding this comment.
Just extracted this logic from storageAsync an added some tests on it
| export class ConnectedPeerChannel | ||
| implements IncomingPeerChannel, OutgoingPeerChannel | ||
| { | ||
| buffer: (SyncMessage | DisconnectedError)[] = []; |
There was a problem hiding this comment.
Changed the Peer channels to use a callback-based communication instead of relying on async iterators.
Used a buffer to collect all the messages pushed before the connection with the syncManager.
Jazz pre-releasePackages:{
"cojson": "https://pkg.pr.new/garden-co/jazz/cojson@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"cojson-storage-indexeddb": "https://pkg.pr.new/garden-co/jazz/cojson-storage-indexeddb@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"cojson-storage-sqlite": "https://pkg.pr.new/garden-co/jazz/cojson-storage-sqlite@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"cojson-transport-ws": "https://pkg.pr.new/garden-co/jazz/cojson-transport-ws@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"create-jazz-app": "https://pkg.pr.new/garden-co/jazz/create-jazz-app@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"cursor-docs": "https://pkg.pr.new/garden-co/jazz/cursor-docs@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"hash-slash": "https://pkg.pr.new/garden-co/jazz/hash-slash@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"jazz-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-auth-betterauth@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"jazz-betterauth-client-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-client-plugin@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"jazz-betterauth-server-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-server-plugin@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"jazz-react-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-react-auth-betterauth@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"jazz-run": "https://pkg.pr.new/garden-co/jazz/jazz-run@9b1d52d183f2afd796ba24061056e6ae778c8dcf",
"jazz-tools": "https://pkg.pr.new/garden-co/jazz/jazz-tools@9b1d52d183f2afd796ba24061056e6ae778c8dcf"
} |
Jazz pre-releasePackages:{
"cojson": "https://pkg.pr.new/garden-co/jazz/cojson@85dc6ba148758ffe217723057f1a05839ca3412d",
"cojson-storage-indexeddb": "https://pkg.pr.new/garden-co/jazz/cojson-storage-indexeddb@85dc6ba148758ffe217723057f1a05839ca3412d",
"cojson-storage-sqlite": "https://pkg.pr.new/garden-co/jazz/cojson-storage-sqlite@85dc6ba148758ffe217723057f1a05839ca3412d",
"cojson-transport-ws": "https://pkg.pr.new/garden-co/jazz/cojson-transport-ws@85dc6ba148758ffe217723057f1a05839ca3412d",
"create-jazz-app": "https://pkg.pr.new/garden-co/jazz/create-jazz-app@85dc6ba148758ffe217723057f1a05839ca3412d",
"cursor-docs": "https://pkg.pr.new/garden-co/jazz/cursor-docs@85dc6ba148758ffe217723057f1a05839ca3412d",
"hash-slash": "https://pkg.pr.new/garden-co/jazz/hash-slash@85dc6ba148758ffe217723057f1a05839ca3412d",
"jazz-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-auth-betterauth@85dc6ba148758ffe217723057f1a05839ca3412d",
"jazz-betterauth-client-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-client-plugin@85dc6ba148758ffe217723057f1a05839ca3412d",
"jazz-betterauth-server-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-server-plugin@85dc6ba148758ffe217723057f1a05839ca3412d",
"jazz-react-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-react-auth-betterauth@85dc6ba148758ffe217723057f1a05839ca3412d",
"jazz-run": "https://pkg.pr.new/garden-co/jazz/jazz-run@85dc6ba148758ffe217723057f1a05839ca3412d",
"jazz-tools": "https://pkg.pr.new/garden-co/jazz/jazz-tools@85dc6ba148758ffe217723057f1a05839ca3412d"
} |
Jazz pre-releasePackages:{
"cojson": "https://pkg.pr.new/garden-co/jazz/cojson@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"cojson-storage-indexeddb": "https://pkg.pr.new/garden-co/jazz/cojson-storage-indexeddb@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"cojson-storage-sqlite": "https://pkg.pr.new/garden-co/jazz/cojson-storage-sqlite@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"cojson-transport-ws": "https://pkg.pr.new/garden-co/jazz/cojson-transport-ws@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"create-jazz-app": "https://pkg.pr.new/garden-co/jazz/create-jazz-app@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"cursor-docs": "https://pkg.pr.new/garden-co/jazz/cursor-docs@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"hash-slash": "https://pkg.pr.new/garden-co/jazz/hash-slash@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"jazz-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-auth-betterauth@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"jazz-betterauth-client-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-client-plugin@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"jazz-betterauth-server-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-server-plugin@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"jazz-react-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-react-auth-betterauth@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"jazz-run": "https://pkg.pr.new/garden-co/jazz/jazz-run@6f6663d8257bd669edbfa33b46428973cf9aedb2",
"jazz-tools": "https://pkg.pr.new/garden-co/jazz/jazz-tools@6f6663d8257bd669edbfa33b46428973cf9aedb2"
} |
Jazz pre-releasePackages:{
"cojson": "https://pkg.pr.new/garden-co/jazz/cojson@ffebb4fdaf58667cf4799fb5535751d445273afe",
"cojson-storage-indexeddb": "https://pkg.pr.new/garden-co/jazz/cojson-storage-indexeddb@ffebb4fdaf58667cf4799fb5535751d445273afe",
"cojson-storage-sqlite": "https://pkg.pr.new/garden-co/jazz/cojson-storage-sqlite@ffebb4fdaf58667cf4799fb5535751d445273afe",
"cojson-transport-ws": "https://pkg.pr.new/garden-co/jazz/cojson-transport-ws@ffebb4fdaf58667cf4799fb5535751d445273afe",
"create-jazz-app": "https://pkg.pr.new/garden-co/jazz/create-jazz-app@ffebb4fdaf58667cf4799fb5535751d445273afe",
"cursor-docs": "https://pkg.pr.new/garden-co/jazz/cursor-docs@ffebb4fdaf58667cf4799fb5535751d445273afe",
"hash-slash": "https://pkg.pr.new/garden-co/jazz/hash-slash@ffebb4fdaf58667cf4799fb5535751d445273afe",
"jazz-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-auth-betterauth@ffebb4fdaf58667cf4799fb5535751d445273afe",
"jazz-betterauth-client-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-client-plugin@ffebb4fdaf58667cf4799fb5535751d445273afe",
"jazz-betterauth-server-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-server-plugin@ffebb4fdaf58667cf4799fb5535751d445273afe",
"jazz-react-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-react-auth-betterauth@ffebb4fdaf58667cf4799fb5535751d445273afe",
"jazz-run": "https://pkg.pr.new/garden-co/jazz/jazz-run@ffebb4fdaf58667cf4799fb5535751d445273afe",
"jazz-tools": "https://pkg.pr.new/garden-co/jazz/jazz-tools@ffebb4fdaf58667cf4799fb5535751d445273afe"
} |
fix: fixes InvalidSignature errors that could happen during streaming
Jazz pre-releasePackages:{
"cojson": "https://pkg.pr.new/garden-co/jazz/cojson@114c10bc77675979326b90ccff8247eb14e54edd",
"cojson-storage-indexeddb": "https://pkg.pr.new/garden-co/jazz/cojson-storage-indexeddb@114c10bc77675979326b90ccff8247eb14e54edd",
"cojson-storage-sqlite": "https://pkg.pr.new/garden-co/jazz/cojson-storage-sqlite@114c10bc77675979326b90ccff8247eb14e54edd",
"cojson-transport-ws": "https://pkg.pr.new/garden-co/jazz/cojson-transport-ws@114c10bc77675979326b90ccff8247eb14e54edd",
"create-jazz-app": "https://pkg.pr.new/garden-co/jazz/create-jazz-app@114c10bc77675979326b90ccff8247eb14e54edd",
"cursor-docs": "https://pkg.pr.new/garden-co/jazz/cursor-docs@114c10bc77675979326b90ccff8247eb14e54edd",
"hash-slash": "https://pkg.pr.new/garden-co/jazz/hash-slash@114c10bc77675979326b90ccff8247eb14e54edd",
"jazz-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-auth-betterauth@114c10bc77675979326b90ccff8247eb14e54edd",
"jazz-betterauth-client-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-client-plugin@114c10bc77675979326b90ccff8247eb14e54edd",
"jazz-betterauth-server-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-server-plugin@114c10bc77675979326b90ccff8247eb14e54edd",
"jazz-react-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-react-auth-betterauth@114c10bc77675979326b90ccff8247eb14e54edd",
"jazz-run": "https://pkg.pr.new/garden-co/jazz/jazz-run@114c10bc77675979326b90ccff8247eb14e54edd",
"jazz-tools": "https://pkg.pr.new/garden-co/jazz/jazz-tools@114c10bc77675979326b90ccff8247eb14e54edd"
} |
|
|
||
| // Delay the initiation of the queue processing to accumulate messages | ||
| // before sending them, in order to do prioritization and batching | ||
| await new Promise<void>((resolve) => setTimeout(resolve, 10)); |
There was a problem hiding this comment.
I'm dubious of this: doesn't it always add 10ms delay?
Do we get "enough" batching if we wait for 1ms? Can we skip batching if the queue is "deep enough"?
There was a problem hiding this comment.
I'm dubious of this: doesn't it always add 10ms delay?
Only for the first message of the batch.
Choosen 10ms because I thought it as a negligible delay for a batch.
Do we get "enough" batching if we wait for 1ms?
We get a smaller window, but still effective when handling stress situations.
Can we skip batching if the queue is "deep enough"?
The queue start growing after this line. Without this await the queue can't be longer than 1 item.
EDIT: It can actually grow when the outgoing buffer is full or when the WebSocket is connecting
|
|
||
| // We check if we have blocked the main thread for too long | ||
| // and if so, we schedule a timer task to yield to the event loop | ||
| if (currentTimer - lastTimer > 50) { |
There was a problem hiding this comment.
Doesn't this mean that under load, the response to the first message will only be sent out after at least 50ms? Can we try yielding more often and seeing how that affects performance? Maybe make it an env var so we can experiment in prod?
There was a problem hiding this comment.
I can setup a config that can be modified from the outside.
the response to the first message will only be sent out after at least 50ms?
We could remove the await from the outgoing messages and the response would be scheduled immediately when the buffer is free.
Can we try yielding more often and seeing how that affects performance?
In my stress tests 50ms looked a good compromise, but can do some more tests to see if things are changing when lowering the budget.
…s scheduling budget
Jazz pre-releasePackages:{
"cojson": "https://pkg.pr.new/garden-co/jazz/cojson@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"cojson-storage-indexeddb": "https://pkg.pr.new/garden-co/jazz/cojson-storage-indexeddb@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"cojson-storage-sqlite": "https://pkg.pr.new/garden-co/jazz/cojson-storage-sqlite@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"cojson-transport-ws": "https://pkg.pr.new/garden-co/jazz/cojson-transport-ws@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"create-jazz-app": "https://pkg.pr.new/garden-co/jazz/create-jazz-app@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"cursor-docs": "https://pkg.pr.new/garden-co/jazz/cursor-docs@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"hash-slash": "https://pkg.pr.new/garden-co/jazz/hash-slash@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"jazz-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-auth-betterauth@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"jazz-betterauth-client-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-client-plugin@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"jazz-betterauth-server-plugin": "https://pkg.pr.new/garden-co/jazz/jazz-betterauth-server-plugin@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"jazz-react-auth-betterauth": "https://pkg.pr.new/garden-co/jazz/jazz-react-auth-betterauth@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"jazz-run": "https://pkg.pr.new/garden-co/jazz/jazz-run@bbb1c44977a05b538684cd7ebf9c39a668eddfe0",
"jazz-tools": "https://pkg.pr.new/garden-co/jazz/jazz-tools@bbb1c44977a05b538684cd7ebf9c39a668eddfe0"
} |
Description
We have noticed that under load our sync server tends to accumulate incoming messages and stops responding until a big chunk of them is processed.
This leads to low throughput and to make the service look like not operative.
In this PR I'm refactoring the internal message scheduling in order to migrate all the async code in the hot-paths into something that we fully control:
setTimeout, giving to the event loop the space to handle the rest of the tasksWhile testing these changes I've also observerd some preformance improvements in both the sync server processing and the browser loading time.
The change on the peer communication has also affected the testing API of jazz-tools, now everything happen in sync so the multi-user tests should become easier to write.
Manual testing instructions
I've created a stress-test application that I've used to check the performance and the behavoir of the sync server under load.
It's inside
tests/stress-test:pnpm build:packages # ensure that the packages dist are aligned with the codecd tests/stress-testpnpm devIt spawns both a vite dev build and a local sync server.
chrome://inspect/#devicescan be used to connect the Chrome dev tools to the sync server and do profiling.Tests
Checklist