Skip to content

Conversation

@jackyzha0
Copy link
Member

@jackyzha0 jackyzha0 commented Mar 30, 2025

Why

still getting invariant violation: would have sent out of order msg

What changed

track last 10 sent messages along with their stack trace

this unfortunately degrades message throughput from ~46krps to ~16kps but should be more than enough for prod as is, we just need to find the bug and fix it and then remove this again

Versioning

  • Breaking protocol change
  • Breaking ts/js API change

@jackyzha0 jackyzha0 requested a review from a team as a code owner March 30, 2025 17:26
@jackyzha0 jackyzha0 requested review from blast-hardcheese and removed request for a team March 30, 2025 17:26
@jackyzha0 jackyzha0 changed the title rip still happening, add last messages tracking add more debug for message ordering violation Mar 30, 2025
@jackyzha0 jackyzha0 requested a review from masad-frost March 30, 2025 17:57
Copy link
Member

@masad-frost masad-frost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Muh RPS

Hopefully that's all we need. Strange buggggg

@jackyzha0 jackyzha0 merged commit f1b9ac4 into main Mar 31, 2025
6 checks passed
@jackyzha0 jackyzha0 deleted the jackyzha0/more-logging branch March 31, 2025 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants