Reach functional backend using jupyter_collaboration frontend

[dlqqq]

Description

• First proof-of-concept of our backend implementation.
• Adds a new dev environment rtccore-jcollab-frontend to allow reviewers to independently verify this branch.
• Fixes (most) backend exceptions raised when the server extension starts.
• Enables the YRoomWebsocket handler.
• Adds a YRoomSessionHandler to allow usage with the jupyter_collaboration frontend.

UPDATE: Also got auto-saving working.

• Right now, it saves as fast as the ContentsManager allows it to. We only run one save task at a time, and don't send multiple save() calls concurrently. We can increase the throttling, but it works quite well as-is.

UPDATE: Cleaned up the logs & rebased onto latest main.

UPDATE: Also fixed the long latency when opening a document. Jialin was right, this was fixed by closing JupyterLab:globalAwareness websockets more gracefully.

Demo

Screen.Recording.2025-05-13.at.2.20.47.PM.mov

How to test this branch

1. Clone this branch
2. Create & activate a new dev environment with the jupyter_collaboration frontend by running:

micromamba create -f devenv-jcollab-frontend.yml
micromamba activate rtccore-jcollab-frontend

3. Start JupyterLab and open a file. You should see a loading spinner for ~10-30 seconds; that is expected. That spinner should eventually resolve, and you should be able to make have edits synchronized across your current tab & another tab in incognito mode.

IMO, the latency is either because we haven't implemented the global awareness websocket yet, or it is because the jupyter_collaboration frontend is inherently slow. It doesn't seem like it's caused by our server extension, since we don't get the initial client SS1 message until a few seconds before the spinner resolves. I'll spend some time to check on this.

[jzhang20133]

Looks good to me!

[dlqqq]

I spent an hour trying to figure out why the client was auto-closing every WS right after we write a message to it. Turns out, you need to pass binary=True to write byte streams. Otherwise, Tornado will automatically encode byte streams as UTF-8, which the client will reject with an opaque error log. 🤦

From the documentation:

def write_message(
self, message: Union[bytes, str, Dict[str, Any]], binary: bool = False
) -> "Future[None]":
"""Sends the given message to the client of this Web Socket.

The message may be either a string or a dict (which will be
encoded as json). If the binary argument is false, the
message will be sent as utf8; in binary mode any byte string
is allowed.

[dlqqq]

@ellisonbg @jzhang20133 Hey team! I've:

• Cleaned up the logging statements,
• Rebased onto latest main & addressed merge conflicts,
• and also fixed the latency issue we noticed by handling our websockets more gracefully.

Check out how fast the new documents load:

Screen.Recording.2025-05-13.at.4.30.10.PM.mov

Feel free to approve and merge at your convenience!

[jzhang20133]

Approve to unblock! Awesome job!