Add compatibility with Jupyter Collaboration v4

[dlqqq]

Description

• Closes Add compatibility with jupyter_collaboration in the same environment #102
• Closes Emit to jupyter_events #105

This PR allows jupyter_server_documents to be installed on top of jupyter_collaboration~=4.0. This means that any package depending on jupyter_collaboration~=4.0 can optionally install jupyter_server_documents to upgrade to the new RTC experience provided by this extension. 🎉

The backend has also been updated to provide compatibility with server extensions depending on jupyter_collaboration~=4.0. Specifically:

• The extension now emits events to the same event schema IDs as jupyter_collaboration. This allows consumers to get notified when a new room is created.

• The extension now provides the same API as jupyter_collaboration under self.settings["jupyter_server_ydoc"]. This allows consumers to get a JupyterYDoc for a room.

Details

• The changes to jupyter-config/server-config/jupyter_server_documents.json disable the jupyter_server_ydoc server extension when this package is installed.

• The changes to package.json disable the @jupyter/docprovider-extension labextension, which we override entirely. This required adding an extra optional plugin; see changes to src/index.ts for more details.

• Events are emitted to jupyter_events via a new YRoomEventsAPI class. Each YRoom instance initializes one to use.

• Fixes the plugin IDs for each plugin provided by our extension. Plugin IDs should always take the syntax <npm-package-name>:<plugin-name>.

Reviewer instructions

I've added a new devenv-jcollab.yaml file that replaces the older dev environments. This creates an environment with jupyterlab>=4.4 & jupyter_collaboration>=4.0. You can use this environment file to test this branch:

# checkout this branch locally
git fetch dlqqq && git switch jcollab-compatibility

# create new dev env w/ JCollab 4
conda create -f devenv-jcollab.yaml

# dev install JSD on top of JCollab 4
jlpm dev:install

Everything should work as expected.

[ellisonbg]

Nice! QQ: what do you think about jupyter_server_documents overriding settings["jupyter_server_ydoc"] with an API compatible object that has the needed get_documents function so server extension just work?

[dlqqq]

@ellisonbg I can explore if "perfect compatibility" is possible. Will look into it and make the changes if they're simple enough. 👍

[dlqqq]

@ellisonbg Done. The backend API should now be compatible with any extension using jupyter_collaboration~=4.0. 👍

[dlqqq]

Rebased onto main and addressed conflicts w/ #103. 👍

[3coins]

@dlqqq
This looks great 🚀 . Tested, and everything with server docs works.

One minor question about the store in the yroom events.