RTC - Save the file on the disk (Auto-save)

[MorphSeur]

Hi @Darshan808, @krassowski, @davidbrochart!

Issue:
During RTC in Jupyter Lab started using: jupyter lab --collaborative

• When adding, editing, deleting cells in a Notebook, during RTC, the JSON of the Notebook in the disk is not changing despite manual save using both cmd + s or File > Save Notebook.
• Although the UI shows Saving completed message, the dot remains visible in the Notebook tab indicating that the file is not saved on the disk.
• When opening Jupyter Lab, a pop-up shows up to build krassowski/jupyterlab-lsp.

What is done:

• Have download the .zip of the Jupyter Collaboration with the latest changes of the #479.
• Have download the .zip of the Jupyterlab with the latest changes of the #17595.
• Install both of them with pip install . as the releases were not triggered for both the repositories, i.e., changes are not included in the last releases.

Goal:

• Are the savings happen in a latent space?
• Ideal use case: when saving cmd + s, the file will be saved in the disk with the latest changes.

P.S.: (And of course, I understand how easy it is to raise issues compared to the effort it takes to maintain and merge PRs—thank you all for your work!)

[krassowski]

I suspect installing from .zip file will not get you the correct versions because of the monorepo setup. You would want to use dev installation instructions instead. If you can wait a few hours the new version should be up by then.

[krassowski]

@MorphSeur you can now install jupyterlab 4.4.3 and jupyter-collaboration 4.1.0b0 (pre-release, requires --pre if upgrading with pip). Your help in testing and feedback would be appreciated!

[MorphSeur]

Hi @krassowski!

Thanks a lot for all this!

Have installed both jupyterlab and jupyter-collaboration using pip as follows:

pip install jupyterlab==4.4.3
pip install jupyter-collaboration==4.1.0b0

Below, the installed packages:

jsonschema-specifications 2024.10.1
jupyter_client            8.6.3
jupyter-collaboration     4.1.0b0
jupyter-collaboration-ui  2.1.0b0
jupyter_core              5.8.1
jupyter-docprovider       2.1.0b0
jupyter-events            0.12.0
jupyter-lsp               2.2.5
jupyter_server            2.16.0
jupyter_server_fileid     0.9.3
jupyter_server_terminals  0.5.3
jupyter-server-ydoc       2.1.0b0
jupyter-ydoc              3.0.5
jupyterlab                4.4.3
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.3
langchain                 0.2.15

Have started jupyter lab using the following

jupyter lab --no-browser --collaborative

Notes:

• When opening jupyter lab, still a pop-up shows up to build krassowski/jupyterlab-lsp.
• The cmd+s works correctly.
• MINOR - When a new cell is added by a user, the scroll bar unexpectedly jumps to the top of the notebook (need to manually scroll down to locate it).
• After running a few cells, a pop-up window with the following contents appears:

---

File Changed
"examples/test/test.ipynb" has changed on disk since the last time it was opened or saved. Do you want to overwrite the file on disk with the version open here, or load the version on disk (revert)?

Cancel Revert Overwrite

---

[krassowski]

When opening jupyter lab, still a pop-up shows up to build krassowski/jupyterlab-lsp.

This is unrelated, I guess you have an old installation of jupyterlab-lsp lingering. I would try removing it (twice in case if there is a local and global installation) and installing it afresh following notes in https://github.com/jupyter-lsp/jupyterlab-lsp (the modern versions do not require build nor are using my name anymore).

After running a few cells, a pop-up window with the following contents appears:

This would be concerning (it would suggest we are missing a hash update), I will try to reproduce.

[MorphSeur]

Thank you @krassowski!

This is unrelated, I guess you have an old installation of jupyterlab-lsp lingering. I would try removing it (twice in case if there is a local and global installation) and installing it afresh following notes in https://github.com/jupyter-lsp/jupyterlab-lsp (the modern versions do not require build nor are using my name anymore).

The following is done for removing jupyter related libs before the install (use of a conda environment).

python -m pip uninstall -y jupyter-collaboration jupyter-collaboration-monorepo jupyter-collaboration-ui jupyterlab jupyterlab-lsp jupyter_client jupyter_core jupyter-docprovider jupyter-events jupyter-lsp jupyter_server jupyter_server_fileid jupyter_server_terminals jupyter-server-ydoc jupyter-ydoc jupyterlab_pygments jupyterlab_server

Will definitely check the global installation!

Thank you again!

[krassowski]

To debug where the old jupyterlab-lsp might be coming from I would also suggest checking output of jupyter labextension list command (after uninstalling the on coming from pip)

[MorphSeur]

@krassowski,

This is unrelated, I guess you have an old installation of jupyterlab-lsp lingering. I would try removing it (twice in case if there is a local and global installation) and installing it afresh following notes in https://github.com/jupyter-lsp/jupyterlab-lsp (the modern versions do not require build nor are using my name anymore).

Have followed that and removed all the extensions in the global installation.

Details:

• Install the libs:

  pip install jupyterlab==4.4.3
  pip install jupyter-collaboration==4.1.0b0
  pip install jupyterlab-lsp
  pip install python-lsp-server[all]
  

• The installed lib versions:

   jsonschema-specifications 2024.10.1
   jupyter_client            8.6.3
   jupyter-collaboration     4.1.0b0
   jupyter-collaboration-ui  2.1.0b0
   jupyter_core              5.8.1
   jupyter-docprovider       2.1.0b0
   jupyter-events            0.12.0
   jupyter-lsp               2.2.5
   jupyter_server            2.16.0
   jupyter_server_fileid     0.9.3
   jupyter_server_terminals  0.5.3
   jupyter-server-ydoc       2.1.0b0
   jupyter-ydoc              3.0.5
   jupyterlab                4.4.3
   jupyterlab-lsp            5.1.0
   jupyterlab_pygments       0.3.0
   jupyterlab_server         2.27.3
   langchain                 0.2.15
  

Notes:

• Still having the pop-up related to the following build: krassowski/jupyterlab-lsp.

[krassowski]

@MorphSeur for these two issues:

MINOR - When a new cell is added by a user, the scroll bar unexpectedly jumps to the top of the notebook (need to manually scroll down to locate it).

After running a few cells, a pop-up window with the following contents appears:

Are you able to reproduce them on the Binder https://gist.github.com/krassowski/c732d13e086db326e499849a62b213eb ? If so can you provide more details? I tried did not encounter either, so more details on reproduction path would be highly valuable.

Also, what browser are you using?

[MorphSeur]

Hi @krassowski!

Have used the Binder; the pop-up window related to the overwrite is not showing up.
This is due to the fact that I'm the lone working on the Notebook.

Have tried to reproduce the same behavior as in my local system, i.e., 2 persons working on the same Notebook, and it was not successful. This means that the fact of opening a Notebook in Binder, you cannot find the same Notebook if you open the same Binder in Incognito tab or in another browser (Brave browser).

The browser I use is Chrome: Version 136.0.7103.114 (Official Build) (arm64).

[krassowski]

You can use this button to get URL with token to share from Binder:

But with the hint that it happens with 2 users I was able to reproduce the warning dialog. I believe this is interaction of manual save and autosave (when multiple users are connected autosave gets enabled).

@Darshan808 should we restore the guard preventing manual save when (a) autosave is enabled and (b) autosave is disabled but gets auto-enabled because another user joined? Should we also indicate that the notebook will be auto-saving somehow? Maybe changing the icon of "save"?

[krassowski]

Or maybe we just need to propagate the latest hash to all clients on all kinds of save? It looks like something is amiss here indeed.

[krassowski]

Testing a bit more, I see that hash propagates well, although an interesting things happens on manual Ctrl + S:

• hash gets set to null in user's browser after sending the send request (here).
• once save request is completed the backend emits a save event which restores the hash to whatever was saved

This is not a problem itself; the warning only shows up on an actual mismatch of hashes. This mismatch is possible if the server-emitted hash comes with a delay. I wonder if this is down to lack of await in:

jupyter-collaboration/packages/docprovider/src/ydrive.ts

Lines 138 to 146 in 7a3986a

	if (provider) {
	provider.wsProvider?.ws?.send(SAVE_MESSAGE);
	const fetchOptions: Contents.IFetchOptions = {
	type: options.type,
	format: options.format,
	content: false
	};
	return this.get(localPath, fetchOptions);
	}

We could:

1. await for round-trip of provider.wsProvider?.ws?.send(SAVE_MESSAGE) to know that document actually got saved
2. maybe include hash parameter in fetchOptions?