Improved outputs handling when reading/writing from/to disk

[ellisonbg]

This PR implements notebook output loading functionality that allows jupyter-server-documents to process and manage notebook outputs stored on disk rather than in the notebook file itself.

• Process outputs in notebooks on disk - Added core functionality to load notebook outputs from disk storage when notebooks are opened
• Enhanced OutputsManager - Extended manager with methods to process loaded and saving notebooks, including placeholder output handling and metadata management
• Improved placeholder handling - Better handling of placeholder outputs with support for both dict and Map formats
• Integration with YRoom system - Connected OutputsManager throughout the room management system for seamless output processing

Key Changes

• OutputsManager now processes notebooks on load/save with process_loaded_notebook() and process_saving_notebook() methods
• Added placeholder_outputs metadata flag to control output processing behavior
• Enhanced placeholder output creation with flexible return types (dict vs Map)
• Integrated OutputsManager across YRoom, YRoomFileAPI, and YRoomManager classes
• Fixed missing parent argument initialization in app.py

Test Plan

• Verify notebooks with placeholder_outputs: true load outputs from disk correctly
• Test notebook saving clears outputs and sets placeholder metadata
• Confirm stream output limiting works as expected
• Validate output URLs are generated correctly
• Test both new and existing notebooks handle outputs appropriately

[ellisonbg]

@dlqqq This mostly works, but I am running into an exception by pycrdt when I try to make the changes to the YDoc. The transaction if failing saying the Ydoc isn't part of a room yet. My sense is that there is a race condition between the YRoom creation/initialization and document loading in this case. No hurry on this, but you are probably the right person to dive into this.

[dlqqq]

@ellisonbg This PR looks great! Left some suggestions below.

I wasn't able to reproduce the race condition you're describing on 6 different notebooks. I don't see how this PR could introduce a race condition. The OutputsManager.load_notebook() method is synchronous, so adding a call to that method shouldn't change anything regarding concurrency.

[ellisonbg]

Here is the exception I am seeing:

Task exception was never retrieved
future: <Task finished name='Task-2746' coro=<YRoomFileAPI._load_content() done, defined at /Users/brgrange/git/jupyter-ai-contrib/jupyter-server-documents/jupyter_server_documents/rooms/yroom_file_api.py:187> exception=RuntimeError('Not integrated in a document yet')>
Traceback (most recent call last):
  File "/Users/brgrange/git/jupyter-ai-contrib/jupyter-server-documents/jupyter_server_documents/rooms/yroom_file_api.py", line 209, in _load_content
    jupyter_ydoc.source = file_data['content']
    ^^^^^^^^^^^^^^^^^^^
  File "/Users/brgrange/micromamba/envs/serverdocs/lib/python3.13/site-packages/jupyter_ydoc/ybasedoc.py", line 100, in source
    return self.set(value)
           ~~~~~~~~^^^^^^^
  File "/Users/brgrange/micromamba/envs/serverdocs/lib/python3.13/site-packages/jupyter_ydoc/ynotebook.py", line 257, in set
    self._ycells.extend([self.create_ycell(cell) for cell in cells])
                         ~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/brgrange/micromamba/envs/serverdocs/lib/python3.13/site-packages/jupyter_ydoc/ynotebook.py", line 168, in create_ycell
    if output.get("output_type") == "stream":
       ~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/brgrange/micromamba/envs/serverdocs/lib/python3.13/site-packages/pycrdt/_map.py", line 226, in get
    with self.doc.transaction():
         ^^^^^^^^
  File "/Users/brgrange/micromamba/envs/serverdocs/lib/python3.13/site-packages/pycrdt/_base.py", line 158, in doc
    raise RuntimeError("Not integrated in a document yet")
RuntimeError: Not integrated in a document yet

The notebook I am trying is this one:

https://github.com/jakevdp/PythonDataScienceHandbook/blob/master/notebooks/04.00-Introduction-To-Matplotlib.ipynb

[ellisonbg]

OK, this is updated and ready for review.

[dlqqq]

@ellisonbg Thank you for fixing that bug you found!

I've approved to unblock, but it would be great if you could downgrade the critical log & remove the asyncio.Event added in yroom_file_api.py before merging. 👍

[JGuinegagne]

how about: Mutate notebook dict in place, return updated value.

As for the "before saving" usage info, how about renaming the method to process_notebook_for_saving_to_disk ?

[3coins]

@ellisonbg
This looks great!, outputs work like magic now, I spent 5 mins looking for them in the notebook file 😆 .

Some minor comments to improve error handling and readability, but good overall.

[3coins]

Should we add error handling here?

Suggested change

	placeholder = self.write(
	file_id,
	cell_id,
	output,
	display_id,
	asdict=True,
	)
	try:
	placeholder = self.write(
	file_id,
	cell_id,
	output,
	display_id,
	asdict=True,
	)
	except Exception e:
	self.log.warning(f"Failed to write output for cell {cell_id}: {e}")
	placeholder = output

[ellisonbg]

OK, this is ready for final review.

[3coins]

Looks good!

[ellisonbg]

I found a few more edge cases that I am fixing.

[ellisonbg]

@3coins can you have a look at my latest commit, review and merge (if ready)?