Pre-proposal: Adding optional field `execution_state` to `kernel_info_reply` representing shell execution status

[dby-tmwctw]

Background

Currently, when a new client gets connected to an existing ipykernel (e.g. when refreshing web browser), it is not possible for the client to get the kernel's status. The client will have to make assumptions about the kernel status which, if the assumption is wrong, could lead to hang commands.

In addition, when the IOPub channel gets overloaded (for example, when there are a lot of display messages, hitting the ZeroMQ high wartermark limit), the status update message kernel send could be dropped. In that case, current clients will be unable to learn about kernel status as well.

Proposal

1. Add additional optional field in kernel_info_reply named execution_state for kernel status. When a new client gets connected, it will be able to poll kenel status with kernel_info_request and inspect the execution_state in the reply to get current kernel status. This field is made optional for backward compatibility.
2. Since there is currently no source of truth in the kernel class to track execution status, I propose to add a new class variable execution_state (type: str) to track the execution status on the main shell thread. Since kernel_info_request could be sent and processed on Control channel, reflecting status on the control channel is less useful. Reflecting status of the subshell status is useful, but we could start with a single string value that represent main shell status to validate the idea, and then expand to a map of (channel, status) later to stay forward compatible with subshell status.

Major Alternatives

1. We could choose to stipulate that before dispatch_shell returns, it MUST send a status update to IOPub channel. However, this requires us to parse out headers of the message and, in the situation of deserialization issue (which could indicate message tampering), could compromise ipykernel's security posture. Also, this does not solve the problem of overloaded IOPub channels.
2. We could choose to implement a kernel-tending process that tracks the status of the kernel. The downside is that this introduces questions like how do we handle multiple threads of execution, whether adding a new process will add kernel overhead, etc.

Shepherd: @jasongrout

[minrk]

Sending kernel_info_request on the shell channel as it used to always be, the answer would always be busy. But now that it can be sent on the control channel, I think this makes perfect sense.

We could choose to stipulate that before dispatch_shell returns, it MUST send a status update to IOPub channel

We should already be guaranteeing that the processing of all messages on the shell channel is preceded by status: busy and succeeded by status: idle, so I'm not sure what exactly this suggestion is about.

Now that we have the idea of sub-shells, we should probably also make sure that this is accessible for each shell, not just the main one.

[krassowski]

Is this responsibility of the kernel or of the server? There is a prior discussion which should be accounted for:

• [Proposal] Jupyter Server should handle resolving kernel lifecycle and execution states. jupyter-server/jupyter_server#990

Also, this is a solved problem with server-side execution because the execution_state gets synced via shared doc:

• Add cell execution_state jupyter-server/jupyter_ydoc#197

We could choose to implement a kernel-tending process that tracks the status of the kernel. The downside is that this introduces questions like how do we handle multiple threads of execution, whether adding a new process will add kernel overhead, etc.

Variants of this are/were explored in:

• Proposal: Restore Notebook execution progress when a browser page is reloaded jupyter-server/jupyter_server#1274
• https://github.com/jupyter-server/message_replay

[minrk]

Is this responsibility of the kernel or of the server?

Not all kernels are behind a server, and I think this makes sense in general.

[dby-tmwctw]

Hi @minrk , thanks for your support! To address your question:

We should already be guaranteeing that the processing of all messages on the shell channel is preceded by status: busy and succeeded by status: idle, so I'm not sure what exactly this suggestion is about

We do have several of such situations (return without status published) in dispatch_shell. One particular instance is that if the deserialization step fails, we do return without publishing a status (for right reasons though, since the message might be tempered).

@krassowski Thanks for your comment and supplement, I feel I echo @minrk 's point that not every use cases have server alongside kernel, as people might sometimes prefer interacting with kernel directly without additional layer of indirection.

[ianthomas23]

This was discussed in today's Jupyter Server/Kernels meeting and the consensus was approval for this in principle and encouragement to progress to a JEP and possible prototype. Here are some notes from the meeting.

Subshells are indeed relevant and the ipykernel implementation is close to being released (probably as 7.0.0) so it would be good if the solution supports subshells straight away, or at least is implemented in a way that allows them to be easily included in future. We thought of use cases for, in roughly decreasing order of importance:

1. What is the status of the main shell?
2. Are any subshells busy?
3. What is the total number of subshells that are busy (cpuload proxy when combined with main shell status).
4. Is a specific subshell busy?

There was also a reminder that it is not just a case of "busy" vs "idle" but there is also a "starting" or "initialising" state. This is not currently implemented for subshells but it would be sensible to add.

For implementation the kernel_info_reply could either:
a. Return a single map with status of main shell and all subshells, indexed by subshell_id (or None for main shell).
b. Always return main shell status, and optionally return a map of all subshell statuses provided at least one exists.

[krassowski]

I guess this could ultimately fix jupyterlab/jupyterlab#14578?