You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- Services are just serve extensions with dependencies.
34
-
- Add dependency injection system.
35
-
5. Namespacing static files and REST API urls.
37
+
1. Namespacing static files and REST API urls.
36
38
- Each extension serves static files at the `/static/<extension>` url.
37
-
-
38
-
6. Migrate configuration.
39
+
-New `ExtensionHandlerApp` class
40
+
1. Migrate configuration.
39
41
- Notebook App configuration stays in `jupyter_notebook_config.py`
40
42
- Server and services configurations move to `jupyter_server_config.py`
41
43
- Add a `migrate` application to automate this migration.
42
-
7. Add necessary documentation to notebook and jupyter_server repos.
44
+
1. Add necessary documentation to notebook and jupyter_server repos.
43
45
44
46
## Detailed Explanation
45
47
46
-
### Create a`jupyter_server` repository
48
+
### Separate`jupyter_server` repository
47
49
48
50
The first thing to do is fork `notebook`. A new `jupyter_server` repo will keep the server specific logic and remove:
49
51
1. the notebook frontend code,
50
52
2. deprecated notebook server APIs, and
51
53
3. tornado handlers to the classic notebook interface. These pieces will stay in the `notebook` repository.
52
54
53
-
Things that stay in notebook:
54
-
55
+
Modules that stay in notebook:
55
56
-`edit` module: the handlers for the classic notebook text editor.
56
57
-`templates` directory: all html templates for the classic notebook
57
58
-`terminal` module: handlers for the classic notebook terminal application.
58
59
-`view` module: handlers for file viewer component.
59
60
-`static` directory: all js and style sheets for notebook frontend.
60
61
-`tree` module: a classic notebook file browser+handlers.
61
-
-`auth` module? *(Should this move to jupyter_server?)*
62
62
63
-
Things that move to jupyter_server:
63
+
Modules that move to jupyter_server:
64
64
-`services` module: all jupyter server services, managers, and handlers
65
65
-`bundler`: handlers for building download bundles
66
66
-`files`: handlers for serving files from contents manager.
67
67
-`kernelspec`: handler for getting kernelspec
68
68
-`base`: base handler for jupyter apps.
69
69
-`i18n`: module for internationalizing the jupyter server
70
-
70
+
-`auth` module
71
71
72
72
Preliminary work resides in [jupyter_server](https://github.com/jupyter/jupyter_server).
73
73
74
-
### Server Extensions
74
+
### Clearly define the smallest unit that is a "Jupyter Server"
75
+
76
+
*(This section needs a lot more discussion.)*
75
77
76
-
The extension mechanism for the *jupyter server* will be the main area where server extensions differ from notebook's server extensions.
78
+
The server-notebook split is an opportunity to clearly define Jupyter's core services. The old notebook server was comprised of many services, and it could be extended by separate "server extensions". However, it isn't clear what should be a core service versus a server extension. Some extensions were "grand-fathered" in as core services to make the notebook application more full-featured (e.g. nbconvert, terminals, contents, ...). Now that we are separating the server from the classic notebook, we need to reevaluate what services are core to the Jupyter Server.
77
79
78
-
Enabled server extensions will still be loaded using the `load_jupyter_server_extension` approach when the jupyter server is started.
80
+
Jupyter server aims to be a core building block for other applications (like dashboards, standalone widgets, etc.). To achieve this, we need to define the simplest "unit" that defines a Jupyter Server: `kernels`, `kernelspec`, and `sessions`. Other services become extensions to the core server (using the `load_jupyter_server_extension` mechanism defined below).
79
81
80
-
**Server extensions as applications**
82
+
### Extensions as Applications
81
83
82
-
In the proposed jupyter_server, extension developers may also create an application from their extension, using a new `JupyterServerExtensionApp` class. Extension developers can subclass the new base class to make server extensions into Jupyter applications (configurable and launchable from the commmand line). This new class loads extension config from file and parses configuration set from the command line.
84
+
A new `ExtensionApp` class will be available in `jupyter_server.extensions.extensionapp`. It enables developers to make server extensions behave like standalone Jupyter applications. Jupyterlab is an example of this kind of server extension. It can be configured, initialized, and launched from the command line. When Lab is launched, it first initializes and configures a jupyter (notebook) server, then loads the configured jupyterlab extension and redirects the use to the Lab landing page.
83
85
84
-
For example, the legacy notebook could be started: 1) as an enabled extension or 2) by running the traitlets application from the command line.
86
+
`ExtensionApp` handles the boilerplate code to make Jupyter applications that configure and launch server extensions directly. It inherits `jupyter_core.JupyterApp` and exposes a command line entry point: `jupyter <extension-name>`. When an extension is launched, it first configures and starts a jupyter_server (`ServerApp`); then, it loads the configured extension and redirects users to the extension's landing page. Extension developers simply inherit the `ExtensionApp` and add the extension's `load_jupyter_server_extension` as a `staticmethod`.
85
87
86
-
Example extension:
87
88
```python
89
+
from jupyter_server.extensionapp import ExtensionApp
88
90
from .extension import load_jupyter_server_extension
89
91
90
-
classNotebookApp(JupyterServerExtensionApp):
92
+
classMyExtension(ExtensionApp):
91
93
92
-
name ='notebook'
93
-
description ='NotebookApp as a server extension.'
94
+
name ='myextension'
95
+
description ='My server extension as a Jupyter app.'
`JupyterServerExtensionApp` subclasses are configurable using Jupyter's configuration system. Users can generate config files for each extension in the Jupyter config path (see `jupyter --paths`). Each extension's configuration is loaded when the server is initialized or the extension application is launched.
98
-
99
-
As will all jupyter applications, users can autogenerate a configuration file for their extension using `jupyter my_extension --generate-config`.
99
+
`ExtensionApp`s can be configured by `jupyter_<extension-name>_config.py|json` as well. When the server extension is loaded by a server or launched from the command line, it searches the list of `jupyter --paths` for configured traits in these files.
100
100
101
101
Initial experimental work resides in [`jupyter_server_extension`](https://github.com/Zsailer/jupyter_server_extension).
102
102
103
103
**Classic notebook server extension**
104
104
105
-
The classic `NotebookApp` will become a server extension. It will inherit `JupyterServerExtensionApp`. Users can enable the notebook using the extension install/enable mechanism or enable the `notebook` in their `jupyter_server_config.py` file:
105
+
The classic `NotebookApp` will become a server extension. It will inherit `ExtensionApp`. Users can enable the notebook using the extension install/enable mechanism or enable the `notebook` in their `jupyter_server_config.py` file:
106
106
```python
107
107
c.ServerApp.jpserver_extensions = {
108
108
'notebook': True
109
109
}
110
110
```
111
111
Users can also launch the notebook application using the (usual)`jupyter notebook` command line interface.
112
112
113
-
**Extension installing/enabling mechanism**
113
+
### New server extensions mechanism.
114
114
115
115
The new extension mechanism in the *jupyter server* will differ from notebook's server extensions.
116
116
@@ -125,58 +125,54 @@ The new extension mechanism in the *jupyter server* will differ from notebook's
125
125
126
126
(Possibly) when an extension is enabled at a given precedence level, it may only look for the version of the extension installed at the same or lower precedence level. For example, if an extension `foobar` is installed and enabled system wide, but a user installs a version with `--user`, this version will only be picked up if the user also enables it with `--user`.
127
127
128
-
### Services become server extensions (with dependency injection)
129
-
130
-
Right now, there are two ways to extend and customize the jupyter server: services and server extensions. This is a bit confusing for new contributors. The main difference is that services often (but not always) **depend on other services**.
131
-
132
-
On example is the `sessions` service, which depends on the `kernels` and `kernelspec` services. Without these two services, the `sessions` service doesn't work (or even make sense).
133
-
134
-
We could reduce complexity around `jupyter_server` by making *everything* a server extension. We would need to add a **dependency injection** system to allow extensions to depend on other extensions. Some good options are [pyinject](https://github.com/google/pinject) or [python-dependency-injector](https://github.com/ets-labs/python-dependency-injector).
135
-
136
-
To port a service, it will need to be refactored using extension mechanism mentioned above.
137
-
138
128
### Add namespacing to `static` endpoints and REST API urls.
139
129
140
130
Currently, the notebook tornado application serves all static files underneath the `/static/` prefix. Jupyter server will add namespacing under the static url and extension REST API urls. Each extension will serve their static files under the `/static/<extension-name>` prefix and their API handlers behind a `/extension/api/<extension-name>` prefix.
141
131
142
132
For example, the classic notebook server extension will add static handlers that reroute requests to the `/static/notebook/` endpoints.
143
133
144
-
The jupyter_server will provide a new `JupyterExtensionHandler` base class that reroute requests to the extension's namespaced static and REST API endpoints.
134
+
A new `ExtensionHandler` class will be available in `jupyter_server.extensions.handlers`. This class inherits `JupyterHandler`. It handles the boilerplate code to reroute requests extension's namespaced static and REST API endpoints.
145
135
146
136
Preliminary experimental work resides in the [`jupyter_server_extension`](https://github.com/Zsailer/jupyter_server_extension) repository.
147
137
148
138
### Configuration System
149
139
150
-
The configuration of the server and the legacy notebook are currently tightly coupled in a single NotebookApp configurable. The proposed jupyter server has server-specific configurations in a separate config system from the classic notebook (and jupyterlab) application configurations.
151
-
152
-
The table below show all the classic notebook traits and where they will live after migration.
153
-
154
-
**Overview of configuration structure**
155
-
156
-
The notebook and server configuration both live in the `jupyter_notebook_config.py` file. Extensions are configured in separate files (named after the extension) in the `jupyter_notebook_config.d` directory:
157
-
```
158
-
~/.jupyter/
159
-
├── jupyter_notebook_config.py
160
-
└── jupyter_notebook_config.d
161
-
└── my_extension.json
162
-
```
163
-
164
-
**New proposed configuration structure**
165
-
166
-
The jupyter_server configuration lives in the `jupyter_server_config.py` file. Extensios are configured in separate files in the `jupyter_server_config.d` folder. The notebook configuration is stored in a `notebook.py` file, just like other extensions.
167
-
```
168
-
~/.jupyter/
169
-
├── jupyter_server_config.py
170
-
└── jupyter_server_config.d
171
-
├── notebook.py|json
172
-
├── lab.py|json
173
-
└── my_extension.py|json
174
-
```
140
+
Splitting the server-specific pieces from the classic notebook affects Jupyter's configuration system. This is a non-trivial problem. Changing Jupyter's configuration system affects everyone. We need to consider how to make this transition as painless as possible. At the end of this section, we list some steps that make reduce the friction.
141
+
142
+
Here is a list of things the changes on the configuration system:
143
+
* Move server-specific configuration from `jupyter_notebook_config.py|json` into `jupyter_server_config.py|json`.
144
+
* Notebook configuration will stay in `jupyter_notebook_config.py|json`.
145
+
* Server extensions configurations move from `jupyter_notebok_config.d` to `jupyter_server_config.d`.
146
+
* The tornado server and webapp configurable applications move to `jupyter_server`. They become `ServerApplication` and `ServerWebApp`
147
+
* The `NotebookApp` becomes a server extension. It would only load notebook specific configuration/traitlets, from `jupyter_notebook_config.py|json`.
148
+
* Server extensions are found using the `jpserver_extensions` trait instead of the `nbserver_extensions` trait in the `ServerApp`.
149
+
* Extension configuration files in `jupyter_server_config.d` must be enabled using the `jpserver_extensions` trait. They are enabled by JSON config files in `jupyter_server_config.d`.
150
+
* Extensions can create their own configuration files in `{sys-prefix}/etc/jupyter/` or `~/.jupyter`, i.e. `jupyter_<my-extension>_config.py|json`.
151
+
* General `jupyter_config.py|json` files must update to set server-specific configuration using the `ServerApp` and notebook specific configuration using `NotebookApp`.
152
+
153
+
Some traits will stay in `jupyter_notebook_config.py|json`. Here is a list of those traits (everything else moves to server config files):
154
+
* extra_nbextensions_path
155
+
* enable_mathjax
156
+
* max_body_size
157
+
* max_buffer_size
158
+
* notebook_dir
159
+
* mathjax_url
160
+
* get_secure_cookie_kwargs
161
+
* mathjax_config
162
+
* ignore_minified_js
163
+
164
+
Here are some steps we can take to reduce the friction for transitioning:
165
+
***Copy (not move)**`jupyter_notebook_config.py|json` to `jupyter_server_config.py|json`
166
+
***Copy (not move)**`jupyter_notebook_config.d/` to `jupyter_server_config.d`.
167
+
*`NotebookApp` becomes `ServerApp` in all copied files.
168
+
* Leftover server traits in `jupyter_notebook_config.py|json` get ignored when the notebook extension is initialized. Server traits are only read from `jupyter_server_config.py|json`.
169
+
* Document like crazy!
170
+
171
+
For an thorough overview of the configuration system and a side-by-side diff between the (current) notebook and (future) server config layout, take a look at [Zsailer/jupyter_config_overview](https://github.com/Zsailer/jupyter_config_overview).
175
172
176
173
**Migration application**
177
174
178
-
To make migration easier on users, the jupyter server will include a `migrate` application to split notebook and server configurations into their appropriate locations (listed above). This application will find any `jupyter_notebook_config.py|json` files in `jupyter --paths`, read configured traits, sort server vs. notebook traits, and write them to the appropriate `jupyter_server_config.py|json` and `jupyter_notebook_config.py|json` files.
179
-
175
+
To make migration easier on users, a `migrate` application will be available to split notebook and server configurations into their appropriate locations (listed above).
180
176
181
177
### How this effects other projects
182
178
@@ -188,7 +184,7 @@ In short, the classic notebook will become a server extension application. The r
188
184
Jupyter lab will also become a server extension application. The new classes described above should simplify the way JupyterLab interfaces with the server.
189
185
190
186
[**Kernel Gateway**]()
191
-
Kernel Gateway writes custom `kernel` and `kernelmanager` services and load them as server extensions.
187
+
Kernel Gateway can use the new Jupyter Server to server kernels as a service. The new Jupyter server will remove the unwanted services that Kernel Gateway currently removes by using a custom server application. KG will also be able to swap out the kernels and kernelspec manager in the Jupyter Server with its custom classes.
192
188
193
189
[**Kernel Nanny**]()
194
190
@@ -209,10 +205,11 @@ Kernel Gateway writes custom `kernel` and `kernelmanager` services and load them
209
205
210
206
## Relevent Issues, PRs, and discussion
211
207
212
-
Moving to a `conf.d` approach.
208
+
Jupyter's configuration system.
213
209
* PR [3116](https://github.com/jupyter/notebook/pull/3116), `jupyter/notebook`: extension config in `config.d` directory.
214
210
* PR [3782](https://github.com/jupyter/notebook/issues/3782), `jupyter/notebook`: server extension use `conf.d` approach
0 commit comments