Support multiple backends by andrii-i · Pull Request #596 · jupyter-server/jupyter-scheduler

andrii-i · 2025-12-10T05:14:57Z

Summary

Fixes Support multiple backends #597, part of Jupyter Scheduler 3.0.0 release plan #599

Update Jupyter Scheduler to support scheduling and running arbitrary file types beyond notebooks. Multiple backends can now run simultaneously, allowing different file types to be handled by their respective backends within the same UI.

Introduces the BaseBackend abstraction, allowing backend authors to define capabilities declaratively (supported file types, output formats, scheduler/executor classes) with automatic discovery via entry points. Custom backends before could have been defined as sets of classes; this PR formalizes the pattern into a BaseBackend class with declarative configuration and automatic discovery via entry points.

Key Features

New `BaseBackend` abstraction

from jupyter_scheduler.base_backend import BaseBackend

class MyBackend(BaseBackend):
    id = "my_backend"
    name = "My Custom Backend"
    description = "Execute notebooks on my infrastructure"
    scheduler_class = "my_package.scheduler:MyScheduler"
    execution_manager_class = "my_package.executors:MyExecutionManager"
    file_extensions = ["ipynb"]
    output_formats = [{"id": "ipynb", "label": "Notebook"}]

Available backends discovery via new REST API endpoint

New GET /scheduler/backends endpoint returns available backends with their capabilities

Backends Provided by Default with Jupyter Scheduler

jupyter_server_nb - Execute notebooks via nbconvert (refactoring of an existing notebook execution logic)
jupyter_server_py - Execute Python scripts via local subprocess

Backend Discovery via Entry Points

Backends are registered using Python entry points in pyproject.toml:

[project.entry-points."jupyter_scheduler.backends"]
my_backend = "my_package.backends:MyBackend"

Backend Selection UI

Backend picker dropdown in "Create Job" form (when multiple backends support the file type)
Shows backend name and description
Auto-selects based on file extension

Configuration Options

Route legacy jobs (pre-3.0 UUID-only IDs) to a specific backend

c.SchedulerApp.legacy_job_backend = "jupyter_server_nb"

Set preferred backend per file extension

c.SchedulerApp.preferred_backends = {"ipynb": "k8s_backend"}

Code Changes

New files:

jupyter_scheduler/base_backend.py - Base class for backends
jupyter_scheduler/backend_registry.py - Registry for managing backends
jupyter_scheduler/backend_utils.py - backends discovery logic via entry points
jupyter_scheduler/job_id.py - Job ID encoding (backend_id:uuid) logic
src/util/backend-utils.ts - backend utilities

Modified:

extension.py - backend discovery and initialization
handlers.py - logic to route requests to correct backend
create-job.tsx - added backend picker UI
handler.ts - added new /scheduler/backends API endpoint

User-facing changes

Backwards-incompatible changes

None.

Pre-v3 pre-multiple-backend jobs have UUID-only IDs and are routed to legacy_job_backend which be default is set to pre-v3 local notebook execution logic. So without installing additional backends default installation works identically (single jupyter_server_nb backend).

Testing

All pre-existing tests pass
Added new tests including Playwright E2E tests for multi-backend UI with multiple mocked multiple backends

…end definition and discovery

for more information, see https://pre-commit.ci

…se database_manager_class to detect SQL storage needs

for more information, see https://pre-commit.ci

dlqqq

@andrii-i Thanks for working on this over the last month! This is an impressive amount of code. 💪

I left some code review feedback on the backend below. However, I haven't reviewed the backend or done local testing. Let's talk more about this PR Thursday (will miss standup tomorrow). Hopefully that will give @JGuinegagne time to review as well.

Regardless of we release this in v2 or v3, it would be great to publish a pre-release before publishing an official release. There are a lot of changes here, and I think we will need a bug bash to validate all of them.

jupyter_scheduler/backend_registry.py

jupyter_scheduler/backend_utils.py

jupyter_scheduler/extension.py

jupyter_scheduler/handlers.py

jupyter_scheduler/orm.py

jupyter_scheduler/backend_registry.py

andrii-i · 2026-01-28T01:07:08Z

Thank you for the review David. Added "Fixes #597, part of #599" to the top of PR description to clarify that this is planned for release as a part of v3

JGuinegagne

Lots of minor suggestions/sanity-checks.

Caveat: I'm not familiar with this codebase, some comments may be irrelevant or missing context. Feel free to use judgement on what to address.

jupyter_scheduler/tests/test_job_files_manager.py

jupyter_scheduler/tests/test_job_id.py

jupyter_scheduler/tests/test_python_executor.py

jupyter_scheduler/backend_registry.py

jupyter_scheduler/python_executor.py

jupyter_scheduler/scheduler.py

src/mainviews/create-job.tsx

…backend_id for legacy jobs at the backend, update snapshots

for more information, see https://pre-commit.ci

… support it logic more readable

for more information, see https://pre-commit.ci

JGuinegagne

LGTM w/ optional comments.

Evaluate whether to return error details for 5xx (disallowed in a normal service, but may be okay in a client application like jupyter-scheduler).

JGuinegagne · 2026-02-06T18:05:22Z

jupyter_scheduler/tests/utils.py

-            if expected_message != message:
-                return False
-        return True
+# Test utilities module (currently empty - add helpers here as needed)


non-blocking: remove module?

JGuinegagne · 2026-02-06T18:06:11Z

jupyter_scheduler/backend_registry.py


    config: BackendConfig
-    scheduler: BaseScheduler
+    scheduler: Any  # BaseScheduler at runtime, but Any to support test mocks


hum, compromising typing integrity for the sake of unit tests doesn't sound right...

JGuinegagne · 2026-02-06T18:10:41Z

jupyter_scheduler/backend_registry.py

+            if cfg.id in seen_ids:
+                raise ValueError(f"Duplicate backend ID: '{cfg.id}'")
+            if ":" in cfg.id:
+                raise ValueError(f"Backend ID cannot contain ':': '{cfg.id}'")


optional: performing two validation operations in the same loop, this is a bit odd.
Any way we could use pydantic models to validate the ID regex?

pydantic most likely supports disallowing duplicates in list.

JGuinegagne · 2026-02-06T18:18:03Z

jupyter_scheduler/handlers.py


-    def get_scheduler(self, job_id: str):
+    def get_scheduler(self, job_id: str) -> BaseScheduler:
        """Get scheduler for a job ID. Raises HTTPError(400) if backend unavailable."""


non-blocking: I know it's not from this PR, but docstring formalities would call for:

"""Return scheduler for a job ID. Raises: HTTPError(400) if backend unavailable.

JGuinegagne · 2026-02-06T18:19:15Z

jupyter_scheduler/handlers.py

-        """Resolve backend from payload['backend'] or auto-select by file extension."""
-        backend_id = payload.get("backend")
+        """Resolve backend from payload['backend_id'] or auto-select by file extension."""
+        backend_id = payload.get("backend_id")


optional: can backend_id possibly be nullish here?

jupyter_scheduler/handlers.py

JGuinegagne · 2026-02-06T18:23:26Z

jupyter_scheduler/handlers.py

        except Exception as e:
            self.log.exception(e)
-            raise HTTPError(500, "Unexpected error occurred during creation of job.") from e
+            raise HTTPError(500, f"Unexpected error during creation of job: {e}") from e


optional: per DRY principle, consider exploring context manager for error handling

JGuinegagne · 2026-02-06T18:24:38Z

jupyter_scheduler/python_executor.py

        if result.returncode != 0:
            raise RuntimeError(
-                f"Script exited with code {result.returncode}\nstderr: {result.stderr[:500]}"
+                f"Script exited with code {result.returncode}. See 'Errors' output for full error trace."


in that case, it would help to provide the file path.

JGuinegagne · 2026-02-06T18:25:41Z

ui-tests/tests/jupyter_scheduler.spec.ts

    await page.waitForSelector('text=Saving Completed', { state: 'hidden' });
-    await scheduler.assertSnapshot(FILENAMES.CREATE_JOB_VIEW);
+    // Flaky: file names and timestamps vary by environment
+    // await scheduler.assertSnapshot(FILENAMES.CREATE_JOB_VIEW);


dlqqq

Thank you for addressing our feedback Andrii!

I'm approving this PR because the main branch will continue to evolve, and I don't think see any issues worth blocking on right now. The testing you've added inspires confidence that this new feature works, so I think it's fine to merge for now.

Before the v3.0 release, we should revisit the architecture as a team, remove any unnecessary / excessively complex components, and simplify as much as possible to minimize maintenance burden. We should also think more deeply about schema changes for the local scheduler database, and whether we should add a DB migration script for users.

andrii-i · 2026-02-07T02:03:53Z

Thanks for the review and approval @dlqqq. Would be happy to have additional discussions.

whether we should add a DB migration script for users

The current update_db_schema function handles automatic column additions via ALTER TABLE - new nullable columns are added transparently on startup. That said, happy to discuss if we need something more robust.

andrii-i force-pushed the multiple-backends branch from ed1708d to 05779aa Compare December 10, 2025 05:15

andrii-i added the enhancement New feature or request label Dec 10, 2025

andrii-i force-pushed the multiple-backends branch 8 times, most recently from 694b782 to b0d7a63 Compare December 12, 2025 16:23

andrii-i and others added 9 commits December 12, 2025 08:23

initial implementation

ea89b2c

Use entry points instead of jupyter server setting traitlets for back…

6553a04

…end definition and discovery

[pre-commit.ci] auto fixes from pre-commit.com hooks

7926459

for more information, see https://pre-commit.ci

adjust details page field display order

ceb1246

add advancedOptionsOverride token to avoid token mismatch in extensions

b9cb64f

Eencode backend into job IDs, add backend field to job definitions, u…

658d52e

…se database_manager_class to detect SQL storage needs

[pre-commit.ci] auto fixes from pre-commit.com hooks

d4a21f9

for more information, see https://pre-commit.ci

rename legacy backend, cleanup tests and comments

720d26d

add python backend

3e48af6

andrii-i force-pushed the multiple-backends branch from b0d7a63 to 3e48af6 Compare December 12, 2025 16:23

pre-commit-ci bot and others added 4 commits December 12, 2025 16:24

[pre-commit.ci] auto fixes from pre-commit.com hooks

ea2fb73

for more information, see https://pre-commit.ci

add dynamic context menu registration

09e3fac

Only validate notebooks have a kernel for .ipynb files

939a2aa

add stdour and stderr for py files

b092f48

andrii-i force-pushed the multiple-backends branch from bb7cd2e to b092f48 Compare December 12, 2025 19:54

andrii-i added 5 commits December 12, 2025 12:11

Auto-select backend by file extension, fall back to default

a37a4df

Create stdout/stderr files only when there's actual content

dd4ce1c

Only add output files that actually exist to job_files / output files

8e24c36

hide backend picker only while loading

a82594f

rename default backends

e96b854

dlqqq reviewed Jan 28, 2026

View reviewed changes

JGuinegagne reviewed Jan 29, 2026

View reviewed changes

andrii-i mentioned this pull request Feb 2, 2026

Add typing to traitlets #602

Open

andrii-i and others added 9 commits February 3, 2026 02:28

add comments

2ebe15c

change id formulation

1da0e91

rename backend to backend_id, remove local fallback logic, fill in …

52b7125

…backend_id for legacy jobs at the backend, update snapshots

[pre-commit.ci] auto fixes from pre-commit.com hooks

3114e4b

for more information, see https://pre-commit.ci

comment out flakey snapshot comparasions

d5ba046

implement comments

3293cd2

[pre-commit.ci] auto fixes from pre-commit.com hooks

24dca85

for more information, see https://pre-commit.ci

return error strings with 500

508c5d9

properly map ValidationError to 400

3c10450

andrii-i force-pushed the multiple-backends branch from c8b5c07 to 3c10450 Compare February 4, 2026 19:36

pre-commit-ci bot and others added 8 commits February 4, 2026 19:36

[pre-commit.ci] auto fixes from pre-commit.com hooks

bbbf1fd

for more information, see https://pre-commit.ci

update logger.error to logger.exception to catch trace

33b6f55

check for absence of colon in the backend_id

91e4052

add typing

2d791e3

make auto-seleciong of a valid backend when preferred backend doesn't…

7a77a99

… support it logic more readable

reference strerr rather than embed part of the output

82a0ea8

fix test assertions

e217c4e

[pre-commit.ci] auto fixes from pre-commit.com hooks

878e984

for more information, see https://pre-commit.ci

andrii-i requested review from JGuinegagne and dlqqq February 5, 2026 15:11

JGuinegagne approved these changes Feb 6, 2026

View reviewed changes

dlqqq approved these changes Feb 6, 2026

View reviewed changes

andrii-i merged commit 955883c into jupyter-server:main Feb 9, 2026
6 checks passed

andrii-i deleted the multiple-backends branch February 9, 2026 17:33

andrii-i mentioned this pull request Feb 9, 2026

Multiple backends suggestions followups #605

Draft

Conversation

andrii-i commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Features

New BaseBackend abstraction

Available backends discovery via new REST API endpoint

Backends Provided by Default with Jupyter Scheduler

Backend Discovery via Entry Points

Backend Selection UI

Configuration Options

Code Changes

User-facing changes

Backwards-incompatible changes

Testing

Uh oh!

dlqqq left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andrii-i commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JGuinegagne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JGuinegagne left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dlqqq left a comment

Choose a reason for hiding this comment

Uh oh!

andrii-i commented Feb 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

andrii-i commented Dec 10, 2025 •

edited

Loading

New `BaseBackend` abstraction

andrii-i commented Jan 28, 2026 •

edited

Loading