Migrating Observable Javascript notebooks to vanilla Javascript?

[gnestor]

When I use notebooks download NOTEBOOK_URL to download a notebook to my filesystem, it's downloading the Observable-flavored Javascript which the runtime in notebook-kit doesn't support (e.g. viewof, import...with, mutable, and standard library). Should notebooks download handle the transpilation from OJS to vanilla JS? Are there currently any tools to do this?

[mbostock]

Notebooks 2.0 supports Observable JavaScript as a cell mode (and you can import 1.0 notebooks as described in #9), so there’s no reason for you to convert your Observable JavaScript to vanilla JavaScript unless you want to.

My opinion is that it’s generally better to convert Observable JavaScript to vanilla JavaScript by hand; you know best how to produce the most idiomatic result. For example, because 2.0 allows cells to declare multiple top-level variables, you often want to combine cells when converting from 1.0 to 2.0. Still, perhaps there’s an opportunity for the community to contribute codemods to automate or partially automate conversion.

Here’s a draft of a guide I’ve been working on…

Converting Observable JavaScript to vanilla JavaScript

This guide covers how to convert from Observable JavaScript, the nonstandard syntax introduced with Observable Notebooks 1.0, to vanilla JavaScript, the standard syntax now supported by Observable Notebooks 2.0. (Observable Framework also uses vanilla JavaScript; you might also find Framework’s Converting notebooks guide helpful.)

Note: you don’t have to convert to vanilla JavaScript! Observable JavaScript is still supported and should — with a few quirks — “just work.” That said, we highly recommend adopting vanilla JavaScript because it’s the standard. Vanilla JavaScript makes your code more easily understood by others. And vanilla JavaScript syntax allows cells to define multiple top-level variables (via standard const and let), which gives more flexibility in organizing code. And vanilla JavaScript makes it easier to reuse code between notebooks and standard web applications.

Observable Notebooks 2.0 allows intermingling of Observable and vanilla JavaScript within the same notebook via cell modes. An Observable JavaScript cell is denoted by ojs in the top-right corner of the cell editor, whereas a vanilla JavaScript cell is denoted by js. (Markdown and HTML cells are also common.) You can change the current cell’s mode via the Edit menu, or use the Command-1 keyboard shortcut to quickly convert the current cell to vanilla JavaScript.

Remove cell declarations; replace return with display.

Remove the curly braces surrounding the cell, and the cell name assignment if any. Replace the return statement with a call to display.

Before:

chart = {
  const svg = d3.create("svg");
  return svg.node();
}

After:

const svg = d3.create("svg");
display(svg.node());

Note: example above, the chart declaration is dropped because it’s not needed by the rest of the notebook; the notebook can now access the svg variable if desired. If the chart variable is referenced by other cells, you can instead say:

const svg = d3.create("svg");
const chart = display(svg.node());

If you want to hide the svg variable from other cells, you could say:

const chart = display((() => {
  const svg = d3.create("svg");
  return svg.node();
})());

Add const or let as needed.

In place of Observable JavaScript’s cell declaration, you can use a standard const or let variable declaration. Also consider merging cells, since cells can now declare multiple top-level variables.

Before:

x = d3.scaleLinear().range([marginLeft, width - marginRight])

y = d3.scaleLinear().range([height - marginTop, marginBottom])

After:

const x = d3.scaleLinear().range([marginLeft, width - marginRight]);
const y = d3.scaleLinear().range([height - marginTop, marginBottom]);

Replace viewof with view.

Before:

viewof gain = Inputs.range([0, 11], {value: 5, step: 0.1, label: "Gain"})

After:

const gain = view(Inputs.range([0, 11], {value: 5, step: 0.1, label: "Gain"}));

Replace mutable with Mutable.

Before:

mutable foo = 42

After:

const foo = Mutable(42);
const setFoo = (x) => (foo.value = x);

Add the observable: to notebook imports.

If your notebook imports another notebook from Observable, you’ll need to add the observable: protocol to the import declaration (to distinguish it from importing a standard JavaScript module, which is now supported in vanilla JavaScript).

Before:

import {BarChart} from "@d3/bar-chart"

After:

import {BarChart} from "observable:@d3/bar-chart";

[Optional] Replace require with static import.

While require is still available in Notebooks 2.0, the previous AMD-based implementation has been replaced under the hood with a standard JavaScript import. We’ve deprecated the use of require because it’s not as reliable or as performant as a standard import. If you’re using require to load a library from npm, in many cases require should continue to work (as a dynamic import), but we recommend that you switch to a standard JavaScript import declaration instead.

Before:

regl = require("regl")

After:

import regl from "npm:regl";

If you really need the AMD-based implementation of require, you can import it explicitly into your notebook, replacing the one provided by the Observable standard library:

import {require} from "npm:d3-require";

[Optional] Replace dynamic import with static import.

Observable JavaScript syntax doesn’t support static JavaScript imports for libraries; it can only be used to import Observable Notebooks. In vanilla JavaScript syntax, you use the observable: protocol to denote notebook imports, which means that you can now use standard JavaScript imports for libraries.

Before:

isoformat = import("https://esm.sh/isoformat")

After:

import * as isoformat from "npm:isoformat";

[rreusser]

Very useful note, thanks! Regarding translating notebooks, one subtle behavior I noticed is that variables don't have to be unique unless you actually use them. I was initially worried about polluting some global variable namespace with all of my internal cell variables, but then I realized this is perfectly valid:

const x = 5;

const x = 6;

and only becomes invalid if you introduce ambiguity by referencing x, e.g.

// RuntimeError: x is defined more than once
x + 1

IIFEs can be a nice way to encapsulate things in the style of Observable JavaScript, but I eased off and started worrying a bit less when I figured out some additional details about how notebooks 2.0 handle vanilla js. Kinda tangential, but I'm not sure all these obscure details are fully documented just yet.

[mbostock]

Good point re. ambiguous variables! I’ll include that when I publish a guide.

[mbostock]

Oh, also note that there was a bug where you couldn’t import a viewof or mutable in Observable JavaScript cells, but that was fixed on August 3 in 4262531. I meant to publish a new release sooner but I almost have database connectors working, so I’m trying to wrap that up first. And you can import viewofs and mutables in vanilla JavaScript cells by using the viewof$ and mutable$ prefix, like so:

import {figure, viewof$rotation} from "observable:@rreusser/drawing-3d-objects-with-svg";

More in this thread: https://bsky.app/profile/ocks.org/post/3lvjhyimjuc2x

[gnestor]

I tried 1.1.0-rc.4 and it's working now 👍

[gganapat]

Is this upgrade for Desktop also or just notebook-kit?

[mbostock]

I’ve only released a rc (release candidate) for Notebook Kit. Desktop will come very soon.