Full JavaScript/TypeScript Support in Quarto

[dereckmezquita]

Description

I'd like to propose adding comprehensive support for JavaScript and TypeScript as first-class languages within Quarto, similar to the current support for R and Python. This feature would allow users to leverage JS/TS as powerful tools for data analysis, visualisation, and general-purpose programming within Quarto documents.

Key Features

1. Native JS/TS Runtime Support: Enable users to choose their preferred runtime (Node.js, Deno, or Bun) for executing JS/TS code chunks.
2. Persistent State: Maintain state across different code chunks within a document, allowing for complex, multi-step analyses.
3. Module Support: Enable importing and using modules from other JS/TS files or npm packages.
4. Local Execution: Perform all computations locally in a JS/TS runtime such as Node.js, Deno, or Bun, without relying on Observable JS (OJS).
5. Full Language Feature Set: Support the entire JS/TS language specification, including latest ECMAScript features and TypeScript types.
6. D3.js Integration: Add support for creating and rendering D3.js visualisations within Quarto documents.

Use Cases and Examples

Here are some examples to illustrate the potential of this feature:

---
title: "JavaScript Data Analysis in Quarto"
format: html
engine: node  # Or 'deno' or 'bun'
---

## Data Processing

```{js}
const data = [1, 2, 3, 4, 5];
const mean = data.reduce((sum, val) => sum + val, 0) / data.length;
console.log(`Mean: ${mean}`);
```

## D3.js Visualisation

```{js}
import * as d3 from 'd3';

const svg = d3.select('body').append('svg')
  .attr('width', 200)
  .attr('height', 100);

svg.selectAll('circle')
  .data(data)
  .enter()
  .append('circle')
  .attr('cx', (d, i) => i * 40 + 20)
  .attr('cy', 50)
  .attr('r', d => d * 5)
  .attr('fill', 'blue');

display(svg.node());
```

---
title: "TypeScript with modules in Quarto"
format: html
engine: bun
---

## TypeScript with modules

```{ts}
import { pythagorean } from './math';

interface Point {
  x: number;
  y: number;
}

function distance(p1: Point, p2: Point): number {
  return Math.sqrt((p2.x - p1.x)**2 + (p2.y - p1.y)**2);
}

const point1: Point = { x: 0, y: 0 };
const point2: Point = { x: 3, y: 4 };

console.log(`Distance: ${distance(point1, point2)}`);

const hypotenuse = pythagorean(3, 4);
console.log(`Hypotenuse: ${hypotenuse}`);
```

Implementation Considerations

1. Engine Selection: Allow users to specify the JS/TS runtime (Node.js, Deno, or Bun) in the document metadata or chunk options.
2. Execution Model: Implement a mechanism to maintain state between chunks, possibly using a persistent runtime process.
3. Module Resolution: Develop a system for resolving and loading both local and npm modules.
4. Output Handling: Capture and display various types of output (console logs, rendered visualisations, etc.) appropriately.
5. TypeScript Integration: Implement TypeScript compilation and type-checking within the Quarto pipeline or force the user to use a TS compatible runtime i.e. Deno, Bun.
6. Asynchronous Execution: Address the asynchronous nature of JavaScript to enable a more synchronous-like execution flow.

Handling Asynchronous JavaScript

One of the anticipated obstacles to overcome is the asynchronous nature of JavaScript. Unlike R or Python, which typically execute in a top-down manner, JavaScript often relies on asynchronous operations. To provide a more familiar experience for users, we could implement a solution that allows for a more linear execution flow. Some potential approaches include:

1. Automatic Promise Resolution: Automatically resolve promises at the end of each chunk, allowing users to write asynchronous code without explicit await statements.
2. Chunk-level Async Wrappers: Implicitly wrap each chunk in an async function, enabling the use of await within chunks without additional boilerplate.
3. Sequential Chunk Execution: Implement a mechanism to ensure that chunks are executed sequentially, waiting for all asynchronous operations in one chunk to complete before moving to the next.
4. Education and Best Practices: Provide guidance on writing asynchronous JavaScript in a literate programming context, encouraging the use of async/await and other best practices.
5. Mark chunks as async: Allow users to mark specific chunks as asynchronous, indicating that the subsequent chunks should wait for their completion.

Here's an example of how this might look in practice:

// This chunk would automatically wait for the fetch to complete
const response = await fetch('https://api.example.com/data');
const data = await response.json();
console.log(data);

// This chunk would only execute after the previous chunk has fully completed
const processedData = data.map(item => item.value * 2);
console.log(processedData);

Rationale

This feature would significantly enhance Quarto's capabilities, allowing users to leverage the vast JavaScript ecosystem for data science, web development, and general-purpose programming tasks. It would provide a seamless experience for developers familiar with JS/TS, enabling them to use these languages alongside R and Python in literate programming documents. The addition of D3.js support would particularly enhance the data visualisation capabilities within Quarto.

Discussion

I'd appreciate insights into Quarto's internal architecture, particularly regarding language engine implementation. How does Quarto currently handle different language engines, and what would be involved in adding full JS/TS support?

Differences from Existing Solutions

1. Local Runtime Execution: While I'm aware of existing solutions like Observable JS in Quarto and Rmd, I'm specifically requesting native JS/TS support using standard local runtimes (Node.js, Deno, or Bun). This approach differs from Observable JS in several key ways:
   • Code execution occurs entirely on the local machine during document build, not in the browser when the document is viewed.
   • It allows for full access to the local file system and environment, enabling more complex data processing and file operations.
   • The execution environment more closely mirrors typical JavaScript development workflows.
2. Full Language Support: This proposal aims to support the entire JS/TS language specification, including features that may not be available or easily accessible in browser-based environments like Observable.
3. Module and Package Integration: By using local runtimes, this feature would allow seamless integration with the npm ecosystem and local module systems, which is more limited in current browser-based solutions.
4. Build-time vs. Runtime Execution: Unlike solutions that run JavaScript in the browser when the document is opened, this proposal focuses on executing all JS/TS code during the document build process, similar to how R and Python code is currently handled in Quarto.

Advantages of Full Local Runtime Support

1. Development Workflow Alignment: Using standard JS/TS runtimes allows developers to work with familiar tools, testing frameworks, and debugging processes.
2. Enhanced Performance: Local execution can offer better performance for computationally intensive tasks compared to browser-based solutions.
3. Offline Capabilities: Documents can be built and executed without an internet connection, which is crucial for certain use cases and environments.
4. Security and Privacy: Sensitive data and operations can be processed locally without exposure to browser environments.
5. Integration with Local Tools and Services: Direct access to local databases, APIs, and other services that may not be accessible from a browser environment.

While Observable JS offers unique and powerful features, particularly for interactive data exploration, this proposal aims to provide a complementary solution that caters to different use cases and developer preferences. The goal is to expand Quarto's capabilities to fully embrace JavaScript and TypeScript as first-class citizens in the ecosystem, alongside R and Python.

I'm open to discussing this feature further and potentially contributing to its implementation if there's interest from the Quarto team. I believe this addition would significantly enhance Quarto's appeal to a broader range of developers and data scientists, particularly those deeply invested in the JavaScript ecosystem.

[mcanouil]

You could possibly contribute such engine to Quarto.
For instance, the native Julia engine is a community contribution.

[dereckmezquita]

Okay wonderful to know there is precedent here. Would it be these PR's? I am looking for an example of the code so I can get an idea of how to implement this.

• Native Julia engine based on QuartoNotebookRunner.jl #8645
• Julia engine - update schemas #9319

[cscheid]

I'm just going to say that these are really deep waters that I wouldn't recommend folks to attempt swimming in, unless they're very comfortable with Quarto. I estimate that a feature like this would take me a couple of months of full-time work to get it into a state that we would be willing to merge into main.

[edenian-prince]

I would love this. Using javascript in Quarto is a pain and has made me consider other publishing frameworks