OPFS: Error calling db.open with coi worker

[thesmartwon]

What happens?

duckdb-browser-coi.worker.ts crashes after page reload with:

DataCloneError: Failed to execute 'postMessage' on 'Worker': FileSystemSyncAccessHandle object could not be cloned.
    at ha.registerFileHandle (duckdb-browser-coi.worker.ts:33:16)
    at ha.registerFileHandleAsync (duckdb-browser-coi.worker.ts:33:16)
    at async ha.prepareDBFileHandle (duckdb-browser-coi.worker.ts:33:16)
    at async xo.onMessage (duckdb-browser-coi.worker.ts:33:16)
    at async globalThis.onmessage (duckdb-browser-coi.worker.ts:33:16)

To Reproduce

import * as duckdb from "@duckdb/duckdb-wasm";
import duckdb_coi from "@duckdb/duckdb-wasm/dist/duckdb-coi.wasm?url";
import duckdb_coi_worker_pthread from "@duckdb/duckdb-wasm/dist/duckdb-browser-coi.pthread.worker.js?url";
import duckdb_coi_worker from "@duckdb/duckdb-wasm/dist/duckdb-browser-coi.worker.js?url";

const MANUAL_BUNDLES: duckdb.DuckDBBundles = {
	coi: {
		mainModule: duckdb_coi,
		mainWorker: duckdb_coi_worker,
		pthreadWorker: duckdb_coi_worker_pthread,
	},
};

const bundle = await duckdb.selectBundle(MANUAL_BUNDLES);
const worker = new Worker(bundle.mainWorker!);
const logger = new duckdb.ConsoleLogger();
const db = new duckdb.AsyncDuckDB(logger, worker);
await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
await db.open({
	path: "opfs://persisted.db",
	accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
});
const conn = await db.connection();
await c.query("CREATE TABLE IF NOT EXISTS ttable AS SELECT 1");

Browser/Environment:

any

Device:

any

DuckDB-Wasm Version:

1.30.0

DuckDB-Wasm Deployment:

Unreleased

Full Name:

thesmartwon

Affiliation:

none

[thesmartwon]

I'm not sure why each worker thread needs to register a file. Only one thread should ever do that, since FileSystemFileHandle is exclusive in OPFS (except in Chrome with the mode option).

Simply removing these lines works just fine with db.open("opfs://") and db.registerFileText():

duckdb-wasm/packages/duckdb-wasm/src/bindings/bindings_base.ts

Lines 600 to 614 in 7a914dc

	if (this.pthread) {
	for (const worker of this.pthread.runningWorkers) {
	worker.postMessage({
	cmd: 'registerFileHandle',
	fileName: name,
	fileHandle: handle,
	});
	}
	for (const worker of this.pthread.unusedWorkers) {
	worker.postMessage({
	cmd: 'dropFileHandle',
	fileName: name,
	});
	}
	}

[thesmartwon]

Relevant commit is 00ed1c3 .

Don't know if you're still around, but do you have any pointers @ankoh 😅 ? My use case is accessing the same persisted duckdb instance from multiple tabs.

Right now I cannot decide between using one shared instance of the multithreaded coi API or one shared writer of the eh API with many readers (which requires #1962).

[thesmartwon]

Right now I cannot decide between using one shared instance of the multithreaded coi API or one shared writer of the eh API with many readers (which requires #1962).

After reading and testing the DuckDB concurrency model, it looks like the only way to handle concurrency with the ability to write is to have a SINGLE read/write connection. Otherwise I'd have to close all the readonly databases before opening a read/write one. That means this issue is blocking persistence and concurrency in WASM.