Skip to content

SQLite Async API #59109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
geeksilva97 wants to merge 12 commits into
base: main
Choose a base branch
from
Open

SQLite Async API #59109

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
081d7c5
src: add async api
geeksilva97 Nov 28, 2025
7acd4eb
src: cleanup sqlite async api
geeksilva97 Nov 28, 2025
cc6263a
test,sqlite: test async classes
geeksilva97 Nov 28, 2025
58ca77f
sqlite: add sqlite property to async database
geeksilva97 Nov 28, 2025
02d12b0
sqlite: fix linter isues
geeksilva97 Nov 28, 2025
56be6da
test: fix database async tests
geeksilva97 Nov 29, 2025
ddec6fb
src: remove unused variable
geeksilva97 Nov 29, 2025
5bec78e
sqlite: set threading mode to multithreaded
geeksilva97 Dec 1, 2025
cada892
src: avoid custom functions in async mode
geeksilva97 Jan 15, 2026
28ba2bd
src: handle race condition
geeksilva97 Jan 15, 2026
45baba7
sqlite: test async ordering
geeksilva97 Jan 15, 2026
3c04b3e
test: fix lint issues
geeksilva97 Jan 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions deps/sqlite/sqlite.gyp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
'SQLITE_ENABLE_RBU',
'SQLITE_ENABLE_RTREE',
'SQLITE_ENABLE_SESSION',
'SQLITE_THREADSAFE=2',
Copy link
Contributor

@louwers louwers Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a performance penalty for purely single-thread use cases?

If so, would it make sense to allow setting the threading mode at runtime?

Copy link
Contributor Author

@geeksilva97 geeksilva97 Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know for sure about any performance penalti. But as discusses in the issue, due to the node.js nature, this will work fine for sync. Better-sqlite uses it.

Copy link
Contributor

@louwers louwers Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. Thanks! This can be closed.

If anything I think this will speed up single-threaded use cases, because apparently by default SQLite uses internal serialization. Did not test it though.

Copy link
Contributor

@mceachen mceachen Feb 12, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I was going to post this on the main thread, but this is the context that matters, so I'll add it here) (edit: I rewrote for brevity):

Synchronous node:sqlite operations can run while your new async operations are executing. This will cause concurrent access to the same SQLite connection from multiple threads. This violates SQLITE_THREADSAFE=2 requirements and can cause db corruption undefined behavior (I could have sworn I read about this on the how-to-corrupt page but I don't see it now.)

SQLITE_THREADSAFE=2 requires no connection be used by multiple threads simultaneously (docs). Currently, nothing prevents sync methods (main thread) from running while async operations are on the thread pool.

Suggested fix: throw from sync methods when has_running_task_ is true:

diff --git a/src/node_sqlite.cc b/src/node_sqlite.cc
--- a/src/node_sqlite.cc
+++ b/src/node_sqlite.cc
@@ -1257,6 +1257,8 @@ void Database::Prepare(const FunctionCallbackInfo<Value>& args) {
   ASSIGN_OR_RETURN_UNWRAP(&db, args.This());
   Environment* env = Environment::GetCurrent(args);
   THROW_AND_RETURN_ON_BAD_STATE(env, !db->IsOpen(), "database is not open");
+  THROW_AND_RETURN_ON_BAD_STATE(env, db->has_running_task_,
+    "Cannot call sync methods while async operation is running");

Same check needed in Exec() (sync path), Function(), Aggregate(), SetAuthorizer(), and LoadExtension(). Tested locally — works as expected.

(SQLITE_THREADSAFE=1 would also prevent undefined behavior, but the main thread would silently block on SQLite's internal mutex until the async op completes, defeating the purpose of async.)

Copy link

@BurningEnlightenment BurningEnlightenment Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the in-depth examples. I'd like to mention another option (which I thought has been discussed in the issue, but maybe we were talking past each other):
Have one sqlite connection for synchronous access and maintain a pool of (seperate) sqlite3 db connections for asynchronous access. For each asynchronous access one connection from the pool would be exclusively used. The main difficulty with this approach is obviously db.prepare(). One would have to track which db connection owns the prepared statement instance and only dispatch operations from that prepared statement object to its owning db connection. Another drawback of that design is the inherent potential of SQLITE_BUSY failures for concurrent write access.

Copy link
Contributor

@mceachen mceachen Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, if you don’t want to cross the streams, you need multiple database instances opened. Rather than transparently managing this magically begins the scenes in a pool, though, I think it may be less surprising to actually push the complexity of managing per -database handles to the user—they can figure out what makes sense in their app, and when one query wedges another, it’s their fault.

],
'include_dirs': ['.'],
'sources': [
Expand Down
1 change: 1 addition & 0 deletions deps/sqlite/unofficial.gni
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ template("sqlite_gn_build") {
"SQLITE_ENABLE_RBU",
"SQLITE_ENABLE_RTREE",
"SQLITE_ENABLE_SESSION",
"SQLITE_THREADSAFE=2",
]
}

Expand Down
1 change: 1 addition & 0 deletions src/env_properties.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,7 @@
V(space_stats_template, v8::DictionaryTemplate) \
V(sqlite_column_template, v8::DictionaryTemplate) \
V(sqlite_statement_sync_constructor_template, v8::FunctionTemplate) \
V(sqlite_statement_async_constructor_template, v8::FunctionTemplate) \
V(sqlite_statement_sync_iterator_constructor_template, v8::FunctionTemplate) \
V(sqlite_session_constructor_template, v8::FunctionTemplate) \
V(srv_record_template, v8::DictionaryTemplate) \
Expand Down
Loading