rust-driver-sync.mdx

name
2025-12-24-rust-driver-sync

Turso - is the SQLite compatible database written in Rust. One of the important features of the Turso - is native ability to sync database with the Cloud in both directions (push local changes and pull remote changes).

Your task is to generate EXTRA functionality on top of the existing Rust driver which will extend regular embedded database with sync capability. Do not modify existing driver - its already implemented in the lib.rs Your task is to write extra code which will use abstractions from lib.rs and build sync support in the Rust on top of it in the lib.rs file.

Rules

General rules for driver implementation you MUST follow and never go against these rules:

• USE already implemented driver - DO NOT copy it
• SET async_io=True for the driver database configuration - because partial sync support requires TURSO_IO to handled externally from the bindings
• STRUCTURE of the implementation
  • Declaration order of elements and semantic blocks MUST be exsactly the same
  • (details and full enumerations omited in the example for brevity but you must generate full code)

/// A builder for `Database`.
pub struct Builder {
    path: String,
    remote_url: String,
    auth_token: Option<String>,
    client_name: Option<String>,
    long_poll_timeout: Option<Duration>,
    bootstrap_if_empty: bool,
    partial_sync_config_experimental: Option<PartialSyncOpts>,
}

pub struct Database { ... }

impl Database {
    pub async fn push(&self) -> crate::Result<()> { ... }
    pub async fn pull(&self) -> crate::Result<bool> { ... }
    pub async fn checkpoint(&self) -> crate::Result<()> { ... }
    pub async fn stats(&self) -> crate::Result<DatabaseSyncStats> { ... }
    pub async fn connect(&self) -> crate::Result<Connection> { ... }
}

impl Builder {
    pub fn new_remote(path: &str, remote_url: &str) -> Self { ... }
    /// ... more methods to configure builder with extra parameters ...
    
    /// Build the synced database.
    pub async fn build(self) -> crate::Result<Database> { ... }
}

• STREAM data from the http request to the completion in chunks and spin async operation in between in order to prevent loading whole response in memory
• FOCUS on code readability: extract helper functions if it will contribute to the code readability (do not overoptimize - it's fine to have some logic inlined especially if it is not repeated anywhere else)
• WATCH OUT for variables scopes and do not use variables which are no longer accessible
• USE hyper to perform HTTP(s) requests
• ACCEPT https://, http:// protocols and also extra libsql:// protocol which internally should be translated to https://
• You MUST implement custom poll future to interact with TursoDatabaseAsyncOperation
• You MUST create separate tokio thread for processing HTTP requests queue
• You MUST provide extra_io callback which will send IO request from the SyncEngineIO queue to separate IO thread
• You MUST call step_io_callbacks in the IO thread after making some progress with IO (pushing more data, finishing request, etc)
• As IO worker don't have a guaranteed way to track relation between IO and request - you MUST await all futures when IO progressed from IO thread
  • Do this in the IO worker main loop
• EXTRACT ALL constants at the top of the sync file
• DO NOT LOAD full http response body in memory - instead stream it to the completion with push_buffer method
• You MUST support TLS (HTTPS) and add necessary TLS connnector for that purpose

Implementation

• Annotate public API with types
• Add comments about public API fields/functions to clarify meaning and usage scenarios
• Use turso_sync_database_create() method for creation of the synced database for now - DO NOT use init + open pair
• Be careful with hyper typing: RequestBuilder is typed by the body type and this can cause conflict if you are using Full and Empty as body
• Implement proper waking machinery which will connect IoWorker with pending futures
• Use Client<HttpsConnector<HttpConnector>, Full<Bytes>> for hyper client
  • With HttpConnector from hyper_util::client::legacy::connect::HttpConnector
• Use turso-sync-rust as client_name if not set by user
• Write and Read files (FullRead / FullWrite) in NON-CHUNKED mode - since files are usually small (but HTTP payloads can be huge)

driver

Current dreiver implementation consist of following main components:

sdk-kit

Your implementation must use sdk-kit for embedded db and sync extension Look at the rust API of the kit here:

Sync engine provide simple "step"-based API and you MUST integrate it to the Rust driver async:

Be careful with TursoDatabaseAsyncOperation ownership as it is wrapped in unique Box container. You must use it from single place.

hyper

Use following example to get up-to date understanding of Hyper API:

For HTTPS project already installed hyper_tls dep. You can inspect example here: