Docs feature branch: SQLite in DO (#16903)

SQLite in DO docs

---------

Co-authored-by: Kenton Varda <[email protected]>
Co-authored-by: justin-mp <[email protected]>
Co-authored-by: Jun Lee <[email protected]>

Author: 4 people

public/_redirects

@@ -356,6 +356,7 @@
 /durable-objects/reference/error-handling/ /durable-objects/best-practices/error-handling/ 301
 /durable-objects/reference/troubleshooting/ /durable-objects/observability/troubleshooting/ 301
 /durable-objects/reference/graphql-analytics/ /durable-objects/observability/graphql-analytics/ 301
+/durable-objects/api/transactional-storage-api/ /durable-objects/api/storage-api/ 301
 
 # email-routing
 /email-routing/enable-email-routing/ /email-routing/get-started/enable-email-routing/ 301

src/content/changelogs/durable-objects.yaml

@@ -5,6 +5,15 @@ productLink: "/durable-objects/"
 productArea: Developer platform
 productAreaLink: /workers/platform/changelog/platform/
 entries:
+  - publish_date: "2024-09-26"
+    title: (Beta) SQLite storage backend & SQL API availabe on new Durable Object classes 
+    description: |-
+      The new beta version of Durable Objects is available where each Durable Object has a private, embedded SQLite database. When deploying a new Durable Object class, users can [opt-in to a SQLite storage backend](/durable-objects/best-practices/access-durable-objects-storage/#sqlite-storage-backend) in order to access new [SQL API](/durable-objects/api/storage-api/#sqlexec) and [point-in-time-recovery API](/durable-objects/api/storage-api/#point-in-time-recovery), part of Durable Objects Storage API.
+
+      You cannot enable a SQLite storage backend on an existing, deployed Durable Object class. Automatic migration of deployed classes from their key-value storage backend to SQLite storage backend will be available in the future.
+
+      During the initial beta, Storage API billing is not enabled for Durable Object classes using SQLite storage backend. SQLite-backed Durable Objects will incur [charges for requests and duration](/durable-objects/platform/pricing/#billing-metrics). We plan to enable Storage API billing for Durable Objects using SQLite storage backend in the first half of 2025 after advance notice with the following [pricing](/durable-objects/platform/pricing/#sqlite-storage-backend).
+
   - publish_date: "2024-06-24"
     description: |-
       [Exceptions](/durable-objects/best-practices/error-handling) thrown from Durable Object internal operations and tunneled to the caller may now be populated with a `.retryable: true` property if the exception was likely due to a transient failure, or populated with an `.overloaded: true` property if the exception was due to [overload](/durable-objects/observability/troubleshooting/#durable-object-is-overloaded).

src/content/docs/durable-objects/api/alarms.mdx

@@ -8,7 +8,7 @@ sidebar:
 
 ## Background
 
-Durable Objects alarms allow you to schedule the Durable Object to be woken up at a time in the future. When the alarm's scheduled time comes, the `alarm()` handler method will be called. Alarms are modified using the [Transactional Storage API](/durable-objects/api/transactional-storage-api/), and alarm operations follow the same rules as other storage operations.
+Durable Objects alarms allow you to schedule the Durable Object to be woken up at a time in the future. When the alarm's scheduled time comes, the `alarm()` handler method will be called. Alarms are modified using the [Storage API](/durable-objects/api/storage-api/), and alarm operations follow the same rules as other storage operations.
 
 Notably:
 
@@ -28,7 +28,7 @@ Alarms are directly scheduled from within your Durable Object. Cron Triggers, on
 
 Alarms can be used to build distributed primitives, like queues or batching of work atop Durable Objects. Alarms also provide a mechanism to guarantee that operations within a Durable Object will complete without relying on incoming requests to keep the Durable Object alive. For a complete example, refer to [Use the Alarms API](/durable-objects/examples/alarms-api/).
 
-## Transactional Storage methods
+## Storage methods
 
 ### getAlarm
 
@@ -110,7 +110,7 @@ export class AlarmExample {
   }
   async alarm() {
     // The alarm handler will be invoked whenever an alarm fires.
-    // You can use this to do work, read from the Transactional Storage API, make HTTP calls
+    // You can use this to do work, read from the Storage API, make HTTP calls
     // and set future alarms to run using this.storage.setAlarm() from within this handler.
   }
 }
@@ -120,4 +120,4 @@ export class AlarmExample {
 
 * Understand how to [use the Alarms API](/durable-objects/examples/alarms-api/) in an end-to-end example.
 * Read the [Durable Objects alarms announcement blog post](https://blog.cloudflare.com/durable-objects-alarms/).
-* Review the [Transactional Storage API](/durable-objects/api/transactional-storage-api/) documentation for Durable Objects.
+* Review the [Storage API](/durable-objects/api/storage-api/) documentation for Durable Objects.

src/content/docs/durable-objects/api/transactional-storage-api.mdx renamed to src/content/docs/durable-objects/api/storage-api.mdx

@@ -1,18 +1,20 @@
 ---
-title: Transactional Storage
+title: Storage API
 pcx_content_type: concept
 sidebar:
   order: 2
 
 ---
 
-The Transactional Storage API allows you to achieve consistent key-value storage.
+import { Render } from "~/components";
 
-Durable Objects gain access to a persistent Transactional Storage API via `state`, the first parameter passed to the Durable Object constructor.
+The Storage API allows Durable Objects to access transactional and strongly consistent storage. A Durable Object's attached storage is private to its unique instance and cannot be accessed by other objects. 
+
+Durable Objects gain access to a persistent Storage API via `ctx.storage`, on the `ctx` parameter passed to the Durable Object constructor.
 
 While access to a Durable Object instance is single-threaded, request executions can still interleave with each other when they wait on I/O, such as when waiting on the promises returned by persistent storage methods or `fetch()` requests.
 
-The following code snippet shows you how to store and retrieve data using the Transactional Storage API.
+The following code snippet shows you how to store and retrieve data using the Storage API.
 
 ```js
 export class Counter {
@@ -39,13 +41,20 @@ export class Counter {
 
 ## Methods
 
-The Transactional Storage API comes with several methods.
+:::note[SQLite in Durable Objects Beta]
 
-Each method is implicitly wrapped inside a transaction, such that its results are atomic and isolated from all other storage operations, even when accessing multiple key-value pairs.
+The ew beta version of Durable Objects is available where each Durable Object has a private, embedded SQLite database. When deploying a new Durable Object class, users can [opt-in to a SQLite storage backend](/durable-objects/best-practices/access-durable-objects-storage/#sqlite-storage-backend) in order to access new [SQL API](/durable-objects/api/storage-api/#sqlexec). Otherwise, a Durable Object class has a key-value storage backend.
 
-### get
+:::
 
+The Storage API comes with several methods, including key-value (KV) API, SQL API, and point-in-time-recovery (PITR) API. 
 
+- Durable Object classes with the default, key-value storage backend can use KV API.
+- Durable Object classes with the [SQLite storage backend](/durable-objects/best-practices/access-durable-objects-storage/#sqlite-storage-backend) can use KV API, SQL API, and PITR API. KV API methods like `get()`, `put()`, `delete()`, or `list()` store data in a hidden SQLite table. 
+
+Each method is implicitly wrapped inside a transaction, such that its results are atomic and isolated from all other storage operations, even when accessing multiple key-value pairs.
+
+### get
 
 * <code>get(keystring, optionsObjectoptional)</code> : Promise\<any>
 
@@ -105,7 +114,7 @@ Each method is implicitly wrapped inside a transaction, such that its results ar
 
   * After any write, subsequent network messages may be slightly delayed. Some applications may consider it acceptable to communicate on the basis of unconfirmed writes. Some programs may prefer to allow network traffic immediately. In this case, set `allowUnconfirmed` to `true` to opt out of the default behavior.
 
-  * If you want to allow some outgoing network messages to proceed immediately but not others, you can use the allowUnconfirmed option to avoid blocking the messages that you want to proceed and then separately call the [`sync()`](/durable-objects/api/transactional-storage-api/#sync) method, which returns a promise that only resolves once all previous writes have successfully been persisted to disk.
+  * If you want to allow some outgoing network messages to proceed immediately but not others, you can use the allowUnconfirmed option to avoid blocking the messages that you want to proceed and then separately call the [`sync()`](/durable-objects/api/storage-api/#sync) method, which returns a promise that only resolves once all previous writes have successfully been persisted to disk.
 
 * <code>noCache</code>boolean
 
@@ -188,7 +197,7 @@ The `put()` method returns a `Promise`, but most applications can discard this p
 
   * Synchronizes any pending writes to disk.
 
-  * This is similar to normal behavior from automatic write coalescing. If there are any pending writes in the write buffer (including those submitted with [the `allowUnconfirmed` option](/durable-objects/api/transactional-storage-api/#supported-options-1)), the returned promise will resolve when they complete. If there are no pending writes, the returned promise will be already resolved.
+  * This is similar to normal behavior from automatic write coalescing. If there are any pending writes in the write buffer (including those submitted with [the `allowUnconfirmed` option](/durable-objects/api/storage-api/#supported-options-1)), the returned promise will resolve when they complete. If there are no pending writes, the returned promise will be already resolved.
 
 ### getAlarm
 
@@ -198,7 +207,7 @@ The `put()` method returns a `Promise`, but most applications can discard this p
 
 #### Supported options
 
-* Same options as [`get()`](/durable-objects/api/transactional-storage-api/#get), but without `noCache`.
+* Same options as [`get()`](/durable-objects/api/storage-api/#get), but without `noCache`.
 
 ### setAlarm
 
@@ -216,9 +225,103 @@ The `put()` method returns a `Promise`, but most applications can discard this p
 
 #### Supported options
 
-* `setAlarm()` and `deleteAlarm()` support the same options as [`put()`](/durable-objects/api/transactional-storage-api/#put), but without `noCache`.
+* `setAlarm()` and `deleteAlarm()` support the same options as [`put()`](/durable-objects/api/storage-api/#put), but without `noCache`.
+
+### sql.exec
+
+:::note[SQLite in Durable Objects Beta]
+
+SQL API methods accessed with `ctx.storage.sql` are only allowed on [Durable Object classes with SQLite storage backen](/durable-objects/reference/durable-objects-migrations/#enable-sqlite-storage-backend-on-create-durable-object-class-migration) and will return an error if called on Durable Object classes with a key-value storage backend.
+
+:::
+
+<code>ctx.storage.sql.exec(query: string, ...bindings: any[])</code> : SqlStorageCursor
+
+#### Parameters
+* `query`: string 
+  * The SQL query string to be executed. `query` can contain `?` placeholders for parameter bindings.
+* `bindings`: any[] Optional
+  * Optional variable number of arguments that correspond to the `?` placeholders in `query`.
+
+#### Returns 
+A cursor (`SqlStorageCursor`) to iterate over query row results as objects. `SqlStorageCursor` is a JavaScript [Iterable](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterable_protocol), which supports iteration using `for (let row of cursor)`. `SqlStorageCursor` is also a JavaScript [Iterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterator_protocol), which supports iteration using `cursor.next()`.
+
+`SqlStorageCursor` supports the following methods:
+  
+* `next()`
+  * Returns an object representing the next value of the cursor. The returned object has  `done` and `value` properties adhering to the JavaScript [Iterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterator_protocol). `done` is set to `false` when a next value is present, and `value` is set to the next row object in the query result. `done` is set to `true` when the entire cursor is consumed, and no `value` is set. 
+* `toArray()`
+  * Iterates through remaining cursor value(s) and returns an array of returned row objects. 
+* `one()`
+  * Returns a row object if query result has exactly one row. If query result has zero rows or more than one row, `one()` throws an exception. 
+* `raw()`: Iterator
+  * Returns an Iterator over the same query results, with each row as an array of column values (with no column names) rather than an object. 
+  * Returned Iterator supports `next()`, `toArray()`, and `one()` methods above.
+  * Returned cursor and `raw()` iterator iterate over the same query results and can be combined. For example:
+
+```ts
+  let cursor = this.sql.exec("SELECT * FROM artist ORDER BY artistname ASC;");
+  let rawResult = cursor.raw().next();
+
+  if (!rawResult.done) {
+    console.log(rawResult.value); // prints [ 123, 'Alice' ]
+  } else {
+    // query returned zero results
+  }
+
+  console.log(cursor.toArray()); // prints [{ artistid: 456, artistname: 'Bob' },{ artistid: 789, artistname: 'Charlie' }]
+```
+`SqlStorageCursor` had the following properties:
+
+* `columnNames`: string[]
+  * The column names of the query in the order they appear in each row array returned by the `raw` iterator.
+*  `rowsRead`: number
+  * The number of rows read so far as part of this SQL `query`. This may increase as you iterate the cursor. The final value is used for [SQL billing](/durable-objects/platform/pricing/#sqlite-storage-backend).
+*  `rowsWritten`: number
+  * The number of rows written so far as part of this SQL `query`. This may increase as you iterate the cursor. The final value is used for [SQL billing](/durable-objects/platform/pricing/#sqlite-storage-backend).
 
+#### Examples
 
+<Render file="durable-objects-sql" />
+
+### sql.databaseSize
+
+<code>ctx.storage.sql.databaseSize</code> : number
+
+#### Returns
+The current SQLite database size in bytes.
+
+```ts
+  let size = ctx.storage.sql.databaseSize;
+```
+
+### Point in time recovery
+
+For [Durable Objects classes with SQL storage](/durable-objects/reference/durable-objects-migrations/#enable-sqlite-storage-backend-on-new-durable-object-class-migration), the following point-in-time-recovery (PITR) API methods are available to restore a Durable Object's embedded SQLite database to any point in time in the past 30 days. These methods apply to the entire SQLite database contents, including both the object's stored SQL data and stored key-value data using the key-value `put()` API. The PITR API is not supported in local development because a durable log of data changes is not stored locally. 
+
+The PITR API represents points in times using "bookmarks". A bookmark is a mostly alphanumeric string like `0000007b-0000b26e-00001538-0c3e87bb37b3db5cc52eedb93cd3b96b`. Bookmarks are designed to be lexically comparable: a bookmark representing an earlier point in time compares less than one representing a later point, using regular string comparison.
+
+<code>ctx.storage.getCurrentBookmark()</code> : Promise\<string>
+
+  * Returns a bookmark representing the current point in time in the object's history.
+
+<code>ctx.storage.getBookmarkForTime(timestamp: number | Date)</code>: Promise\<string>
+
+  * Returns a bookmark representing approximately the given point in time, which must be within the last 30 days. If the timestamp is represented as a number, it is converted to a date as if using `new Date(timestamp)`.
+
+<code>ctx.storage.onNextSessionRestoreBookmark(bookmark: string)</code>: Promise\<string>
+
+  * Configure the Durable Object so that the next time it restarts, it should restore its storage to exactly match what the storage contained at the given bookmark. After calling this, the application should typically invoke `ctx.abort()` to restart the Durable Object, thus completing the point-in-time recovery.
+
+  This method returns a special bookmark representing the point in time immediately before the recovery takes place (even though that point in time is still technically in the future). Thus, after the recovery completes, it can be undone by performing a second recovery to this bookmark.
+
+
+```ts
+  let now = new Date();
+  // restore to 2 days ago
+  let bookmark = ctx.storage.getBookmarkForTime(now - 2);
+  ctx.storage.onNextSessionRestoreBookmark(bookmark);
+```
 
 ### Related resources
 

src/content/docs/durable-objects/api/websockets.mdx

@@ -32,7 +32,7 @@ To learn more about WebSocket Hibernation, refer to [Build a WebSocket server wi
 
   * Keeps a copy of `value` in memory (not on disk) to survive hibernation. The value can be any type supported by the [structured clone algorithm](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm), which is true of most types.
 
-  * If you modify `value` after calling this method, those changes will not be retained unless you call this method again. The serialized size of `value` is limited to 2,048 bytes, otherwise this method will throw an error. If you need larger values to survive hibernation, use the [Transactional Storage API](/durable-objects/api/transactional-storage-api/) and pass the corresponding key to this method so it can be retrieved later.
+  * If you modify `value` after calling this method, those changes will not be retained unless you call this method again. The serialized size of `value` is limited to 2,048 bytes, otherwise this method will throw an error. If you need larger values to survive hibernation, use the [Storage API](/durable-objects/api/storage-api/) and pass the corresponding key to this method so it can be retrieved later.