-
Notifications
You must be signed in to change notification settings - Fork 9.8k
[DO] Updating Durable Objects storage API docs #24964
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+572
−438
Merged
Changes from 5 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
c21989f
Splitting Storage API chapter into two
Oxyjun 7ec7875
Fixing links
Oxyjun 44b2810
Fixing broken links
Oxyjun e4a3ba5
Apply suggestions from code review
Oxyjun 48e79c0
Implementing feedback
Oxyjun 9b9a991
Apply suggestions from code review
Oxyjun 78733c1
Apply suggestions from code review
Oxyjun 984bde8
Using spaces instead of tabs
Oxyjun 44565d9
Using anchorheadings and self-referencing links in partial
Oxyjun d66204f
Update src/content/partials/durable-objects/api-sync-kv.mdx
Oxyjun bee1e0b
Update src/content/partials/durable-objects/api-storage-introduction.mdx
Oxyjun 5245cd2
Update src/content/partials/durable-objects/api-storage-introduction.mdx
Oxyjun 3316d2c
Merge commit 'dcebdd8f43119732744f207a4dc62b9fdd04382c' into jun/do/s…
Oxyjun File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
63 changes: 63 additions & 0 deletions
63
src/content/docs/durable-objects/api/legacy-kv-storage-api.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| --- | ||
| title: KV-backed Durable Object Storage (Legacy) | ||
| pcx_content_type: concept | ||
| sidebar: | ||
| order: 7 | ||
| --- | ||
|
|
||
| import { Render, Type, MetaInfo, GlossaryTooltip, TypeScriptExample, Details } from "~/components"; | ||
|
|
||
| :::note | ||
| This page documents the storage API for legacy KV-backed Durable Objects. | ||
|
|
||
| For the newer SQLite-backed Durable Object storage API, refer to [SQLite-backed Durable Object Storage](/durable-objects/api/sqlite-storage-api). | ||
| ::: | ||
|
|
||
| <Render file="api-storage-introduction" product="durable-objects" /> | ||
|
|
||
| <Render file="recommend-sqlite-do" product="durable-objects" /> | ||
|
|
||
| ## Access storage | ||
|
|
||
| Durable Objects gain access to Storage API via the `DurableObjectStorage` interface and accessed by the `DurableObjectState::storage` property. This is frequently accessed via `this.ctx.storage` with the `ctx` parameter passed to the Durable Object constructor. | ||
|
|
||
| The following code snippet shows you how to store and retrieve data using the Durable Object Storage API. | ||
|
|
||
| <TypeScriptExample> | ||
| ```ts | ||
| export class Counter extends DurableObject { | ||
| constructor(ctx: DurableObjectState, env: Env) { | ||
| super(ctx, env); | ||
| } | ||
|
|
||
| async increment(): Promise<number> { | ||
| let value: number = (await this.ctx.storage.get("value")) || 0; | ||
| value += 1; | ||
| await this.ctx.storage.put("value", value); | ||
| return value; | ||
| } | ||
| } | ||
| ``` | ||
| </TypeScriptExample> | ||
|
|
||
| JavaScript is a single-threaded and event-driven programming language. This means that JavaScript runtimes, by default, allow requests to interleave with each other which can lead to concurrency bugs. The Durable Objects runtime uses a combination of <GlossaryTooltip term="input gate">input gates</GlossaryTooltip> and <GlossaryTooltip term="output gate">output gates</GlossaryTooltip> to avoid this type of concurrency bug when performing storage operations. Learn more in our [blog post](https://blog.cloudflare.com/durable-objects-easy-fast-correct-choose-three/). | ||
|
|
||
| ## Asynchronous KV API | ||
|
|
||
| KV-backed Durable Objects provide KV API methods which are asynchronous. | ||
|
|
||
| <Render file="api-async-kv-legacy" product="durable-objects" /> | ||
|
|
||
| ## Alarms | ||
|
|
||
| <Render file="api-storage-alarms" product="durable-objects" /> | ||
|
|
||
| ## Other | ||
|
|
||
| <Render file="api-storage-other-methods" product="durable-objects" /> | ||
|
|
||
| ## Related resources | ||
|
|
||
| - [Durable Objects: Easy, Fast, Correct Choose Three](https://blog.cloudflare.com/durable-objects-easy-fast-correct-choose-three/) | ||
| - [Zero-latency SQLite storage in every Durable Object blog](https://blog.cloudflare.com/sqlite-in-durable-objects/) | ||
| - [WebSockets API](/durable-objects/best-practices/websockets/) |
210 changes: 210 additions & 0 deletions
210
src/content/docs/durable-objects/api/sqlite-storage-api.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,210 @@ | ||
| --- | ||
| title: SQLite-backed Durable Object Storage | ||
| pcx_content_type: concept | ||
| sidebar: | ||
| order: 6 | ||
| --- | ||
|
|
||
| import { Render, Type, MetaInfo, GlossaryTooltip, TypeScriptExample, Details } from "~/components"; | ||
|
|
||
| :::note | ||
| This page documents the storage API for the newer SQLite-backed Durable Objects. | ||
|
|
||
| For the legacy KV-backed Durable Object storage API, see [KV-backed Durable Object Storage (Legacy)](/durable-objects/api/legacy-kv-storage-api/). | ||
Oxyjun marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ::: | ||
|
|
||
| <Render file="api-storage-introduction" product="durable-objects" /> | ||
|
|
||
| <Render file="recommend-sqlite-do" product="durable-objects" /> | ||
|
|
||
| <Render file="do-sqlite-storage-no-bill-note" product="durable-objects" /> | ||
|
|
||
| ## Access storage | ||
|
|
||
| Durable Objects gain access to Storage API via the `DurableObjectStorage` interface and accessed by the `DurableObjectState::storage` property. This is frequently accessed via `this.ctx.storage` with the `ctx` parameter passed to the Durable Object constructor. | ||
|
|
||
| The following code snippet shows you how to store and retrieve data using the Durable Object Storage API. | ||
|
|
||
| <TypeScriptExample> | ||
| ```ts | ||
Oxyjun marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| export class Counter extends DurableObject { | ||
| constructor(ctx: DurableObjectState, env: Env) { | ||
| super(ctx, env); | ||
| } | ||
|
|
||
| async increment(): Promise<number> { | ||
| let value: number = (await this.ctx.storage.get('value')) || 0; | ||
| value += 1; | ||
| await this.ctx.storage.put('value', value); | ||
| return value; | ||
| } | ||
|
|
||
| } | ||
|
|
||
| ``` | ||
| </TypeScriptExample> | ||
|
|
||
| JavaScript is a single-threaded and event-driven programming language. This means that JavaScript runtimes, by default, allow requests to interleave with each other which can lead to concurrency bugs. The Durable Objects runtime uses a combination of <GlossaryTooltip term="input gate">input gates</GlossaryTooltip> and <GlossaryTooltip term="output gate">output gates</GlossaryTooltip> to avoid this type of concurrency bug when performing storage operations. Learn more in our [blog post](https://blog.cloudflare.com/durable-objects-easy-fast-correct-choose-three/). | ||
|
|
||
| ## SQL API | ||
|
|
||
| The `SqlStorage` interface encapsulates methods that modify the SQLite database embedded within a Durable Object. The `SqlStorage` interface is accessible via the [`sql` property](/durable-objects/api/sqlite-storage-api/#sql) of `DurableObjectStorage` class. | ||
|
|
||
| For example, using `sql.exec()` a user can create a table and insert rows. | ||
|
|
||
| ```ts | ||
| import { DurableObject } from "cloudflare:workers"; | ||
|
|
||
| export class MyDurableObject extends DurableObject { | ||
| sql: SqlStorage; | ||
| constructor(ctx: DurableObjectState, env: Env) { | ||
| super(ctx, env); | ||
| this.sql = ctx.storage.sql; | ||
|
|
||
| this.sql.exec(` | ||
| CREATE TABLE IF NOT EXISTS artist( | ||
| artistid INTEGER PRIMARY KEY, | ||
| artistname TEXT | ||
| ); | ||
| INSERT INTO artist (artistid, artistname) VALUES | ||
| (123, 'Alice'), | ||
| (456, 'Bob'), | ||
| (789, 'Charlie'); | ||
| `); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| - SQL API methods accessed with `ctx.storage.sql` are only allowed on [Durable Object classes with SQLite storage backend](/durable-objects/best-practices/access-durable-objects-storage/#create-sqlite-backed-durable-object-class) and will return an error if called on Durable Object classes with a KV-storage backend. | ||
| - When writing data, every row update of an index counts as an additional row. However, indexes may be beneficial for read-heavy use cases. Refer to [Index for SQLite Durable Objects](/durable-objects/best-practices/access-durable-objects-storage/#index-for-sqlite-durable-objects). | ||
| - Writing data to [SQLite virtual tables](https://www.sqlite.org/vtab.html) also counts towards rows written. | ||
|
|
||
| <Render file="d1-do-supported-sqlite-extensions" product="d1" params={{ product: "Durable Objects support" }}/> | ||
|
|
||
| ### `exec` | ||
|
|
||
| <code>exec(query: <Type text="string" />, ...bindings: <Type text="any[]" />)</code>: <Type text="SqlStorageCursor" /> | ||
|
|
||
| #### Parameters | ||
|
|
||
| - `query`: <Type text ='string' /> | ||
| - The SQL query string to be executed. `query` can contain `?` placeholders for parameter bindings. Multiple SQL statements, separated with a semicolon, can be executed in the `query`. With multiple SQL statements, any parameter bindings are applied to the last SQL statement in the `query`, and the returned cursor is only for the last SQL statement. | ||
| - `...bindings`: <Type text='any[]' /> <MetaInfo text='Optional' /> | ||
| - Optional variable number of arguments that correspond to the `?` placeholders in `query`. | ||
|
|
||
| #### Returns | ||
|
|
||
| A cursor (`SqlStorageCursor`) to iterate over query row results as objects. `SqlStorageCursor` is a JavaScript [Iterable](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterable_protocol), which supports iteration using `for (let row of cursor)`. `SqlStorageCursor` is also a JavaScript [Iterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterator_protocol), which supports iteration using `cursor.next()`. | ||
|
|
||
| `SqlStorageCursor` supports the following methods: | ||
|
|
||
| - `next()` | ||
| - Returns an object representing the next value of the cursor. The returned object has `done` and `value` properties adhering to the JavaScript [Iterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterator_protocol). `done` is set to `false` when a next value is present, and `value` is set to the next row object in the query result. `done` is set to `true` when the entire cursor is consumed, and no `value` is set. | ||
| - `toArray()` | ||
| - Iterates through remaining cursor value(s) and returns an array of returned row objects. | ||
| - `one()` | ||
| - Returns a row object if query result has exactly one row. If query result has zero rows or more than one row, `one()` throws an exception. | ||
| - `raw()`: <Type text='Iterator' /> | ||
| - Returns an Iterator over the same query results, with each row as an array of column values (with no column names) rather than an object. | ||
| - Returned Iterator supports `next()` and `toArray()` methods above. | ||
| - Returned cursor and `raw()` iterator iterate over the same query results and can be combined. For example: | ||
|
|
||
| ```ts | ||
| let cursor = this.sql.exec("SELECT * FROM artist ORDER BY artistname ASC;"); | ||
| let rawResult = cursor.raw().next(); | ||
|
|
||
| if (!rawResult.done) { | ||
| console.log(rawResult.value); // prints [ 123, 'Alice' ] | ||
| } else { | ||
| // query returned zero results | ||
| } | ||
|
|
||
| console.log(cursor.toArray()); // prints [{ artistid: 456, artistname: 'Bob' },{ artistid: 789, artistname: 'Charlie' }] | ||
| ``` | ||
|
|
||
| `SqlStorageCursor` has the following properties: | ||
|
|
||
| - `columnNames`: <Type text='string[]' /> | ||
| - The column names of the query in the order they appear in each row array returned by the `raw` iterator. | ||
| - `rowsRead`: <Type text='number' /> | ||
| - The number of rows read so far as part of this SQL `query`. This may increase as you iterate the cursor. The final value is used for [SQL billing](/durable-objects/platform/pricing/#sqlite-storage-backend). | ||
| - `rowsWritten`: <Type text='number' /> | ||
| - The number of rows written so far as part of this SQL `query`. This may increase as you iterate the cursor. The final value is used for [SQL billing](/durable-objects/platform/pricing/#sqlite-storage-backend). | ||
| - Any numeric value in a column is affected by JavaScript's 52-bit precision for numbers. If you store a very large number (in `int64`), then retrieve the same value, the returned value may be less precise than your original number. | ||
|
|
||
| :::note[SQL transactions] | ||
| Note that `sql.exec()` cannot execute transaction-related statements like `BEGIN TRANSACTION` or `SAVEPOINT`. Instead, use the [`ctx.storage.transaction()`](/durable-objects/api/sqlite-storage-api/#transaction) or [`ctx.storage.transactionSync()`](/durable-objects/api/sqlite-storage-api/#transactionsync) APIs to start a transaction, and then execute SQL queries in your callback. | ||
| ::: | ||
|
|
||
| #### Examples | ||
|
|
||
| <Render file="durable-objects-sql" product="durable-objects" /> | ||
|
|
||
| ### `databaseSize` | ||
|
|
||
| `databaseSize`: <Type text ='number' /> | ||
|
|
||
| #### Returns | ||
|
|
||
| The current SQLite database size in bytes. | ||
|
|
||
| ```ts | ||
| let size = ctx.storage.sql.databaseSize; | ||
| ``` | ||
|
|
||
| ## PITR (Point In Time Recovery) API | ||
|
|
||
| For [SQLite-backed Durable Objects](/durable-objects/best-practices/access-durable-objects-storage/#create-sqlite-backed-durable-object-class), the following point-in-time-recovery (PITR) API methods are available to restore a Durable Object's embedded SQLite database to any point in time in the past 30 days. These methods apply to the entire SQLite database contents, including both the object's stored SQL data and stored key-value data using the key-value `put()` API. The PITR API is not supported in local development because a durable log of data changes is not stored locally. | ||
|
|
||
| The PITR API represents points in time using 'bookmarks'. A bookmark is a mostly alphanumeric string like `0000007b-0000b26e-00001538-0c3e87bb37b3db5cc52eedb93cd3b96b`. Bookmarks are designed to be lexically comparable: a bookmark representing an earlier point in time compares less than one representing a later point, using regular string comparison. | ||
Oxyjun marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### `getCurrentBookmark` | ||
|
|
||
| <code>ctx.storage.getCurrentBookmark()</code>: <Type text="Promise<string>" /> | ||
|
|
||
| - Returns a bookmark representing the current point in time in the object's history. | ||
|
|
||
| ### `getBookmarkForTime` | ||
|
|
||
| <code>ctx.storage.getBookmarkForTime(timestamp: <Type text="number | Date" />)</code>: <Type text="Promise<string>" /> | ||
|
|
||
| - Returns a bookmark representing approximately the given point in time, which must be within the last 30 days. If the timestamp is represented as a number, it is converted to a date as if using `new Date(timestamp)`. | ||
|
|
||
| ### `onNextSessionRestoreBookmark` | ||
|
|
||
| <code>ctx.storage.onNextSessionRestoreBookmark(bookmark: <Type text="string" />)</code>: <Type text="Promise<string>" /> | ||
|
|
||
| - Configures the Durable Object so that the next time it restarts, it should restore its storage to exactly match what the storage contained at the given bookmark. After calling this, the application should typically invoke `ctx.abort()` to restart the Durable Object, thus completing the point-in-time recovery. | ||
|
|
||
| This method returns a special bookmark representing the point in time immediately before the recovery takes place (even though that point in time is still technically in the future). Thus, after the recovery completes, it can be undone by performing a second recovery to this bookmark. | ||
|
|
||
| ```ts | ||
| let now = new Date(); | ||
| // restore to 2 days ago | ||
| let bookmark = ctx.storage.getBookmarkForTime(now - 2); | ||
| ctx.storage.onNextSessionRestoreBookmark(bookmark); | ||
| ``` | ||
|
|
||
| ## Synchronous KV API | ||
|
|
||
| <Render file="api-sync-kv" product="durable-objects" /> | ||
|
|
||
| ## Alarms | ||
|
|
||
| <Render file="api-storage-alarms" product="durable-objects" /> | ||
|
|
||
| ## Other | ||
|
|
||
| <Render file="api-storage-other-methods" product="durable-objects" /> | ||
|
|
||
| ## Storage properties | ||
|
|
||
| ### `sql` | ||
|
|
||
| `sql` is a readonly property of type `DurableObjectStorage` encapsulating the [SQL API](/durable-objects/api/sqlite-storage-api/#synchronous-sql-api). | ||
|
|
||
| ## Related resources | ||
|
|
||
| - [Durable Objects: Easy, Fast, Correct Choose Three](https://blog.cloudflare.com/durable-objects-easy-fast-correct-choose-three/) | ||
| - [Zero-latency SQLite storage in every Durable Object blog](https://blog.cloudflare.com/sqlite-in-durable-objects/) | ||
| - [WebSockets API](/durable-objects/best-practices/websockets/) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.