-
Notifications
You must be signed in to change notification settings - Fork 10.4k
[D1] Docs for automatic read-only retries in D1 #25087
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
1030840
Docs for automatic read-only retries in D1
Oxyjun 0418154
Adding image to changelog, minor update.
Oxyjun d83e3b7
Apply suggestions from code review
Oxyjun 5d5c830
Accepting review wording
Oxyjun 3ac7599
Cross linking to new page
Oxyjun 408c373
Apply suggestions from code review
Oxyjun ae987c3
Update src/content/docs/d1/best-practices/retry-queries.mdx
Oxyjun 48c8c53
Apply suggestions from code review
Oxyjun File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
22 changes: 22 additions & 0 deletions
22
src/content/changelog/d1/2025-09-11-d1-automatic-read-retries.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| --- | ||
| title: D1 automatically retries read-only queries | ||
| description: D1 now detects read-only queries and automatically attempts up to 2 retries to execute those queries in the event of failures with retryable errors. | ||
| products: | ||
| - d1 | ||
| - workers | ||
| date: 2025-09-11 | ||
| --- | ||
|
|
||
| D1 now detects read-only queries and automatically attempts up to 2 retries to execute those queries in the event of failures with retryable errors. You can view the number of execution attempts in the returned [response metadata](/d1/worker-api/return-object/#d1result) property `total_attempts`. | ||
Oxyjun marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| At the moment, only read-only queries are retried, that is, queries containing only the following SQLite keywords: `SELECT`, `EXPLAIN`, `WITH`. Queries containing any [SQLite keyword](https://sqlite.org/lang_keywords.html) that leads to database writes are not retried. | ||
|
|
||
| The retry success ratio among read-only retryable errors varies from 5% all the way up to 95%, depending on the underlying error and its duration (like network errors or other internal errors). | ||
|
|
||
| The retry success ratio among all retryable errors is lower, indicating that there are write-queries that could be retried. Therefore, we recommend D1 users to continue applying [retries in their own code](/d1/best-practices/retry-queries/) for queries that are not read-only but are idempotent according to the business logic of the application. | ||
|
|
||
|  | ||
|
|
||
| D1 ensures that any retry attempt does not cause database writes, making the automatic retries safe from side-effects, even if a query causing changes slips through the read-only detection. D1 achieves this by checking for modifications after every query execution, and if any write occurred due to a retry attempt, the query is rolled back. | ||
|
|
||
| The read-only query detection heuristics are simple for now, and there is room for improvement to capture more cases of queries that can be retried, so this is just the beginning. | ||
Oxyjun marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| --- | ||
| title: Retry queries | ||
| pcx_content_type: concept | ||
| sidebar: | ||
| order: 3 | ||
|
|
||
| --- | ||
|
|
||
| import { GitHubCode } from "~/components"; | ||
|
|
||
| It is useful to retry write queries from your application when you encounter a transient [error](/d1/observability/debug-d1/#error-list). From the list of `D1_ERROR`s, refer to the Recommended action column to determine if a query should be retried. | ||
|
|
||
| :::note | ||
| D1 automatically retries read-only queries up to 2 more times when it encounters a retryable error. | ||
Oxyjun marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ::: | ||
|
|
||
| ## Example of retrying queries | ||
|
|
||
| Consider the following example, taken from the [D1 read replication starter template](https://github.com/cloudflare/templates/blob/main/d1-starter-sessions-api-template/src/index.ts#L108). | ||
Oxyjun marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
Oxyjun marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| You should make sure your retries apply an exponential backoff with jitter strategy for more successful retries. | ||
| You can use libraries abstracting that already like [`@cloudflare/actors`](https://github.com/cloudflare/actors), or [copy the retry logic](https://github.com/cloudflare/actors/blob/9ba112503132ddf6b5cef37ff145e7a2dd5ffbfc/packages/core/src/retries.ts#L18) in your own code directly. | ||
|
|
||
| ```ts | ||
| import { tryWhile } from "@cloudflare/actors"; | ||
|
|
||
| function queryD1Example(d1: D1Database, sql: string) { | ||
| const result = await tryWhile(async () => { | ||
Oxyjun marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| return await d1.prepare(sql).run(); | ||
| }, shouldRetry); | ||
| } | ||
|
|
||
| function shouldRetry(err: unknown, nextAttempt: number) { | ||
Oxyjun marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| const errMsg = String(err); | ||
| const isRetryableError = | ||
| errMsg.includes("Network connection lost") || | ||
| errMsg.includes("storage caused object to be reset") || | ||
| errMsg.includes("reset because its code was updated"); | ||
| if (nextAttempt <= 5 && isRetryableError) { | ||
| return true; | ||
| } | ||
| return false; | ||
| } | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.