Skip to content

Commit 06960e2

Browse files
Oxyjunlambrospetroumarciocloudflare
authored
[D1] Docs for automatic read-only retries in D1 (#25087)
* Docs for automatic read-only retries in D1 * Adding image to changelog, minor update. * Apply suggestions from code review Co-authored-by: Lambros Petrou <[email protected]> * Accepting review wording * Cross linking to new page * Apply suggestions from code review Co-authored-by: Lambros Petrou <[email protected]> * Update src/content/docs/d1/best-practices/retry-queries.mdx Co-authored-by: Lambros Petrou <[email protected]> * Apply suggestions from code review Co-authored-by: marciocloudflare <[email protected]> --------- Co-authored-by: Lambros Petrou <[email protected]> Co-authored-by: marciocloudflare <[email protected]>
1 parent dacbc13 commit 06960e2

File tree

5 files changed

+78
-1
lines changed

5 files changed

+78
-1
lines changed
352 KB
Loading
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
title: D1 automatically retries read-only queries
3+
description: D1 now detects read-only queries and automatically attempts up to two retries to execute those queries in the event of failures with retryable errors.
4+
products:
5+
- d1
6+
- workers
7+
date: 2025-09-11
8+
---
9+
10+
D1 now detects read-only queries and automatically attempts up to two retries to execute those queries in the event of failures with retryable errors. You can access the number of execution attempts in the returned [response metadata](/d1/worker-api/return-object/#d1result) property `total_attempts`.
11+
12+
At the moment, only read-only queries are retried, that is, queries containing only the following SQLite keywords: `SELECT`, `EXPLAIN`, `WITH`. Queries containing any [SQLite keyword](https://sqlite.org/lang_keywords.html) that leads to database writes are not retried.
13+
14+
The retry success ratio among read-only retryable errors varies from 5% all the way up to 95%, depending on the underlying error and its duration (like network errors or other internal errors).
15+
16+
The retry success ratio among all retryable errors is lower, indicating that there are write-queries that could be retried. Therefore, we recommend D1 users to continue applying [retries in their own code](/d1/best-practices/retry-queries/) for queries that are not read-only but are idempotent according to the business logic of the application.
17+
18+
![D1 automatically query retries success ratio](~/assets/images/changelog/d1/d1-auto-retry-success-ratio.png)
19+
20+
D1 ensures that any retry attempt does not cause database writes, making the automatic retries safe from side-effects, even if a query causing changes slips through the read-only detection. D1 achieves this by checking for modifications after every query execution, and if any write occurred due to a retry attempt, the query is rolled back.
21+
22+
The read-only query detection heuristics are simple for now, and there is room for improvement to capture more cases of queries that can be retried, so this is just the beginning.
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
title: Retry queries
3+
pcx_content_type: concept
4+
sidebar:
5+
order: 3
6+
7+
---
8+
9+
import { GitHubCode } from "~/components";
10+
11+
It is useful to retry write queries from your application when you encounter a transient [error](/d1/observability/debug-d1/#error-list). From the list of `D1_ERROR`s, refer to the Recommended action column to determine if a query should be retried.
12+
13+
:::note
14+
D1 automatically retries read-only queries up to two more times when it encounters a retryable error.
15+
:::
16+
17+
## Example of retrying queries
18+
19+
Consider the following example of a `shouldRetry(...)` function, taken from the [D1 read replication starter template](https://github.com/cloudflare/templates/blob/main/d1-starter-sessions-api-template/src/index.ts#L108).
20+
21+
You should make sure your retries apply an exponential backoff with jitter strategy for more successful retries.
22+
You can use libraries abstracting that already like [`@cloudflare/actors`](https://github.com/cloudflare/actors), or [copy the retry logic](https://github.com/cloudflare/actors/blob/9ba112503132ddf6b5cef37ff145e7a2dd5ffbfc/packages/core/src/retries.ts#L18) in your own code directly.
23+
24+
```ts
25+
import { tryWhile } from "@cloudflare/actors";
26+
27+
function queryD1Example(d1: D1Database, sql: string) {
28+
return await tryWhile(async () => {
29+
return await d1.prepare(sql).run();
30+
}, shouldRetry);
31+
}
32+
33+
function shouldRetry(err: unknown, nextAttempt: number) {
34+
const errMsg = String(err);
35+
const isRetryableError =
36+
errMsg.includes("Network connection lost") ||
37+
errMsg.includes("storage caused object to be reset") ||
38+
errMsg.includes("reset because its code was updated");
39+
if (nextAttempt <= 5 && isRetryableError) {
40+
return true;
41+
}
42+
return false;
43+
}
44+
```

src/content/docs/d1/observability/debug-d1.mdx

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ While some D1 errors can be resolved by retrying the operation, retrying is only
5959

6060
Before retrying any failed operation:
6161
- Verify your query is idempotent (for example, read-only operations, or queries such as `CREATE TABLE IF NOT EXISTS`).
62-
- Consider implementing application-level checks to identify if the operation can be retried, and retrying only when it is safe and necessary.
62+
- Consider [implementing application-level checks](/d1/best-practices/retry-queries/) to identify if the operation can be retried, and retrying only when it is safe and necessary.
6363
:::
6464

6565
| `D1_ERROR` type | Description | Recommended action |
@@ -79,6 +79,16 @@ Before retrying any failed operation:
7979

8080
</Example>
8181

82+
## Automatic retries
83+
84+
D1 detects read-only queries and automatically attempts up to two retries to execute those queries in the event of failures with retryable errors.
85+
86+
D1 ensures that any retry attempt does not cause database writes, making the automatic retries safe from side-effects, even if a query causing modifications slips through the read-only detection. D1 achieves this by checking for modifications after every query execution, and if any write occurred due to a retry attempt, the query is rolled back.
87+
88+
:::note
89+
Only read-only queries (queries containing only the following SQLite keywords: `SELECT`, `EXPLAIN`, `WITH`) are retried. Queries containing any [SQLite keyword](https://sqlite.org/lang_keywords.html) that leads to database writes are not retried.
90+
:::
91+
8292
## View logs
8393

8494
View a stream of live logs from your Worker by using [`wrangler tail`](/workers/observability/logs/real-time-logs#view-logs-using-wrangler-tail) or via the [Cloudflare dashboard](/workers/observability/logs/real-time-logs#view-logs-from-the-dashboard).

src/content/docs/d1/worker-api/return-object.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ The methods [`D1PreparedStatement::run`](/d1/worker-api/prepared-statements/#run
3737
size_after: number, // the size of the database after the query is successfully applied
3838
rows_read: number, // the number of rows read (scanned) by this query
3939
rows_written: number // the number of rows written by this query
40+
total_attempts: number //the number of total attempts to successfully execute the query, including retries
4041
}
4142
results: array | null, // [] if empty, or null if it does not apply
4243
}

0 commit comments

Comments
 (0)