Skip to content

Commit 7a51218

Browse files
mremre
committed
docs: document retry policy for network errors
Closes #149 - Add a 'Retry Policy' section to the Network Errors troubleshooting page - Include an SVG-icon table listing which error types are retried and which are not (based on the logic in lychee-lib/src/retry.rs) - Document the --max-retries and --retry-wait-time options with their defaults (3 and 1s) - Add 'Connection reset by server' as an example error message - Minor wording improvements throughout
1 parent 1875626 commit 7a51218

File tree

1 file changed

+161
-7
lines changed

1 file changed

+161
-7
lines changed

‎src/content/docs/troubleshooting/network-errors.mdx‎

Lines changed: 161 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,16 @@ title: Network Errors
44

55
import { Aside } from "@astrojs/starlight/components";
66

7-
If you run lychee and some links fail then it could be certificate-related issue.
8-
Certificates are used to verify the identity of a website and to establish a secure connection.
9-
Different operating systems and tools have different ways to handle certificates.
7+
If you run lychee and some links fail, it could be a transient network issue or
8+
a certificate-related problem. Certificates are used to verify the identity of a
9+
website and to establish a secure connection. Different operating systems and
10+
tools have different ways to handle certificates.
1011

1112
If lychee is unable to verify the certificate of a website, it will show a
1213
network error. This could be due to an expired certificate, an invalid
1314
certificate, or a certificate from an unknown authority.
1415

15-
Examples of network errors that could be certificate-related:
16+
Examples of network errors:
1617

1718
```bash
1819
Failed: Network error: error sending request for url
@@ -22,6 +23,157 @@ Failed: Network error: error sending request for url
2223
Network error: Forbidden
2324
```
2425

26+
```bash
27+
Network error: Connection reset by server. Server forcibly closed connection
28+
```
29+
30+
## Retry Policy
31+
32+
Lychee automatically retries failed requests before declaring a link broken.
33+
Not every error is worth retrying — some are permanent, while others are
34+
transient and may succeed on a second attempt.
35+
36+
The table below summarises which error types trigger a retry:
37+
38+
<table>
39+
<thead>
40+
<tr>
41+
<th>Error type</th>
42+
<th>Example</th>
43+
<th>Retried?</th>
44+
</tr>
45+
</thead>
46+
<tbody>
47+
<tr>
48+
<td>Timeout</td>
49+
<td><code>Network error: operation timed out</code></td>
50+
<td>
51+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="Yes" style="color: #22c55e">
52+
<polyline points="20 6 9 17 4 12"/>
53+
</svg>
54+
</td>
55+
</tr>
56+
<tr>
57+
<td>Too Many Requests (429)</td>
58+
<td><code>Network error: too many requests</code></td>
59+
<td>
60+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="Yes" style="color: #22c55e">
61+
<polyline points="20 6 9 17 4 12"/>
62+
</svg>
63+
</td>
64+
</tr>
65+
<tr>
66+
<td>Server error (5xx)</td>
67+
<td><code>Response: 503 Service Unavailable</code></td>
68+
<td>
69+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="Yes" style="color: #22c55e">
70+
<polyline points="20 6 9 17 4 12"/>
71+
</svg>
72+
</td>
73+
</tr>
74+
<tr>
75+
<td>Request timeout (408)</td>
76+
<td><code>Response: 408 Request Timeout</code></td>
77+
<td>
78+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="Yes" style="color: #22c55e">
79+
<polyline points="20 6 9 17 4 12"/>
80+
</svg>
81+
</td>
82+
</tr>
83+
<tr>
84+
<td>Incomplete response</td>
85+
<td><code>Network error: connection closed before message completed</code></td>
86+
<td>
87+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="Yes" style="color: #22c55e">
88+
<polyline points="20 6 9 17 4 12"/>
89+
</svg>
90+
</td>
91+
</tr>
92+
<tr>
93+
<td>Connection reset / aborted</td>
94+
<td><code>Network error: Connection reset by server</code></td>
95+
<td>
96+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="Yes" style="color: #22c55e">
97+
<polyline points="20 6 9 17 4 12"/>
98+
</svg>
99+
</td>
100+
</tr>
101+
<tr>
102+
<td>Connection refused / error</td>
103+
<td><code>Network error: connection refused</code></td>
104+
<td>
105+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="No" style="color: #ef4444">
106+
<line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/>
107+
</svg>
108+
</td>
109+
</tr>
110+
<tr>
111+
<td>Certificate / TLS error</td>
112+
<td><code>Network error: invalid certificate</code></td>
113+
<td>
114+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="No" style="color: #ef4444">
115+
<line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/>
116+
</svg>
117+
</td>
118+
</tr>
119+
<tr>
120+
<td>Redirect error</td>
121+
<td><code>Failed: Too many redirects</code></td>
122+
<td>
123+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="No" style="color: #ef4444">
124+
<line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/>
125+
</svg>
126+
</td>
127+
</tr>
128+
<tr>
129+
<td>Client error (4xx, except 408/429)</td>
130+
<td><code>Response: 404 Not Found</code></td>
131+
<td>
132+
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-label="No" style="color: #ef4444">
133+
<line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/>
134+
</svg>
135+
</td>
136+
</tr>
137+
</tbody>
138+
</table>
139+
140+
### Configuring retries
141+
142+
By default, lychee retries failed requests up to **3 times** with an
143+
exponential backoff starting at **1 second** (i.e. 1 s, 2 s, 4 s).
144+
145+
You can tune this behaviour with two options:
146+
147+
| Option | CLI flag | Config key | Default |
148+
|---|---|---|---|
149+
| Maximum retries | `--max-retries <NUM>` | `max_retries` | `3` |
150+
| Initial wait between retries | `--retry-wait-time <SECS>` | `retry_wait_time` | `1` |
151+
152+
For example, to retry up to 5 times with a 5-second initial wait:
153+
154+
```bash
155+
lychee --max-retries 5 --retry-wait-time 3 https://example.com
156+
```
157+
158+
Or in `lychee.toml`:
159+
160+
```toml
161+
max_retries = 5
162+
retry_wait_time = 3
163+
```
164+
165+
To disable retries entirely, set `--max-retries 0`:
166+
167+
```bash
168+
lychee --max-retries 0 https://example.com
169+
```
170+
171+
<Aside>
172+
Retries only apply to requests where lychee judges the failure to be
173+
transient (see the table above). Permanent errors such as a `404 Not Found`
174+
or a certificate problem are reported immediately without any retry.
175+
</Aside>
176+
25177
## What now?
26178

27179
### Double-check with a different tool
@@ -36,7 +188,8 @@ If this works, then the issue is with lychee and not the website.
36188
In that case, please open an issue on the lychee GitHub repository.
37189

38190
If it doesn't work, then the issue is with the website or your system.
39-
It might be related to the certificate or the user agent. The site might also use a bot detection such as Cloudflare Bot Management. Read on to find out more.
191+
It might be related to the certificate or the user agent. The site might also
192+
use bot detection such as Cloudflare Bot Management. Read on to find out more.
40193

41194
### Use the `--insecure` flag
42195

@@ -82,7 +235,7 @@ use a service like [BuiltWith](https://builtwith.com/). Just enter the website
82235
URL and check if it uses any bot detection services like Cloudflare Bot Management or
83236
Cloudflare Challenge.
84237

85-
If the website uses a bot detection service, which is blocking lychee, there's
238+
If the website uses a bot detection service which is blocking lychee, there's
86239
little you can do. You can try contacting the website administrator and ask them
87240
to whitelist lychee.
88241

@@ -113,4 +266,5 @@ Also, see this [Stack Overflow
113266
discussion](https://stackoverflow.com/a/24618403/270334) for additional
114267
information.
115268

116-
After that, try running lychee again and see if the issue is resolved. If not, please open an issue on the lychee GitHub repository.
269+
After that, try running lychee again and see if the issue is resolved. If not,
270+
please open an issue on the lychee GitHub repository.

0 commit comments

Comments
 (0)