Handle external links during mdbook-linkcheck better during CI #21
Description
Since the age of AI, https://developer.hashicorp.com/ and https://www.terraform.io/ have decided to consistently throttle our linkchecker. That means running nix run .#serve fails every time because of 429 Too Many Requests. The short-term solution is to disable linkcheck so that mdbook serve works. The medium-term solution is to decouple linkchecking and serving so that you can still run linkcheck, but separate from serve. The long-term solution is to keep linkchecking in CI, but handle 429s as "pass" in CI, since we cannot confirm or deny that it's a valid link; perhaps look into another crawler, since the site in question doesn't explicitly deny crawlers.
$ nix run .#serve
2025-09-11 18:05:50 [INFO] (mdbook::book): Book building has started
2025-09-11 18:05:50 [INFO] (mdbook::book): Running the html backend
2025-09-11 18:05:50 [INFO] (mdbook::book): Running the linkcheck backend
2025-09-11 18:05:50 [INFO] (mdbook::renderer): Invoking the "linkcheck" renderer
error: Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/syntax/json
┌─ index.md:3:55
│
3 │ Terranix works as a replacement for HCL by generating [Terraform JSON][tf-json].
│ ^^^^^^^^^^^^^^^^^^^^^^^^^ Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/syntax/json
error: Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/providers
┌─ index.md:12:3
│
12 │ [Terraform reference material](https://www.terraform.io/docs/providers/index.html)
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/providers
error: Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform
┌─ documentation/what-is-terranix.md:6:1
│
6 │ [Terraform](https://terraform.io) is a tool to interact with APIs via declarative files.
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform
error: Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/providers
┌─ documentation/what-is-terranix.md:21:1
│
21 │ [the Terraform website](https://www.terraform.io/docs/providers/index.html).
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/providers
error: Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/syntax/json
┌─ documentation/what-is-terranix.md:25:37
│
25 │ Here is an example of how you would [define multiple providers of the same kind](https://www.terraform.io/language/syntax/json#provider-blocks)
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/syntax/json
error: Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/expressions
┌─ documentation/terranix-vs-hcl.md:82:1
│
82 │ [multi line strings](https://www.terraform.io/docs/configuration/expressions.html#string-literals)
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Server returned 429 Too Many Requests for https://developer.hashicorp.com/terraform/language/expressions
Error: One or more incorrect links
2025-09-11 18:05:54 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2025-09-11 18:05:54 [ERROR] (mdbook::utils): Error: Rendering failed
2025-09-11 18:05:54 [ERROR] (mdbook::utils): Caused By: The "linkcheck" renderer failed