feat(docs): Add docusaurus plugin to autogerate llms.txt (#16362)

Adds a plugin to the docs to autogenerate an llms.txt file and an
llms-full.txt file to be published with the docs site according to the
standards defined at https://llmstxt.org/.

Also adds page descriptions to pages that are missing them, since this
helps with generating the llms.txt.

Note that this is set up to generate the file for the first versioned
defined in the versions.json file, which should be the default version
for the docs site.

Also removes an outdated sandbox features page.

closes: AztecProtocol/dev-rel#597

Author: critesjosh

docs/CLAUDE.md

@@ -152,18 +152,39 @@ Use these terms consistently throughout:
 
 Every guide should include:
 
-1. **Title** - Clear, descriptive, and action-oriented
-2. **Overview** - Brief description of what the guide covers
-3. **Prerequisites** - Required knowledge, tools, or access
-4. **Steps/Content** - Main body with clear headings
-5. **Verification** - How to confirm successful completion
-6. **Troubleshooting** - Common issues and solutions (where applicable)
-7. **Next Steps** - Related guides or advanced topics
+1. **Front-matter** - YAML metadata block with required `description` field
+2. **Title** - Clear, descriptive, and action-oriented
+3. **Overview** - Brief description of what the guide covers
+4. **Prerequisites** - Required knowledge, tools, or access
+5. **Steps/Content** - Main body with clear headings
+6. **Verification** - How to confirm successful completion
+7. **Troubleshooting** - Common issues and solutions (where applicable)
+8. **Next Steps** - Related guides or advanced topics
+
+### Front-matter Requirements
+
+Every markdown file MUST include front-matter with a `description` field:
+
+```yaml
+---
+title: "Page Title"
+description: "Brief, SEO-friendly description of the page content (50-160 characters recommended)"
+---
+```
+
+The description should:
+
+- Clearly summarize the page's purpose and content
+- Be between 50-160 characters for optimal SEO
+- Use active voice and be user-focused
+- Avoid redundancy with the page title
+- Help users understand what they'll learn or accomplish
 
 ## Special Instructions
 
 ### Do Review For:
 
+- ✅ Missing front-matter or missing `description` field in front-matter
 - ✅ Ambiguous instructions that could confuse users
 - ✅ Missing context or assumptions about user knowledge
 - ✅ Outdated screenshots or version references

docs/docs/aztec/concepts/accounts/index.md

@@ -2,6 +2,7 @@
 title: Accounts
 sidebar_position: 1
 tags: [accounts]
+description: Learn about Aztec's native account abstraction, where every account is a smart contract, and how wallets interact with account contracts for authentication and transaction execution.
 ---
 
 Aztec has native account abstraction. Every account in Aztec is a smart contract.

docs/docs/aztec/concepts/accounts/keys.md

@@ -1,6 +1,7 @@
 ---
 title: Keys
 tags: [accounts, keys]
+description: Understand the key pairs used in Aztec accounts - nullifier keys, address keys, incoming viewing keys, and signing keys - and how they enable privacy and authentication.
 ---
 
 import Image from "@theme/IdealImage";

docs/docs/aztec/concepts/advanced/authwit.md

@@ -3,6 +3,7 @@ title: Authentication Witness (Authwit)
 tags: [accounts, authwit]
 sidebar_position: 2
 keywords: [authwit, authentication witness, accounts]
+description: Learn about Aztec's Authentication Witness scheme that enables secure third-party actions on behalf of users, providing a privacy-preserving alternative to traditional token approvals.
 ---
 
 import Image from "@theme/IdealImage";

docs/docs/aztec/concepts/advanced/circuits/index.md

@@ -2,6 +2,7 @@
 title: Circuits
 sidebar_position: 2
 tags: [protocol, circuits]
+description: Explore Aztec's core protocol circuits that enforce privacy rules and transaction validity through zero-knowledge proofs, enabling private state and function execution.
 ---
 
 Central to Aztec's operations are 'circuits' derived both from the core protocol and the developer-written Aztec.nr contracts.

docs/docs/aztec/concepts/advanced/circuits/kernels/private_kernel.md

@@ -1,6 +1,7 @@
 ---
 title: Private Kernel Circuit
 tags: [protocol, circuits]
+description: Learn about the Private Kernel Circuit, the only zero-knowledge circuit in Aztec that handles private data and ensures transaction privacy by executing on user devices.
 ---
 
 This circuit is executed by the user, on their own device. This is to ensure private inputs to the circuit remain private!

docs/docs/aztec/concepts/advanced/circuits/kernels/public_kernel.md

@@ -1,6 +1,7 @@
 ---
 title: Public Kernel Circuit
 tags: [protocol, circuits]
+description: Understand the Public Kernel Circuit executed by sequencers that manages public state transitions in Aztec.
 ---
 
 This circuit is executed by a Sequencer, since only a Sequencer knows the current state of the [public data tree](../../../storage/state_model.md#public-state) at any time. A Sequencer might choose to delegate proof generation to the Prover pool.

docs/docs/aztec/concepts/advanced/circuits/rollup_circuits/index.md

@@ -1,6 +1,7 @@
 ---
 title: Rollup Circuits
 tags: [protocol, circuits]
+description: Learn how Rollup Circuits compress thousands of transactions into a single SNARK proof using a binary tree topology for efficient verification on Ethereum.
 ---
 
 The primary purpose of the Rollup Circuits is to 'squish' all of the many thousands of transactions in a rollup into a single SNARK, which can then be efficiently and verified on Ethereum.

docs/docs/aztec/concepts/advanced/index.md

@@ -2,6 +2,7 @@
 title: Advanced Concepts
 sidebar_position: 9
 tags: [protocol]
+description: Explore advanced concepts of the Aztec Network including authentication witnesses and other protocol-level details for experienced developers.
 ---
 
 In this section, you'll learn about the more advanced concepts of the Aztec Network. It is not required to understand these in order to start building on Aztec.

docs/docs/aztec/concepts/advanced/storage/indexed_merkle_tree.mdx

@@ -2,6 +2,7 @@
 title: Indexed Merkle Tree (Nullifier Tree)
 tags: [storage, concepts, advanced]
 sidebar_position: 2
+description: Learn about indexed merkle trees, an efficient data structure for nullifier trees that enables fast non-membership proofs and batch insertions in Aztec.
 ---
 
 import Image from "@theme/IdealImage";
@@ -183,7 +184,7 @@ Suppose we want to show that the value `20` doesn't exist in the tree. We just r
 - Prove the low leaf exists in the tree: `n` hashes.
 - Check the new value 'would have' belonged in the range given by the low leaf: `2` range checks.
   - If ($low\_nullifier_{\textsf{next\_index}} == 0$):
-    - Special case, the low leaf is at the very end, so the new\_value must be higher than all values in the tree:
+    - Special case, the low leaf is at the very end, so the new_value must be higher than all values in the tree:
     - $assert(low\_nullifier_{\textsf{value}} < new\_value_{\textsf{value}})$
   - Else:
     - $assert(low\_nullifier_{\textsf{value}} < new\_value_{\textsf{value}})$