aptos-labs · gregnazario · Mar 20, 2026 · Mar 20, 2026 · Mar 20, 2026 · Mar 20, 2026
diff --git a/.cursor/skills/llm-seo-readiness/SKILL.md b/.cursor/skills/llm-seo-readiness/SKILL.md
@@ -0,0 +1,50 @@
+---
+name: llm-seo-readiness
+description: >-
+  Validates Aptos docs changes for LLM consumption (llms.txt, curated feeds, Markdown exports, HTML→Markdown)
+  and for SEO (metadata, sitemap, robots, structured data). Use when adding or restructuring docs, editing
+  llms routes or curation, changing Head/meta/OG, updating robots.txt or sitemap behavior, or when the user
+  mentions LLMs.txt, AI tool ingestion, crawlers, discoverability, or SEO.
+---
+
+# LLM and SEO readiness (Aptos docs)
+
+## LLM / machine-readable
+
+- **Frontmatter**: Every new or renamed doc needs accurate `title` and `description` in English; they surface in `/llms.txt` and `.md` exports. Mirror updates in `src/content/docs/zh/` per agent guidelines (Spanish `es/` is out of scope for agent-maintained docs).
+- **Curated lists**: If a page should appear in the LLM index or early in full corpus, update `src/lib/llms-curated-ids.ts` (`LLMS_INDEX_SECTIONS`, `LLMS_SMALL_DOC_IDS`, `FULL_PRIORITY_DOC_IDS` as appropriate). Build fails if an id is missing from English non-draft docs.
+- **Index copy**: User-facing explanations live in `src/content/docs/llms-txt.mdx`, `src/content/docs/build/ai.mdx`, and the Chinese `zh/` counterparts—keep URLs and feed names aligned with `src/pages/llms-index.ts`.
+- **HTML → Markdown**: Shared logic is `src/lib/llms-html-sanitize.ts`. When minifying for `llms-small.txt`, collapse **spaces/tabs only**—never all `\s` (newlines must survive for fenced code and Markdown structure).
+- **Route wiring**: Custom handlers are swapped in via `src/integrations/llms-txt-index.ts`; endpoint implementations live under `src/pages/llms-index.ts`, `src/endpoints/llms-small.txt.ts`, `src/endpoints/llms-full.txt.ts`.
+- **Robots**: `public/robots.txt` should stay consistent with sitemap URL and, when feeds change, the commented LLMs.txt pointers at the bottom.
+
+## SEO
+
+- **Per-page metadata**: Starlight frontmatter `title` / `description` feed OG, Twitter, and schema in `src/starlight-overrides/Head.astro`—avoid empty or placeholder descriptions on public pages.
+- **Sitemap**: Produced by `@astrojs/sitemap` in `astro.config.mjs`; ensure new top-level routes or major URL changes still make sense for indexing.
+- **Hreflang / alternates**: Head override builds language alternates from `SUPPORTED_LANGUAGES`—new locales need config updates, not only content folders.
+- **Draft / hidden content**: Do not rely on curated LLM ids or public sitemap for content that must stay off production; follow existing draft filtering in the `.md` pipeline and curation tests.
+
+## Verification
+
+After substantive changes:
+
+```bash
+pnpm test tests/llms-curated-ids.test.ts tests/llms-html-sanitize.test.ts
+pnpm lint && pnpm check
+```
+
+For full coverage: `pnpm test` and a production `pnpm build` when touching routes or integrations.
+
+## File map
+
+| Area | Primary locations |
+|------|-------------------|
+| Curation | `src/lib/llms-curated-ids.ts`, `src/lib/llms.ts` |
+| llms.txt body | `src/pages/llms-index.ts` |
+| Feed endpoints | `src/endpoints/llms-small.txt.ts`, `src/endpoints/llms-full.txt.ts` |
+| Plugin override | `src/integrations/llms-txt-index.ts` |
+| Markdown export / sanitize | `src/lib/llms-html-sanitize.ts`, `src/pages/[...slug].md.ts` |
+| User docs | `src/content/docs/llms-txt.mdx`, `build/ai.mdx`, `zh/` |
+| SEO head | `src/starlight-overrides/Head.astro` |
+| Crawlers / sitemap hint | `public/robots.txt` |
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -4,7 +4,11 @@ This document provides essential guidelines for AI agents working on the Aptos D
 
 ## Project Overview
 
-This repository contains the official Aptos Developer Documentation, built using [Astro](https://astro.build/) and [Starlight](https://starlight.astro.build/). The documentation is available in multiple languages including English, Spanish (es), and Chinese (zh).
+This repository contains the official Aptos Developer Documentation, built using [Astro](https://astro.build/) and [Starlight](https://starlight.astro.build/). Published languages include English and Chinese (zh). Agent workflows here do **not** include creating or updating Spanish (`es`) documentation.
+
+## Machine-readable documentation for agents
+
+Production docs are indexed for LLMs and coding agents at [https://aptos.dev/llms.txt](https://aptos.dev/llms.txt) (same content as [https://aptos.dev/.well-known/llms.txt](https://aptos.dev/.well-known/llms.txt)). For IDE access to Aptos APIs and on-chain data, use the Aptos MCP server (`npx @aptos-labs/aptos-mcp`); see the live [AI tools](https://aptos.dev/build/ai) page.
 
 ## Development Setup
 
@@ -38,7 +42,6 @@ pnpm dev            # Start development server (http://localhost:4321)
 src/
 ├── content/
 │   ├── docs/           # Main English documentation
-│   │   ├── es/         # Spanish translations
 │   │   └── zh/         # Chinese translations
 │   ├── i18n/           # UI translations
 │   └── nav/            # Sidebar translations
@@ -47,6 +50,8 @@ src/
 └── assets/             # Site assets
 ```
 
+A legacy `docs/es/` tree or URLs may still exist for redirects; do not add or maintain Spanish doc content under these agent guidelines.
+
 ## Critical Guidelines
 
 ### 1. Linting and Formatting
@@ -62,18 +67,16 @@ Run these commands after making changes to ensure code quality.
 
 ### 2. Translation Requirements
 
-**All documentation changes must be translated and verified in both Chinese and Spanish versions.**
+**Documentation changes that need localization must include the Chinese (`zh`) version.** Do not add or update Spanish (`es`) translations as part of agent work.
 
 - **English docs:** `src/content/docs/`
-- **Spanish docs:** `src/content/docs/es/`
 - **Chinese docs:** `src/content/docs/zh/`
 
 When modifying documentation:
 
 1. Make the change in the English version first
-2. Create or update the corresponding Spanish translation in `es/`
-3. Create or update the corresponding Chinese translation in `zh/`
-4. Ensure all internal links use the correct language prefix (e.g., `/es/network/` for Spanish)
+2. Create or update the corresponding Chinese translation in `zh/`
+3. Ensure Chinese internal links use the `/zh/...` prefix (see Internal Links below)
 
 ### 3. Commit Message Requirements
 
@@ -101,7 +104,7 @@ Example:
 docs: add glossary entry for BlockSTM
 
 Added definition and example for BlockSTM parallel execution engine.
-Updated Spanish and Chinese translations accordingly.
+Updated Chinese translation accordingly.
 ```
 
 ### 4. Grammar and Style
@@ -121,7 +124,6 @@ Updated Spanish and Chinese translations accordingly.
 The glossary is located at:
 
 - **English:** `src/content/docs/network/glossary.mdx`
-- **Spanish:** `src/content/docs/es/network/glossary.mdx`
 - **Chinese:** `src/content/docs/zh/network/glossary.mdx`
 
 When adding a new term to the glossary:
@@ -160,7 +162,6 @@ sidebar:
 Use relative paths without the file extension:
 
 - English: `/network/blockchain/accounts`
-- Spanish: `/es/network/blockchain/accounts`
 - Chinese: `/zh/network/blockchain/accounts`
 
 ## Content Guidelines
@@ -190,21 +191,20 @@ module example::hello {
 
 1. Create the MDX file in the appropriate directory
 2. Add frontmatter with title and description
-3. Create Spanish translation in `es/` subdirectory
-4. Create Chinese translation in `zh/` subdirectory
-5. Update sidebar configuration if needed (`astro.sidebar.ts`)
-6. Run `pnpm lint` and `pnpm format`
+3. Create Chinese translation in `zh/` subdirectory
+4. Update sidebar configuration if needed (`astro.sidebar.ts`)
+5. Run `pnpm lint` and `pnpm format`
 
 ### Updating the Glossary
 
 1. Add the term to `src/content/docs/network/glossary.mdx`
-2. Add Spanish translation to `src/content/docs/es/network/glossary.mdx`
-3. Add Chinese translation to `src/content/docs/zh/network/glossary.mdx`
-4. Ensure alphabetical ordering within each section
-5. Include definition, examples, and links to related content
+2. Add Chinese translation to `src/content/docs/zh/network/glossary.mdx`
+3. Ensure alphabetical ordering within each section
+4. Include definition, examples, and links to related content
 
 ## Resources
 
+- [LLM and SEO readiness (Cursor skill)](.cursor/skills/llm-seo-readiness/SKILL.md) — checklists for `llms.txt`, curated feeds, `.md` exports, and metadata/crawlers
 - [Astro Documentation](https://docs.astro.build/)
 - [Starlight Documentation](https://starlight.astro.build/)
 - [MDX Documentation](https://mdxjs.com/)

diff --git a/astro.config.mjs b/astro.config.mjs
@@ -139,6 +139,11 @@ export default defineConfig({
             // Exclude autogenerated content and non-translatable static resources
             const excludePaths = ["/rest-api", "/move-reference", "/gas-profiling", "/scripts"];
 
+            // Plain-text LLM exports (injected routes; no HTML page for the validator to crawl)
+            if (link.includes("/llms-small.txt") || link.includes("/llms-full.txt")) {
+              return true;
+            }
+
             // Exclude specific problematic links from external move-reference content
             const excludeLinks = [
               "https://aptos.dev/move/book/SUMMARY",
@@ -151,42 +156,10 @@ export default defineConfig({
             );
           },
         }),
+        // Registers /llms.txt, /llms-small.txt, /llms-full.txt routes; local handlers override output
+        // (see src/integrations/llms-txt-index.ts). Curation lives in src/lib/llms-curated-ids.ts + src/lib/llms.ts.
         starlightLlmsTxt({
           rawContent: true,
-          description:
-            "Developer documentation for the Aptos blockchain — Move smart contracts, SDKs, APIs, indexer, node operations, and AI tools.",
-          details: [
-            "For AI coding tools, install the Aptos MCP server: `npx @aptos-labs/aptos-mcp`",
-            "It gives your IDE direct access to Aptos APIs, on-chain data, and Move contract helpers.",
-            "See <https://aptos.dev/build/ai> for setup guides (Claude Code, Cursor, and more).",
-          ].join("\n"),
-          optionalLinks: [
-            {
-              label: "Aptos MCP Server",
-              url: "https://www.npmjs.com/package/@aptos-labs/aptos-mcp",
-              description: "MCP server for AI coding tools",
-            },
-            {
-              label: "Aptos GitHub",
-              url: "https://github.com/aptos-labs",
-              description: "Official source code repositories",
-            },
-          ],
-          promote: [
-            "index*",
-            "get-started",
-            "network/glossary",
-            "build/smart-contracts*",
-            "network/blockchain*",
-            "build/sdks*",
-            "build/apis*",
-            "build/cli*",
-            "build/indexer*",
-            "build/guides*",
-            "build/ai*",
-          ],
-          demote: ["404"],
-          exclude: ["404"],
         }),
         ...(hasAlgoliaConfig
           ? [
@@ -217,9 +190,9 @@ export default defineConfig({
       sidebar,
       customCss: ["./src/styles/global.css", "katex/dist/katex.min.css"],
     }),
-    // Override the starlight-llms-txt plugin's /llms.txt index with a structured
-    // version that lists page titles, descriptions, and per-page .md URLs.
-    // Must be after Starlight so our injected route takes priority.
+    // Override the starlight-llms-txt plugin's generated llms routes with
+    // local handlers so we can curate the index and tune the small/full exports.
+    // Must be after Starlight so our injected routes take priority.
     llmsTxtIndex(),
     sitemap({
       serialize(item) {

diff --git a/public/robots.txt b/public/robots.txt
@@ -1,13 +1,34 @@
 User-agent: Algolia Crawler
 Allow: /
 
+# OpenAI search crawler
+User-agent: OAI-SearchBot
+Allow: /
+
+# OpenAI training crawler
+User-agent: GPTBot
+Allow: /
+
+# Anthropic
+User-agent: ClaudeBot
+Allow: /
+
+# Perplexity
+User-agent: PerplexityBot
+Allow: /
+
+# Google Gemini / grounding
+User-agent: Google-Extended
+Allow: /
+
 User-agent: *
 Allow: /
 
 Sitemap: https://aptos.dev/sitemap-index.xml
 
 # LLMs.txt - Machine-readable documentation for AI tools
 # https://aptos.dev/llms.txt
+# https://aptos.dev/llms-small.txt
 # https://aptos.dev/llms-full.txt
 
 # Algolia-Crawler-Verif: F9CC1F2E397396BF
diff --git a/scripts/fix-i18n-links/README.md b/scripts/fix-i18n-links/README.md
@@ -4,7 +4,7 @@ A high-performance Rust tool for automatically fixing localized documentation li
 
 ## Problem
 
-Localized documentation (Spanish, Chinese, etc.) often contains internal links that lack proper locale prefixes. For example, a link to `/guide/setup` in Spanish docs should be `/es/guide/setup`.
+Localized documentation (Chinese, etc.) often contains internal links that lack proper locale prefixes. For example, a link to `/guide/setup` in Chinese docs should be `/zh/guide/setup`.
 
 ## Solution
 
@@ -41,7 +41,7 @@ pnpm fix-i18n-links
 
 The tool will:
 
-1. Auto-discover locale directories (e.g., `es/`, `zh/`)
+1. Auto-discover locale directories (e.g., `zh/`)
 2. Process all `.mdx` files in each locale
 3. Fix internal links by adding locale prefixes
 4. Report the number of files and links fixed

diff --git a/scripts/fix-i18n-links/src/main.rs b/scripts/fix-i18n-links/src/main.rs
@@ -58,7 +58,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
     }
 
     if locales.is_empty() {
-        println!("{}", "ℹ️  No locale directories found (looking for 2-letter codes like es, zh)".yellow());
+        println!("{}", "ℹ️  No locale directories found (looking for 2-letter locale folders like zh)".yellow());
         return Ok(());
     }
 

diff --git a/src/components/SearchFallback.astro b/src/components/SearchFallback.astro
@@ -19,7 +19,6 @@
       fr: ["en"], // French falls back to English
       zh: ["en"], // Chinese falls back to English
       ja: ["en"], // Japanese falls back to English
-      es: ["en"], // Spanish falls back to English
     };
 
     function enhanceSearchWithFallbacks() {