Add more documentation for docs-builder CLI and its basic building blocks

docs/_docset.yml

@@ -46,26 +46,19 @@ toc:
       - file: branching-strategy.md
       - file: add-repo.md
       - file: release-new-version.md
-  - folder: migration
+  - folder: building-blocks
     children:
       - file: index.md
-      - folder: freeze
-        children:
-          - file: index.md
-          - file: gh-action.md
-      - file: syntax.md
-      - file: ia.md
-      - file: versioning.md
-      - file: engineering.md
-      - folder: guide
-        children:
-          - file: index.md
-          - file: working-in-docs-content.md
-          - file: automated.md
-          - file: tooling.md
-          - file: move-ref-docs.md
-          - file: mapping.md
-          - file: how-to-set-up-docs-previews.md
+      - file: documentation-set.md
+      - file: assembled-documentation.md
+      - file: distributed-documentation.md
+      - file: link-service.md
+      - file: link-index.md
+      - file: link-catalog.md
+      - file: outbound-crosslinks.md
+      - file: inbound-crosslinks.md
+      - file: documentation-set-navigation.md
+      - file: global-navigation.md
   - folder: configure
     children:
       - file: index.md
@@ -74,9 +67,9 @@ toc:
           - file: index.md
           - file: content.md
           - file: navigation.md
+          - file: products.md
           - file: versions.md
           - file: legacy-url-mappings.md
-          - file: products.md
       - folder: content-set
         children:
           - file: index.md
@@ -121,6 +114,57 @@ toc:
       - file: tabs.md
       - file: tagged_regions.md
       - file: titles.md
+  - folder: cli
+    children:
+      - file: index.md
+      - folder: docset
+        children:
+          - file: index.md
+          - file: build.md
+          - file: diff-validate.md
+          - file: index-command.md
+          - file: mv.md
+          - file: serve.md
+      - folder: assembler
+        children:
+          - file: index.md
+          - file: assemble.md
+          - file: assembler-bloom-filter-create.md
+          - file: assembler-bloom-filter-lookup.md
+          - file: assembler-build.md
+          - file: assembler-clone.md
+          - file: assembler-config-init.md
+          - file: assembler-content-source-match.md
+          - file: assembler-content-source-validate.md
+          - file: assembler-deploy-apply.md
+          - file: assembler-deploy-plan.md
+          - file: assembler-deploy-update-redirects.md
+          - file: assembler-index.md
+          - file: assembler-navigation-validate.md
+          - file: assembler-navigation-validate-link-reference.md
+          - file: assembler-serve.md
+      - folder: links
+        children:
+          - file: index.md
+          - file: inbound-links-validate.md
+          - file: inbound-links-validate-all.md
+          - file: inbound-links-validate-link-reference.md
+  - folder: migration
+    children:
+      - file: index.md
+      - file: syntax.md
+      - file: ia.md
+      - file: versioning.md
+      - file: engineering.md
+      - folder: guide
+        children:
+          - file: index.md
+          - file: working-in-docs-content.md
+          - file: automated.md
+          - file: tooling.md
+          - file: move-ref-docs.md
+          - file: mapping.md
+          - file: how-to-set-up-docs-previews.md
   # nested TOCs are only allowed from docset.yml by default
   # to prevent them from being nested deeply arbitrarily
   # use max_toc_depth to allow deeper nesting (Expert mode, consult with docs team)

docs/building-blocks/assembled-documentation.md

@@ -0,0 +1,50 @@
+---
+navigation_title: Assembled Documentation
+---
+
+# Assembled Documentation
+
+**Assembled documentation** is the product of building many [documentation sets](documentation-set.md) and weaving them into a global navigation to produce the fully assembled documentation site.
+
+## How it works
+
+The assembler:
+
+1. Clones multiple documentation repositories
+2. Reads its [configuration files](../configure/site/index.md) and builds [a global navigation](../configure/site/navigation.md)
+3. Builds each documentation set independently using the global configuration and navigation to inform path prefixes
+4. Produces a unified documentation website
+
+## Configuration
+
+Assembled documentation is configured through the site configuration, which defines:
+
+* [assembler.yml](../configure/site/index.md) Which repositories to include and [their branching strategy](../contribute/branching-strategy.md)
+* [navigation.yml](../configure/site/index.md) Navigation and url prefixes for TOC's.
+* [versions.yml](../configure/site/versions.md) Defines the various versioning schemes of products/solutions being documented
+* [products.yml](../configure/site/products.md) Defines the product catalog (id, name) and ties it to a specific versioning scheme
+
+See [Site Configuration](../configure/site/index.md) for complete details on configuring assembled documentation.
+
+Important to note that `docs-builder` makes no assumptions about how repositories, products, solutions and versions tie into each other.
+
+## Build process
+
+The typical build process for assembled documentation:
+
+1. **Clone** - Clone all configured repositories using `docs-builder assembler clone`
+2. **Build** - Build all documentation sets using `docs-builder assembler build`
+3. **Serve** - Serve the documentation on http://localhost:4000 using `docs-builder assembler serve`
+
+This uses the embedded configuration files inside the `docs-builder` binary. To build a specific configuration:
+
+1. **Init Config** - Fetch the latest config to `$(pwd)/config` `docs-builder assembler config init --local`
+2. **Clone** - Clone all configured repositories using `docs-builder assembler clone --local`
+3. **Build** - Build all documentation sets using `docs-builder assembler build --local`
+4. **Serve** - Serve the documentation on http://localhost:4000 using `docs-builder assembler serve`
+
+## Related concepts
+
+* [Documentation Set](documentation-set.md) - The individual units being assembled
+* [Distributed Documentation](distributed-documentation.md) - How documentation sets work independently
+* [Link Catalog](link-catalog.md) - How the assembler knows what to include

docs/building-blocks/distributed-documentation.md

@@ -0,0 +1,63 @@
+---
+navigation_title: Distributed Documentation
+---
+
+# Distributed Documentation
+
+**Distributed documentation** is the architectural approach that allows repositories to build their own documentation independently.
+
+## Purpose
+
+The separation between building individual documentation sets and assembling them enables distributed builds, where:
+
+* Each repository builds its own documentation independently
+* Builds don't block each other
+* Teams maintain full autonomy over their documentation
+* Cross-repository links are validated without requiring synchronized builds
+
+## How it works
+
+### Link Index publication
+
+Each time a documentation set is built successfully on its default integration branch, it produces and publishes a `links.json` file ([Link Index](link-index.md)) to a central location ([Link Service](link-service.md)).
+
+This Link Index contains all the linkable resources in that repository at that specific commit.
+
+### Example
+
+For instance, [Elasticsearch's links.json](https://elastic-docs-link-index.s3.us-east-2.amazonaws.com/elastic/elasticsearch/main/links.json) represents all linkable resources in the Elasticsearch repository's main branch.
+
+## Benefits
+
+This distributed approach provides several key advantages:
+
+### Link validation
+
+* **Outbound links** - Validate links to other repositories ahead of time, even during local `docs-builder` builds
+* **Inbound links** - Know when changes to your documentation would break links from other repositories
+
+### Resilient builds
+
+* **Isolation** - Documentation errors in one repository won't affect builds of other repositories
+* **Fallback mechanism** - When a repository has build failures on its integration branch, the assembler falls back to the last known good commit
+* **Snapshot builds** - Assembled builds only use commits that successfully produced a Link Index
+
+### Independent iteration
+
+* Teams can iterate on their documentation independently
+* No coordination required for routine updates
+* Faster feedback loops for documentation changes
+
+## Architecture components
+
+The distributed documentation system relies on several key components:
+
+* [Link Index](link-index.md) - Per-repository file of linkable resources
+* [Link Service](link-service.md) - Central storage for Link Index files
+* [Link Catalog](link-catalog.md) - Catalog of all available Link Index files
+* [Outbound Crosslinks](outbound-crosslinks.md) - Links from one repository to another
+* [Inbound Crosslinks](inbound-crosslinks.md) - Links from other repositories to yours
+
+## Local development
+
+Even during local development, `docs-builder` can access the [Link Service](link-service.md) to validate cross-repository links without requiring you to clone and build all related repositories.