Big docs tidy up in preparation for launch (#364)

## Motivation and Context
Docs currently don't have very clear organisation, and don't explain
core flows.

This fixes #89 (although we still might want to push these to
`https://modelcontextprotocol.io/docs/`), based on the plan there.

One minor change from that plan is that I merged tutorials and guides,
because they ended up basically being duplicates which I thought would
be annoying for us to keep in sync. Additionally in the Diátaxis
framework, tutorials and guides are separated by whether they are
focused on acquisition of skills (e.g. in a learning environment) or
application of skills (e.g. in a work environment). I expect the
registry to be something that will be used occasionally in a work
environment, rather than a fundamental 'learnable' skill so I think it's
reasonable not to have tutorials and fold these into guides.

## How Has This Been Tested?
- Read through all these updated docs
- Unit and integration tests (that previously used examples.md and use
the schema) working
- Ran `npx markdown-link-check .` to check all the links work

## Breaking Changes
Links to old docs will break, but I don't think there are any. All
internal links have been checked and updated.

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update

Author: domdomegg

README.md

@@ -1,30 +1,14 @@
 # MCP Registry
 
-> [!WARNING]  
-> The registry is under [active development](#development-status). The registry API spec is unstable and the official MCP registry database may be wiped at any time.
-
-📖 **[API Documentation](https://staging.registry.modelcontextprotocol.io/docs)** | 📚 **[Technical Docs](./docs)**
-
-## What is the registry?
-
-The registry will provide MCP clients with a list of MCP servers, like an app store for MCP servers. (In the future it might do more, like also hosting a list of clients).
-
-There are two parts to the registry project:
-1. 🟦 **The MCP registry spec**: An [API specification](./docs/server-registry-api/) that allows anyone to implement a registry.
-2. 🟥 **The Official MCP registry**: A hosted registry following the MCP registry spec at [`registry.modelcontextprotocol.io`](https://registry.modelcontextprotocol.io). This serves as the **authoritative repository** for publicly-available MCP servers. Server creators publish once, and all consumers (MCP clients, aggregators, marketplaces) reference the same canonical data. This is owned by the MCP open-source community, backed by major trusted contributors to the MCP ecosystem such as Anthropic, GitHub, PulseMCP and Microsoft.
-
-The registry is built around the [`server.json`](./docs/server-json/) format - a standardized way to describe MCP servers that works across discovery, initialization, and packaging scenarios.
+The MCP registry provides MCP clients with a list of MCP servers, like an app store for MCP servers.
 
-In time, we expect the ecosystem to look a bit like this:
-
-![](./docs/ecosystem-diagram.excalidraw.svg)
-
-Note that MCP registries are _metaregistries_. They host metadata about packages, but not the package code or binaries. Instead, they reference other package registries (like NPM, PyPi or Docker) for this.
-
-Additionally, we expect clients pull from _subregistries_. These subregistries add value to the registry ecosystem by providing curation, or extending it with additional metadata. The Official MCP registry expects a lot of API requests from ETL jobs from these subregistries.
+📖 **[Full documentation](./docs)**
 
 ## Development Status
 
+> [!WARNING]  
+> The registry is under [active development](#development-status). The registry API spec is unstable and the official MCP registry database may be wiped at any time.
+
 **2025-09-04 update**: We're targeting a 'preview' go-live on 8th September. This may still be unstable and not provide durability guarantees, but is a step towards being more solidified. A general availability (GA) release will follow later.
 
 Current key maintainers:
@@ -106,7 +90,7 @@ make publisher
 ./cmd/publisher/bin/mcp-publisher --help
 ```
 
-See [the publisher README](./cmd/publisher/README.md) for more details.
+See [the publisher guide](./docs/guides/publishing/publish-server.md) for more details.
 
 #### Other commands
 
@@ -130,9 +114,7 @@ For Claude and other AI tools: Always prefer make targets over custom commands w
 │   └── publisher/           # Server publishing tool
 ├── data/                    # Seed data
 ├── deploy/                  # Deployment configuration (Pulumi)
-├── docs/                    # Technical documentation
-│   ├── server-json/         # server.json specification & examples
-│   └── server-registry-api/ # API specification
+├── docs/                    # Documentation
 ├── internal/                # Private application code
 │   ├── api/                 # HTTP handlers and routing
 │   ├── auth/                # Authentication (GitHub OAuth, JWT, namespace blocking)
@@ -165,4 +147,4 @@ The registry validates namespace ownership when publishing. E.g. to publish...:
 
 ## More documentation
 
-See the [docs](./docs) folder for more details if your question has not been answered here!
+See the [documentation](./docs) for more details if your question has not been answered here!

cmd/publisher/README.md

@@ -2,7 +2,7 @@
 
 CLI tool for publishing MCP servers to the registry.
 
-> These docs are for contributors. See the [Publisher User Guide](../../docs/publisher/guide.md) for end-user documentation.
+> These docs are for contributors. See the [Publisher User Guide](../../docs/guides/publishing/publish-server.md) for end-user documentation.
 
 ## Quick Development Setup
 

docs/README.md

@@ -1,18 +1,16 @@
-# Official Registry Documentation
+# MCP Registry Documentation
 
-This extends [the main README](../README.md).
+The MCP registry provides MCP clients with a list of MCP servers, like an app store for MCP servers.
 
-## Project Documentation
+## I want to...
 
-- [`faq.md`](./faq.md) - Frequently asked questions about the MCP Registry
-- [`design_principles.md`](./design_principles.md) - Core constraints and principles guiding the registry design
-- [`roadmap.md`](./roadmap.md) - High-level roadmap for the MCP Registry development
-- [`MCP Developers Summit 2025 - Registry Talk Slides.pdf`](./MCP%20Developers%20Summit%202025%20-%20Registry%20Talk%20Slides.pdf) - Slides from a talk given at the MCP Developers Summit on May 23, 2025, with an up-to-date vision of how we are thinking about the official registry.
+- **📤 Publish my MCP server** → [Publishing Guide](guides/publishing/publish-server.md)
+- **📥 Consume registry data** → [API Usage Guide](guides/consuming/use-rest-api.md)
+- **🔌 Understand the registry's purpose** → [Ecosystem vision](explanations/ecosystem-vision.md)
+- **📋 Look up specific information** → [server.json spec](reference/server-json/generic-server-json.md) | [API spec](reference/api/generic-registry-api.md) | [CLI reference](reference/cli/commands.md)
 
-## API & Technical Specifications
+## Documentation Index
 
-- [`openapi.yaml`](./server-registry-api/openapi.yaml) - OpenAPI specification for the official registry API
-- [`api_examples.md`](./api_examples.md) - Examples of what data will actually look like coming from the official registry API
-- [`architecture.md`](./architecture.md) - Technical architecture, deployment strategies, and data flows
-- [`server.json` README](./server-json/README.md) - description of the `server.json` purpose and schema
-- [`new_package_registry.md`](./new_package_registry.md) - steps to add a new package registry for local server packages
+- 🛠️ [How-To Guides: Task-focused instructions](./guides/)
+- 💡 [Explanations: Understanding-oriented content](./explanations/)
+- 📖 [Reference: Technical specifications and lookup material](./reference/)

docs/explanations/README.md

@@ -0,0 +1,10 @@
+# Explanations
+
+Understanding-oriented documentation that clarifies and illuminates the MCP Registry.
+
+- [Design principles](design-principles.md)
+- [Ecosystem vision](ecosystem-vision.md)
+- [Roadmap](roadmap.md)
+- [Tech architecture](tech-architecture.md)
+- [Namespacing](namespacing.md)
+- [Versioning](versioning.md)

docs/design_principles.md docs/explanations/design-principles.mddocs/design_principles.md renamed to docs/explanations/design-principles.md

@@ -0,0 +1,56 @@
+# Ecosystem Vision
+
+How the MCP Registry fits into the broader ecosystem and our vision for the future.
+
+## The Registry Ecosystem
+
+The MCP registry provides MCP clients with a list of MCP servers, like an app store for MCP servers. (In the future it might do more, like also hosting a list of clients).
+
+There are two parts to the registry project:
+
+1. 🟦 **The MCP registry spec**: An [API specification](../reference/api/) that allows anyone to implement a registry.
+2. 🟥 **The Official MCP registry**: A hosted registry following the MCP registry spec at [`registry.modelcontextprotocol.io`](https://registry.modelcontextprotocol.io). This serves as the **authoritative repository** for publicly-available MCP servers. Server creators publish once, and all consumers (MCP clients, aggregators, marketplaces) reference the same canonical data. This is owned by the MCP open-source community, backed by major trusted contributors to the MCP ecosystem such as Anthropic, GitHub, PulseMCP and Microsoft.
+
+The registry is built around the [`server.json`](../reference/server-json/) format - a standardized way to describe MCP servers that works across discovery, initialization, and packaging scenarios.
+
+In time, we expect the ecosystem to look a bit like this:
+
+![](./ecosystem-diagram.excalidraw.svg)
+
+Note that MCP registries are _metaregistries_. They host metadata about packages, but not the package code or binaries. Instead, they reference other package registries (like NPM, PyPi or Docker) for this.
+
+Additionally, we expect clients pull from _subregistries_. These subregistries add value to the registry ecosystem by providing curation, or extending it with additional metadata. The Official MCP registry expects a lot of API requests from ETL jobs from these subregistries.
+
+## Registry vs Package Registry
+
+Key distinction: MCP Registries are **metaregistries**.
+
+- **Package registries** (npm, PyPI, Docker Hub) host actual code/binaries
+- **The MCP Registry** hosts metadata pointing to those packages
+
+```
+MCP Registry: "weather-server v1.2.0 is at npm:weather-mcp"
+NPM Registry: [actual weather-mcp package code]
+```
+
+## Official vs Community Registries
+
+**Official MCP Registry** (`registry.modelcontextprotocol.io`):
+- Canonical source for publicly-available servers
+- Community-owned, backed by trusted contributors
+- Focuses on discoverability and basic metadata
+
+**Subregistries** (Smithery, PulseMCP, etc.):
+- Add value through curation, ratings, enhanced metadata
+- ETL from official registry + additional annotations
+- Serve specific communities or use cases
+
+## How Servers Are Represented
+
+Each server entry contains:
+- **Identity**: Unique name (`io.github.user/server-name`)
+- **Packages**: Where to download it (`npm`, `pypi`, `docker`, etc.)
+- **Runtime**: How to execute it (args, env vars)
+- **Metadata**: Description, capabilities, version
+
+This is stored in a standardized `server.json` format that works across discovery, installation, and execution.

docs/ecosystem-diagram.excalidraw.svg …nations/ecosystem-diagram.excalidraw.svgdocs/ecosystem-diagram.excalidraw.svg renamed to docs/explanations/ecosystem-diagram.excalidraw.svg docs/ecosystem-diagram.excalidraw.svg renamed to docs/explanations/ecosystem-diagram.excalidraw.svg

@@ -0,0 +1,45 @@
+# Namespacing and Security
+
+Namespacing prevents name squatting, makes impersonation and typo squatting much harder, and provides attribution through domain-based identity.
+
+For practical steps on how to authenticate for different namespaces, see the [publishing guide](../guides/publishing/publish-server.md#authenticate).
+
+## Namespace Format
+
+All server names follow reverse-DNS format. Examples:
+- `io.github.alice/weather-server` - GitHub user `alice`
+- `com.acme/internal-tool` - Company domain `acme.com`
+- `org.nonprofit.research/data-analyzer` - Subdomain `research.nonprofit.org`
+
+## Security Model
+
+### Ownership Verification
+
+Publishing to a namespace requires proving you control the corresponding identity:
+
+**GitHub namespaces** (`io.github.*`):
+- OAuth login to GitHub account/organization
+- OIDC tokens in GitHub Actions workflows
+
+**Domain namespaces** (`com.company.*`):
+- DNS verification: TXT record at `_mcp-registry.company.com`
+- HTTP verification: File at `https://company.com/.well-known/mcp-registry-auth`
+
+## Namespace Scoping
+
+Different authentication methods grant different namespace access:
+
+**GitHub OAuth/OIDC**:
+- `io.github.username/*` (for personal accounts)
+- `io.github.orgname/*` (for organizations)
+
+**DNS verification**:
+- `com.domain/*` and `com.domain.*/*` (domain + all subdomains)
+
+**HTTP verification**:
+- `com.domain/*` only (exact domain, no subdomains)
+
+## Limitations
+
+**Domain ownership changes**: If someone loses/sells a domain, they lose publishing rights. Similarly if someone gains a domain, they gain publishing rights.
+**Package validation**: Registry validates namespace ownership, but actual packages may still be malicious etc.