RFC: Llama Stack Documentation Improvements

[reluctantfuturist]

0 Why do this?

Incomplete / inconsistent documentation is a persistent pain point for developers, and we need to make significant investments in documentation for new users to close these gaps, especially given our steer towards OpenAI compatibility. The purpose of this doc is to summarize key documentation improvements and a new proposed doc structure.

1 Scope & Guiding Principles

• Retain Sphinx infrastructure (all changes are structural/content‑only).
• Target audience: app developers already familiar with core AI concepts and cloud AI platforms.
• For these developers, docs should facilitate seamless and quick onboarding and migration.
• Workflow‑first coverage – document the APIs that developers hit in day‑to‑day production flows first (inference, embeddings, safety, telemetry, RAG & agentic APIs).

2 Feature Requirements

• Documentation versioning
• Language tabs (curl, JS, Python)
• API reference generation

3 Key Guides

• [P0] Using Llama Stack with popular AI frameworks:

  • LlamaIndex
  • Langchain
  • Crew.AI
  • CopilotKit
  • Haystack*

• [P0] Onboarding guides for hi-pri distros

  • ollama
  • postgres demo with vllm

* We’re already pursuing a production integration with Deepset, so we can reuse this effort.

4 Proposed Nav Structure

└── Landing page (value prop, link to GitHub, latest release banner, link to Quickstart)

Get Started
├── Quickstart ('Hello Llama' 5‑min intro)
├── Setup & Auth
├── 'Migrating to Llama Stack' Cheat Sheet
└── Libraries (SDKs)

Concepts
├── Llama Stack Architecture
├── API Providers
├── Resources
├── Distributions

API Guides
├── Inference
├── Embeddings
├── Vector Stores
├── Safety
└── Telemetry

Build & Test
├── Build a RAG Bot
├── Build an Agent
├── Use MCP Servers
└── Llama Stack Playground

Deploy & Operate
├── Use on Partner Platforms (Dell, IBM, OCI)
├── Configure Llama Stack
├── Deploy on Kubernetes
├── Authorization
├── Build a Custom Distribution
└── External Providers

Advanced APIs (coming soon)
├── Batch Inference
├── Batch Agents
├── Post‑Training (fine‑tune)
├── Synthetic Data Generation
└── Evals

Examples
└── Reference Apps index (links out to llama-stack-apps)

API Reference
├── REST Endpoints
├── Python SDK
└── CLI Commands

Changelog / Release Notes

Documentation Version Selector
└── v1.2 • v1.1 • ... • Latest```

[leseb]

@kelbrown20 FYI

[kelbrown20]

Hello @reluctantfuturist I think this is a great content topic map! One thing I’ve noticed while going through the Llama Stack documentation is that similar content often appears in multiple formats and descriptions, which can make it difficult to locate specific information, especially when titles are similar or even the same.

I just have a question about the "Setup and Auth" section in "Getting Started" and the
"Configure Llama Stack"/ "Authorization" sections in "Deploy". Im wondering if we can consolidate these sections, unless there might be more of a difference in process here i'm unaware of, and have something like this:

Get Started
├── QuickStart ('Hello Llama' 5‑min intro)
├── Setup and configuring Llama Stack 
├──  Authorization
├── 'Migrating to Llama Stack' Cheat Sheet
└── Libraries (SDKs)

Deploying and Operating
├── Use on Partner Platforms (Dell, IBM, OCI)
├── Deploy on Kubernetes
├── Build a Custom Distribution
└── External Providers
^^ Each section will outline any specific necessary configurations.

Or even something like where setup and configuration are distinct

Get Started
├── QuickStart ('Hello Llama' 5‑min intro)
├── Setup and Authorization
├── 'Migrating to Llama Stack' Cheat Sheet
└── Libraries (SDKs)

Deploying and Operating
├── Configuring Llama stack
├── Use on Partner Platforms (Dell, IBM, OCI)
├── Deploy on Kubernetes
├── Build a Custom Distribution
└── External Providers

[reluctantfuturist]

Consolidating these makes sense to me, thank you for flagging!

[mattf]

@reluctantfuturist fyi https://pypi.org/project/langchain-llama-stack/

[reluctantfuturist]

Thank you @mattf !