diff --git a/docs/_docset.yml b/docs/_docset.yml index 96aac9673..1e7785616 100644 --- a/docs/_docset.yml +++ b/docs/_docset.yml @@ -32,6 +32,16 @@ toc: - file: index.md - file: locally.md - file: on-the-web.md + - folder: api-docs-contribution + children: + - file: index.md + - file: overview.md + - file: checklist.md + - file: guidelines.md + - file: organize-annotate.md + - file: workflows.md + - file: quickstart.md + - file: help.md - file: move.md - file: redirects.md - file: cumulative-docs.md diff --git a/docs/contribute/api-docs-contribution/checklist.md b/docs/contribute/api-docs-contribution/checklist.md new file mode 100644 index 000000000..fcad8e471 --- /dev/null +++ b/docs/contribute/api-docs-contribution/checklist.md @@ -0,0 +1,35 @@ +--- +navigation_title: Checklist +--- + +# API docs checklist + +Use this checklist to verify the quality, completeness, and consistency of your API docs contributions. + +## Content quality and completeness + + +- ☑️ Write clear [summaries](./guidelines.md#write-summaries) (30 characters max, start with a verb) +- ☑️ Write detailed [descriptions](./guidelines.md#write-descriptions) that explain purpose, context, and usage +- ☑️ Document all [path parameters](./guidelines.md#document-path-parameters) with constraints and formats +- ☑️ Provide descriptions for non-obvious [enum values](./guidelines.md#document-enum-values) +- ☑️ Specify [default values](./organize-annotate.md#set-default-values) for optional parameters +- ☑️ Include realistic [examples](./guidelines.md#add-examples) with helpful descriptions +- ☑️ Add [links](./guidelines.md#add-links) to related operations and documentation + +## Structure, organization, and metadata +- ☑️ Include required [OpenAPI document info](./organize-annotate.md#add-open-api-document-info) +- ☑️ Include [OpenAPI specification version](./organize-annotate.md#add-openapi-specification-version) +- ☑️ Define unique [operation identifiers](./organize-annotate.md#add-operation-identifiers) using camelCase +- ☑️ Use consistent [tags](./organize-annotate.md#group-apis-with-tags) to group related operations +- ☑️ Document [API lifecycle status](./organize-annotate.md#specify-api-lifecycle-status) (availability, stability, version information) +- ☑️ Mark deprecated APIs and properties with appropriate notices +- ☑️ Document [required permissions](./organize-annotate.md#document-required-permissions) for each operation + +## Quality assurance + +- ☑️ Preview your changes locally before submitting +- ☑️ [Lint your API docs](guidelines.md#lint-your-api-docs) to identify and fix issues +- ☑️ Check all links to ensure they work correctly +- ☑️ Ensure examples are realistic and error-free +- ☑️ Validate that your OpenAPI document is well-formed diff --git a/docs/contribute/api-docs-contribution/guidelines.md b/docs/contribute/api-docs-contribution/guidelines.md new file mode 100644 index 000000000..671eb524a --- /dev/null +++ b/docs/contribute/api-docs-contribution/guidelines.md @@ -0,0 +1,625 @@ +# Add core content + +This page covers the core guidelines for excellent API docs. Learn how to write clear summaries and descriptions, create helpful examples, and add useful links. + +## Write summaries and descriptions + +Most objects in your Open API specification accept both concise summaries and detailed descriptions to help users understand their purpose and usage. + +### Write summaries + +Clear, single-sentence summaries help users understand the components of your API at a glance. They appear in various contexts like IDEs, search results, and documentation overviews. + +Here are some principles for writing effective summaries: + +- **Be concise:** Keep summaries short (between 5-45 characters) because they appear in different contexts where space is limited +- **Start with a verb:** Use action words like "Get", "Update", "Delete", "Create" +- **Use simple verbs:** Use simple verbs (Get, Update, Delete) rather than verbose alternatives (Retrieve, Return, List) +- **Include articles:** "Delete a space", "Delete spaces", "Delete all spaces" +- **Use sentence case:** Capitalize only the first word and proper nouns + +#### Summary examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +In OpenAPI specifications, summaries are defined in the `summary` field: + +```yaml +paths: + /indices/{index}: + get: + summary: Get index information + description: Retrieve configuration and mapping information... +``` +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +In the Elasticsearch TypeScript definitions, summaries are added as JSDoc comments above class definitions: + +```ts +/** + * Get index information + */ +class GetIndexRequest { + // ... +} +``` + +**Notes:** +- Use 30 characters maximum to avoid text wrapping in generated output +- Don't add a period at the end +::: + +::::: + +### Write descriptions + +> **"Documentation can be added almost everywhere using a `description` field".** - [OpenAPI Specification docs](https://learn.openapis.org/specification/docs.html#providing-long-descriptions-in-yaml) + +Elastic APIs are generally complex, and summaries alone are not enough to explain their purpose and usage. +Descriptions enable you to add detailed explanations of your OpenAPI objects. + +They are essential for transforming machine-readable information into nicely formatted, human-readable prose. You can add paragraph breaks, links, or lists to your descriptions. + +Here are some principles for effective descriptions: + +- **Explain purpose and impact:** What does this operation/parameter do and why would users need it? +- **Provide context:** What are the prerequisites or related concepts users should know? +- **Use formatting:** Use paragraph breaks, lists, and other formatting options to make descriptions readable +- **Reference related operations:** [Link](#add-links) to complementary APIs or prerequisite steps +- **Provide usage guidance:** How should users typically use this parameter or operation? +- **Document constraints:** What are the valid values, formats, or limitations? +- **Explain data relationships:** For list parameters, clarify how multiple values are handled (comma-separated, arrays, etc.) +- **Document special formats:** Include expected formats for timestamps (ISO-8601), patterns for wildcards, etc. +- **Ensure comprehensive coverage:** Ensure all parameters, tags, and information sections include clear descriptions +- **Use inclusive language:** Avoid problematic terminology (blacklist, whitelist, execute, kill, etc.) and use inclusive language throughout + +#### Description examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +**Operation descriptions** + +Add descriptions to operation definitions: + +```yaml +paths: + /books: + get: + summary: List books + description: | + Retrieve a paginated list of books from the library catalog. + Use query parameters to filter by author, genre, or publication year. + Results are sorted by title by default. +``` + +**Parameter descriptions** + +Add descriptions to parameter definitions: + +```yaml +parameters: + - name: author + in: query + description: | + Filter books by author name. Supports partial matching - + searching for "smith" will find "John Smith" and "Jane Smithson". + schema: + type: string + example: "Jane Austen" +``` + +**Schema property descriptions** + +Add descriptions directly to schema properties: + +```yaml +components: + schemas: + Book: + type: object + properties: + publishedDate: + type: string + format: date + description: | + The date when the book was first published in ISO 8601 format. + Used for filtering and sorting operations. +``` +:::{tip} +Learn more about providing long descriptions in YAML in the [OpenAPI docs](https://learn.openapis.org/specification/docs.html#providing-long-descriptions-in-yaml). +::: +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +**Operation descriptions** + +Write comments above class definitions: + +```ts +/** + * Get index information + * + * Retrieve configuration and mapping information for one or more indices. + * You can use this API to check index settings, mappings, and other metadata + * before performing operations that might affect the index structure. + */ +class GetIndexRequest { + // ... +} +``` + +**Parameter descriptions** + +Write comments inline: + +```ts +class SearchRequest { + /** + * A comma-separated list of data streams, indices, and index aliases used to limit the request. + * Wildcard expressions (*) are supported. + */ + index: string +} + +class AlertRule { + /** + * An ISO-8601 timestamp that indicates when the event was detected. + */ + detectedAt: string +} +``` +::: + +:::: + +## Document path parameters + +Path parameters are variables in URL paths that users must provide to access specific resources. Proper documentation helps users understand what values are expected and how to construct valid requests. + +Here are some principles for documenting path parameters: + +- **Write clear descriptions:** Follow the general guidance for [writing descriptions](#write-descriptions) +- **All path parameters are required:** the `required` field must be present and it must be `true` in the OpenAPI doc +- **Provide alternative paths for optional variations:** If you need optional parameters, create separate endpoint definitions or list the variations in the description +- **Document parameter constraints:** Specify valid formats, patterns, or value ranges +- **Explain parameter relationships:** Clarify how multiple path parameters work together and any dependencies between them + +### Examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +Define path parameters directly in the path specification: + +```yaml +paths: + /users/{userId}/posts/{postId}: + get: + summary: Get a specific post + parameters: + - name: userId + in: path + required: true + description: The unique identifier of the user + schema: + type: string + example: "12345" + - name: postId + in: path + required: true + description: The unique identifier of the post + schema: + type: string + example: "post-abc" +``` + +For optional variations, create separate paths: + +```yaml +paths: + /books: + get: + summary: List all books + /books/{category}: + get: + summary: List books by category + parameters: + - name: category + in: path + required: true + description: The book category to filter by + schema: + type: string + example: "fiction" +``` +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +For endpoints that can accept optional parameters, define multiple paths in the `urls` array: + +```ts +urls: [ + { + path: "/{index}/_doc/{id}", + methods: ["PUT", "POST"] + }, + { + path: "/{index}/_doc", + methods: ["POST"] + } +] +``` + +**Notes:** +- The `make transform-to-openapi` output shows these two paths as separate entities +- The `make transform-to-openapi-for-docs` output contains only the most complex variation of the paths (and mentions the others in its description) +::: + +:::: + +## Document enum values + +Enumerated types (enums) define a fixed set of allowed values for a property. Well-documented enum values help users understand the purpose of each option and when to use it. + +Here are some principles for documenting enum values: + +- **Clarify non-obvious meanings:** Document when the purpose isn't clear from the name alone +- **Provide usage guidance:** Explain when to use each option and any performance implications +- **Document special behavior:** Note if some values are deprecated, experimental, or have unique characteristics +- **Keep it concise:** Use brief, focused descriptions that explain the essential differences + +:::{tip} +You can skip documenting enum values that are self-explanatory (like `true`/`false`) or follow standard conventions (like HTTP status codes). +::: + +### Examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +In OpenAPI specifications, enum descriptions are documented in schema definitions: + +```yaml +components: + schemas: + SearchType: + type: string + enum: + - query_then_fetch + - dfs_query_then_fetch + - query_and_fetch + description: | + The search execution strategy: + - `query_then_fetch`: Searches across all fields using default settings + - `dfs_query_then_fetch`: Performs distributed frequency scoring before fetching results + - `query_and_fetch`: Fetches results immediately without scoring optimization +``` +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +In TypeScript definitions, enum descriptions are added as JSDoc comments above each value: + +```ts +export enum SearchType { + /** Searches across all fields using default settings. */ + query_then_fetch = "query_then_fetch", + + /** Performs distributed frequency scoring before fetching results. */ + dfs_query_then_fetch = "dfs_query_then_fetch", + + /** Fetches results immediately without scoring optimization. */ + query_and_fetch = "query_and_fetch" +} +``` + +The compiler automatically extracts these descriptions and formats them as lists in the property documentation: + +- `query_then_fetch`: Searches across all fields using default settings +- `dfs_query_then_fetch`: Performs distributed frequency scoring before fetching results +- `query_and_fetch`: Fetches results immediately without scoring optimization +::: + +:::: + + +## Add examples + +Examples help users understand how to use your API with realistic request and response data. Well-written descriptions transform examples from code snippets into effective learning tools. + +Here are some principles for effective examples: + +- **Use realistic data:** Provide examples that reflect actual use cases rather than placeholder values +- **Write clear summaries:** Introduce the example with a very brief summary that explains its purpose (reused as dropdown label in the docs) +- **Write clear descriptions:** Explain what the example accomplishes in more detail and why it's useful +- **Include edge cases:** Show how to handle optional parameters, error conditions, and different response types +- **Point out key details:** Highlight important aspects users might miss +- **Show variations:** Demonstrate alternative approaches or related concepts +- **Provide realistic response bodies:** Each response body should have a realistic example. It must not contain any sensitive or confidential data +- **Include success responses:** Include at least one example for each success response (HTTP 200) + +#### Generated examples + +If you don't provide examples, Bump.sh automatically generates them from your API schema. Because it's hard to randomly generate meaningful examples, this has been disabled for Elasticsearch APIs. + +It's preferable to include curated examples in your OpenAPI document with a realistic combination of property values. You can provide more than one example object in the examples field, so consider adding examples that reflect the most common use cases. + +Bump.sh also generates a single curl example for each API. To override that generated example and optionally provide an equivalent Console or language-specific example, use the `x-codeSamples` extension described in [Custom code examples in Bump.sh](https://docs.bump.sh/help/specification-support/doc-code-samples). + +:::{tip} +When you use validation tools to check your API specification, examples are helpful for ensuring that you haven't missed things like nullable data types. + +Learn more: + +- [The example object](https://spec.openapis.org/oas/latest#example-object) +- [Add examples](https://swagger.io/docs/specification/adding-examples) +::: + +### Examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +The OpenAPI specification supports examples in several ways: + +**Individual parameter examples** can be defined directly in specifications: + +```yaml +components: + schemas: + SearchRequest: + type: object + properties: + department: + type: string + description: Filter employees by department + example: "engineering" + hireDate: + type: string + format: date + description: Minimum hire date in ISO 8601 format + example: "2020-01-01" +``` + +For more complex examples, OpenAPI provides the [Example Object](https://spec.openapis.org/oas/v3.0.3#example-object), which can be attached to request bodies, responses, or parameters. +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +**File-based examples** use a structured folder approach alongside your API specification: + +``` +/specification/your-api/examples/ + ├── request/ + │ ├── basic-search.yml + │ └── advanced-filters.yml + └── response/ + ├── success.yml + └── error.yml +``` + +Each example file follows the [OpenAPI Example object specification](https://spec.openapis.org/oas/v3.0.3#example-object) in YAML format: + +```yaml +summary: Search with filters +method_request: GET /employees/_search +description: > + Combine multiple conditions using a bool query. This example finds employees + in the engineering department hired after 2020. Notice how the bool query's + "must" clause requires both conditions to match. +value: |- + { + "query": { + "bool": { + "must": [ + {"match": {"department": "engineering"}}, + {"range": {"hire_date": {"gte": "2020-01-01"}}} + ] + } + } + } +``` + +**Required fields:** +- `summary` - Brief label that appears in the docs +- `value` - The actual request/response body +- `method_request` - HTTP method and path (request examples only) + +:::{warning} +For Elasticsearch APIs, you only need to provide Console examples. Examples for other programming languages (including cURL) are **generated automatically** and added to [docs/examples/languageExamples.json](https://github.com/elastic/elasticsearch-specification/blob/main/docs/examples/languageExamples.json). +::: +::: +::: + +:::: + +## Add links + +Links help users navigate between related APIs and find additional context in narrative documentation. Strategic linking improves discoverability and helps users understand how different operations work together. + +Here are some principles for effective linking: + +- **Link to prerequisite concepts:** Connect APIs to foundational concepts users need to understand first +- **Reference related operations:** Point to complementary APIs that users typically need together +- **Provide narrative context:** Link to guides that explain when and how to use the API effectively +- **Use descriptive link text:** Choose meaningful labels that indicate what users will find +- **Validate link targets:** Ensure links point to current, accurate documentation + +### Examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +Use the `externalDocs` field to link to narrative guides in the main Elastic documentation: + +```yaml +paths: + /search: + post: + summary: Search documents + description: Execute a search query against one or more indices + externalDocs: + description: Try the hands-on Query DSL tutorial + url: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html +``` + +**Related operation links** can be referenced in descriptions using standard markdown: + +```yaml +paths: + /indices/{index}: + put: + summary: Create an index + description: | + Creates a new index with optional settings and mappings. + See also: [Delete Index](delete-index) and [Update Index Sets](update-index-settings). +``` +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +**External documentation links** use the `@ext_doc_id` annotation to connect to narrative guides. This is transformed into an OpenAPI `externalDocs` field in the [compilation process](overview.md#example-elasticsearch): + +```ts +/** + * @variants container + * @non_exhaustive + * @ext_doc_id query-dsl + */ +export class QueryContainer { + // ... +} +``` + +**API reference links** use the `@doc_id` annotation for language client generation and Console integration: + +```ts +/** + * @rest_spec_name indices.create + * @doc_id indices-create-index + */ +export interface CreateIndexRequest extends RequestBase { + // ... +} +``` + +Both annotations require corresponding entries in [`specification/_doc_ids/table.csv`](https://github.com/elastic/elasticsearch-specification/blob/main/specification/_doc_ids/table.csv): + +:::{note} +Each endpoint can only have one `@ext_doc_id`. For multiple links, use inline markdown in descriptions. +::: +::: + +:::: + +## Set default values + +Default values can only be applied to optional parameters or properties in OpenAPI specifications using the `default` field. + +### Examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +Specify defaults directly in schema definitions: + +```yaml +components: + schemas: + SearchSettings: + properties: + size: + type: integer + default: 10 + timeout: + type: string + default: "1m" +``` + +For arrays: + +```yaml +parameters: + - name: fields + in: query + schema: + type: array + items: + type: string + default: ["id", "name", "created"] +``` +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +Use the `@server_default` annotation for optional properties: + +```ts +class Foo { + /** @server_default "hello" */ + baz?: string + + /** @server_default ["hello", "world"] */ + tags?: string[] +} +``` + +:::{note} +Default values only work on optional properties and appear in parameter documentation without affecting client behavior. +::: +::: + +:::: + +## Lint your API docs + +Linting your API docs helps ensure consistency, correctness, and adherence to best practices. It catches common issues like missing descriptions, inconsistent naming, and formatting errors. + +::::{tab-set} +:group: implementations +::::{tab-item} Elasticsearch +:sync: elasticsearch + +The [Elasticsearch API specification](https://github.com/elastic/elasticsearch-specification/tree/main/docs/linters) uses the following linters with custom rules: + +- **Spectral**: Configuration in `.spectral.yaml` +- **Redocly**: Configuration in `redocly.yaml` + +Refer to [the Elasticsearch quickstart](quickstart.md#format-generate-and-validate-your-changes) to learn how to run the linter locally. + +:::: +:::: diff --git a/docs/contribute/api-docs-contribution/help.md b/docs/contribute/api-docs-contribution/help.md new file mode 100644 index 000000000..4172ba8b3 --- /dev/null +++ b/docs/contribute/api-docs-contribution/help.md @@ -0,0 +1,32 @@ +# Report issues and get help + +## Report issues + +### API docs issues + +When you find a bug or gap in the API docs: + +- If you're comfortable making the change, open a PR in the [relevant repo](workflows.md#quick-reference) +- Alternatively, open an issue in either: + - [elastic/docs-content](https://www.github.com/elastic/docs-content/issues) (public repo) + - [elastic/docs-content-internal](https://www.github.com/elastic/docs-content-internal/issues) (Elastic employees only) + +### Contribution guide issues + +If you find a bug in the contribution guide itself, please open a PR. +For small edits that only touch a single page, click the **Edit this page** button. This will take you to the source file in GitHub, where you can make your changes and submit a PR. + +## Get help + +:::{note} +Elastic employees only. +::: + +The API docs workflow is pretty complicated, and there's lots of moving parts. It takes a village to produce excellent API docs, so don't hesitate to ask for help if you get stuck or have questions. + +If you're not sure who owns a specific API, check out the [API ownership table](workflows.md#quick-reference) to find the right team. + +If you don't find what you need on these pages, Elasticians can reach out in the following Slack channels: + +- [**#docs**](https://elastic.slack.com/archives/C0JF80CJZ): If you need to chat with a technical writer about the API docs. +- [**#devtools**](https://elastic.slack.com/archives/C0D8Y4ZJ9): If you're having issues with the Elasticsearch API documentation tooling or need help with the OpenAPI specification. \ No newline at end of file diff --git a/docs/contribute/api-docs-contribution/images/api-docs-general-pipeline.png b/docs/contribute/api-docs-contribution/images/api-docs-general-pipeline.png new file mode 100644 index 000000000..34c0543dc Binary files /dev/null and b/docs/contribute/api-docs-contribution/images/api-docs-general-pipeline.png differ diff --git a/docs/contribute/api-docs-contribution/images/es-api-docs-pipeline.png b/docs/contribute/api-docs-contribution/images/es-api-docs-pipeline.png new file mode 100644 index 000000000..8dcb3e58f Binary files /dev/null and b/docs/contribute/api-docs-contribution/images/es-api-docs-pipeline.png differ diff --git a/docs/contribute/api-docs-contribution/index.md b/docs/contribute/api-docs-contribution/index.md new file mode 100644 index 000000000..f7515617c --- /dev/null +++ b/docs/contribute/api-docs-contribution/index.md @@ -0,0 +1,21 @@ +--- +navigation_title: Contribute to API docs +--- + +# Contribute to Elastic API docs + +Learn how to contribute to Elastic's Open API docs across our various product repositories. While each team uses different toolchains and workflows, the core principles of how to write excellent API docs are the same. + +1. [**How Elastic API docs work**](./overview.md) - Understand how API docs work at Elastic today, from source files to OpenAPI documents and published documentation. + +2. [**Checklist**](./checklist.md) - Use the checklist to make sure your API docs aren't missing anything important. + +3. [**Core guidelines**](./guidelines.md) - Learn universal principles for writing clear, informative, and valid API docs, including how to write summaries and descriptions, and how to add links and examples. + +4. [**Organize and annotate**](./organize-annotate.md) - Learn how to document API lifecycle status, deployment availability, and required permissions. + +5. [**Contribute locally: Elasticsearch**](./quickstart.md) - Follow a step-by-step guide to contribute to Elasticsearch API docs, including environment setup, local development workflow, and preview generation. + +6. [**Find your workflow**](./workflows.md) - Find the workflow for the specific Elastic API docs you're interested in and learn how the OpenAPI docs are generated, validated, and published. + +7. [**Report issues and get help**](./help.md) - Find out where to ask questions, report issues, and get support from the Elastic community and documentation team across all our API docs workflows. \ No newline at end of file diff --git a/docs/contribute/api-docs-contribution/organize-annotate.md b/docs/contribute/api-docs-contribution/organize-annotate.md new file mode 100644 index 000000000..77779be55 --- /dev/null +++ b/docs/contribute/api-docs-contribution/organize-annotate.md @@ -0,0 +1,452 @@ +--- +navigation_title: Organize and annotate docs +--- + +# Organize and annotate your API docs + +This page explains how to organize your API documentation and add technical metadata to help users understand your APIs. You'll find guidance on: + +- Adding required OpenAPI document information +- Adding structure with operation IDs and tags +- Documenting API availability and lifecycle status +- Specifying required permissions and privileges +- Extending OpenAPI with custom properties + +For guidance on writing effective API content like summaries, descriptions, and examples, see [Core guidelines](./guidelines.md). + +## Add Open API document info + +The published OpenAPI documents must have the following metadata in the [`info` object](https://spec.openapis.org/oas/latest#info-object): + +| **Field** | **Description** | +| -------------- | --------------- | +| `description` | A descriptive overview of the API's purpose and functionality. This should be a concise summary that helps users understand what the API does. | +| `license` | [License information](https://spec.openapis.org/oas/latest#license-object "https://spec.openapis.org/oas/latest#license-object") for the API (which is distinct from the documentation license).
Until we have a custom extension, add the documentation license ([Deed - Attribution-NonCommercial-NoDerivatives 4.0 International - Creative Commons](https://creativecommons.org/licenses/by-nc-nd/4.0/)) to `info.description` | +| `title` | The title of the API | +| `version` | The version of the document (which is distinct from the OpenAPI specification version and the API implementation version) | +| `x-feedbackLink` | This specialization extension enables us to collect feedback about the document. The URL should take readers to a GitHub issue template so they can create an issue. Refer to [Get feedback from users](https://docs.bump.sh/help/publish-documentation/feedback/) | + + +## Add OpenAPI specification version + +You must specify the version number of the OpenAPI specification that the OpenAPI document uses. This value is not related to the API `version` string in the [document info](#add-required-open-api-document-info). + +[Bump.sh](https://bump.sh/openapi "https://bump.sh/openapi") supports all versions from Swagger 2.0 to OpenAPI 3.1. + +Learn more about the [OpenAPI object](https://spec.openapis.org/oas/latest#openapi-object). + +## Add operation identifiers + +The `operationId` uniquely identifies the operation. + +It is case sensitive and must be unique in the OpenAPI document. + +When adding operation identifiers: +- Ensure each operation has a unique `operationId` +- Make `operationId`s valid for use in URLs (no special characters) +- Use consistent naming patterns across related operations + +## Group APIs with tags + +We use [tags](https://swagger.io/docs/specification/v3_0/grouping-operations-with-tags/) to group APIs by feature. + +There are two types of tags: + +| Tag type | Purpose | Notes | +| --- | --- | --- | +| **Document-level** | Define the available tag categories for the entire API | - Use human-readable names
- Add descriptions
- Order affects navigation | +| **Operation-level** | Assign each endpoint to a category | - Use tags from document-level list
- Each operation needs at least one tag | + +**Best practices:** +- Use **consistent tag names** across your entire API +- Use **one tag per operation** for cleaner navigation +- Use **sentence case** for tag names (use `x-displayName` to override the actual tag name if necessary) +- **Order tags logically** to create intuitive navigation flow + +### Examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +```yaml +paths: + /indices: + post: + summary: Create an index + tags: ["Index management"] + description: Creates a new index with optional settings and mappings + /indices/{index}: + delete: + summary: Delete an index + tags: ["Index management"] + description: Removes an index and all its data + /indices/{index}/_settings: + put: + summary: Update index settings + tags: ["Index management"] + description: Modifies settings for an existing index +``` + +**Define tag metadata** in the top-level `tags` section. Write a description for each tag to clarify its purpose: + +```yaml +tags: + - name: "Index management" + description: Operations for creating, configuring, and managing indices + - name: "Search" + description: Operations for querying and retrieving documents + - name: "Document operations" + description: Operations for creating, updating, and deleting individual documents +``` +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +**Default namespace behavior** + +The compiler extracts the namespace from your endpoint name using everything before the first dot: + +- `indices.create` → `"indices"` tag +- `search.template` → `"search"` tag +- `cluster.health` → `"cluster"` tag +- `ping` → `"ping"` tag (no dot, uses full name) + +This automatic grouping works well when your namespace matches how users think about the API functionality. + +**Overriding defaults with `@doc_tag`** + +Use the `@doc_tag` annotation when the automatic namespace doesn't match user expectations or when you want more descriptive groupings: + +```ts +/** + * @rest_spec_name indices.create + * @doc_tag "Index management" + */ +export interface Request extends RequestBase { + // ... +} +``` + +This moves the operation from an "indices" section to an "Index management" section. + +**Add new tags** + +If you create a new tag value, you must also add it to the [elasticsearch-shared-overlays.yaml](https://github.com/elastic/elasticsearch-specification/blob/main/docs/overlays/elasticsearch-shared-overlays.yaml) file. You can see all existing tag values in that file. + +Use **sentence case** for tag names. If you need different casing in the final docs, use the `x-displayName` extension in the overlay file. + +:::{note} +Each operation can only have one tag, even though the OpenAPI specification supports multiple tags. This is a deliberate system constraint to keep the docs navigation clean and predictable. +::: +::: + +::::: + +## Extend OpenAPI documents + +Per [OpenAPI Specification v3.0.3](https://spec.openapis.org/oas/v3.0.3#specification-extensions): + +_"While the OpenAPI Specification tries to accommodate most use cases, additional data can be added to extend the specification at certain points."_ + +The extensions properties are implemented as patterned fields that are always prefixed by `x-`. The following extensions are supported by Bump.sh: + +| **Field** | **Description** | +|-----------------|-----------------| +| `x-beta` | Indicates that something is in beta. **Note:** Use `x-state` instead. | +| `x-codeSamples` | Defines code examples and their programming language. [Docs](https://docs.bump.sh/help/specification-support/doc-code-samples/) | +| `x-displayName` | Overrides tag names at the document level to align with documentation standards. | +| `x-feedbackLink` | Adds a link for users to send feedback. [Docs](https://docs.bump.sh/help/publish-documentation/feedback/) | +| `x-model` | Used to abbreviate deeply nested/recursive API sections (e.g., Elasticsearch query DSL). Should include an `externalDocs` link. Currently only applied via overlays. | +| `x-state` | Indicates lifecycle state (e.g., “Technical preview; added in 9.1.0”). Appears next to the operation/property. | +| `x-topics` | Adds extra pages shown below the introduction. [Docs](https://docs.bump.sh/help/enhance-documentation-content/topics/) | + +:::{note} +For more information, refer to the [Bump.sh page about Extensions](https://docs.bump.sh/help/specification-support/extensions/) +::: + +## Specify API lifecycle status + +In OpenAPI, lifecycle status is communicated through extension properties like `x-state` and the standard `deprecated` field. These annotations indicate deployment compatibility, stability level, and version information. + +### Examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +**Availability and stability** is specified using the `x-state` extension property: + +```yaml +paths: + /indices/{index}: + get: + summary: Get index information + x-state: "Generally available; Added in 7.10.0" + # ... + + /ml/anomaly_detection/{jobId}/_forecast: + post: + summary: Create anomaly detection forecast + x-state: "Beta; Added in 8.5.0" + # ... +``` +**Deprecation notices** use the standard OpenAPI `deprecated` boolean field: + +```yaml +components: + schemas: + IndexSettings: + type: object + properties: + blocks: + type: object + deprecated: true + description: | + Deprecated in 8.0.0. Use 'index.blocks.read_only' instead. + Controls read/write blocks on the index. +``` +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +**API availability** is specified using the `@availability` annotation: + +```ts +/** + * @availability stack + * @availability serverless + */ +class FooRequest { + bar: string + /** + * @availability stack + * @availability serverless + */ + baz: string + faz: string +} +``` + +**When to use parameter/property-level availability** + +Parameter and property-level `@availability` annotations are necessary in specific scenarios: + +1. **Mixed deployment availability**: When an API supports both serverless and stateful deployments, but some parameters/properties are only available in one deployment type +2. **Later additions**: When a parameter or property is added after the initial API release, making its availability later than the API-level availability + +**API stability** is indicated using the `stability` parameter: + +```ts +/** + * @availability stack since=8.0.0 stability=beta + * @availability serverless stability=experimental + */ +class BetaRequest { + // ... +} +``` + +The stability values map to the following `x-state` labels: +- **`stability=stable`** (default) → "Generally available" +- **`stability=beta`** → "Beta" +- **`stability=experimental`** → "Technical preview" + +**Deployment-specific APIs** can be restricted to certain deployment types: + +```ts +export class Example { + /** + * Available in both (default when no annotations) + */ + fieldBoth1: integer + + /** + * @availability serverless + */ + fieldServerlessOnly: integer + + /** + * @availability stack + */ + fieldStackOnly: integer +} +``` + +This example shows a mixed deployment scenario where the API itself is available to both deployment types, but individual parameters have different availability constraints. + +**Version information** uses the `since=` parameter: + +```ts +/** + * @availability stack since=7.10.0 + * @availability serverless + */ +class FooRequest { + /** + * Added with the original API + */ + originalField: string + + /** + * @availability stack since=7.11.0 + * @availability serverless + */ + laterAddition: string +} +``` + +This example shows the "later addition" scenario where `laterAddition` was added after the initial API release, requiring a parameter-level availability annotation with a later version than the API-level availability. + +**Deprecation notices** use the `@deprecated` annotation: + +```ts +class Foo { + bar: string + /** @deprecated 7.0.0 */ + baz?: string + faz: string +} +``` + +You can add an optional description explaining the deprecation: + +```ts +class Foo { + bar: string + /** @deprecated 7.0.0 'baz' has been deprecated, use 'bar' instead */ + baz?: string + faz: string +} +``` +::: + +:::: + +:::{important} +Since we now only publish API docs for major versions (v8, v9) and not for minor versions (8.1, 8.2, etc.), always include the full version information in your `x-state` labels. For example: `Technical preview; added in 9.1.0". +::: + +## Document required permissions + +Required permissions aren't part of the standard OpenAPI specification, but they can be documented in operation descriptions or through custom extensions. In Elasticsearch, we use annotations that generate structured permission documentation. + +### Examples + +::::{tab-set} +:group: implementations + +:::{tab-item} OpenAPI +:sync: openapi + +Add permission requirements to the operation description using markdown formatting: + +```yaml +paths: + /indices/{index}: + put: + summary: Create index + description: | + Creates a new index with optional settings and mappings. + + ## Required authorization + + * Index privileges: `create_index`, `manage` +``` + +For APIs requiring multiple permission types: + +```yaml +paths: + /indices/{index}/_stats: + get: + summary: Get index statistics + description: | + Returns statistics about one or more indices. + + ## Required authorization + + * Index privileges: `monitor`, `view_index_metadata` + * Cluster privileges: `monitor` +``` +::: + +:::{tab-item} Elasticsearch +:sync: elasticsearch + +**Index-level permissions** use the `@index_privileges` annotation: + +```ts +/** + * @rest_spec_name indices.create + * @availability stack since=0.0.0 stability=stable + * @index_privileges create_index, manage + */ +export interface Request extends RequestBase { + // ... +} +``` + +This generates the following markdown section in the operation description: + +```markdown +## Required authorization + +* Index privileges: `create_index`, `manage` +``` + +**Cluster-level permissions** use the `@cluster_privileges` annotation: + +```ts +/** + * @rest_spec_name cluster.state + * @availability stack since=1.3.0 stability=stable + * @cluster_privileges monitor, manage + */ +export interface Request extends RequestBase { + // ... +} +``` + +This adds: + +```markdown +## Required authorization + +* Cluster privileges: `monitor`, `manage` +``` + +**Combined privileges** use both annotations together: + +```ts +/** + * @rest_spec_name indices.analyze + * @index_privileges read + * @cluster_privileges monitor + */ +export interface Request extends RequestBase { + // ... +} +``` + +This creates: + +```markdown +## Required authorization + +* Index privileges: `read` +* Cluster privileges: `monitor` +``` + +:::{note} +The annotation system doesn't support "OR" relationships between privileges. When multiple privileges are listed, they appear as if all are required. Use only the minimum necessary privileges for each operation to avoid confusion. +::: +::: + +:::: \ No newline at end of file diff --git a/docs/contribute/api-docs-contribution/overview.md b/docs/contribute/api-docs-contribution/overview.md new file mode 100644 index 000000000..f732fb40b --- /dev/null +++ b/docs/contribute/api-docs-contribution/overview.md @@ -0,0 +1,109 @@ +# How Elastic API docs work + +This page explains at a high-level how API docs work at Elastic today, with context on how this differs from our previous approaches and where we're heading. Use this page to understand the core primitives and workflows that apply across all Elastic teams, so you can contribute effectively. + +While the implementation details [vary significantly across teams](workflows.md), at a high level: + +- We use [OpenAPI](https://spec.openapis.org/oas/latest.html) files to generate REST API documentation +- Our API docs are hosted by [Bump.sh](https://bump.sh/) +- The published docs live at [elastic.co/docs/api/](https://www.elastic.co/docs/api/) + +## Context and evolution + +This table gives a simple overview of how API docs were produced in the past, how we do it today, and where we're heading. + +This evolution seeks to elevate documentation to being a first-class citizen in the API development lifecycle. + +| Era | Developer Tasks | Approach | Publication | +| --- | --- | --- | --- | +| **Yesterday (8.x and earlier)** | Code + spec + manual API docs | Manual maintenance | - Hand-written in Asciidoc
- Published at [elastic.co/guide](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/rest-apis.html)) | +| **Today (9.0+)** | Code + spec | Docs generated from spec |- Hosted by [Bump.sh](https://www.elastic.co/docs/api/)
- Separate from [elastic.co/docs](https://www.elastic.co/docs) + +## High-level process + +All Elastic API docs follows this general pattern: + + B[OpenAPI documents] + B --> C[Published documentation] + + style A fill:#fff2cc + style B fill:#e1d5e7 + style C fill:#dae8fc +--> + +:::{image} images/api-docs-general-pipeline.png +:alt: High-level API docs workflow pipeline diagram, showing the flow from source files to OpenAPI documents and published documentation. +:width: 490px +::: + +1. **Source files** can be: + - TypeScript definitions with JSDoc comments (Elasticsearch) + - Code annotations in application source (some Kibana teams) + - Manual YAML/JSON specifications (Logstash) + - Generated specifications from service code (Cloud teams) +2. **OpenAPI documents** are either generated from source files or edited manually. This is the common intermediate format regardless of origin. +3. **Published documentation** lives at [elastic.co/docs/api/](https://www.elastic.co/docs/api/) and is hosted by Bump.sh. + +## Example: Elasticsearch + +The Elasticsearch API specification workflow is particularly complex because it combines multiple source files and serves various downstream consumers. + +When adding a new API, Elasticsearch engineers first create a very basic spec in the [elasticsearch repo](https://github.com/elastic/elasticsearch/tree/main/rest-api-spec). Those specs are mirrored, and made more robust and detailed in [elasticsearch-specification](https://github.com/elastic/elasticsearch-specification/tree/main/docs). + +The generated Schema JSON and OpenAPI documents feed into client libraries (and their docs), the Dev Tools Console, and the [Elasticsearch API reference](https://www.elastic.co/docs/api/doc/elasticsearch/) (including the [Serverless API reference](https://www.elastic.co/docs/api/doc/elasticsearch-serverless/)). Here's how the pipeline works: + +% Mermaid source code +|compiled| B[Schema JSON] + F[JSON spec files] -->|merged| B + G[Examples YAML files] -->|processed| B + B -->|converted| C1[OpenAPI for Documentation] + B -->|converted| C2[OpenAPI for Clients, Dev Tools Console, etc.] + H[Overlays] -->|applied| C1 + C1 -->|published| E["HTML published by Bump.sh"] + C2 -->|used by| I[Clients & Dev Tools Console] + B -->|generated| D[Client Libraries] + + style A fill:#fff2cc + style F fill:#fff2cc + style G fill:#fff2cc + style B fill:#d5e8d4 + style C1 fill:#e1d5e7 + style C2 fill:#e1d5e7 + style H fill:#fff2cc + style D fill:#dae8fc + style E fill:#dae8fc + style I fill:#dae8fc +--> + + +:::{image} images/es-api-docs-pipeline.png +:alt: API docs generation pipeline diagram, showing the flow from TypeScript specifications to JSON schema, OpenAPI transformation, and HTML publishing. +:width: 750px +:align: center +::: + +:::{important} +The Dev Tools and Elasticsearch teams are working together to merge the two specifications into the `elasticsearch` repo in 2025. +::: + +### Input sources + +Four types of files feed into the compilation process: +- **TypeScript API definitions** contain JSDoc comments with descriptions, annotations, and type information +- **JSON spec files** define endpoint URLs, HTTP methods, and parameters for each API +- **Example YAML files** provide realistic request/response demonstrations +- **OpenAPI overlays** allow customization of the generated OpenAPI documents for specific consumers + +**Schema compilation:** All sources are processed and merged into a single Schema JSON file. TypeScript definitions are compiled and validated, JSON specs are merged in, and examples are parsed from YAML and attached to their corresponding endpoints. Compilation failures indicate missing types, malformed specs, or invalid examples that must be fixed. + +**OpenAPI generation:** The Schema JSON is converted into OpenAPI documents optimized for different consumers. [OpenAPI overlays](https://github.com/OAI/Overlay-Specification?tab=readme-ov-file#overlay-specification) are then applied to the OpenAPI documents for API docs, to handle publisher-specific requirements or work around rendering limitations. + +**Publication:** The final OpenAPI documents are published as HTML documentation via Bump.sh. A separate generation process creates client libraries directly from the Schema JSON. + +## Next steps + +Now that you understand how API docs work at Elastic at a high level, explore the following sections to learn how to contribute effectively. \ No newline at end of file diff --git a/docs/contribute/api-docs-contribution/quickstart.md b/docs/contribute/api-docs-contribution/quickstart.md new file mode 100644 index 000000000..a563c374f --- /dev/null +++ b/docs/contribute/api-docs-contribution/quickstart.md @@ -0,0 +1,116 @@ +--- +navigation_title: Contribute locally (Elasticsearch) +--- + +# Contribute locally: Elasticsearch quickstart + +The Elasticsearch APIs are the foundation of the Elastic Stack and the largest API set we maintain. Because the workflow is quite complex, we created this quickstart guide to help you get started. + +This is a step-by-step local development workflow. While CI runs these steps automatically on PR branches in the `elasticsearch-specification` repo (see [Makefile](https://github.com/elastic/elasticsearch-specification/blob/main/Makefile)), working locally enables you to validate, preview and debug before submitting your changes. For a complete list of available make targets, run `make help`. + +For the official Elasticsearch specification contribution guidance, see [`CONTRIBUTING.md`](https://github.com/elastic/elasticsearch-specification/blob/main/CONTRIBUTING.md#contributing-to-the-elasticsearch-specification). + +:::::{stepper} + +::::{step} Prepare your environment + +Run this command to set up your Node.js environment: + +```shell +nvm use +``` +If you don't have Node.js installed, refer to the [setup guide](https://github.com/elastic/elasticsearch-specification/tree/main?tab=readme-ov-file#prepare-the-environment). +:::: + +::::{step} Clone the specification repo +```shell +git clone https://github.com/elastic/elasticsearch-specification.git +cd elasticsearch-specification +``` +:::{warning} +You must [create PRs from a branch](https://github.com/elastic/elasticsearch-specification/blob/main/CONTRIBUTING.md#send-your-pull-request-from-a-branch) in the `elasticsearch-specification` repo, not a fork. +::: +:::: + +::::{step} Install dependencies +```shell +make setup +``` + +:::{important} +You should run `make setup` every time you begin work on a contribution, because the `elasticsearch-specification` repository is under active development. This ensures you have the latest dependencies and tools. +::: + +:::: + +::::{step} Make your docs changes +Edit the relevant TypeScript files in the `specification` directory. Use JSDoc comments to describe your API interfaces, following the [guidelines](./guidelines.md). Add or update summaries, descriptions, tags, metadata, links, and examples as needed. + +:::{important} +If you're adding a new API, you must first create a REST API specification file in the [`specification/_json_spec`](https://github.com/elastic/elasticsearch-specification/tree/main/specification/_json_spec) directory. +::: + +::::{step} Format, generate and validate your changes +```shell +make contrib +``` +This command runs multiple steps in sequence: + +1. Formats your code (`spec-format-fix`) +2. Generates the schema JSON (`generate`) +3. Transforms to OpenAPI format for language clients (`transform-to-openapi`) +4. Filters for serverless (`filter-for-serverless`) +5. Lints the language clients OpenAPIdocs (`lint-docs`) + +:::{note} +Some of the linter errors at this stage may be false alarms, and are fixed by path consolidation and overlays. You'll need to run `make lint` later against the docs-specific OpenAPI files. +::: +:::: + +::::{step} Generate docs-specific OpenAPI files +```shell +make transform-to-openapi-for-docs +``` +Generates the OpenAPI files specifically for docs purposes. This step also runs `generate-language-examples` to autogenerate examples for the various language clients and `curl`. + +:::{note} +The `transform-to-openapi` command (run by `make contrib`) is used for client libraries and does not generate the JSON schema files needed for docs purposes. +::: +:::: + +::::{step} Lint your docs + +Run this command to lint your docs-specific OpenAPI files: +```shell +make lint-docs +``` + +You should try to fix all linter warnings and not just errors. +:::: + +::::{step} Apply overlays + +[OpenAPI overlays](https://github.com/OAI/Overlay-Specification?tab=readme-ov-file#overlay-specification) are used to handle publisher-specific requirements or work around rendering limitations. For example, they sort the list of tags alphabetically and apply `x-model` extensions to abbreviate deeply nested/recursive schema objects. + +```shell +make overlay-docs +``` + +::::{step} Preview your changes +Generate a preview of how your docs will appear: +```shell +npm install -g bump-cli +bump preview output/openapi/elasticsearch-openapi-docs-final.json # Preview Elasticsearch API docs +bump preview output/openapi/elasticsearch-serverless-openapi-docs-final.json # Preview Elasticsearch serverless API docs +``` +This creates a temporary URL to preview your changes and share with others. +:::: + +::::{step} Open a pull request + +Once you're satisfied with your docs changes: +1. Create a pull request from a branch on your local clone +2. The CI will validate your OpenAPI specs +:::: + +::::: \ No newline at end of file diff --git a/docs/contribute/api-docs-contribution/workflows.md b/docs/contribute/api-docs-contribution/workflows.md new file mode 100644 index 000000000..cd888110a --- /dev/null +++ b/docs/contribute/api-docs-contribution/workflows.md @@ -0,0 +1,258 @@ +--- +navigation_title: Find your team's workflow +--- +# Find the workflow for your API docs + +:::{warning} +This is WIP and apart from the quick reference is just a quick-and-dirty copy/paste from the Confluence doc +::: + +Quickly find the team-specific workflows related to creating, reviewing, and publishing API docs for various Elastic products. Each team has its own unique process. + +:::{important} +The bulk of the content on this page is relevant to Elastic employees only. Most links are to internal resources or private repos. +::: + +## Quick reference + +| Product | Repo | OpenAPI spec file | Live docs | Publishing process | Automated? | Status | +|---------|------------|---------------------|-----------|-------------------|------------|--------| +| Elasticsearch | [elasticsearch-specification](https://github.com/elastic/elasticsearch-specification) | [elasticsearch-openapi.json](https://github.com/elastic/elasticsearch-specification/blob/main/output/openapi/elasticsearch-openapi.json) | [elasticsearch](https://www.elastic.co/docs/api/doc/elasticsearch) | Manual generation + Bump.sh deployment | No | Partial; docs-specific files are not checked in | +| Elasticsearch Serverless | [elasticsearch-specification](https://github.com/elastic/elasticsearch-specification) | [elasticsearch-serverless-openapi.json](https://github.com/elastic/elasticsearch-specification/blob/main/output/openapi/elasticsearch-serverless-openapi.json) | [elasticsearch-serverless](https://www.elastic.co/docs/api/doc/elasticsearch-serverless) | Manual generation + Bump.sh deployment | No | Partial; docs-specific files are not checked in | +| Kibana | [kibana/oas_docs](https://github.com/elastic/kibana/tree/main/oas_docs) | [bundle.json](https://github.com/elastic/kibana/blob/main/oas_docs/bundle.json) | [kibana](https://www.elastic.co/docs/api/doc/kibana) | Auto-generated, manual release | Partial | Full | +| Kibana Serverless | [kibana/oas_docs](https://github.com/elastic/kibana/tree/main/oas_docs) | [bundle.serverless.json](https://github.com/elastic/kibana/blob/main/oas_docs/bundle.serverless.json) | [serverless](https://www.elastic.co/docs/api/doc/serverless) | Auto-generated, manual release | Partial | Full | +| Elastic Cloud | [cloud repo](https://github.com/elastic/cloud) | [apidocs-user.json](https://github.com/elastic/cloud/blob/master/scala-services/adminconsole/src/main/resources/apidocs-user.json) | [cloud](https://www.elastic.co/docs/api/doc/cloud) | Manual overlay + Bump.sh deployment | No | Partial; Missing overlays | +| Elastic Cloud Enterprise | [cloud repo](https://github.com/elastic/cloud) | [apidocs.json](https://github.com/elastic/cloud/blob/master/scala-services/adminconsole/src/main/resources/apidocs.json) | [cloud-enterprise](https://www.elastic.co/docs/api/doc/cloud-enterprise) | Manual overlay + Bump.sh deployment | No | Partial; Missing overlays | +| Elastic Cloud Billing | [cloud repo](https://github.com/elastic/cloud) | [billing-service.external.yaml](https://github.com/elastic/cloud/blob/master/python-services-v3/openapi/billing-service.external.yaml) | [cloud-billing](https://www.elastic.co/docs/api/doc/cloud-billing) | Direct file upload to Bump.sh | No | Full | +| Elastic Cloud Serverless | Multiple repos | Multiple files | [elastic-cloud-serverless](https://www.elastic.co/docs/api/doc/elastic-cloud-serverless) | Fully automated | Yes | Full | +| Logstash | [logstash repo](https://github.com/elastic/logstash) | [logstash-api.yaml](https://github.com/elastic/logstash/blob/main/docs/static/spec/openapi/logstash-api.yaml) | [logstash](https://www.elastic.co/docs/api/doc/logstash) | Manual Bump.sh deployment | No | Full | +| Observability Intake | [apm-managed-service](https://github.com/elastic/apm-managed-service) | [bundled-apm-mis-openapi.json](https://github.com/elastic/apm-managed-service/blob/main/docs/spec/openapi/bundled-apm-mis-openapi.json) | [observability-serverless](https://www.elastic.co/docs/api/doc/observability-serverless) | Manual generation + Bump.sh deployment | No | Full | + +## Processes + +Most of the openAPI documents are not yet automatically published. Until that is the case, they must be manually published as described in the following sections. For a related docs engineering page, check out [https://elasticco.atlassian.net/wiki/spaces/DE/pages/746193088](https://elasticco.atlassian.net/wiki/spaces/DE/pages/746193088) (Elastic employees only). + +### Elastic Cloud and Elastic Cloud Enterprise + +:::{note} +This process only applies overlays to existing generated OpenAPI spec files (authored in Scala code annotations). Regenerating these spec files requires a complete Cloud development environment. If you need to make changes to endpoint or property descriptions, or author new descriptions, then you might need to ask a cloud developer to regenerate the spec for you from your branch. For more details, refer to [https://elasticco.atlassian.net/wiki/spaces/DOC/pages/61605944](https://elasticco.atlassian.net/wiki/spaces/DOC/pages/61605944) (Elastic employees only). +::: + +To publish content to [https://www.elastic.co/docs/api/doc/cloud](https://www.elastic.co/docs/api/doc/cloud) and [https://www.elastic.co/docs/api/doc/cloud-enterprise](https://www.elastic.co/docs/api/doc/cloud-enterprise): + +1. Check out the appropriate branch in the [cloud repo](https://github.com/elastic/cloud) (i.e. whatever ms-xxx branch is current according to the [release schedule](https://docs.google.com/spreadsheets/d/1ohPWWkUy5Cc2k4HJSx3XBKXY5nxeuiehApFLFHdHM3U/edit?gid=1929311770#gid=1929311770) (Elastic employees only)). + +2. Generate the API document by using [Makefile](https://github.com/elastic/cloud/blob/master/docs/Makefile) commands: + + 1. Go to the `docs` folder + + 2. Run `make api-docs-overlay` + The output files are: + + - Elastic Cloud: `docs/openapi/output/apidocs-user.new.json` + + - Elastic Cloud Enterprise: `docs/openapi/output/apidocs.new.json` + If you encounter an error you might need to create that output directory then re-run the command. + + 3. Run `make api-docs-lint` + If there are errors, they should be addressed before deployment. Warnings and info messages can be ignored. + +3. Get assistance from someone with admin authority in Bump.sh (reach out on #next-api-reference Slack channel) to deploy the new output file as described in [https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/](https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/) and [https://docs.google.com/document/d/1xP_n2AusBhJcLmmJeCZZkAztzeykvkZtJ6la2XHz7Aw/edit?tab=t.0](https://docs.google.com/document/d/1xP_n2AusBhJcLmmJeCZZkAztzeykvkZtJ6la2XHz7Aw/edit?tab=t.0) (Elastic employees only). + + - If the file that you upload is unchanged from the previous one, you will receive a message that there's nothing new to upload. + + +### Elastic Cloud Billing + +To publish content to [https://www.elastic.co/docs/api/doc/cloud-billing](https://www.elastic.co/docs/api/doc/cloud-billing) : + +1. Check out the appropriate branch in the [cloud repo](https://github.com/elastic/cloud) (i.e. "master"). + +2. Get someone with Bump.sh admin or maintainer authority (or ask on #next-api-reference) to upload and release the following file: [https://github.com/elastic/cloud/blob/master/python-services-v3/openapi/billing-service.external.yaml](https://github.com/elastic/cloud/blob/master/python-services-v3/openapi/billing-service.external.yaml). For admin Bump.sh users, refer to [Bump.sh API admin instructions](https://docs.google.com/document/d/1xP_n2AusBhJcLmmJeCZZkAztzeykvkZtJ6la2XHz7Aw/edit?tab=t.0) (Elastic employees only). + + +### Elastic Cloud Serverless + +The content in [https://www.elastic.co/docs/api/doc/elastic-cloud-serverless](https://www.elastic.co/docs/api/doc/elastic-cloud-serverless) is automatically generated and deployed per [2024 - Serverless Api Spec publication](https://docs.google.com/presentation/d/1y5BRJ8SY41Rj-DEMcIxASEFXL-zuPqLsfwKsbzfAtVs/edit#slide=id.p13) (Elastic employees only) + +The source files are in several repos, such as [https://github.com/elastic/project-api](https://github.com/elastic/project-api), [https://github.com/elastic/cluster-api](https://github.com/elastic/cluster-api), and [https://github.com/elastic/serverless-api-specification](https://github.com/elastic/serverless-api-specification). For example, the `info.description` can be edited in [https://github.com/elastic/serverless-api-specification/blob/main/templates/production/public-user-serverless-header.yml](https://github.com/elastic/serverless-api-specification/blob/main/templates/production/public-user-serverless-header.yml) + +### Elasticsearch + +For info about the structure of the Elasticsearch specifications, refer to [elasticsearch-specification/docs](https://github.com/elastic/elasticsearch-specification/tree/main/docs). + +To learn how to set up your environment and contribute to Elasticsearch API documentation, refer to the [Elasticsearch quickstart](./quickstart.md). + +#### Publish + +To publish content to [https://www.elastic.co/docs/api/doc/elasticsearch-serverless](https://www.elastic.co/docs/api/doc/elasticsearch-serverless) and [https://www.elastic.co/docs/api/doc/elasticsearch](https://www.elastic.co/docs/api/doc/elasticsearch): + +1. Check out the appropriate branch in the [elasticsearch-specification repo](https://github.com/elastic/elasticsearch-specification) (e.g. "main", or the current v9 or v8 branch). + +2. (Optional) Regenerate the elasticsearch specification and OpenAPI documents by using [Makefile](https://github.com/elastic/elasticsearch-specification/blob/main/Makefile) commands. + Refer to [readme](https://github.com/elastic/elasticsearch-specification?tab=readme-ov-file#how-to-generate-the-openapi-representation) for the definitive details. + The output files are generally up-to-date since there are automated tasks that re-generate them. + + 1. `make setup` + + 2. `make generate` + +3. Generate the docs-specific OpenAPI documents, apply overlays, and run lint checks: + + :::{note} + Unlike the Kibana process, these files are not checked into the repo so you must do this task manually before uploading the files to Bump.sh. + ::: + + 1. `make transform-to-openapi-for-docs` (new command added for 8.18+) + + 2. `make overlay-docs` + The output files are: + + 1. Elasticsearch: `output/openapi/elasticsearch-openapi-docs-final.json` (new output file names as of 8.18+) + + 2. Elasticsearch Serverless: `output/openapi/elasticsearch-serverless-openapi-docs-final.json` + + 3. Run `make lint-docs` + If there are errors, they should be addressed before deployment. Warnings and info messages can be ignored. + + 1. There is an error in the Elasticsearch document about two paths being ambiguous, which is true but also not something that needs to be fixed. It can be ignored for now. + +4. Get assistance from someone with admin authority in Bump.sh (reach out on #next-api-reference Slack channel) to deploy the new output file as described in [https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/](https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/) and [Bump.sh API admin instructions](https://docs.google.com/document/d/1xP_n2AusBhJcLmmJeCZZkAztzeykvkZtJ6la2XHz7Aw/edit?tab=t.0) (Elastic employees only). + + :::{important} + You must ensure that you upload the file to the appropriate version in Bump.sh ("main", "v9", or "v8") as described in [https://docs.bump.sh/help/publish-documentation/branching/#deploy-to-a-specific-branch](https://docs.bump.sh/help/publish-documentation/branching/#deploy-to-a-specific-branch) + ::: + + - If the file that you upload is unchanged from the previous one, you will receive a message that there's nothing new to deploy. + + +### Kibana + +For info about how the OpenAPI documents are generated, refer to [kibana/oas_docs/readme](https://github.com/elastic/kibana/tree/main/oas_docs#readme). + +#### Review + +##### Review generated APIs + +1. If you're reviewing Kibana APIs generated from code (per [Generating OAS for HTTP APIs](https://docs.elastic.dev/kibana-dev-docs/genereating-oas-for-http-apis)), check for: + + - API-level details: + + + - **tags**: ['oas-tag: my tag'], which defines which group the API will belong to. Output as tag. + + - **availability**: { since '9.1.0', stability: 'experimental'}, which defines the lifecycle for each API. Output as `x-state`. + + - **security**: { authz: { requiredPrivileges: ...}, which lists the specific Kibana security privileges required. Output in operation description. + + - **summary**, which is output as the [operation summary](https://elasticco.atlassian.net/wiki/spaces/DOC/pages/450494532/API+reference+docs#Summaries) (Elastic employees only). + + - **description**, which is output as the operation [description](https://elasticco.atlassian.net/wiki/spaces/DOC/pages/450494532/API+reference+docs#Descriptions) (Elastic employees only). - Request and response examples, per the "Adding examples" section under [Route definitions](https://docs.elastic.dev/kibana-dev-docs/genereating-oas-for-http-apis#2-route-definitions). + + - **Parameter- and property-level details**: + + - Descriptions for all properties and parameters. For style guidance, refer to [Descriptions](https://elasticco.atlassian.net/wiki/spaces/DOC/pages/450494532/API+reference+docs#Descriptions) (Elastic employees only). + +##### Review OpenAPI files for individual features + +If you're reviewing Kibana APIs that are derived from OpenAPI files for individual features, check for: + + - the general OpenAPI [documentation requirements](https://elasticco.atlassian.net/wiki/spaces/DOC/pages/450494532/API+reference+docs#Documentation-requirements) + + - the presence of x-state with API lifecycle details, per [Specification extensions](https://elasticco.atlassian.net/wiki/spaces/DOC/pages/450494532/API+reference+docs#Specification-extensions). + +2. If you're suggesting changes in a pull request, make your suggestions against the source files (e.g. in the typescript or feature-specific yaml files instead of the oas_docs folder). + +3. If you notice that an entire API is being removed, it means that there will potentially be broken links to that page (either from docs or UIs or other places that link to APIs or in the SEO results). There are discussions about how to automatically take action when this occurs, but for now, reach out in #docs and we'll likely need to add a redirect in [https://github.com/elastic/docs-infra/blob/main/modules/elastic-docs-v3/api-redirects.json](https://github.com/elastic/docs-infra/blob/main/modules/elastic-docs-v3/api-redirects.json) + + +Refer to [oas_doccs/scripts](https://github.com/elastic/kibana/tree/main/oas_docs/scripts) for the list of individual files that are merged into the final Kibana API documents. + +#### Publish + +Followed these steps to publish content to [https://www.elastic.co/docs/api/doc/kibana](https://www.elastic.co/docs/api/doc/kibana) or [https://www.elastic.co/docs/api/doc/serverless](https://www.elastic.co/docs/api/doc/serverless): + +- Option 1: Publish the content that was automatically deployed (this should be the default path) + + - Just ask from someone with admin authority in Bump.sh (reach out `#docs` Slack channel) to deploy the latest documents. + + - Why so easy? There is a buildkite script ([publish_oas_docs.sh](https://github.com/elastic/kibana/blob/main/.buildkite/scripts/steps/openapi_publishing/publish_oas_docs.sh)) that deploys the OpenAPI documents to Bump.sh when a PR that affects those files is merged. This final step to release them is required because we've still got [Manual release mode](https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/#manually-release-a-deployment) enabled but that will eventually be turned off too. + + - Note it's not yet completely foolproof because when a new version branch is forked or the "current" branch changes, the buildkite script needs updates. For an example of how to update that file, check out: [https://github.com/elastic/kibana/pull/219864](https://github.com/elastic/kibana/pull/219864) + +- Option 2: Generate and deploy the content manually (shouldn't be necessary any longer) + + 1. Check out the appropriate branch in the [kibana repo](https://github.com/elastic/kibana) (e.g. "main" or the current v8 or v9 branch). + + 2. Generate the output by using [Makefile](https://github.com/elastic/kibana/blob/main/oas_docs/makefile) commands: + + 1. Go to the `oas_docs` folder. + + 2. (Optional) Regenerate the OpenAPI content that is derived from the code. This generally shouldn't be necessary but if you need to force it to occur, refer to the commands in [https://github.com/elastic/kibana/blob/main/.buildkite/scripts/steps/checks/capture_oas_snapshot.sh](https://github.com/elastic/kibana/blob/main/.buildkite/scripts/steps/checks/capture_oas_snapshot.sh) + If you get questions from dev teams about the source files or process for adding new APIs, refer to: + + 1. [https://docs.elastic.dev/kibana-dev-docs/genereating-oas-for-http-apis](https://docs.elastic.dev/kibana-dev-docs/genereating-oas-for-http-apis) in general + + 2. [https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/openapi/README.md](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/openapi/README.md) for Security APIs + + 3. Run `make api-docs` (bundles all the disparate source files and applies overlays). + + 1. For Kibana, the output file is `oas_docs/output/kibana.yaml` + + 2. For Kibana Serverless, the output file is `oas_docs/output/kibana.serverless.yaml` + + 3. If you get errors, it might mean you need to refresh the cache and/or bootstrap Kibana per [https://www.elastic.co/guide/en/kibana/current/development-getting-started.html#_install_dependencies](https://www.elastic.co/guide/en/kibana/current/development-getting-started.html#_install_dependencies) + + 4. Run `make api-docs-lint` + If there are errors, they should be addressed before deployment. Warnings and info messages can be ignored. + + 3. Preview the output in Bump.sh. After you [install the Bump.sh CLI](https://docs.bump.sh/help/continuous-integration/cli/#installation) you can generate a preview from the command line. For example: bump preview output/kibana.yaml + + 4. Get assistance from someone with admin authority in Bump.sh (reach out on #next-api-reference Slack channel) to deploy the new output file as described in [https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/](https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/) and [Bump.sh API admin instructions](https://docs.google.com/document/d/1xP_n2AusBhJcLmmJeCZZkAztzeykvkZtJ6la2XHz7Aw/edit?tab=t.0) (Elastic employees only). + + :::{important} + You must ensure that you upload the file to the appropriate version in Bump.sh ("main", "v9", or "v8") as described in [https://docs.bump.sh/help/publish-documentation/branching/#deploy-to-a-specific-branch](https://docs.bump.sh/help/publish-documentation/branching/#deploy-to-a-specific-branch) + + If the file that you upload is unchanged from whatever was last automatically deployed, you will receive a message that there's nothing new to upload and should just release the latest existing file (for the appropriate branch). + ::: + + + +### Logstash + +To publish content to [https://www.elastic.co/docs/api/doc/logstash](https://www.elastic.co/docs/api/doc/logstash) : + +1. Check out the appropriate branch in the [logstash repo](https://github.com/elastic/logstash/tree/main) (e.g. "main" ). + +2. Find the OpenAPI document in the `docs/static/spec/openapi` folder. For example: [https://github.com/elastic/logstash/blob/main/docs/static/spec/openapi/logstash-api.yaml](https://github.com/elastic/logstash/blob/main/docs/static/spec/openapi/logstash-api.yaml) + +3. Optionally, lint and preview the document. For example, install [redocly](https://redocly.com/docs/cli/installation) and [bump.sh](https://docs.bump.sh/help/continuous-integration/cli/) CLI, then run: + + 1. `redocly lint logstash-api.yaml --config redocly.yaml` + + 2. `bump preview logstash-api.yaml` + +4. Get assistance from someone with admin authority in Bump.sh (reach out on #next-api-reference Slack channel) to deploy the new output file as described in [https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/](https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/) and [Bump.sh API admin instructions](https://docs.google.com/document/d/1xP_n2AusBhJcLmmJeCZZkAztzeykvkZtJ6la2XHz7Aw/edit?tab=t.0) (Elastic employees only). + + - If the file that you upload is unchanged from the previous one, you will receive a message that there's nothing new to deploy. + + +### Observability Intake (serverless) + +To publish content to [https://www.elastic.co/docs/api/doc/observability-serverless](https://www.elastic.co/docs/api/doc/observability-serverless) + +1. Check out the main branch in the [apm-managed-service repo](https://github.com/elastic/apm-managed-service). + +2. Generate the output by using [Makefile](https://github.com/elastic/apm-managed-service/blob/main/Makefile) commands: + + 1. Go to the root folder. + + 2. Run `make api-docs` (bundles all the manually-maintained source files). + The output file is `docs/spec/openapi/bundled-apm-mis-openapi.yaml`. + + 3. Run `make api-docs-lint` + If there are errors, they should be addressed before deployment. Warnings and info messages can be ignored. + +3. Get assistance from someone with admin authority in Bump.sh (reach out on #next-api-reference Slack channel) to deploy the new output file as described in [https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/](https://docs.bump.sh/help/publish-documentation/deploy-and-release-management/) and [Bump.sh API admin instructions](https://docs.google.com/document/d/1xP_n2AusBhJcLmmJeCZZkAztzeykvkZtJ6la2XHz7Aw/edit?tab=t.0) (Elastic employees only). + + - If the file that you upload is unchanged from the previous one, you will receive a message that there's nothing new to deploy.