Skip to content

Commit 1927c72

Browse files
leemthompoleemthompo
authored
[DOCS] Update esql docs readme with 9.x+ version differentiation guidance (elastic#130340) (elastic#130437)
(cherry picked from commit 137eac5) # Conflicts: # docs/reference/query-languages/esql/README.md
1 parent ef9e66e commit 1927c72

File tree

1 file changed

+182
-0
lines changed

1 file changed

+182
-0
lines changed

‎docs/reference/query-languages/esql/README.md‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,185 @@ The `kibana` directory contains `definition` and `docs` sub-directories that are
4848
* `kibana/docs` - the inline docs for kibana
4949

5050
These are also generated as part of the unit tests described above.
51+
52+
## Under the hood
53+
54+
There are three overlapping mechanisms for generating the content:
55+
* The `AbstractFunctionTestCase` class generates the content for all the functions and most operators.
56+
This class makes use of the `DocsV3Support` class to generate the content.
57+
It uses the `@FunctionInfo` and `@Param` annotations on function and operator classes to know what content should be generated.
58+
All tests that extend this class will automatically generate the content for the functions they test.
59+
* Some operators do not have a clear class or test class, and so the content is generated by custom
60+
tests that do not extend the `AbstractOperatorTestCase` class. See, for example, operators such as `Cast ::`,
61+
which uses `CastOperatorTests` to call directly into the `DocsV3Support` class to generate the content.
62+
* Commands do not have dedicated classes or test classes with annotation that can be used.
63+
For this reason, the command documentation is generated by the `CommandDocsTests` class.
64+
Currently, this only covers tested examples used in the documentation, and all other commands
65+
content is static.
66+
Since there are no annotations to mark which examples to use, the command documentation
67+
relies on the docs author providing the knowledge of which examples to use by creating subdirectories
68+
and examples files that match the csv-spec files and tags to include.
69+
70+
To help differentiate between the static and generated content, the generated content is prefixed with a comment:
71+
```
72+
% This is generated by ESQL's AbstractFunctionTestCase. Do no edit it. See ../README.md for how to regenerate it.
73+
```
74+
75+
## Version differentiation in Docs V3
76+
77+
> [!IMPORTANT]
78+
> Starting with 9.0, we no longer publish separate documentation branches for every minor release (`9.0`, `9.1`, `9.2`, etc.).
79+
> This means there won't be a different page for `9.1`, `9.2`, and so on. Instead, all changes landing in subsequent minor releases **will appear on the same page**.
80+
81+
Because we now publish just one docs set off of the `main` branch, we use the [`applies_to` metadata](https://elastic.github.io/docs-builder/syntax/applies/) to differentiate features and their availability across different versions. This is a [cumulative approach](https://elastic.github.io/docs-builder/contribute/#cumulative-docs): instead of creating separate pages for each product and release, we update a **single page** with product- and version-specific details over time.
82+
83+
`applies_to` allows us to clearly communicate when features are introduced, when they transition from preview to GA, and which versions support specific functionality.
84+
85+
This metadata accepts a lifecycle and an optional version.
86+
87+
### Functions and operators
88+
89+
Use the `@FunctionAppliesTo` annotation within the `@FunctionInfo` annotation on function and operator classes to specify the lifecycle and version for functions and operators.
90+
91+
For example, to indicate that a function is in technical preview and applies to version 9.0.0, you would use:
92+
93+
```java
94+
@FunctionInfo(
95+
returnType = "boolean",
96+
appliesTo = {
97+
@FunctionAppliesTo(lifeCycle = FunctionAppliesToLifecycle.PREVIEW, version = "9.0.0")
98+
},
99+
...
100+
)
101+
```
102+
103+
When a feature evolves from preview in `9.0` to GA in `9.2`, add a new entry alongside the existing preview entry and remove the `preview = true` boolean:
104+
105+
```java
106+
@FunctionInfo(
107+
returnType = "boolean",
108+
preview = false, // the preview boolean can be removed (or flipped to false) when the function becomes GA
109+
appliesTo = {
110+
@FunctionAppliesTo(lifeCycle = FunctionAppliesToLifecycle.PREVIEW, version = "9.0.0"),
111+
@FunctionAppliesTo(lifeCycle = FunctionAppliesToLifecycle.GA, version = "9.2.0")
112+
},
113+
...
114+
)
115+
```
116+
117+
We updated [`DocsV3Support.java`](https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/expression/function/DocsV3Support.java) to generate the `applies_to` metadata correctly for functions and operators.
118+
119+
### Inline `applies_to` metadata
120+
121+
Use [inline annotations](https://elastic.github.io/docs-builder/syntax/applies/#inline-annotations) to specify `applies_to` metadata in descriptions, parameter lists, etc.
122+
123+
For example, the second item in this list is in technical preview as of version 9.2:
124+
125+
```markdown
126+
- Item 1
127+
- Item 2 {applies_to}`stack: preview 9.2.`
128+
```
129+
130+
### Key rules
131+
132+
1. **Use the `preview = true` boolean** for any tech preview feature - this is required for the Kibana inline docs
133+
- **Remove `preview = true`** only when the feature becomes GA on serverless and is _definitely_ going GA in the next minor release
134+
2. **Never delete `appliesTo` entries** - only add new ones as features evolve from preview to GA
135+
3. **Use specific versions** (`9.0.0`, `9.1.0`) when known, or just `PREVIEW` without a version if timing is uncertain
136+
4. **Add `applies_to` to examples** where necessary
137+
138+
> [!IMPORTANT]
139+
> We don't use `applies_to` in the legacy asciidoc system for 8.x and earlier versions.
140+
141+
### Supported lifecycles
142+
143+
- `PREVIEW` - Feature is in technical preview
144+
- `GA` - Feature is generally available
145+
- `DEPRECATED` - Feature is deprecated and will be removed in a future release
146+
- `UNAVAILABLE` - Feature is not available in the current version, but may be available in future releases
147+
148+
> [!NOTE]
149+
> Unreleased version information is automatically sanitized in the docs build output. For example, say you specify `preview 9.3.0`:
150+
> - Before `9.3.0` is released, the live documentation will display "Planned for a future release" instead of the specific version number.
151+
> - This will be updated automatically when the version is released.
152+
153+
## Tutorials
154+
155+
### Adding a new command
156+
157+
When adding a new command, for example adding the `CHANGE_POINT` command, do the following:
158+
1. Create a new file in the `_snippets/commands/layout` directory with the name of the command, for example `change_point.md`.
159+
2. Ensure to specify what versions the command applies to. See [Version differentiation in Docs V3](#version-differentiation-in-docs-v3) for details. [Example PR](https://github.com/elastic/elasticsearch/pull/130314/files#diff-0ab90b6202c5d9eeea75dc95a7cb71dc4d720230342718bff887816771a5a803R3-R6).
160+
3. Add the content for the command to the file. See other files in this directory for examples.
161+
4. Add the command to the list in `_snippets/lists/processing-commands.md`.
162+
5. Add an include directive to the `commands/processing-commands.md` file to include the new command.
163+
6. Add tested examples to the `_snippets/commands/examples` directory. See below for details.
164+
165+
### Adding examples to commands
166+
167+
When adding tested examples to a command, for example adding an example to the `CHANGE_POINT` command, do the following:
168+
* Make sure you have an example in an appropriate csv-spec file in the `x-pack/plugin/esql/qa/testFixtures/src/main/resources/` directory.
169+
* Make sure the example has a tag that is unique in that file, and matches the intent of the test, or the docs reason for including that test.
170+
* If you only want to show the query, and no results, then do not tag the results table,
171+
otherwise tag the results table with a tag that has the same name as the query tag, but with the suffix `-result`.
172+
* Create a file with the name of the tag in a subdirectory with the name of the csv-spec file
173+
in the `_snippets/commands/examples` directory. While you could add the content to that file, it is not necessary, merely that the file exists
174+
* Run the test `CommandDocsTests` in the `x-pack/plugin/esql` module to generate the content.
175+
176+
For example, we tag the following test in change_point.csv-spec:
177+
178+
```
179+
example for docs
180+
required_capability: change_point
181+
182+
// tag::changePointForDocs[]
183+
ROW key=[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]
184+
| MV_EXPAND key
185+
| EVAL value = CASE(key<13, 0, 42)
186+
| CHANGE_POINT value ON key
187+
| WHERE type IS NOT NULL
188+
// end::changePointForDocs[]
189+
;
190+
191+
// tag::changePointForDocs-result[]
192+
key:integer | value:integer | type:keyword | pvalue:double
193+
13 | 42 | step_change | 0.0
194+
// end::changePointForDocs-result[]
195+
;
196+
```
197+
198+
Then we create the file `_snippets/commands/examples/change_point.csv-spec/changePointForDocs.md` with the content:
199+
```
200+
This should be overwritten
201+
```
202+
203+
Then we run the test `CommandDocsTests` in the `x-pack/plugin/esql` module to generate the content.
204+
205+
Now the content of the changePointForDocs.md file should have been updated:
206+
207+
```
208+
% This is generated by ESQL's AbstractFunctionTestCase. Do no edit it. See ../README.md for how to regenerate it.
209+
210+
\```esql
211+
ROW key=[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]
212+
| MV_EXPAND key
213+
| EVAL value = CASE(key<13, 0, 42)
214+
| CHANGE_POINT value ON key
215+
| WHERE type IS NOT NULL
216+
\```
217+
218+
| key:integer | value:integer | type:keyword | pvalue:double |
219+
| --- | --- | --- | --- |
220+
| 13 | 42 | step_change | 0.0 |
221+
```
222+
223+
Finally include this file in the `CHANGE_POINT` command file `_snippets/commands/layout/change_point.md`:
224+
225+
```
226+
**Examples**
227+
228+
The following example shows the detection of a step change:
229+
230+
:::{include} ../examples/change_point.csv-spec/changePointForDocs.md
231+
:::
232+
```

0 commit comments

Comments
 (0)