Fix cluster.put_component_template API #5370

pquentin · 2025-09-30T06:25:47Z

I believe this was added in elastic/elasticsearch#117357, and the YAML tests added in elastic/elasticsearch#117357 helped surface the issue in the specification validation.

I'd be happy to have a better description.

github-actions · 2025-09-30T06:29:47Z

Following you can find the validation changes against the target branch for the APIs.

API	Status	Request	Response
`cluster.put_component_template`	🔴 → 🟢	34/38 → 38/38	38/38
`indices.put_index_template`	🔴	147/171 → 169/171	171/171

You can validate these APIs yourself by using the make validate target.

specification/indices/_types/IndexState.ts

This reverts commit 69c9eee.

…ate`

nielsbauman

I reverted your commit and pushed some other changes; I hope you don't mind :). My intention with my commit is to indicate what the spec should look like for these changes. I'll let it up to you if you want to make code style changes, extract parts, etc. I ran make contrib right before pushing, so I'm hoping the generated files should be ok like this. Let me know what you think!

nielsbauman · 2025-09-30T18:52:22Z

specification/cluster/put_component_template/ClusterPutComponentTemplateRequest.ts

 import { Metadata, Name, VersionNumber } from '@_types/common'
 import { Duration } from '@_types/Time'
-import { IndexState } from '@indices/_types/IndexState'
+import { IndexTemplateMapping } from '@indices/put_index_template/IndicesPutIndexTemplateRequest'


This import isn't particularly pretty. I'll leave it up to you whether this is fine or if we should extract the IndexTemplateMapping class to a dedicated file. If we're extracting it (which I'm in favor of), we should probably also rename it.

Yes, we should move to a shared place. Options include:

specification/indices/_types

specification/cluster/_types

specification/_types

Not sure which is best. Probably cluster since it's where Template lives in the Elasticsearch code. Unfortunately, moving a class is a breaking change, so that will be 9.3 only.

Yep, moving it to specification/cluster/_types works for me 👍. I'm having trouble coming up with a good name though... IndexTemplate is already used for retrieving templates. Is there some kind of naming convention for types that are used for PUT requests?

No naming convention, I'm not sure what to use. Is keeping IndexTemplateMapping that bad? It will reduce breakage for clients that don't care about the namespace of types.

I mean, I don't think the name itself really makes any sense, but if it avoids breakage, I'm fine with leaving it as is. Then let's just stick to moving the class. Could you take care of that perhaps? 🙏

Sure! So I did it locally, which made me realize that we already have a case of an endpoint doing this:

https://github.com/elastic/elasticsearch-specification/blob/6566f6962beceaa27c50ac31100ac22f62aea822/specification/indices/simulate_template/IndicesSimulateTemplateRequest.ts#L25C10-L25C30

So let's merge without doing this for now, and I'll ask to @l-trotta what's the best thing to do here, as the location of types mostly affects the Java client.

specification/cluster/put_component_template/ClusterPutComponentTemplateRequest.ts

specification/indices/_types/IndexTemplate.ts

specification/indices/put_index_template/IndicesPutIndexTemplateRequest.ts

…plate

pquentin

Thank you for this great work, and sorry for the huge delay.

There's one failing test left for cluster.put_component_template:

// Test file: /test/free/cluster.component_template/10_basic.yml
// Test name: Add data stream lifecycle with downsampling

import { expectAssignable } from 'tsd'
import * as T from '../types'

expectAssignable<T.ClusterPutComponentTemplateRequest>({
  "body": {
    "template": {
      "lifecycle": {
        "data_retention": "10d",
        "downsampling": [
          {
            "after": "1d",
            "fixed_interval": "5m"
          }
        ],
        "downsampling_method": "last_value"
      }
    }
  },
  "name": "test-lifecycle"
})

The issue is on line 12:

Property 'rounds' is missing in type '{ after: string; fixed_interval: string; }[]' but required in type 'IndicesDataStreamLifecycleDownsampling'.

Also, are we missing created_date and modified_date in the cluster.put_component_template body? Server code

specification/cluster/put_component_template/ClusterPutComponentTemplateRequest.ts

specification/indices/put_index_template/IndicesPutIndexTemplateRequest.ts

nielsbauman · 2025-12-04T11:03:34Z

@pquentin regarding the failing test related to downsampling, I think @gmarouli's #5593 should fix that.

nielsbauman · 2025-12-04T11:12:25Z

Also, are we missing created_date and modified_date in the cluster.put_component_template body?

Nope, we don't allow specifying those in the API: https://github.com/elastic/elasticsearch/blob/7cb125be9e971dee0fbd8b0c8d2d0d796b126d39/server/src/main/java/org/elasticsearch/action/admin/indices/template/put/PutComponentTemplateAction.java#L82-L93
Only ES itself can set those fields.

nielsbauman · 2025-12-04T11:17:20Z

I noticed another API that used IndexTemplate instead of IndexTemplateMapping incorrectly. I modifed it in this PR, but I'm also ok with extracting that change to another PR, but I'll let that up to you.

nielsbauman · 2025-12-04T11:20:10Z

specification/indices/put_index_template/IndicesPutIndexTemplateRequest.ts

   * @availability stack since=8.11.0 stability=stable
   * @availability serverless stability=stable
   */
  lifecycle?: DataStreamLifecycle


Actually, this lifecycle field in the template object can also explicitly be set to null, and the same holds for all the fields inside DataStreamLifecycle. Do you want to address that in this PR or a follow-up one?

Happy to do this as a follow-up, as long as this is before 9.3, to avoid breaking those APIs twice.

pquentin · 2025-12-04T11:57:53Z

specification/indices/simulate_index_template/IndicesSimulateIndexTemplateRequest.ts

  }
  /** @codegen_name index_template */
-  body?: IndexTemplate
+  body?: IndexTemplateMapping


This causes two new failures with composed_of that indices.simulate_index_template supports. Here's one:

// Test file: /test/free/indices.simulate_index_template/10_basic.yml // Test name: Simulate index template specifying a new template import { expectAssignable } from 'tsd' import * as T from '../types' expectAssignable<T.IndicesSimulateIndexTemplateRequest>({ "body": { "composed_of": [ "ct" ], "index_patterns": "test*", "priority": 15, "template": { "aliases": { "test_alias": {} }, "settings": { "index": { "blocks": { "write": true } } } } }, "name": "test" })

It looks like indices.simulate_index_template accepts a ComposableIndexTemplate object, while cluster.put_component_template accepts a ComponentTemplate.

Ahh yeah I was mistaken indeed. The current value of IndexTemplate is not really correct, because it contains things like created_date_millis which you're not allowed to specify in the request, but it's not a disaster. I'll remove that commit, thanks for flagging.

pquentin · 2025-12-04T11:58:32Z

specification/indices/put_index_template/IndicesPutIndexTemplateRequest.ts

   * @availability stack since=8.11.0 stability=stable
   * @availability serverless stability=stable
   */
  lifecycle?: DataStreamLifecycle


Happy to do this as a follow-up, as long as this is before 9.3, to avoid breaking those APIs twice.

pquentin · 2025-12-04T12:06:57Z

specification/cluster/put_component_template/ClusterPutComponentTemplateRequest.ts

 import { Metadata, Name, VersionNumber } from '@_types/common'
 import { Duration } from '@_types/Time'
-import { IndexState } from '@indices/_types/IndexState'
+import { IndexTemplateMapping } from '@indices/put_index_template/IndicesPutIndexTemplateRequest'


No naming convention, I'm not sure what to use. Is keeping IndexTemplateMapping that bad? It will reduce breakage for clients that don't care about the namespace of types.

…ateRequest`" This reverts commit 85a3def.

pquentin · 2025-12-05T11:04:29Z

Thanks for updating the branch @gmarouli! We have one failure left related to downsampling, with downsampling_method missing: elastic/elasticsearch#137023. This can be treated in a different PR.

nielsbauman · 2025-12-05T11:05:35Z

@pquentin, @gmarouli already opened #5594 for that :)

…plate

pquentin · 2025-12-05T14:02:09Z

Thanks! We merged this one, which indeed fixed the issue (and fixed three other APIs along the way!)

The only item left is #5370 (comment)

…plate

github-actions · 2025-12-08T13:42:40Z

The backport to 8.19 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.19 8.19
# Navigate to the new working tree
cd .worktrees/backport-8.19
# Create a new branch
git switch --create backport-5370-to-8.19
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1cd36d69ee3f8e4b113a93dcc43092859e4e5cee
# Push it to GitHub
git push --set-upstream origin backport-5370-to-8.19
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.19

Then, create a pull request where the base branch is 8.19 and the compare/head branch is backport-5370-to-8.19.

github-actions · 2025-12-08T13:42:43Z

The backport to 9.0 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-9.0 9.0
# Navigate to the new working tree
cd .worktrees/backport-9.0
# Create a new branch
git switch --create backport-5370-to-9.0
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1cd36d69ee3f8e4b113a93dcc43092859e4e5cee
# Push it to GitHub
git push --set-upstream origin backport-5370-to-9.0
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-9.0

Then, create a pull request where the base branch is 9.0 and the compare/head branch is backport-5370-to-9.0.

github-actions · 2025-12-08T13:42:44Z

The backport to 9.1 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-9.1 9.1
# Navigate to the new working tree
cd .worktrees/backport-9.1
# Create a new branch
git switch --create backport-5370-to-9.1
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1cd36d69ee3f8e4b113a93dcc43092859e4e5cee
# Push it to GitHub
git push --set-upstream origin backport-5370-to-9.1
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-9.1

Then, create a pull request where the base branch is 9.1 and the compare/head branch is backport-5370-to-9.1.

* Fix cluster.put_component_template API * Revert "Fix cluster.put_component_template API" This reverts commit 69c9eee. * Properly use `DataStreamOptions` and fix `cluster.put_component_template` * Also use `IndexTemplateMapping` in `IndicesSimulateIndexTemplateRequest` * Allow `null` for `data_stream_options` * Revert "Also use `IndexTemplateMapping` in `IndicesSimulateIndexTemplateRequest`" This reverts commit 85a3def. --------- Co-authored-by: Niels Bauman <[email protected]> Co-authored-by: Mary Gouseti <[email protected]> (cherry picked from commit 1cd36d6) # Conflicts: # output/openapi/elasticsearch-openapi.json # output/openapi/elasticsearch-serverless-openapi.json # output/schema/schema.json

pquentin · 2025-12-08T13:52:23Z

💔 Some backports could not be created

Status	Branch	Result
✅	9.2
✅	9.1
❌	8.19	Conflict resolution was aborted by the user

Manual backport

To create the backport manually run:

backport --pr 5370

Questions ?

Please refer to the Backport tool documentation

pquentin · 2025-12-09T06:22:06Z

8.19 backport currently blocked on #5807

Fix cluster.put_component_template API

69c9eee

pquentin added backport 8.19 backport 9.0 backport 9.1 labels Sep 30, 2025

github-actions bot added the specification label Sep 30, 2025

pquentin requested a review from a team September 30, 2025 06:37

nielsbauman reviewed Sep 30, 2025

View reviewed changes

specification/indices/_types/IndexState.ts Outdated Show resolved Hide resolved

nielsbauman added 2 commits September 30, 2025 15:27

Revert "Fix cluster.put_component_template API"

1de905b

This reverts commit 69c9eee.

Properly use DataStreamOptions and fix `cluster.put_component_templ…

30176b8

…ate`

nielsbauman reviewed Sep 30, 2025

View reviewed changes

Merge remote-tracking branch 'origin/main' into fix-put-component-tem…

42a64bc

…plate

pquentin commented Dec 3, 2025

View reviewed changes

specification/cluster/put_component_template/ClusterPutComponentTemplateRequest.ts Show resolved Hide resolved

specification/indices/put_index_template/IndicesPutIndexTemplateRequest.ts Outdated Show resolved Hide resolved

nielsbauman added 2 commits December 4, 2025 12:15

Also use IndexTemplateMapping in IndicesSimulateIndexTemplateRequest

85a3def

Allow null for data_stream_options

ce2a8a1

nielsbauman reviewed Dec 4, 2025

View reviewed changes

pquentin commented Dec 4, 2025

View reviewed changes

nielsbauman and others added 2 commits December 4, 2025 13:33

Revert "Also use IndexTemplateMapping in `IndicesSimulateIndexTempl…

62d1739

…ateRequest`" This reverts commit 85a3def.

Merge branch 'main' into fix-put-component-template

0a98b37

Merge remote-tracking branch 'origin/main' into fix-put-component-tem…

9ea3abf

…plate

Merge remote-tracking branch 'origin/main' into fix-put-component-tem…

0dcd6ee

…plate

nielsbauman approved these changes Dec 8, 2025

View reviewed changes

pquentin merged commit 1cd36d6 into main Dec 8, 2025
10 checks passed

pquentin deleted the fix-put-component-template branch December 8, 2025 13:40

pquentin mentioned this pull request Dec 8, 2025

[9.2] Fix cluster.put_component_template API (#5370) #5803

Open

pquentin mentioned this pull request Dec 8, 2025

[9.1] Fix cluster.put_component_template API (#5370) #5804

Open

Fix cluster.put_component_template API #5370

Fix cluster.put_component_template API #5370

Uh oh!

Conversation

pquentin commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

nielsbauman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pquentin Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pquentin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nielsbauman commented Dec 4, 2025

Uh oh!

nielsbauman commented Dec 4, 2025

Uh oh!

nielsbauman commented Dec 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pquentin Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pquentin commented Dec 5, 2025

Uh oh!

nielsbauman commented Dec 5, 2025

Uh oh!

pquentin commented Dec 5, 2025

Uh oh!

Uh oh!

github-actions bot commented Dec 8, 2025

Uh oh!

github-actions bot commented Dec 8, 2025

Uh oh!

github-actions bot commented Dec 8, 2025

Uh oh!

pquentin commented Dec 8, 2025

💔 Some backports could not be created

Manual backport

Questions ?

Uh oh!

pquentin commented Dec 9, 2025

Uh oh!

Reviewers

pquentin commented Sep 30, 2025 •

edited

Loading

github-actions bot commented Sep 30, 2025 •

edited

Loading

pquentin Dec 3, 2025 •

edited

Loading

pquentin left a comment •

edited

Loading

pquentin Dec 4, 2025 •

edited

Loading