#104233 Allow Watcher Node Allocation Settings #115251
Conversation
Now accepts index.routing.allocation.* settings but denies changing the allocation setting that keeps watches on data nodes
Now returns index.routing.allocation.* settings explicitly filters out the `index.routing.allocation.include._tier_preference` setting
lukewhiting
added
>bug
>docs
|
Documentation preview: |
elasticsearchmachine
added
Team:Data Management
|
Pinging @elastic/es-docs (Team:Docs) |
|
Pinging @elastic/es-data-management (Team:Data Management) |
| IndexMetadata.INDEX_ROUTING_REQUIRE_GROUP_PREFIX | ||
| ); | ||
|
|
||
| public static final Set<String> EXPLICITLY_DENIED_SETTINGS = Set.of( |
There was a problem hiding this comment.
I assume you're denying this one because we always want the .watches shards in the hot tier? That makes me wonder what other possible values there are for these afix settings, and if we ought to just explicitly allow the index.routing.allocation.exclude.role, index.routing.allocation.include.role, and index.routing.allocation.require.role ones and disallow anything else to avoid causing some unforeseen problems. You've probably looked at all these settings more than I have -- what do you think?
There was a problem hiding this comment.
So I based this restriction on what the default setting for the .watcher index are. I assumed we put it in the hot tier for good reason so locked off the ability to change that. I may be wrong in that assumption.
Locking down the index.routing.allocation.*.* settings further is difficult / undesirable. index.routing.allocation.include.role isn't actually a setting as such but a convention for a wildcard setting. Anything after index.routing.allocation.include. is assumed to be an arbitrary node attribute hence my leaving it as allowing any setting with that prefix as we have no way of knowing what attributes the end user might have set on their nodes and how they would like to use those for scheduling watches.
For instance index.routing.allocation.include.sandwich_filling=turkey is a valid setting for this if the user chose to use a bread based node attributes scheme to group their cluster members :-)
* Update settings endpoint modified Now accepts index.routing.allocation.* settings but denies changing the allocation setting that keeps watches on data nodes * Get settings endpoint modified Now returns index.routing.allocation.* settings explicitly filters out the `index.routing.allocation.include._tier_preference` setting * Tests for modified endpoints * Update docs
💚 Backport successful
|
* Update settings endpoint modified Now accepts index.routing.allocation.* settings but denies changing the allocation setting that keeps watches on data nodes * Get settings endpoint modified Now returns index.routing.allocation.* settings explicitly filters out the `index.routing.allocation.include._tier_preference` setting * Tests for modified endpoints * Update docs
* Update settings endpoint modified Now accepts index.routing.allocation.* settings but denies changing the allocation setting that keeps watches on data nodes * Get settings endpoint modified Now returns index.routing.allocation.* settings explicitly filters out the `index.routing.allocation.include._tier_preference` setting * Tests for modified endpoints * Update docs
* Update settings endpoint modified Now accepts index.routing.allocation.* settings but denies changing the allocation setting that keeps watches on data nodes * Get settings endpoint modified Now returns index.routing.allocation.* settings explicitly filters out the `index.routing.allocation.include._tier_preference` setting * Tests for modified endpoints * Update docs
|
Should we also update the documentation https://www.elastic.co/guide/en/elasticsearch/reference/8.x/watcher-api-update-settings.html ? |
|
@herrBez The documentation has now been updated and will go live with the next release :-) |
4 participants
Fixes #104233
This PR allows the
PUT _watcher/settingsendpoint to accept shard allocation settings while also adding restrictions to prevent users modifying or removing theindex.routing.allocation.include._tier_preferencesetting on the.watchesindex. It also modifies theGET _watcher/settingsto show the newly allowed settings.By controlling shard allocation, You also control which nodes in the cluster actually execute watches. The relevant documentation is also updated to show this new way of adjusting which nodes Watcher runs on.
I chose this approach over removing the reference in the documentation as that felt like a breaking change / removal of previous functionality.