Skip to content

Commit 60573be

Browse files
not-napoleonnot-napoleon
authored
Merge branch 'main' into t-digest-sum-of-empty-change
2 parents 2db174b + 6462cc9 commit 60573be

File tree

172 files changed

+2911
-5107
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

172 files changed

+2911
-5107
lines changed

‎benchmarks/src/main/java/org/elasticsearch/benchmark/_nightly/esql/QueryPlanningBenchmark.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ public void setup() {
102102
mapping.put("field" + i, new EsField("field-" + i, TEXT, emptyMap(), true, EsField.TimeSeriesFieldType.NONE));
103103
}
104104

105-
var esIndex = new EsIndex("test", mapping, Map.of("test", IndexMode.STANDARD), Set.of());
105+
var esIndex = new EsIndex("test", mapping, Map.of("test", IndexMode.STANDARD), Map.of(), Map.of(), Set.of());
106106

107107
var functionRegistry = new EsqlFunctionRegistry();
108108

‎docs/changelog/108875.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
pr: 108875
2+
summary: Break on `FieldData` when building global ordinals
3+
area: Aggregations
4+
type: bug
5+
issues:
6+
- 97075

‎docs/changelog/137558.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 137558
2+
summary: Improve security migration resilience by handling version conflicts
3+
area: Security
4+
type: enhancement
5+
issues: []

‎docs/changelog/138084.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 138084
2+
summary: Handle Query Timeouts During Collector Initialization in `QueryPhase`
3+
area: Search
4+
type: bug
5+
issues: []

‎docs/changelog/138094.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
pr: 138094
2+
summary: "[IRONSCALES] Add `manage`, `create_index`, `read`, `index`, `write`, `delete`, permission for third party agent indices `kibana_system`"
3+
area: Authorization
4+
type: enhancement
5+
issues:
6+
- 138093

‎docs/changelog/138123.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 138123
2+
summary: ES|QL Update CHUNK to support `chunking_settings` as optional argument
3+
area: ES|QL
4+
type: enhancement
5+
issues: []

‎docs/changelog/138380.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
pr: 138380
2+
summary: Do not use Min or Max as Top's surrogate when there is an `outputField`
3+
area: ES|QL
4+
type: bug
5+
issues:
6+
- 134083

‎docs/changelog/138464.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 138464
2+
summary: Always prefer YES over NOT_PREFERRED when allocating unassigned shards
3+
area: Allocation
4+
type: bug
5+
issues: []

‎docs/changelog/138484.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
pr: 138484
2+
summary: "[Inference API] Add `ElasticInferenceServiceDenseTextEmbeddingsServiceSettings`\
3+
\ to `InferenceNamedWriteablesProvider`"
4+
area: Machine Learning
5+
type: bug
6+
issues: []

‎docs/reference/elasticsearch/rest-apis/explain-lifecycle.md‎

Lines changed: 279 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,279 @@
1+
---
2+
mapped_pages:
3+
- https://www.elastic.co/guide/en/elasticsearch/reference/8.19/ilm-explain-lifecycle.html
4+
applies_to:
5+
stack: all
6+
navigation_title: Understand the lifecycle status
7+
---
8+
9+
# Use the explain lifecycle API to understand the index lifecycle status [explain-lifecycle-api]
10+
11+
The [explain lifecycle API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-explain-lifecycle) retrieves the current lifecycle status for one or more indices. For data streams, the API retrieves the lifecycle status for the stream’s backing indices, including the current phase, action, step, and any failures.
12+
13+
## Get the lifecycle status of an index
14+
15+
The following example retrieves the lifecycle state of `my-index-000001`:
16+
17+
<!--
18+
19+
```console
20+
PUT _ilm/policy/my_policy
21+
{
22+
"policy": {
23+
"phases": {
24+
"warm": {
25+
"min_age": "10d",
26+
"actions": {
27+
"forcemerge": {
28+
"max_num_segments": 1
29+
}
30+
}
31+
},
32+
"delete": {
33+
"min_age": "30d",
34+
"actions": {
35+
"delete": {}
36+
}
37+
}
38+
}
39+
}
40+
}
41+
42+
PUT my-index-000001
43+
{
44+
"settings": {
45+
"index.lifecycle.name": "my_policy",
46+
"index.number_of_replicas": 0
47+
}
48+
}
49+
50+
GET /_cluster/health?wait_for_status=green&timeout=10s
51+
```
52+
% TEST
53+
54+
-->
55+
56+
```console
57+
GET my-index-000001/_ilm/explain?human
58+
```
59+
60+
When management of the index is first taken over by {{ilm-init}}, `explain` shows that the index is managed and in the `new` phase:
61+
62+
```console-result
63+
{
64+
"indices": {
65+
"my-index-000001": {
66+
"index": "my-index-000001",
67+
"index_creation_date_millis": 1538475653281, <1>
68+
"index_creation_date": "2018-10-15T13:45:21.981Z",
69+
"time_since_index_creation": "15s", <2>
70+
"managed": true, <3>
71+
"policy": "my_policy", <4>
72+
"lifecycle_date_millis": 1538475653281, <5>
73+
"lifecycle_date": "2018-10-15T13:45:21.981Z",
74+
"age": "15s", <6>
75+
"phase": "new",
76+
"phase_time_millis": 1538475653317, <7>
77+
"phase_time": "2018-10-15T13:45:22.577Z",
78+
"action": "complete"
79+
"action_time_millis": 1538475653317, <8>
80+
"action_time": "2018-10-15T13:45:22.577Z",
81+
"step": "complete",
82+
"step_time_millis": 1538475653317, <9>
83+
"step_time": "2018-10-15T13:45:22.577Z"
84+
}
85+
}
86+
}
87+
```
88+
% TESTRESPONSE[skip:no way to know if we will get this response immediately]
89+
90+
1. When the index was created. This timestamp is used to determine when to roll over the index.
91+
2. The time since the index creation (used for calculating when to rollover the index via the `max_age`).
92+
3. Shows if the index is being managed by {{ilm-init}}. If the index is not managed by {{ilm-init}} the other fields will not be shown.
93+
4. The name of the policy which {{ilm-init}} is using for this index.
94+
5. The timestamp used for the `min_age`.
95+
6. The age of the index (used for calculating when to enter the next phase).
96+
7. When the index entered the current phase.
97+
8. When the index entered the current action.
98+
9. When the index entered the current step.
99+
100+
## View the phase definition applied to an index
101+
102+
Once the policy is running on the index, the response includes a `phase_execution` object that shows the definition of the current phase. Changes to the underlying policy will not affect this index until the current phase completes.
103+
104+
```console-result
105+
{
106+
"indices": {
107+
"test-000069": {
108+
"index": "test-000069",
109+
"index_creation_date_millis": 1538475653281,
110+
"time_since_index_creation": "25.14s",
111+
"managed": true,
112+
"policy": "my_lifecycle3",
113+
"lifecycle_date_millis": 1538475653281,
114+
"lifecycle_date": "2018-10-15T13:45:21.981Z",
115+
"age": "25.14s",
116+
"phase": "hot",
117+
"phase_time_millis": 1538475653317,
118+
"phase_time": "2018-10-15T13:45:22.577Z",
119+
"action": "rollover",
120+
"action_time_millis": 1538475653317,
121+
"action_time": "2018-10-15T13:45:22.577Z",
122+
"step": "attempt-rollover",
123+
"step_time_millis": 1538475653317,
124+
"step_time": "2018-10-15T13:45:22.577Z",
125+
"phase_execution": {
126+
"policy": "my_lifecycle3",
127+
"phase_definition": { <1>
128+
"min_age": "0ms",
129+
"actions": {
130+
"rollover": {
131+
"max_age": "30s",
132+
"max_primary_shard_docs": 200000000, <2>
133+
"min_docs": 1
134+
}
135+
}
136+
},
137+
"version": 3, <3>
138+
"modified_date": "2018-10-15T13:21:41.576Z", <4>
139+
"modified_date_in_millis": 1539609701576 <5>
140+
}
141+
}
142+
}
143+
}
144+
```
145+
% TESTRESPONSE[skip:not possible to get the cluster into this state in a docs test]
146+
147+
1. The JSON phase definition loaded from the specified policy when the index entered this phase
148+
2. The rollover action includes the default `max_primary_shard_docs` and `min_docs` conditions. See [ILM rollover options](../index-lifecycle-actions/ilm-rollover.md#ilm-rollover-options) for more information.
149+
3. The version of the policy that was loaded
150+
4. The date the loaded policy was last modified
151+
5. The epoch time when the loaded policy was last modified
152+
153+
## Check the status of a running step
154+
155+
If {{ilm-init}} is waiting for a step to complete, the response includes status information for the step that's being performed on the index.
156+
157+
```console-result
158+
{
159+
"indices": {
160+
"test-000020": {
161+
"index": "test-000020",
162+
"index_creation_date_millis": 1538475653281,
163+
"time_since_index_creation": "4.12m",
164+
"managed": true,
165+
"policy": "my_lifecycle3",
166+
"lifecycle_date_millis": 1538475653281,
167+
"lifecycle_date": "2018-10-15T13:45:21.981Z",
168+
"age": "4.12m",
169+
"phase": "warm",
170+
"phase_time_millis": 1538475653317,
171+
"phase_time": "2018-10-15T13:45:22.577Z",
172+
"action": "allocate",
173+
"action_time_millis": 1538475653317,
174+
"action_time": "2018-10-15T13:45:22.577Z",
175+
"step": "check-allocation",
176+
"step_time_millis": 1538475653317,
177+
"step_time": "2018-10-15T13:45:22.577Z",
178+
"step_info": { <1>
179+
"message": "Waiting for all shard copies to be active",
180+
"shards_left_to_allocate": -1,
181+
"all_shards_active": false,
182+
"number_of_replicas": 2
183+
},
184+
"phase_execution": {
185+
"policy": "my_lifecycle3",
186+
"phase_definition": {
187+
"min_age": "0ms",
188+
"actions": {
189+
"allocate": {
190+
"number_of_replicas": 2,
191+
"include": {
192+
"box_type": "warm"
193+
},
194+
"exclude": {},
195+
"require": {}
196+
},
197+
"forcemerge": {
198+
"max_num_segments": 1
199+
}
200+
}
201+
},
202+
"version": 2,
203+
"modified_date": "2018-10-15T13:20:02.489Z",
204+
"modified_date_in_millis": 1539609602489
205+
}
206+
}
207+
}
208+
}
209+
```
210+
% TESTRESPONSE[skip:not possible to get the cluster into this state in a docs test]
211+
212+
1. The status of the step that's in progress.
213+
214+
## Diagnose lifecycle errors
215+
216+
If the index is in the ERROR step, something went wrong while executing a step in the policy and you will need to take action for the index to proceed
217+
to the next step. Some steps are safe to automatically be retried in certain circumstances. To help you diagnose the problem, the explain response shows the step that failed, the step info which provides information about the error, and information about the retry attempts executed for the failed step if it's the case.
218+
219+
```console-result
220+
{
221+
"indices": {
222+
"test-000056": {
223+
"index": "test-000056",
224+
"index_creation_date_millis": 1538475653281,
225+
"time_since_index_creation": "50.1d",
226+
"managed": true,
227+
"policy": "my_lifecycle3",
228+
"lifecycle_date_millis": 1538475653281,
229+
"lifecycle_date": "2018-10-15T13:45:21.981Z",
230+
"age": "50.1d",
231+
"phase": "hot",
232+
"phase_time_millis": 1538475653317,
233+
"phase_time": "2018-10-15T13:45:22.577Z",
234+
"action": "rollover",
235+
"action_time_millis": 1538475653317,
236+
"action_time": "2018-10-15T13:45:22.577Z",
237+
"step": "ERROR",
238+
"step_time_millis": 1538475653317,
239+
"step_time": "2018-10-15T13:45:22.577Z",
240+
"failed_step": "check-rollover-ready", <1>
241+
"is_auto_retryable_error": true, <2>
242+
"failed_step_retry_count": 1, <3>
243+
"step_info": { <4>
244+
"type": "cluster_block_exception",
245+
"reason": "index [test-000057/H7lF9n36Rzqa-KfKcnGQMg] blocked by: [FORBIDDEN/5/index read-only (api)",
246+
"index_uuid": "H7lF9n36Rzqa-KfKcnGQMg",
247+
"index": "test-000057"
248+
},
249+
"previous_step_info": { <5>
250+
"type": "cluster_block_exception",
251+
"reason": "index [test-000057/H7lF9n36Rzqa-KfKcnGQMg] blocked by: [FORBIDDEN/5/index read-only (api)",
252+
"index_uuid": "H7lF9n36Rzqa-KfKcnGQMg",
253+
"index": "test-000057"
254+
},
255+
"phase_execution": {
256+
"policy": "my_lifecycle3",
257+
"phase_definition": {
258+
"min_age": "0ms",
259+
"actions": {
260+
"rollover": {
261+
"max_age": "30s"
262+
}
263+
}
264+
},
265+
"version": 3,
266+
"modified_date": "2018-10-15T13:21:41.576Z",
267+
"modified_date_in_millis": 1539609701576
268+
}
269+
}
270+
}
271+
}
272+
```
273+
% TESTRESPONSE[skip:not possible to get the cluster into this state in a docs test]
274+
275+
1. The step that caused the error.
276+
2. Indicates if retrying the failed step can overcome the error. If this is true, {{ilm-init}} will retry the failed step automatically.
277+
3. Shows the number of attempted automatic retries to execute the failed step.
278+
4. What went wrong.
279+
5. Contains a copy of the `step_info` field (when it exists) of the last attempted or executed step for diagnostic purposes, since the `step_info` is overwritten during each new attempt.

0 commit comments

Comments
 (0)