Skip to content

Commit cc13241

Browse files
Merge remote-tracking branch 'origin/main' into feature/mistral-chat-completion-integration
2 parents 34ca847 + d597e50 commit cc13241

File tree

36 files changed

+955
-805
lines changed

36 files changed

+955
-805
lines changed

.buildkite/pipelines/periodic-micro-benchmarks.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
steps:
22
- label: periodic-micro-benchmarks
33
command: |
4-
.ci/scripts/run-gradle.sh -p benchmarks/ run --args 'org.elasticsearch.benchmark._nightly -rf json -rff build/result.json'
4+
.ci/scripts/run-gradle.sh :benchmarks:run --args 'org.elasticsearch.benchmark._nightly -rf json -rff build/result.json'
55
timeout_in_minutes: 300
66
agents:
77
provider: gcp

.ci/scripts/run-gradle.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,4 +41,4 @@ else
4141
fi
4242

4343
set -e
44-
$GRADLEW -S --max-workers=$MAX_WORKERS $@
44+
$GRADLEW -S --max-workers=$MAX_WORKERS "$@"

docs/changelog/128362.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 128362
2+
summary: Avoid unnecessary determinization in index pattern conflict checks
3+
area: Indices APIs
4+
type: bug
5+
issues: []

docs/changelog/128650.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
pr: 128650
2+
summary: Update shardGenerations for all indices on snapshot finalization
3+
area: Snapshot/Restore
4+
type: enhancement
5+
issues:
6+
- 108907

docs/reference/enrich-processor/dissect-processor.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ and result in a document with the following fields:
4646

4747
A dissect pattern is defined by the parts of the string that will be discarded. In the previous example, the first part to be discarded is a single space. Dissect finds this space, then assigns the value of `clientip` everything up until that space. Next, dissect matches the `[` and then `]` and then assigns `@timestamp` to everything in-between `[` and `]`. Paying special attention to the parts of the string to discard will help build successful dissect patterns.
4848

49-
Successful matches require all keys in a pattern to have a value. If any of the `%{{keyname}}` defined in the pattern do not have a value, then an exception is thrown and may be handled by the [`on_failure`](docs-content://manage-data/ingest/transform-enrich/ingest-pipelines.md#handling-pipeline-failures) directive. An empty key `%{}` or a [named skip key](#dissect-modifier-named-skip-key) can be used to match values, but exclude the value from the final document. All matched values are represented as string data types. The [convert processor](/reference/enrich-processor/convert-processor.md) may be used to convert to expected data type.
49+
Successful matches require all keys in a pattern to have a value. If any of the `%{keyname}` defined in the pattern do not have a value, then an exception is thrown and may be handled by the [`on_failure`](docs-content://manage-data/ingest/transform-enrich/ingest-pipelines.md#handling-pipeline-failures) directive. An empty key `%{}` or a [named skip key](#dissect-modifier-named-skip-key) can be used to match values, but exclude the value from the final document. All matched values are represented as string data types. The [convert processor](/reference/enrich-processor/convert-processor.md) may be used to convert to expected data type.
5050

5151
Dissect also supports [key modifiers](#dissect-key-modifiers) that can change dissect’s default behavior. For example you can instruct dissect to ignore certain fields, append fields, skip over padding, etc. See [below](#dissect-key-modifiers) for more information.
5252

@@ -75,7 +75,7 @@ $$$dissect-options$$$
7575

7676
## Dissect key modifiers [dissect-key-modifiers]
7777

78-
Key modifiers can change the default behavior for dissection. Key modifiers may be found on the left or right of the `%{{keyname}}` always inside the `%{` and `}`. For example `%{+keyname ->}` has the append and right padding modifiers.
78+
Key modifiers can change the default behavior for dissection. Key modifiers may be found on the left or right of the `%{keyname}` always inside the `%{` and `}`. For example `%{+keyname ->}` has the append and right padding modifiers.
7979

8080
$$$dissect-key-modifiers-table$$$
8181

@@ -89,9 +89,9 @@ $$$dissect-key-modifiers-table$$$
8989

9090
### Right padding modifier (`->`) [dissect-modifier-skip-right-padding]
9191

92-
The algorithm that performs the dissection is very strict in that it requires all characters in the pattern to match the source string. For example, the pattern `%{{fookey}} %{{barkey}}` (1 space), will match the string "foo bar" (1 space), but will not match the string "foo bar" (2 spaces) since the pattern has only 1 space and the source string has 2 spaces.
92+
The algorithm that performs the dissection is very strict in that it requires all characters in the pattern to match the source string. For example, the pattern `%{fookey} %{barkey}` (1 space), will match the string "foo bar" (1 space), but will not match the string "foo bar" (2 spaces) since the pattern has only 1 space and the source string has 2 spaces.
9393

94-
The right padding modifier helps with this case. Adding the right padding modifier to the pattern `%{fookey->} %{{barkey}}`, It will now will match "foo bar" (1 space) and "foo bar" (2 spaces) and even "foo bar" (10 spaces).
94+
The right padding modifier helps with this case. Adding the right padding modifier to the pattern `%{fookey->} %{barkey}`, It will now will match "foo bar" (1 space) and "foo bar" (2 spaces) and even "foo bar" (10 spaces).
9595

9696
Use the right padding modifier to allow for repetition of the characters after a `%{keyname->}`.
9797

@@ -101,7 +101,7 @@ Right padding modifier example
101101

102102
| | |
103103
| --- | --- |
104-
| **Pattern** | `%{ts->} %{{level}}` |
104+
| **Pattern** | `%{ts->} %{level}` |
105105
| **Input** | 1998-08-10T17:15:42,466 WARN |
106106
| **Result** | * ts = 1998-08-10T17:15:42,466<br>* level = WARN<br> |
107107

@@ -111,7 +111,7 @@ Right padding modifier with empty key example
111111

112112
| | |
113113
| --- | --- |
114-
| **Pattern** | `[%{{ts}}]%{->}[%{{level}}]` |
114+
| **Pattern** | `[%{ts}]%{->}[%{level}]` |
115115
| **Input** | [1998-08-10T17:15:42,466] [WARN] |
116116
| **Result** | * ts = 1998-08-10T17:15:42,466<br>* level = WARN<br> |
117117

@@ -153,7 +153,7 @@ Named skip key modifier example
153153

154154
| | |
155155
| --- | --- |
156-
| **Pattern** | `%{{clientip}} %{?ident} %{?auth} [%{@timestamp}]` |
156+
| **Pattern** | `%{clientip} %{?ident} %{?auth} [%{@timestamp}]` |
157157
| **Input** | 1.2.3.4 - - [30/Apr/1998:22:00:52 +0000] |
158158
| **Result** | * clientip = 1.2.3.4<br>* @timestamp = 30/Apr/1998:22:00:52 +0000<br> |
159159

@@ -167,7 +167,7 @@ Reference key modifier example
167167

168168
| | |
169169
| --- | --- |
170-
| **Pattern** | `[%{{ts}}] [%{{level}}] %{*p1}:%{&p1} %{*p2}:%{&p2}` |
170+
| **Pattern** | `[%{ts}] [%{level}] %{*p1}:%{&p1} %{*p2}:%{&p2}` |
171171
| **Input** | [2018-08-10T17:15:42,466] [ERR] ip:1.2.3.4 error:REFUSED |
172172
| **Result** | * ts = 2018-08-10T17:15:42,466<br>* level = ERR<br>* ip = 1.2.3.4<br>* error = REFUSED<br> |
173173

docs/reference/query-languages/query-dsl/query-dsl-match-query-phrase.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,10 @@ GET /_search
2929
`analyzer`
3030
: (Optional, string) [Analyzer](docs-content://manage-data/data-store/text-analysis.md) used to convert the text in the `query` value into tokens. Defaults to the [index-time analyzer](docs-content://manage-data/data-store/text-analysis/specify-an-analyzer.md#specify-index-time-analyzer) mapped for the `<field>`. If no analyzer is mapped, the index’s default analyzer is used.
3131

32+
`boost`
33+
: (Optional, float) Floating point number used to decrease or increase the [relevance scores](/reference/query-languages/query-dsl/query-filter-context.md#relevance-scores) of the query. Defaults to `1.0`.
34+
Boost values are relative to the default value of `1.0`. A boost value between `0` and `1.0` decreases the relevance score. A value greater than `1.0` increases the relevance score.
35+
3236
`slop`
3337
: (Optional, integer) Maximum number of positions allowed between matching tokens. Defaults to `0`. Transposed terms have a slop of `2`.
3438

docs/release-notes/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,10 @@ To check for security updates, go to [Security announcements for the Elastic sta
2020
% ### Fixes [elasticsearch-next-fixes]
2121
% *
2222

23+
## 9.0.2 [elasticsearch-9.0.2-release-notes]
2324
```{applies_to}
2425
stack: coming 9.0.2
2526
```
26-
## 9.0.2 [elasticsearch-9.0.2-release-notes]
2727

2828
### Features and enhancements [elasticsearch-9.0.2-features-enhancements]
2929

modules/lang-painless/src/main/java/org/elasticsearch/painless/api/Augmentation.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -456,10 +456,10 @@ public static <K, V, T> Map<T, Map<K, V>> groupBy(Map<K, V> receiver, BiFunction
456456
public static String replaceAll(CharSequence receiver, Pattern pattern, Function<Matcher, String> replacementBuilder) {
457457
Matcher m = pattern.matcher(receiver);
458458
if (false == m.find()) {
459-
// CharSequqence's toString is *supposed* to always return the characters in the sequence as a String
459+
// CharSequence's toString is *supposed* to always return the characters in the sequence as a String
460460
return receiver.toString();
461461
}
462-
StringBuffer result = new StringBuffer(initialBufferForReplaceWith(receiver));
462+
StringBuilder result = new StringBuilder(initialBufferForReplaceWith(receiver));
463463
do {
464464
m.appendReplacement(result, Matcher.quoteReplacement(replacementBuilder.apply(m)));
465465
} while (m.find());
@@ -477,7 +477,7 @@ public static String replaceFirst(CharSequence receiver, Pattern pattern, Functi
477477
// CharSequqence's toString is *supposed* to always return the characters in the sequence as a String
478478
return receiver.toString();
479479
}
480-
StringBuffer result = new StringBuffer(initialBufferForReplaceWith(receiver));
480+
StringBuilder result = new StringBuilder(initialBufferForReplaceWith(receiver));
481481
m.appendReplacement(result, Matcher.quoteReplacement(replacementBuilder.apply(m)));
482482
m.appendTail(result);
483483
return result.toString();

qa/rolling-upgrade/build.gradle

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
* License v3.0 only", or the "Server Side Public License, v 1".
88
*/
99

10+
import org.elasticsearch.gradle.Version
1011
import org.elasticsearch.gradle.testclusters.StandaloneRestIntegTestTask
1112

1213
apply plugin: 'elasticsearch.internal-java-rest-test'
@@ -25,6 +26,18 @@ buildParams.bwcVersions.withWireCompatible { bwcVersion, baseName ->
2526
}
2627
}
2728

29+
tasks.register("bcUpgradeTest", StandaloneRestIntegTestTask) {
30+
// We use a phony version here as the real version is provided via `tests.bwc.main.version` system property
31+
usesBwcDistribution(Version.fromString("0.0.0"))
32+
systemProperty("tests.old_cluster_version", "0.0.0")
33+
onlyIf("tests.bwc.main.version system property exists") { System.getProperty("tests.bwc.main.version") != null }
34+
filter {
35+
// filter tests initially for quicker iterations
36+
// TODO remove once expanding the test set to other modules
37+
includeTestsMatching("org.elasticsearch.upgrades.IndexingIT")
38+
}
39+
}
40+
2841
tasks.withType(Test).configureEach {
2942
// CI doesn't like it when there's multiple clusters running at once
3043
maxParallelForks = 1

server/src/main/java/org/elasticsearch/cluster/metadata/MetadataIndexTemplateService.java

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -946,12 +946,15 @@ public static Map<String, List<String>> findConflictingV1Templates(
946946
final String candidateName,
947947
final List<String> indexPatterns
948948
) {
949-
Automaton v2automaton = Regex.simpleMatchToAutomaton(indexPatterns.toArray(Strings.EMPTY_ARRAY));
949+
// No need to determinize the automaton, as it is only used to check for intersection with another automaton.
950+
// Determinization is avoided because it can fail or become very costly due to state explosion.
951+
Automaton v2automaton = Regex.simpleMatchToNonDeterminizedAutomaton(indexPatterns.toArray(Strings.EMPTY_ARRAY));
950952
Map<String, List<String>> overlappingTemplates = new HashMap<>();
951953
for (Map.Entry<String, IndexTemplateMetadata> cursor : project.templates().entrySet()) {
952954
String name = cursor.getKey();
953955
IndexTemplateMetadata template = cursor.getValue();
954-
Automaton v1automaton = Regex.simpleMatchToAutomaton(template.patterns().toArray(Strings.EMPTY_ARRAY));
956+
// No need to determinize the automaton, as it is only used to check for intersection with another automaton.
957+
Automaton v1automaton = Regex.simpleMatchToNonDeterminizedAutomaton(template.patterns().toArray(Strings.EMPTY_ARRAY));
955958
if (Operations.isEmpty(Operations.intersection(v2automaton, v1automaton)) == false) {
956959
logger.debug(
957960
"composable template {} and legacy template {} would overlap: {} <=> {}",

0 commit comments

Comments
 (0)