Skip to content

Commit 2645ad6

Browse files
romuletsromulets
authored
Merge branch 'main' into main
2 parents 5738b5c + 29fa3d8 commit 2645ad6

File tree

50 files changed

+1944
-1375
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+1944
-1375
lines changed

‎docs/changelog/132456.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 132456
2+
summary: Some optimizations for constant blocks
3+
area: ES|QL
4+
type: enhancement
5+
issues: []

‎docs/reference/query-languages/esql/_snippets/commands/examples/rerank.csv-spec/combine.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
% This is generated by ESQL's AbstractFunctionTestCase. Do not edit it. See ../README.md for how to regenerate it.
2+
3+
```esql
4+
FROM books METADATA _score
5+
| WHERE MATCH(description, "hobbit") OR MATCH(author, "Tolkien")
6+
| SORT _score DESC
7+
| LIMIT 100
8+
| RERANK rerank_score = "hobbit" ON description, author WITH { "inference_id" : "test_reranker" }
9+
| EVAL original_score = _score, _score = rerank_score + original_score
10+
| SORT _score
11+
| LIMIT 3
12+
| KEEP title, original_score, rerank_score, _score
13+
```
14+
15+
| title:text | _score:double | rerank_score:double | rerank_score:double |
16+
| --- | --- | --- | --- |
17+
| Poems from the Hobbit | 4.012462615966797 | 0.001396648003719747 | 0.001396648003719747 |
18+
| The Lord of the Rings - Boxed Set | 3.768855094909668 | 0.0010020040208473802 | 0.001396648003719747 |
19+
| Return of the King Being the Third Part of The Lord of the Rings | 3.6248698234558105 | 9.000900317914784E-4 | 0.001396648003719747 |

‎docs/reference/query-languages/esql/_snippets/commands/examples/rerank.csv-spec/simple-query.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
% This is generated by ESQL's AbstractFunctionTestCase. Do not edit it. See ../README.md for how to regenerate it.
2+
3+
```esql
4+
FROM books METADATA _score
5+
| WHERE MATCH(description, "hobbit")
6+
| SORT _score DESC
7+
| LIMIT 100
8+
| RERANK "hobbit" ON description WITH { "inference_id" : "test_reranker" }
9+
| LIMIT 3
10+
| KEEP title, _score
11+
```
12+
13+
| title:text | _score:double |
14+
| --- | --- |
15+
| Poems from the Hobbit | 0.0015673980815336108 |
16+
| A Tolkien Compass: Including J. R. R. Tolkien's Guide to the Names in The Lord of the Rings | 0.007936508394777775 |
17+
| Return of the King Being the Third Part of The Lord of the Rings | 9.960159659385681E-4 |

‎docs/reference/query-languages/esql/_snippets/commands/examples/rerank.csv-spec/two-queries.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
% This is generated by ESQL's AbstractFunctionTestCase. Do not edit it. See ../README.md for how to regenerate it.
2+
3+
```esql
4+
FROM books METADATA _score
5+
| WHERE MATCH(description, "hobbit") OR MATCH(author, "Tolkien")
6+
| SORT _score DESC
7+
| LIMIT 100
8+
| RERANK rerank_score = "hobbit" ON description, author WITH { "inference_id" : "test_reranker" }
9+
| SORT rerank_score
10+
| LIMIT 3
11+
| KEEP title, _score, rerank_score
12+
```
13+
14+
| title:text | _score:double | rerank_score:double |
15+
| --- | --- | --- |
16+
| Return of the Shadow | 2.8181066513061523 | 5.740527994930744E-4 |
17+
| Return of the King Being the Third Part of The Lord of the Rings | 3.6248698234558105 | 9.000900317914784E-4 |
18+
| The Lays of Beleriand | 1.3002015352249146 | 9.36329597607255E-4 |

‎docs/reference/query-languages/esql/_snippets/commands/layout/rerank.md‎

Lines changed: 6 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -100,61 +100,17 @@ If you don't want to increase the timeout limit, try the following:
100100

101101
Rerank search results using a simple query and a single field:
102102

103-
```esql
104-
FROM books
105-
| WHERE MATCH(title, "science fiction")
106-
| SORT _score DESC
107-
| LIMIT 100
108-
| RERANK "science fiction" ON (title) WITH { "inference_id" : "my_reranker" }
109-
| LIMIT 3
110-
| KEEP title, _score
111-
```
112103

113-
| title:keyword | _score:double |
114-
|---------------|---------------|
115-
| Neuromancer | 0.98 |
116-
| Dune | 0.95 |
117-
| Foundation | 0.92 |
104+
:::{include} ../examples/rerank.csv-spec/simple-query.md
105+
:::
118106

119107
Rerank search results using a query and multiple fields, and store the new score
120108
in a column named `rerank_score`:
121109

122-
```esql
123-
FROM movies
124-
| WHERE MATCH(title, "dystopian future") OR MATCH(synopsis, "dystopian future")
125-
| SORT _score DESC
126-
| LIMIT 100
127-
| RERANK rerank_score = "dystopian future" ON (title, synopsis) WITH { "inference_id" : "my_reranker" }
128-
| SORT rerank_score DESC
129-
| LIMIT 5
130-
| KEEP title, _score, rerank_score
131-
```
132-
133-
| title:keyword | _score:double | rerank_score:double |
134-
|-----------------|---------------|---------------------|
135-
| Blade Runner | 8.75 | 0.99 |
136-
| The Matrix | 9.12 | 0.97 |
137-
| Children of Men | 8.50 | 0.96 |
138-
| Akira | 8.99 | 0.94 |
139-
| Gattaca | 8.65 | 0.91 |
110+
:::{include} ../examples/rerank.csv-spec/two-queries.md
111+
:::
140112

141113
Combine the original score with the reranked score:
142114

143-
```esql
144-
FROM movies
145-
| WHERE MATCH(title, "dystopian future") OR MATCH(synopsis, "dystopian future")
146-
| SORT _score DESC
147-
| LIMIT 100
148-
| RERANK rerank_score = "dystopian future" ON (title, synopsis) WITH { "inference_id" : "my_reranker" }
149-
| EVAL original_score = _score, _score = rerank_score + original_score
150-
| SORT _score DESC
151-
| LIMIT 2
152-
| KEEP title, original_score, rerank_score, _score
153-
```
154-
155-
| title:keyword | original_score:double | rerank_score:double | _score:double |
156-
|---------------|-----------------------|---------------------|---------------|
157-
| The Matrix | 9.12 | 0.97 | 10.09 |
158-
| Akira | 8.99 | 0.94 | 9.93 |
159-
160-
115+
:::{include} ../examples/rerank.csv-spec/combine.md
116+
:::

‎docs/reference/query-languages/esql/_snippets/functions/layout/copy_sign.md‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎modules/repository-s3/src/main/java/org/elasticsearch/repositories/s3/S3Repository.java‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@
4242
import org.elasticsearch.repositories.blobstore.MeteredBlobStoreRepository;
4343
import org.elasticsearch.snapshots.SnapshotId;
4444
import org.elasticsearch.snapshots.SnapshotsService;
45+
import org.elasticsearch.snapshots.SnapshotsServiceUtils;
4546
import org.elasticsearch.threadpool.Scheduler;
4647
import org.elasticsearch.threadpool.ThreadPool;
4748
import org.elasticsearch.xcontent.NamedXContentRegistry;
@@ -388,7 +389,7 @@ private static ByteSizeValue objectSizeLimit(ByteSizeValue chunkSize, ByteSizeVa
388389
@Override
389390
public void finalizeSnapshot(final FinalizeSnapshotContext finalizeSnapshotContext) {
390391
final FinalizeSnapshotContext wrappedFinalizeContext;
391-
if (SnapshotsService.useShardGenerations(finalizeSnapshotContext.repositoryMetaVersion()) == false) {
392+
if (SnapshotsServiceUtils.useShardGenerations(finalizeSnapshotContext.repositoryMetaVersion()) == false) {
392393
final ListenableFuture<Void> metadataDone = new ListenableFuture<>();
393394
wrappedFinalizeContext = new FinalizeSnapshotContext(
394395
finalizeSnapshotContext.serializeProjectMetadata(),

‎muted-tests.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -521,6 +521,15 @@ tests:
521521
- class: org.elasticsearch.xpack.esql.inference.completion.CompletionOperatorTests
522522
method: testSimpleCircuitBreaking
523523
issue: https://github.com/elastic/elasticsearch/issues/132382
524+
- class: org.elasticsearch.xpack.esql.ccq.MultiClusterSpecIT
525+
method: test {csv-spec:lookup-join.MvJoinKeyOnFrom}
526+
issue: https://github.com/elastic/elasticsearch/issues/132554
527+
- class: org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT
528+
method: test {csv-spec:lookup-join.MvJoinKeyFromRow}
529+
issue: https://github.com/elastic/elasticsearch/issues/132555
530+
- class: org.elasticsearch.xpack.esql.qa.single_node.EsqlSpecIT
531+
method: test {csv-spec:spatial.ConvertFromStringParseError}
532+
issue: https://github.com/elastic/elasticsearch/issues/132558
524533

525534
# Examples:
526535
#

‎qa/repository-multi-version/src/test/java/org/elasticsearch/upgrades/MultiVersionRepositoryAccessIT.java‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
import org.elasticsearch.common.settings.Settings;
1919
import org.elasticsearch.index.IndexVersion;
2020
import org.elasticsearch.index.IndexVersions;
21-
import org.elasticsearch.snapshots.SnapshotsService;
21+
import org.elasticsearch.snapshots.SnapshotsServiceUtils;
2222
import org.elasticsearch.test.rest.ESRestTestCase;
2323
import org.elasticsearch.test.rest.ObjectPath;
2424
import org.elasticsearch.xcontent.XContentParser;
@@ -183,7 +183,7 @@ public void testUpgradeMovesRepoToNewMetaVersion() throws IOException {
183183
// incompatibility in the downgrade test step. We verify that it is impossible here and then create the repo using verify=false
184184
// to check behavior on other operations below.
185185
final boolean verify = TEST_STEP != TestStep.STEP3_OLD_CLUSTER
186-
|| SnapshotsService.includesUUIDs(minNodeVersion)
186+
|| SnapshotsServiceUtils.includesUUIDs(minNodeVersion)
187187
|| minNodeVersion.before(IndexVersions.V_7_12_0);
188188
if (verify == false) {
189189
expectThrowsAnyOf(EXPECTED_BWC_EXCEPTIONS, () -> createRepository(repoName, false, true));
@@ -208,7 +208,7 @@ public void testUpgradeMovesRepoToNewMetaVersion() throws IOException {
208208
ensureSnapshotRestoreWorks(repoName, "snapshot-2", shards, index);
209209
}
210210
} else {
211-
if (SnapshotsService.includesUUIDs(minNodeVersion) == false) {
211+
if (SnapshotsServiceUtils.includesUUIDs(minNodeVersion) == false) {
212212
assertThat(TEST_STEP, is(TestStep.STEP3_OLD_CLUSTER));
213213
expectThrowsAnyOf(EXPECTED_BWC_EXCEPTIONS, () -> listSnapshots(repoName));
214214
expectThrowsAnyOf(EXPECTED_BWC_EXCEPTIONS, () -> deleteSnapshot(repoName, "snapshot-1"));

‎server/src/internalClusterTest/java/org/elasticsearch/snapshots/ConcurrentSnapshotsIT.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2154,7 +2154,7 @@ public void testDeleteIndexWithOutOfOrderFinalization() {
21542154
final Map<String, SubscribableListener<Void>> otherIndexSnapshotListeners = indexNames.stream()
21552155
.collect(Collectors.toMap(k -> k, k -> new SubscribableListener<>()));
21562156
masterTransportService.<UpdateIndexShardSnapshotStatusRequest>addRequestHandlingBehavior(
2157-
SnapshotsService.UPDATE_SNAPSHOT_STATUS_ACTION_NAME,
2157+
TransportUpdateSnapshotStatusAction.NAME,
21582158
(handler, request, channel, task) -> {
21592159
final var indexName = request.shardId().getIndexName();
21602160
if (indexName.equals(indexToDelete)) {

0 commit comments

Comments
 (0)