Updating documentation for text_similarity_reranker for 8.18 #126890
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… (elastic#124254) (cherry picked from commit 0f58537)
…ic#124107) (elastic#124266) # Backport This will backport the following commits from `main` to `8.18`: - [Avoid hoarding cluster state references during rollover (elastic#124107)](elastic#124107) <!--- Backport version: 9.6.4 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)
* Pulling api spec changes * Fixing test and updating code javadoc (cherry picked from commit 3a472eb) # Conflicts: # x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/inference/inference_crud.yml
…24273) * [Deprecation] Update URL (elastic#124259) Replace the URL with the new updated docs URL. Should probably migrate to ReferenceDocs after 9.1. * Fix checkstyle
…24149) (elastic#124270) * Retry ILM async action after reindexing data stream (elastic#124149) When reindexing a data stream, the ILM metadata is copied from the index metadata of the source index to the destination index. But the ILM state of the new index can be stuck if the source index was in an AsyncAction at the time of reindexing. To un-stick the new index, we call TransportRetryAction to retry the AsyncAction. In the past this action would only run if the index were in the error phase. This change includes an update to TransportRetryAction, which allows it to be run when the index is not in an error phase, if the parameter requireError is set to false. (cherry picked from commit 10a8dcf) # Conflicts: # server/src/main/java/org/elasticsearch/TransportVersions.java # x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/user/InternalUsers.java # x-pack/plugin/ilm/src/main/java/org/elasticsearch/xpack/ilm/action/TransportRetryAction.java # x-pack/qa/rolling-upgrade/src/test/java/org/elasticsearch/upgrades/DataStreamsUpgradeIT.java * index mode cannot be set on v7 indices
(cherry picked from commit 2a006ec)
The main goal of the PR is to fix the "rollover failed" log in the `IndexTemplateRegistry`. We were logging this on every upgrade because `rolloverResponse.isRolledOver()` is always `false` when we do a lazy rollover request - which is the default for index template updates. While I was looking at this, I noticed we don't actually return early when we process a lazy rollover _dry run_. The end result happened to be ok (i.e. side-effects and response), but I figured I might as well fix the behavior to return early. Finally, we add an `INFO` log in the lazy rollover action to aid the debugging process. (cherry picked from commit 9544204) # Conflicts: # server/src/main/java/org/elasticsearch/action/admin/indices/rollover/TransportRolloverAction.java
…astic#124324) * Update synthetic-source.asciidoc * Update source-field.asciidoc
…r type (elastic#124304) (elastic#124326) Our check methods injected by the instrumenter receive "this" as the second parameter. For internal classes like AbstractDelegateHttpsURLConnection we generally use a base type; in this case we were using javax.net.ssl.HttpsURLConnection, which is incorrect as AbstractDelegateHttpsURLConnection derives from java.net.HttpURLConnection. This was not failing in our tests because we don't actually use that parameter in that check function. Also, it was not failing on transform, just on retransformClasses, and only in JDK 24. Apparently, JDK 24 introduced new validation there (to be confirmed). And it was failing just on cloud as the APM agent there (which is loaded before our agent) connects to a https endpoint - our IT tests, and ./gradlew run --with-apm-server, use a http endpoint. Using https makes the JVM load AbstractDelegateHttpsURLConnection, making it one of the classes we need to retransform, triggering the VerifyError.
…#124235) * Entitle inference to access AWS credentials. Copied from the security policy. * more missing entitlements for the inference plugin --------- Co-authored-by: Lorenzo Dematte <[email protected]>
…3761) (elastic#124345) closes: elastic#123661 (cherry picked from commit a1ee3c9)
elastic#123890) (elastic#124346) * do not let ShardBulkInferenceActionFilter unwrap / rewrap ESExceptions
…ic#124353) (elastic#124365) These parameters are confusing and almost entirely unused (apart from one easily-adjusted test, two production locations where they are used incorrectly, and one production location that can be adjusted not to need them). This commit just removes them.
…=reference/cat/nodes/line_361} elastic#124103
…24198) Backporting elastic#122606 to 8.18 branch.
…d` annotation descriptor (elastic#124310) (elastic#124399)
…lastic#123776 and elastic#124023) (elastic#123924) (elastic#124331) * remove duplicate paths in FileAccessTree (elastic#123776) * Remove duplicate exclusive paths (elastic#124023) * Remove duplicate exclusive paths * Normalize paths in tests to support Windows * Remove withMode --------- Co-authored-by: Jack Conradson <[email protected]> Co-authored-by: Lorenzo Dematté <[email protected]>
…for compatibility purposes (elastic#124321) (elastic#124417) * Make NotEntitledException inherit from AccessControlException for compatibility purposes (elastic#124321) Even if the contract for JDK methods using the SecurityManager states that the exception throw is of type SecurityException, many libraries (including our own, apparently!) violates that and use the type actually thrown by SecurityManager, AccessControlException. A prime example is the GCS/CSP libraries. In order to maintain compatibility for them, we need to inherit from the more specific AccessControlException; this is less desirable, as AccessControlException is marked as deprecated for removal alongside the other SecurityManager classes, but we discussed and found this is the best short term solution. More work will be needed -- again, this is a short term solution. Replaces elastic#123984 * cleanup exceptions
…tic#124386) (elastic#124427) (cherry picked from commit b9a9784) # Conflicts: # server/src/internalClusterTest/java/org/elasticsearch/search/SearchCancellationIT.java
elastic#123985)" (elastic#124329) (elastic#124431) This reverts commit 404c9e6.
…6504) Internal UBI images from `docker.elastic.co` are planned to be removed.
…6539) This commit cleans up how entitlement test methods are discovered. It also adds another robustness check to ensure an annotation doesn't exist on a private method.
Co-authored-by: Quentin Pradet <[email protected]> (cherry picked from commit 5f871c5) Co-authored-by: Josh Mock <[email protected]>
) This commit adds a note that ignore_above has a different limit for logsdb indices to the documentation. Also specifies that ignore_above applies to all types of the keyword family. Relates elastic/sdh-elasticsearch#8892
…6680) Backporting elastic#126637 to 8.18 branch. If updating the `index.time_series.end_time` fails for one data stream, then UpdateTimeSeriesRangeService should continue updating this setting for other data streams. The following error was observed in the wild: ``` [2025-04-07T08:50:39,698][WARN ][o.e.d.UpdateTimeSeriesRangeService] [node-01] failed to update tsdb data stream end times java.lang.IllegalArgumentException: [index.time_series.end_time] requires [index.mode=time_series] at org.elasticsearch.index.IndexSettings$1.validate(IndexSettings.java:636) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.index.IndexSettings$1.validate(IndexSettings.java:619) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.common.settings.Setting.get(Setting.java:563) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.common.settings.Setting.get(Setting.java:535) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.datastreams.UpdateTimeSeriesRangeService.updateTimeSeriesTemporalRange(UpdateTimeSeriesRangeService.java:111) ~[?:?] at org.elasticsearch.datastreams.UpdateTimeSeriesRangeService$UpdateTimeSeriesExecutor.execute(UpdateTimeSeriesRangeService.java:210) ~[?:?] at org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1075) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1038) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.cluster.service.MasterService.executeAndPublishBatch(MasterService.java:245) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.lambda$run$2(MasterService.java:1691) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.action.ActionListener.run(ActionListener.java:452) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.run(MasterService.java:1688) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.cluster.service.MasterService$5.lambda$doRun$0(MasterService.java:1283) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.action.ActionListener.run(ActionListener.java:452) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.cluster.service.MasterService$5.doRun(MasterService.java:1262) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023) ~[elasticsearch-8.17.3.jar:?] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27) ~[elasticsearch-8.17.3.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] at java.lang.Thread.run(Thread.java:1575) ~[?:?] ``` Which resulted in a situation, that causes the `index.time_series.end_time` index setting not being updated for any data stream. This then caused data loss as metrics couldn't be indexed, because no suitable backing index could be resolved: ``` the document timestamp [2025-03-26T15:26:10.000Z] is outside of ranges of currently writable indices [[2025-01-31T07:22:43.000Z,2025-02-15T07:24:06.000Z][2025-02-15T07:24:06.000Z,2025-03-02T07:34:07.000Z][2025-03-02T07:34:07.000Z,2025-03-10T12:45:37.000Z][2025-03-10T12:45:37.000Z,2025-03-10T14:30:37.000Z][2025-03-10T14:30:37.000Z,2025-03-25T12:50:40.000Z][2025-03-25T12:50:40.000Z,2025-03-25T14:35:40.000Z ```
…122762) (elastic#123957) * ESQL: Ensure non-zero row size in `EstimatesRowSize` (elastic#122762) Closes elastic#121535 * Removed getFirest() usages --------- Co-authored-by: kanoshiou <[email protected]> Co-authored-by: Costin Leau <[email protected]> Co-authored-by: Iván Cea Fontenla <[email protected]> Co-authored-by: Iván Cea Fontenla <[email protected]>
…lastic#126696) Today we rely on registering the channel after registering the task to be cancelled to ensure that the task is cancelled even if the channel is closed concurrently. However the client may already have processed a cancellable request on the channel and therefore this mechanism doesn't work. With this change we make sure not to register another task after draining the registrations in order to cancel them. Closes elastic#88201
|
Documentation preview: |
|
Pinging @elastic/es-docs (Team:Docs) |
Samiul-TheSoccerFan
added
the
:SearchOrg/Relevance
|
Pinging @elastic/search-eng (Team:SearchOrg) |
|
Pinging @elastic/search-relevance (Team:Search - Relevance) |
Samiul-TheSoccerFan
added
the
auto-backport
Mikep86
reviewed
Apr 16, 2025
|
Thank you @Mikep86 . I will create another PR. I falsely assumed both |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
71 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As part of Semantic Rerank GA, updating the documentation to address that the inference_id field is not required anymore.