Backport 120944 to 8.18 #121969
Closed
Backport 120944 to 8.18 #121969
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…c#120584) (elastic#120932) * Cover custom sorting and routing in randomized testing * [CI] Auto commit changes from spotless * fix reindex tests * fix reindex tests * refactor classes * comment * more refactoring * more refactoring * restore tests with static mappings * reduce diff * reduce diff * Restore single-element array removal in synthetic source * Revert "Restore single-element array removal in synthetic source" This reverts commit e8e99e1. * [CI] Auto commit changes from spotless --------- Co-authored-by: elasticsearchmachine <[email protected]>
(cherry picked from commit 10d09a1)
…0905) * Move scoring in ES|QL out of snapshot (elastic#120354)
…ic#120941) * Use new semantic text format by default (elastic#120813) (cherry picked from commit be8f51a) * Fix changelog
…tion (elastic#120818) (elastic#120919) * Removing default model ID from ElasticsearchInternalService configuration * Update docs/changelog/120818.yaml * Delete docs/changelog/120818.yaml --------- Co-authored-by: Elastic Machine <[email protected]>
…120906) (elastic#120922) Partial rollback of elastic#120859, these data types seem fine.
…0235) (elastic#120916) * [ML] Adding dynamic filtering for EIS configuration (elastic#120235) * Functionality for filtering task types based on acl info for EIS * Fixing compile and test errors * updating with chat_completion * Adding acl call * [CI] Auto commit changes from spotless * working run * Starting to rename * [CI] Auto commit changes from spotless * Writing some tests * rename authorizations endpoint and response fields * Fixing enabled task types bug * Adding timed listener tests * Fixing some test failures * Adding more tests and a mock gateway * Switch sparse embedding name from gateway * Adding supported streaming tasks tests * Trying to fix the javadoc * Removing commented code * Still fixing javadoc * Lets try a break this time * add AuthHandler, AuthRequest, and AuthResponseEntity tests * [CI] Auto commit changes from spotless * Adding tests * Speeding up test * Adding atomic ref * Refactoring * Addressing feedback * Forgot a fix * Removing todo --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Brendan Jugan <[email protected]> (cherry picked from commit 3534ded) # Conflicts: # x-pack/plugin/inference/qa/inference-service-tests/src/javaRestTest/java/org/elasticsearch/xpack/inference/InferenceCrudIT.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/http/sender/RequestTask.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elastic/ElasticInferenceService.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/HttpRequestSenderTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/RequestExecutorServiceTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/http/sender/RequestTaskTests.java * Removing elastic timeout exception * Fixing addFirst
elastic#119003) (elastic#120956) * Add a replicate_for option to the ILM searchable_snapshot action (elastic#119003) * Replace calls to getFirst()
…trips is true (elastic#117504) (elastic#120954) * Fix NPE caused by race condition in async search when minimise round trips is true Previously, the `notifyListShards()` initialised and updated the required pre-requisites (`searchResponse` being amongst them) when a search op began. This function takes in arguments that contain shard-specific details amongst others. Because this information is not immediately available when the search begins, it is not immediately called. In some specific cases, there can be a race condition that can cause the pre-requisities (such as `searchResponse`) to be accessed before they're initialised, causing an NPE. This fix addresses the race condition by splitting the initialisation and subsequent updation amongst 2 different methods. This way, the pre-requisities are always initialised and do not lead to an NPE. * Try: call `notifyListShards()` after `notifySearchStart()` when minimize round trips is true * Add removed code comment * Pass `Clusters` to `SearchTask` rather than using progress listener to signify search start. To prevent polluting the progress listener with unnecessary search specific details, we now pass the `Clusters` object to `SearchTask` when a search op begins. This lets `AsyncSearchTask` access it and use it to initialise `MutableSearchResponse` appropriately. * Use appropriate `clusters` object rather than re-building it * Do not double set `mutableSearchResponse` * Move mutable entities such as shard counts out of `MutableSearchResponse` * Address PR review: revert moving out mutable entities from `MutableSearchResponse` * Update docs/changelog/117504.yaml * Get rid of `SetOnce` for `searchResponse` * Drop redundant check around shards count * Add a test that calls `onListShards()` at last and clarify `updateShardsAndClusters()`'s comment * Fix test: ref count * Address review comment: rewrite comment and test
…MigrationProperlyDetected elastic#115697
…IT testFileSettingsReprocessedOnRestartWithoutVersionChange elastic#120964
…est {p0=data_stream/80_resolve_index_data_streams/Resolve index with hidden and closed indices} elastic#120965
* Entitled plugin * [CI] Auto commit changes from spotless * SuppressForbidden in entitled plugin * Respond to PR comments * Reinstate entitled plugin * Make System_clearProperty package-private --------- Co-authored-by: elasticsearchmachine <[email protected]>
…20963) (elastic#120967) There are many features of the Elasticsearch ecosystem that may malfunction, or fail to work entirely, if these templates are not installed. This commit adds documentation cautioning against disabling the installation of templates.
…20_operator_privileges_disabled/Operator only settings can be set and restored by non-operator user when operator privileges is disabled} elastic#120973
…ceActionFilterIT testBulkOperations {p0=false} elastic#120975
…ackaging.test.DockerTests elastic#120978
…gHeadersOnFailedConversions {ASYNC} elastic#120980
This removes an unnecessary entitlement from reindex that is causing test failures.
…ts testActivateProfileForJWT elastic#120983
…rofileIndexAutoCreation elastic#120987
Co-authored-by: Sylvain Morin <[email protected]>
(cherry picked from commit d91d516) # Conflicts: # docs/reference/esql/functions/description/match.asciidoc # docs/reference/esql/functions/kibana/definition/match.json # docs/reference/esql/functions/kibana/docs/match.md # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/Match.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParser.interp # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/parser/EsqlBaseParser.java # x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LogicalPlanOptimizerTests.java
(cherry picked from commit 974c4e6)
…angeComparisons {ASYNC} elastic#121007
Fix a bug in TOP which surfaces when merging results from ordinals. We weren't always accounting for oversized arrays when checking if we'd ever seen a field. This changes the oversize itself to always size on a bucket boundary. The test for this required random `bucketSize` - without that the oversizing frequently wouldn't cause trouble.
This causes the ESQL heap attack tests to grow their memory usage if they first don't cause a circuit breaking exception. It just tries again with more data. That's slow, but it should stop this from failing quite as much. And it'll give us even more information about failures. Closes elastic#121465
* Rename environment dir accessors (elastic#121803) The node environment has many paths. The accessors for these currently use a "file" suffix, but they are always directories. This commit renames the accessors to make it clear these paths are directories. * [CI] Auto commit changes from spotless --------- Co-authored-by: elasticsearchmachine <[email protected]>
…e_metric_double (elastic#121727) (elastic#121853) Fixes elastic#119696 and elastic#96076
…astic#121833) (elastic#121861) (cherry picked from commit 56cac1b)
…lastic#121851) Unified Chat Completion error responses now forward code, type, and param to in the response payload. `reason` has been renamed to `message`. Notes: - `XContentFormattedException` is a `ChunkedToXContent` so that the REST listener can call `toXContentChunked` to format the output structure. By default, the structure forwards to our existing ES exception structure. - `UnifiedChatCompletionException` will override the structure to match the new unified format. - The Rest, Transport, and Stream handlers all check the exception to verify it is a UnifiedChatCompletionException. - OpenAI response handler now reads all the fields in the error message and forwards them to the user. - In the event that a `Throwable` is a `Error`, we rethrow it on another thread so the JVM can catch and handle it. We also stop surfacing the JVM details to the user in the error message (but it's still logged for debugging purposes).
…nsportActionIT testTimestamp0AddedIfMissing elastic#121745
* Report componentName in ModuleEntitlements. This lets us produce error messages that guide the user to add the right entitlement to the right plugin/server/etc. * Include component names in errors and logs * Name APM agent specifically. Avoids confusion with the entitlements agent. * Entitlement component names package private * [CI] Auto commit changes from spotless --------- Co-authored-by: elasticsearchmachine <[email protected]>
…etUsersWithProfileUid elastic#121483
…astic#121838) We need to explicitly add the incubating vector API module to the third party audit task on Java 24. (cherry picked from commit 0c667ec) # Conflicts: # build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/precommit/ThirdPartyAuditTask.java
…lity` (elastic#121883) Today we use the ID from `Version#CURRENT` in this test, which only works if its ID is no less than that of `TranportVersion#current()`. This commit fixes the test to ensure it always picks a transport version ID that is not from the past.
…elastic#121326) (elastic#121907) * [ML] Support revoking inference default endpoint authorization (elastic#121326) * Starting revoke * Adding integration tests * More integration tests * Adding test for deleting default inference endpoint via rest call * Removing task type any * Addressing feedback and adding test * Fixing tests
This test creates an incorrectly-serialized handshake which cannot be validated, and elastic#121747 made that validation compulsory. This test corrects the serialization. Closes elastic#121816
…stic#121925) Today, the exchange buffer of an exchange source is finished in two cases: (1) when the downstream pipeline has received enough data and (2) when all remote sinks have completed. In the first case, outstanding pages could be safely discarded. In the second case, no new pages should be received after finishing. In both scenarios, discarding all outstanding pages was safe if noMoreInputs was switched while adding pages. However, with the stop API, the buffer may now finish while keeping outstanding pages, and new pages may still be received. This change updates the exchange buffer to discard only the incoming page when noMoreInputs is switched, rather than all pages in the buffer. Closes elastic#120757
…121864) (elastic#121937) This change replaces FileEntitlement with FilesEntitlement so that we can have exactly one entitlement class per module (or possibly future scope). This cleans up our policy files so that all files are located together to allow access, and this opens up the design for future optimizations.
This PR addresses issues around aggregations cancellation, mentioned in elastic#108701 and other places. In brief, during aggregations collection time, we respect cancellation via the mechanisms in the searcher to poison cancelled queries. But once the aggregation finishes collection, there is no further need to interact with the searcher, so we cannot rely on that for cancellation checking. In particular, deeply nested aggregations can spend a long time constructing the results tree. Checking for cancellation is a trade off, as the check itself is somewhat expensive (it involves a volatile read), so we want to balance checking often enough that cancelled queries aren't taking up resources for a long time, but not so frequently that it slows down most aggregation queries. Our first attempt to this is to check once when we go to build sub-aggregations, as the worst cases for this that we've seen involve needing to build deep sub-aggregation trees. Checking at sub-aggregation construction time also provides a conveniently centralized method call to add the check to. --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Nik Everett <[email protected]> Conflicts: test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java
|
Documentation preview: |
|
typed enter at the wrong moment, will reopen correctly |
|
It looks like this PR modifies one or more |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
74 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
gradle check?