Backport/8.x/pr 121110 #121295
Closed
Backport/8.x/pr 121110 #121295
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Test ML model server (#120270) * Fix model downloading for very small models. * Test MlModelServer * Tiny ELSER * unmute TextEmbeddingCrudIT and DefaultEndPointsIT * update ELSER * Improve MlModelServer * tiny E5 * more logging * improved E5 model * tiny reranker * scan for ports * [CI] Auto commit changes from spotless * Serve default models when optimized model is requested * @ClassRule * polish code * Respect dynamic setting ML model repo * fix metadata for optimized models * improve logging --------- Co-authored-by: elasticsearchmachine <[email protected]> * backport HttpHeaderParser --------- Co-authored-by: elasticsearchmachine <[email protected]>
Dried up (and moved to the much faster inline logic) for the summation here for both implementations. Obviously this could have been done even drier but it didn't seem like that was possible without a performance hit (we really don't want to sub-class the leaf-collector I think). Benchmarks suggest this variant is ~10% faster than the previous iteration of `SumAggregator` (probably from making the grow method smaller) and a bigger than that improvement for the `AvgAggregator`.
* Forward port release notes for v8.16.3 * Delete includes * del coming tag --------- Co-authored-by: Liam Thompson <[email protected]>
…eInferencesTriggeringDownloadAndDeploy #117208
…time other than disconnected exceptions (#120236) (#120628) For ES|QL, we are moving to limit the scope of the skip_unavailable setting for remote clusters. Going forward, skip_unavailable will be considered for two scenarios: 1) inability to connect to a remote cluster ("unavailable") 2) whether to fail on execution time errors or not (inline with the upcoming allow_partial_search_results work for ES|QL). This PR reverses the special plan-time handling for skip_unavailable=true clusters that was added in #116348. Remote clusters, regardless of their skip_unavailable setting, will now use the same logic as the local cluster for index expression analysis at plan time, namely: 1) If any concrete index specified is missing from the cluster, a VerificationException will be thrown 2) If no matching index/alias/datastream was found on any cluster (even if all were specified with a wildcard), a VerificationException will be thrown Thus, we no longer require at least one matching index expression for skip_unavailable=false clusters either, as was done in the previous PR referenced above.
Co-authored-by: Liam Thompson <[email protected]>
Muting a yaml bwc test called "alias" apparently is not possible:
```
task.skipTest("esql/190_lookup_join/alias", "LOOKUP JOIN does not support index aliases for now")
```
```
* What went wrong:
Execution failed for task ':x-pack:plugin:yamlRestCompatTestTransform'.
> class com.fasterxml.jackson.databind.node.TextNode cannot be cast to class com.fasterxml.jackson.databind.node.ArrayNode (com.fasterxml.jackson.databind.node.TextNode and com.fasterxml.jackson.databind.node.ArrayNode are in unnamed module of loader org.gradle.internal.classloader.VisitableURLClassLoader$InstrumentingVisitableURLClassLoader @725f7fdd)
```
I need to mute this test in
#120617
…ssion required) (#119898) (#120650) Resolve/cluster allows querying for cluster-info-only (no index expression required) This enhancement provides users with the ability to query the _resolve/cluster API endpoint without specifying an index expression to match against. This allows users to quickly test what remote clusters are configured on a cluster and whether they are available for querying. The new endpoint takes no index expression: ``` GET _resolve/cluster ``` and returns the same information as before except for the "matching_indices" field. Example response: ``` { "remote1": { "connected": false, "skip_unavailable": true }, "remote2": { "connected": true, "skip_unavailable": false, "version": { "number": "8.17.0", "build_flavor": "default", "minimum_wire_compatibility_version": "7.17.0", "minimum_index_compatibility_version": "7.0.0" } } } ``` For backwards compatibility, this new endpoint works with clusters from older versions by querying with the index expression `dummy*` on those older clusters and ignoring the matching_indices value in the response they return.
* Remove index blocks by default in create_from (#120643) Change the default value for the `remove_index_blocks` parameter of the `_create_from` api to true. This means that when a source index has blocks, by default these blocks will be filtered out when creating the destination index. * Fix broken yaml test
This commit combines the different entitlement test plugins used for integration tests into a single plugin. When installing the plugin in the integ test cluster, the entitlement policy and plugin properties are adjusted to test the desired combination of settings. This allows the plugin policy to be dynamically generated by each test, rather than rely on a static policy.
…positoryAnalysisRestIT org.elasticsearch.repositories.blobstore.testkit.analyze.MinioRepositoryAnalysisRestIT #120672
…mplicit casting strings to dates for IN operator} #120155
…N operator with null in list, finds match} #120156
Currently, ES|QL ExecutionInfo is updated in multiple places: on remote clusters, during the execution of data-node requests, in compute listeners, and on the coordinator node. This change consolidates updates to happen only on the coordinator, when receiving compute responses. This change should also make the compute listener easier to understand
This commit refactors the logic introduced in #121092, ensuring that translog assertion logic related to inference fields resides exclusively within `TranslogOperationAsserter`. For mappings that contain inference fields, we regenerate the metadata field's content during peer recovery. This can create discrepancies between the original and regenerated sources, which is expected since indexed values may differ from the original. This change ensures that we compare the "synthetic" version of index operations in the same way we do for mappings with synthetic source enabled. In contrast, real-time get (`realtime-get`) simply filters inference fields from the original source, eliminating the need to synthesize them as we do for synthetic sources.
…ts testClientSecretRotation #120985
…est {p0=data_stream/140_data_stream_aliases/Create data stream alias with filter} #121014
… testAuditorWritesAudits #121241
) Adds non-grouping support for min, max, sum, and count, using CompositeBlock as the underlying block type and an internal FromAggregateMetricDouble function to handle converting from CompositeBlock to the correct metric subfields. Closes #110649
…temFeaturesMigration {cluster=UPGRADED} #121115
This commit adds FileEntitlement to entitlements. It does not add checks for all file access methods yet, instead opting for example read and write methods. Each module contains a sorted array of paths with read and write permissions. Binary search is used to quickly identify the closest path to determine whether a target path can be read or written. Some important things about FileEntitlement are the path can either be a file or a directory. All directories grant recursive permission. The mode is either read or read_write. All operations like create or delete are considered write. relates ES-10354 Co-authored-by: Jack Conradson <[email protected]>
…ts testGrantApiKeyForJWT #121039
…uggestProfileWithData #121258
…121237) This commit introduces the `MappedFieldType#getDefaultHighlighter`, allowing a specific highlighter to be enforced for a field. The semantic field mapper utilizes this new functionality to set the `semantic` highlighter as the default. All other fields will continue to use the `unified` highlighter by default.
…API (#120400) (#121251) * [Inference API] Add node-local rate limiting for the inference API (#120400) * Add node-local rate limiting for the inference API * Fix integration tests by using new LocalStateInferencePlugin instead of InferencePlugin and adjust formatting. * Correct feature flag name * Add more docs, reorganize methods and make some methods package private * Clarify comment in BaseInferenceActionRequest * Fix wrong merge * Fix checkstyle * Fix checkstyle in tests * Check that the service we want to the read the rate limit config for actually exists * [CI] Auto commit changes from spotless * checkStyle apply * Update docs/changelog/120400.yaml * Move rate limit division logic to RequestExecutorService * Spotless apply * Remove debug sout * Adding a few suggestions * Adam feedback * Fix compilation error * [CI] Auto commit changes from spotless * Add BWC test case to InferenceActionRequestTests * Add BWC test case to UnifiedCompletionActionRequestTests * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/common/InferenceServiceNodeLocalRateLimitCalculator.java Co-authored-by: Adam Demjen <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/common/InferenceServiceNodeLocalRateLimitCalculator.java Co-authored-by: Adam Demjen <[email protected]> * Remove addressed TODO * Spotless apply * Only use new rate limit specific feature flag * Use ThreadLocalRandom * [CI] Auto commit changes from spotless * Use Randomness.get() * [CI] Auto commit changes from spotless * Fix import * Use ConcurrentHashMap in InferenceServiceNodeLocalRateLimitCalculator * Check for null value in getRateLimitAssignment and remove AtomicReference * Remove newAssignments * Up the default rate limit for completions * Put deprecated feature flag back in * Check feature flag in BaseTransportInferenceAction * spotlessApply * Export inference.common * Do not export inference.common * Provide noop rate limit calculator, if feature flag is disabled * Add proper dependency injection --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Jonathan Buttner <[email protected]> Co-authored-by: Adam Demjen <[email protected]> * Use .get(0) as getFirst() doesn't exist in 8.18 (probably JDK difference?) --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Jonathan Buttner <[email protected]> Co-authored-by: Adam Demjen <[email protected]>
|
Documentation preview: |
elasticsearchmachine
added
needs:triage
|
It looks like this PR modifies one or more |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
81 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.