[9.0] Migrate initial elasticsearch transport versions for 8.18 (#134158) #134162
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Newlines are system dependent. The release notes generator uses groovy's template engine, which produces system dependent newlines. This commit adjusts the test to account for newlines on both windows and nix systems.
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | docker.elastic.co/wolfi/chainguard-base | | digest | `a02075b` -> `5e3d0d5` | | docker.elastic.co/wolfi/chainguard-base | final | digest | `a02075b` -> `5e3d0d5` | | docker.elastic.co/wolfi/chainguard-base | stage | digest | `a02075b` -> `5e3d0d5` | | docker.elastic.co/wolfi/chainguard-base-fips | | digest | `7f2cebd` -> `048914a` | | docker.elastic.co/wolfi/chainguard-base-fips | final | digest | `7f2cebd` -> `048914a` | | docker.elastic.co/wolfi/chainguard-base-fips | stage | digest | `7f2cebd` -> `048914a` | --- ### Configuration 📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://redirect.github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoiOS4wIiwibGFiZWxzIjpbIjpEZWxpdmVyeS9QYWNrYWdpbmciLCI+bm9uLWlzc3VlIiwiVGVhbTpEZWxpdmVyeSIsImF1dG8tbWVyZ2Utd2l0aG91dC1hcHByb3ZhbCJdfQ==-->
…30431) * Adds Update a document API example page * Update docs/reference/elasticsearch/rest-apis/update-document.md * Adresses suggestions --------- Co-authored-by: Liam Thompson <[email protected]>
… (elastic#130433) * Rename serach connectors to content connectors * Adding support table to documentation * replace 'search' to 'content' * fix links
…ance (elastic#130340) (elastic#130437) (cherry picked from commit 137eac5) # Conflicts: # docs/reference/query-languages/esql/README.md
…tic#130412) (elastic#130444) - Ensure we include sources and javadoc jar to maven publication (cherry picked from commit 50d0474)
…lastic#130468) This documents the new threadpool-based merge scheduler, which is disk space aware, and blocks merges when disk space is low. The code changes were mostly introduced in elastic#120869 and elastic#127613 .
We used to have an assertion during transport version loading that duplicate ids were not found, but it appears to have been lost in refactorings. This commit adds a test to ensure duplicate ids do not occur. relates elastic#130486
… (elastic#130515) This PR ensures the node responses are copied and drained exclusively in onCompletion so that they do not get concurrently modified by cancellation. Resolves: elastic#128852
Previously, out of zealousness for testing efficiency, the mocked filesystems were reused across the test suite class. But this makes tests liable to interference wrt to filesystem stats calls. Moreover, if one test fails, it can trigger failures in other test methods. This PR recreates the mocked filesystems for every test method. Fixes elastic#129296 elastic#130205
…rge (elastic#130078) (elastic#130529) This fixes a race condition in the test scenario, between the merge scheduler closing and the merge task being scheduled to run. The test scenario expects that the merge task runs when the scheduler is closed. If the merge scheduler is closed before the merge task is scheduled, the merge task will instead be scheduled as aborted. Fixes: elastic#125236
…0554) * Add update by query api examples * Update docs/reference/elasticsearch/rest-apis/update-by-query-api.md * add TOC --------- Co-authored-by: Liam Thompson <[email protected]>
…ningMergeTasks (elastic#130545) (elastic#130562) The test submits merge tasks that support IO throttling, and asserts that all the currently running merge tasks are indeed IO throttled after the new one was submitted. The test erroneously tried to assert a property on the set of currently running merge tasks, which is very difficult to do since all merge tasks are possibly backlogged and re-enqueued asynchronously multiple times before they are run or aborted (so looking at the threadpool merge task queue there's no telling which merge task will execute first). Fixes elastic#129531
…ic#130602) This commit trim to size those lists to prevent wasteful heap usage.
…lastic#130410) (elastic#130613) * [9.1] Fixes to null-predicates and lookup-join docs (elastic#130440 elastic#130410) (elastic#130537) * Support types table in lookup join docs (elastic#130410) * Support types table in lookup join docs * Don't show a results column in the join types * Make LOOKUP JOIN types table more compact * Update docs/reference/query-languages/esql/esql-lookup-join.md Co-authored-by: Liam Thompson <[email protected]> Co-authored-by: Alexander Spies <[email protected]> * Fix the ESQL docs for the null predicates (elastic#130440) Fix null predicates docs * This also includes moving the note above the examples in the Kibana inline docs. * Add missing knn docs * Refine newlines in kibana docs --------- Co-authored-by: Liam Thompson <[email protected]> Co-authored-by: Alexander Spies <[email protected]> * Fix Null predicates after cherry-pick * [DOCS] Update ES|QL generated docs to consistently use the `applies_to` metadata (elastic#128576) - Add PREVIEW annotations to all preview functions - Update docs generation logic to use annotations instead of preview boolean * Changed stack: ga 9.1 to stack: coming in multiple places - Match.java: Updated the options parameter description - MultiMatch.java: Updated the options parameter description - ToLower.java: Reformatted parameter description and updated version annotation - ToUpper.java: Removed the appliesTo annotation entirely and reformatted parameter description - updated applies_to annotations to specify both preview 9.0.0 and ga 9.1.0 lifecycle stages - added version-specific documentation examples with applies_to markers for ga 9.1.0 features - enhanced to_lower and to_upper functions to indicate support for multi-valued expressions in ga 9.1.0 - added version guards around function parameters and descriptions using applies_to syntax - updated function parameter descriptions to indicate ga 9.1.0 availability for named parameters - use detailedDescription + and fix match_phrase applies_to syntax - strip inline applies_to from kibana docs - update roundto, matchphrase lifecycles - fix match named params info - various spatial functions are preview - unsigned long is preview - update qstr - Update TO_LOWER/TO_UPPER parameter descriptions for clarity - hide spatial functions per elastic#129839 - added `stack: ga 9.1.0` blocks to match_phrase.md and qstr.md examples - simplified term.md version from `preview 9.0.0` to just `preview` - added `applies_to = "stack: ga 9.1.0"` to matchphrase and querystring java annotations - removed version `9.0.0` from term function annotation - deleted unused `makeCallout()` and `appendLifeCycleAndVersion()` methods Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Liam Thompson <[email protected]> Co-authored-by: Liam Thompson <[email protected]> Co-authored-by: Nik Everett <[email protected]> * First round cleanup of docs after cherry-picking Includes partial support for the grammer fix to the header comment. * Second round, just replacing `no` with `not` * Fix lookupjointypes test after backport --------- Co-authored-by: Liam Thompson <[email protected]> Co-authored-by: Alexander Spies <[email protected]> Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Liam Thompson <[email protected]> Co-authored-by: Nik Everett <[email protected]>
…#130636) * Fix tests involving getBestDowngradeVersion * [CI] Auto commit changes from spotless * Typo in assume message --------- Co-authored-by: elasticsearchmachine <[email protected]> (cherry picked from commit 64568ee) # Conflicts: # muted-tests.yml Co-authored-by: Patrick Doyle <[email protected]>
…kSpaceTests testUnavailableBudgetBlocksNewMergeTasksFromStartingExecution elastic#130205
We should have used the standard warning here because we don't need a title for the admonition (cherry picked from commit ec5254b) # Conflicts: # docs/reference/elasticsearch/mapping-reference/mapping-source-field.md
…30695) In EsqlSession#fieldNames, keep track of wildcard patterns in DROP separately from the references and wildcard patterns in KEEP commands. Only the latter are relevant to determine if we need to ask for all fields for the lookup index in the field caps request. (cherry picked from commit 05fc6f2) # Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java # x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/session/IndexResolverFieldNamesTests.java
(cherry picked from commit f9d96dc) # Conflicts: # server/src/main/java/org/elasticsearch/transport/TransportService.java
…30452) (elastic#130746) By default, the `FilterByFilterAggregator` (Used by the `"filter"` and `"filters"` aggs) was using the `DefaultBulkScorer` (From Lucene), which has no cancellation mechanism. This PR wraps it into a `CancellableBulkScorer`, which instead calls the inner scorer with ranges, and checks cancellation between them. This should solve cases of long-running tasks using these aggregators not being cancelled, or greatly reduce the time they take after cancellation.
…129008) (elastic#129661) (cherry picked from commit e53592e) # Conflicts: # .buildkite/pipelines/periodic-micro-benchmarks.yml # .buildkite/pipelines/periodic-packaging.yml # .buildkite/pipelines/periodic.yml
…ic#130773) Using the VALUES aggregator with ordinals grouping led to accidental quadratic complexity. Queries like FROM .. | STATS ... VALUES(field) ... BY keyword-field are affected by this performance issue. This change caches a sorted structure - previously used to fix a similar O(N^2) problem when emitting the output block - during the merging phase of the OrdinalGroupingOperator.
…lastic#130856) * Instead of creating a new Docker image on-the-fly, it is better to create a persistent image and upload it to Docker Registry. This will help tests dependent on this image be resilient to transient issues such as elastic#126694 and also speed up execution. * fix name (cherry picked from commit 22390d7)
…astic#133525) Versions from branches.json are now used for calculating unreleased versions for BwC tests. Gradle task added to modify branches.json. Possibility to add, remove and update branches in the file with the version field added next to the branch. BwC test configuration has been changed to use changed branches.json file to calculate unreleased ES versions, instead of combining data from Version.java and branches.json.
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | docker.elastic.co/wolfi/chainguard-base | | digest | `4dbe940` -> `ed432bb` | | docker.elastic.co/wolfi/chainguard-base | final | digest | `4dbe940` -> `ed432bb` | | docker.elastic.co/wolfi/chainguard-base | stage | digest | `4dbe940` -> `ed432bb` | | docker.elastic.co/wolfi/chainguard-base-fips | | digest | `d9382de` -> `091dcf0` | | docker.elastic.co/wolfi/chainguard-base-fips | final | digest | `d9382de` -> `091dcf0` | | docker.elastic.co/wolfi/chainguard-base-fips | stage | digest | `d9382de` -> `091dcf0` | --- ### Configuration 📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://elastic.slack.com/archives/C07AMD4CNUR) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://redirect.github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoiOS4wIiwibGFiZWxzIjpbIjpEZWxpdmVyeS9QYWNrYWdpbmciLCI+bm9uLWlzc3VlIiwiVGVhbTpEZWxpdmVyeSIsImF1dG8tbWVyZ2Utd2l0aG91dC1hcHByb3ZhbCJdfQ==-->
…stic#133932) Co-authored-by: Rassyan <[email protected]> Co-authored-by: Moritz Mack <[email protected]>
…3849) (elastic#133865) This refactors ZERO, MINIMUM_COMPATIBLE, and MINIMUM_CCS_VERSION into TransportVersion.VersionsHolder and load them from /transport/constants/....
With this change it takes into account that not all versions have previous minor unreleased version (because we are on the oldest active development branch), or that current version has minor == 0 (e.g. 9.0) so previous minor have to be calculated differently. * Add FwC branch configuration and update periodic trigger logic Ensuring that only relevant branches are considered. * Correct FWC periodic pipeline variables Utilize Buildkite matrix syntax and escape env in command.
…ncer (elastic#133919) (elastic#134052) The static method TrainedModelAssignmentRebalancer.getNodeFreeMemoryExcludingPerNodeOverheadAndNativeInference was used to subtract load.getAssignedNativeInferenceMemory() from load.getFreeMemoryExcludingPerNodeOverhead(). However, in NodeLoad.getFreeMemoryExcludingPerNodeOverhead(), native inference memory was already subtracted as part of the getAssignedJobMemoryExcludingPerNodeOverhead() calculation. This led to double-counting of the native inference memory. Avoiding this double-counting allows us to remove the private method getNodeFreeMemoryExcludingPerNodeOverheadAndNativeInference() entirely.
…lastic#134056) This had to be changed because there is no access to the classes described as alternatives in the deprecated field.
…lastic#133708) (elastic#133825) * Update distro packaging when testing java ea versions * Unify resolving pre release java versions like RCs and EAs This reworks how we deal with pre-release java versions for testing. Passing `-Druntime.java=25-pre` will pick the lastest build that could be either an EA or and RC version. Passing explicitly a build number works by running build via `-Druntime.java=25-pre -Druntime.java.build=36` which as of now would pick a RC build. This also tweaks the archive packaging in case of a defined pre release version. This is used downstream when packaging serverless images including ea / rc versions * Bring back getJavaDetails used when configuring fips * Adopt ReproduceInfoPrinter (cherry picked from commit 5e71f1a)
…33793) (elastic#134112) This PR focuses on the short term solution which add the logs-sentinel_one.application-* and logs-sentinel_one.application_risk-* indices under the kibana_system role with deletion privileges to prevent a failed deletion error when the index enters the deletion phase for the ILM lifecycle, in upcoming PR. As it ships transform pipeline too hence read, write permissions are also required. Current behavior: It shows permission issue while deleting the index. (cherry picked from commit bfde47a)
…lastic#134084) (elastic#134144) This reverts our other bootstrapping of file-based transport versions to make migration simpler moving forward and adds the initial version one prior to the last initial version for each of 9.1, 9.0, 8.19, and 8.18.
…4158) This leaves the latest initial elasticsearch transport version for 8.18 as a marker. ES-12334
jdconrad
added
Team:Core/Infra
jdconrad
removed
the
auto-merge-without-approval
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
67 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backports the following commits to 9.0: