Skip to content

Conversation

ChrisHegarty
Copy link
Contributor

Merge main into lucene_snapshot

pawankartik-elastic and others added 30 commits July 30, 2025 12:00
…stic#131517)

For CPS S2D9, we'd need field caps to refresh potentially lost connections before
executing a query by explicitly establishing a connection with a short timeout to
avoid waiting for large duration. This is similar to what we recently did with
`_search` and will help for ES|QL as well.
Current knn search over nested vectors only supports filtering on
parent's metadata.  This adds support for filtering over nested
metadata.

Closes elastic#106994 Closes elastic#128803
We set a 120s timeout on the repo analysis that runs in these tests, but
the REST client in use has a read timeout of 60s so any analysis that
lasts longer than that will yield an HTTP timeout that contains very
limited information to help further investigation. With this commit we
extend the HTTP timeout to be a few seconds longer than the analysis
timeout so we should get a proper response in these cases.
…UpdateIT testDenseVectorMappingUpdate {initialType=bbq_flat updateType=bbq_disk #2} elastic#132184
The number of plugin archives changed and triggered the assertion.

Changing the assertion to check for `analysis-icu` plugin archive 
instead of asserting on number of archives. This will always fail  when
new plugin is added (like microsoft-graph-authz).

Resolves elastic#116866
…UpdateIT testDenseVectorMappingUpdate {initialType=bbq_hnsw updateType=bbq_disk} elastic#132188
…UpdateIT testDenseVectorMappingUpdate {initialType=int8_flat updateType=bbq_disk #2} elastic#132189
This commit presents a hierarchical layer on top of the DiskBBQ centroids to reduce the number of centroids scored
at search time.
Support for (the deprecated) path.shared_data was previously not covered in any REST test and broke unnoticed when we introduced entitlements. The problem only surfaced when eventually enabling entitlements on internalClusterTests.

This PR adds a basic test to check indexing using a relative (to path.shared_data) data_path index setting succeeds with sufficient entitlements for path.shared_data.

Relates to ES-12447
A failure of a write to a register does not indicate that the write has
definitely completed - it may eventually succeed at some later point in
time.
…131937)

It is possible for a linked remote to get unlinked in
between the containsKey() and get() calls in collectNodes().
This change adds a test that produces the NullPointerException
and adds a fix.
In some contexts ESQL's HeapAttack tests were failing because the
TranslogWriter keeps a map with the contents of the documents in it
around. But only when assertions are enabled. Fun. This flushes the map
away, preventing the odd OOM we were seeing in the HeapAttack tests.
Speeds up the spec tests by:
* Randomize `async`/`sync` instead of both
* Check the took time rarely
* Don't check if index exists every time
* Reduce transfer when checking breaker
* Cache capabilities checks

The first one is most of the speed up but the rest save another couple
of minutes. Time drops from 13m45s to 6m or so on my laptop.
…UpdateIT testDenseVectorMappingUpdate {initialType=int4_hnsw updateType=bbq_hnsw} elastic#132211
…UpdateIT testDenseVectorMappingUpdate {initialType=int8_hnsw updateType=bbq_disk #2} elastic#132213
…UpdateIT testDenseVectorMappingUpdate {initialType=int8_hnsw updateType=bbq_disk} elastic#132214
If index throttling is enabled such that it pauses all indexing threads that try to index into a shard, this can starve other tasks such as relocation that try to acquire all indexing permits. This PR addresses this by suspending throttling to allow the indexing threads that are holding the permits to pass. Addresses ES-11770.
…UpdateIT testDenseVectorMappingUpdate {initialType=int4_flat updateType=bbq_hnsw} elastic#132219
…UpdateIT testDenseVectorMappingUpdate {initialType=hnsw updateType=bbq_disk} elastic#132220
…UpdateIT testDenseVectorMappingUpdate {initialType=int4_flat updateType=bbq_disk} elastic#132221
elasticsearchmachine and others added 19 commits July 31, 2025 00:32
…UpdateIT testDenseVectorMappingUpdate {initialType=int4_hnsw updateType=bbq_disk #2} elastic#132228
…#131981)

The ml-cpp PR elastic/ml-cpp#2846 introduces
changes to how memory values are calculated and reported for Linux
platforms. This PR adjusts test case values accordingly.
… built in Editor and Viewer roles (elastic#129662)

* Adding asset criticality and entity store permissions to built in roles

* Update docs/changelog/129662.yaml

* [CI] Auto commit changes from spotless

* Corrects entity store index pattern

Updates the entity store index pattern to ensure it matches the minimum necessary index name and narrow it down to the correct use case

---------

Co-authored-by: elasticsearchmachine <[email protected]>
…2157)

This is the same failure as observed in elastic#129644 for which the original
fix elastic#129680 did not really work. It did not work because the the
ordering of checks. The shutdown marker is removed after the cluster
passes ready check so that new shards can be allocated. But the cluster
cannot pass the ready check before the shards are allocated. Hence the
circular dependency. In hindsight, there is no need to put shutdown
record for all nodes. It is only needed on the node that upgrades the
last to prevent snapshot from completion during the upgrade process.
This PR does that which ensures there are always 2 nodes for hosting new
shards.

Resolves: elastic#132135 Resolves: elastic#132136 Resolves: elastic#132137
…UpdateIT testDenseVectorMappingUpdate {initialType=int4_flat updateType=bbq_disk #2} elastic#132234
The listener is expected to be called on failure case. This PR does
that.

Resolves: elastic#131979
Reenable INLINESTATS telemetry test that was disabled in elastic#116005.
The test query is now planned and accounted for correctly.
elastic#115813 refactoring disabled a couple of INLINESTATS tests. These are working correctly now.
…stic#132154)

This scorer gives a better tradeoff on scoring precision and speed and only requiring two additional bytes
per vector comparing to int4.
@ChrisHegarty ChrisHegarty requested a review from a team as a code owner July 31, 2025 11:51
@ChrisHegarty ChrisHegarty added the :Search/Search Search-related issues that do not fall into other categories label Jul 31, 2025
@ChrisHegarty ChrisHegarty requested a review from a team as a code owner July 31, 2025 11:51
@ChrisHegarty ChrisHegarty added the Team:Search Meta label for search team label Jul 31, 2025
@ChrisHegarty ChrisHegarty requested a review from a team as a code owner July 31, 2025 11:51
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

Copy link
Contributor

@ChrisHegarty ChrisHegarty merged commit 1c91d5b into elastic:lucene_snapshot Jul 31, 2025
32 of 33 checks passed
@ChrisHegarty ChrisHegarty deleted the lucene_snapshot branch July 31, 2025 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team

Projects

None yet

Development

Successfully merging this pull request may close these issues.