Skip to content

Conversation

@jordan-powers
Copy link
Contributor

Fixed a bug in the construction of the expected source when the value and malformed value counts were both zero.

Closes #125319

@jordan-powers jordan-powers added >test Issues or PRs that are addressing/adding tests :StorageEngine/Mapping The storage related side of mappings v9.1.0 labels Mar 20, 2025
@jordan-powers jordan-powers requested a review from martijnvg March 20, 2025 17:51
@jordan-powers jordan-powers self-assigned this Mar 20, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jordan-powers jordan-powers enabled auto-merge (squash) March 20, 2025 18:13
@jordan-powers jordan-powers merged commit 2ff03ac into elastic:main Mar 20, 2025
17 checks passed
afoucret pushed a commit to afoucret/elasticsearch that referenced this pull request Mar 21, 2025
Fix and unmute testSynthesizeArrayRandomIgnoresMalformed
smalyshev pushed a commit to smalyshev/elasticsearch that referenced this pull request Mar 21, 2025
Fix and unmute testSynthesizeArrayRandomIgnoresMalformed
@jordan-powers
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x

Questions ?

Please refer to the Backport tool documentation

elasticsearchmachine pushed a commit that referenced this pull request Mar 21, 2025
#124594) | Fix ignores malformed testcase (#125337) | Fix offsets not recording duplicate values (#125354) (#125440)

* Natively store synthetic source array offsets for numeric fields (#124594)

This patch builds on the work in #122999 and #113757 to natively store
array offsets for numeric fields instead of falling back to ignored source
when `source_keep_mode: arrays`.

(cherry picked from commit 376abfe)

# Conflicts:
#	server/src/main/java/org/elasticsearch/index/IndexVersions.java
#	server/src/main/java/org/elasticsearch/index/mapper/NumberFieldMapper.java

* Fix ignores malformed testcase (#125337)

Fix and unmute testSynthesizeArrayRandomIgnoresMalformed

(cherry picked from commit 2ff03ac)

# Conflicts:
#	muted-tests.yml

* Fix offsets not recording duplicate values (#125354)

Previously, when calculating the offsets, we just compared the values as-is
without any loss of precision. However, when the values were saved into doc
values and loaded in the doc values loader, they could have lost precision.
This meant that values that were not duplicates when calculating the
offsets could now be duplicates in the doc values loader. This interfered
with the de-duplication logic, causing incorrect values to be returned.

My solution is to apply the precision loss before calculating the offsets,
so that both the offsets calculation and the SortedNumericDocValues
de-duplication see the same values as duplicates.

(cherry picked from commit db73175)
@jordan-powers jordan-powers deleted the fix-ignores-malformed-testcase branch March 25, 2025 21:48
omricohenn pushed a commit to omricohenn/elasticsearch that referenced this pull request Mar 28, 2025
Fix and unmute testSynthesizeArrayRandomIgnoresMalformed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:StorageEngine/Mapping The storage related side of mappings Team:StorageEngine >test Issues or PRs that are addressing/adding tests v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] NativeArrayIntegrationTestCase testSynthesizeArrayRandomIgnoresMalformed failing

3 participants