Skip to content

Conversation

@benwtrent
Copy link
Member

When utilizing synthetic source with nested fields, we attempt to rebuild the child values in addition to all the parent values.

While this generally works well, its potential that certain values might be missing from various child docs. Consequently, we will attempt to iterate the vector values strangely, resulting in seemingly missing values or potentially exceptions indicating EOFs.

closes: #122383

@benwtrent benwtrent added >bug auto-backport Automatically create backport pull requests when merged :Search Relevance/Vectors Vector search :StorageEngine/Mapping The storage related side of mappings v8.18.1 v8.19.0 v9.0.1 v9.1.0 v8.17.3 labels Feb 12, 2025
@elasticsearchmachine elasticsearchmachine added Team:StorageEngine Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Feb 12, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @benwtrent, I've created a changelog YAML for you.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Copy link
Contributor

@lkts lkts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from my side

@benwtrent benwtrent added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Feb 12, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @benwtrent, I've updated the changelog YAML for you.

@elasticsearchmachine elasticsearchmachine merged commit f5c901e into elastic:main Feb 12, 2025
17 checks passed
@benwtrent benwtrent deleted the bugfix/fix-nested-dense-vector-synthetic-behavior branch February 12, 2025 21:20
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.18 Commit could not be cherrypicked due to conflicts
8.x Commit could not be cherrypicked due to conflicts
9.0
8.17 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 122425

benwtrent added a commit to benwtrent/elasticsearch that referenced this pull request Feb 12, 2025
…lds (elastic#122425)

When utilizing synthetic source with nested fields, we attempt to
rebuild the child values in addition to all the parent values.

While this generally works well, its potential that certain values might
be missing from various child docs. Consequently, we will attempt to
iterate the vector values strangely, resulting in seemingly missing
values or potentially exceptions indicating EOFs.

closes: elastic#122383
benwtrent added a commit to benwtrent/elasticsearch that referenced this pull request Feb 12, 2025
…lds (elastic#122425)

When utilizing synthetic source with nested fields, we attempt to
rebuild the child values in addition to all the parent values.

While this generally works well, its potential that certain values might
be missing from various child docs. Consequently, we will attempt to
iterate the vector values strangely, resulting in seemingly missing
values or potentially exceptions indicating EOFs.

closes: elastic#122383
(cherry picked from commit f5c901e)
@benwtrent
Copy link
Member Author

💚 All backports created successfully

Status Branch Result
8.x
8.18
8.17

Questions ?

Please refer to the Backport tool documentation

benwtrent added a commit to benwtrent/elasticsearch that referenced this pull request Feb 12, 2025
…lds (elastic#122425)

When utilizing synthetic source with nested fields, we attempt to
rebuild the child values in addition to all the parent values.

While this generally works well, its potential that certain values might
be missing from various child docs. Consequently, we will attempt to
iterate the vector values strangely, resulting in seemingly missing
values or potentially exceptions indicating EOFs.

closes: elastic#122383
(cherry picked from commit f5c901e)
elasticsearchmachine pushed a commit that referenced this pull request Feb 12, 2025
…lds (#122425) (#122434)

When utilizing synthetic source with nested fields, we attempt to
rebuild the child values in addition to all the parent values.

While this generally works well, its potential that certain values might
be missing from various child docs. Consequently, we will attempt to
iterate the vector values strangely, resulting in seemingly missing
values or potentially exceptions indicating EOFs.

closes: #122383
elasticsearchmachine pushed a commit that referenced this pull request Feb 12, 2025
…lds (#122425) (#122437)

When utilizing synthetic source with nested fields, we attempt to
rebuild the child values in addition to all the parent values.

While this generally works well, its potential that certain values might
be missing from various child docs. Consequently, we will attempt to
iterate the vector values strangely, resulting in seemingly missing
values or potentially exceptions indicating EOFs.

closes: #122383
(cherry picked from commit f5c901e)
elasticsearchmachine pushed a commit that referenced this pull request Feb 12, 2025
…lds (#122425) (#122436)

When utilizing synthetic source with nested fields, we attempt to
rebuild the child values in addition to all the parent values.

While this generally works well, its potential that certain values might
be missing from various child docs. Consequently, we will attempt to
iterate the vector values strangely, resulting in seemingly missing
values or potentially exceptions indicating EOFs.

closes: #122383
(cherry picked from commit f5c901e)
elasticsearchmachine pushed a commit that referenced this pull request Feb 13, 2025
…or fields (#122425) (#122435)

* Fix synthetic source bug that would mishandle nested dense_vector fields (#122425)

When utilizing synthetic source with nested fields, we attempt to
rebuild the child values in addition to all the parent values.

While this generally works well, its potential that certain values might
be missing from various child docs. Consequently, we will attempt to
iterate the vector values strangely, resulting in seemingly missing
values or potentially exceptions indicating EOFs.

closes: #122383
(cherry picked from commit f5c901e)

* fixing backport
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport pending >bug :Search Relevance/Vectors Vector search :StorageEngine/Mapping The storage related side of mappings Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch Team:StorageEngine v8.17.3 v8.18.1 v8.19.0 v9.0.1 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nested synthetic source fails to load vectors correctly

3 participants