ESQL - TO_DENSE_VECTOR function and additional casting

[carlosdelest]

Adds a TO_DENSE_VECTOR conversion function to convert to dense_vectors:

• Multivalued numbers: [0.1, 0.2, 0.3], [1, 2, 3]
• Hex strings: "532af3e"

Also, performs automatic conversion of vector values for VectorFunction instances on the Analyzer:

FROM test
| WHERE knn(vector_field, [0, 1, 2])

or

FROM test
| EVAL s = v_hamming("asdf2342", [0, 1, 2])

The Analyzer will convert automatically keyword fields via castStringLiteral, and add a TO_DENSE_VECTOR function for numeric values.

In order to use a reference attribute for a vector function, we need to use TO_DENSE_VECTOR explicitly, just like other types like IP:

FROM test
| EVAL query = TO_DENSE_VECTOR([1, 2, 3])
| WHERE knn(vector_field, query)

For now, hex strings gets translated to byte dense_vectors. We can create a follow up function TO_BIT_DENSE_VECTOR for using hex strings for bit vectors, or add an optional parameter to TO_DENSE_VECTOR.

[github-actions]

🔍 Preview links for changed docs

• docs/reference/query-languages/esql/kibana/docs/functions/to_dense_vector.md

[github-actions]

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

• Check out the cumulative docs guidelines
• Reach out in the #docs Slack channel

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[kderusso]

Changes look good to me, nice work!

[tteofili]

nice work Carlos, LGTM!

[afoucret]

Should we add text as well (if it is loaded from an ES text field)?

[carlosdelest]

I would say no - it's ok to allow for hex string constants, but I don't think they should be analyzed in any way or retrieved from an analyzed field.

If any time this is needed, they can be converted via the ::keyword casting operator.