Skip to content

Conversation

@afoucret
Copy link
Contributor

@afoucret afoucret commented Jul 1, 2025

This is a POC implementation for a function that allow to embed text as dense vector in ES|QL

TODO list:

  • Implement the InferenceOperator
  • Implement InferenceFunction
    • Basic function implementation (signature and parsing)
    • Add dimensions resolution for the Logical plam
  • Implement logical and physical plan
    • Logical plan
    • Physical plan
    • Logical to physical mapping
    • Logical plan analysis
    • Logical plan optimization (Already implemented for all InferencePlan instances)
    • Physical planning
  • Implement dense vector embedding function rewrite to a logical plan
  • Implement foldable evaluation
  • Implement inference resolution for function

Also missing (not in the scope of the POC):

  • Tests
    • Unit
      • Analyzer, Logical plan optimization
    • Java REST and CSV tests
  • Good documentation

Gaps:

  • DENSE_VECTOR datatype does not support multivalued fields.

@afoucret afoucret force-pushed the esql-dense-vector-embedding branch from f4be9cf to 3d075fd Compare July 1, 2025 11:52
@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2025

🔍 Preview links for changed docs:

🔔 The preview site may take up to 3 minutes to finish building. These links will become live once it completes.

@afoucret afoucret force-pushed the esql-dense-vector-embedding branch from 244a0c9 to ae208a6 Compare July 2, 2025 08:44
@github-actions
Copy link
Contributor

github-actions bot commented Jul 2, 2025

@afoucret afoucret force-pushed the esql-dense-vector-embedding branch from e09409d to 03c6110 Compare July 2, 2025 16:22
@afoucret afoucret closed this Jul 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants