Skip to content

[FEATURE] SQL query support for Analytics engine integration #5248

@dai-chen

Description

@dai-chen

Is your feature request related to a problem?

The Analytics engine integration requires ANSI SQL support to enable SQL queries against non-Lucene indices. However, the unified query API currently only supports PPL V3. The PPL integration (tracked in the sibling issue) establishes the query routing, execution engine, and response formatting infrastructure — this issue covers the additional work needed for SQL support.

Technical requirements:

  • ANSI SQL queries against non-Lucene indices return correct results through _plugins/_sql endpoint
  • SQL explain API (_plugins/_sql/_explain) returns the logical plan
  • OpenSearch full-text search functions (match, match_phrase, query_string) available in ANSI SQL syntax
  • Existing V2 SQL queries are unaffected (backward compatible, ANSI SQL is opt-in)
  • Same response format, error handling, and observability as the PPL unified path

What solution would you like?

Unified query API:

  1. Enable Calcite SQL in unified query API: Add the Calcite native SQL parser path (SqlParserSqlValidatorSqlToRelConverterRelNode) in UnifiedQueryPlanner with proper SQL conformance, case sensitivity, and identifier handling
  2. Register full-text search functions: Register search functions (match, match_phrase, query_string) with same signatures and return types

SQL plugin:

  1. Integrate with SQL REST endpoint: Add ANSI SQL mode routing in RestSqlAction (via mode parameter or auto-detection) through the unified query pipeline established by the PPL issue, with explain support and backward compatibility for existing V2 SQL queries.
  2. Integration tests: End-to-end ITs verifying ANSI SQL query, explain, full-text search functions, response format, and breaking changes in existing V2 SQL queries.What alternatives have you considered?

See parent issue #5246 for the design comparison of Option A (Query Delegation), Option B (Unified Query Pipeline), and Option C (Calcite Schema Adapter).

Do you have any additional context?

  • PoC PR: dai-chen/sql-1#10 — demonstrates ANSI SQL RelNode generation via Calcite's native parser in UnifiedQueryPlanner
  • Depends on PPL Analytics engine integration (sibling issue) for query routing, execution engine, and response formatting infrastructure

Metadata

Metadata

Assignees

Labels

MetaMeta issue, not directly linked to a PRSQLenhancementNew feature or request

Type

No type

Projects

Status

New

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions