[Feature] Add StarRocks Driver Support #8454

EdwardArchive · 2025-12-05T10:49:16Z

Issue : #6839
Related PR : #8390 (priveus)

PR: StarRocks OLAP Connector (Read-Only)

What is StarRocks

https://www.starrocks.io/ is an open-source, high-performance OLAP database designed for real-time analytics. It provides a MySQL-compatible interface with columnar storage
and vectorized query execution, making it ideal for large-scale data analytics workloads.

Summary

Add StarRocks as an OLAP connector to directly query existing StarRocks tables. This is a read-only OLAP connector - Model Execution is not supported.

Changes

1. Backend - StarRocks Driver (`runtime/drivers/starrocks/`) - 5 files

File	Status	Changes
`starrocks.go`	Added	Connection logic, database field optional, OLAP interface only
`olap.go`	Added	Query execution with fully qualified table names
`information_schema.go`	Added	Table/column metadata retrieval with catalog support
`utils.go`	Added	Utility functions for StarRocks-specific SQL
`starrocks_test.go`	Added	Test cases for connection and queries

2. Backend - Query Dialect Support (`runtime/queries/`) - 14 files

File	Status	Changes
`column_cardinality.go`	Modified	Added StarRocks dialect with `approx_count_distinct()`
`column_null_count.go`	Modified	Added StarRocks dialect support
`column_desc_stats.go`	Modified	Added StarRocks with `percentile_approx()`, `stddev_samp()`
`column_time_range.go`	Modified	Added StarRocks dialect support
`column_time_grain.go`	Modified	Added StarRocks-specific time functions
`column_topk.go`	Modified	Added StarRocks dialect support
`column_rug_histogram.go`	Modified	Returns empty result for StarRocks
`column_numeric_histogram.go`	Modified	Returns empty result for StarRocks
`column_timeseries.go`	Modified	Returns empty result for StarRocks (no TEMP TABLE)
`table_head.go`	Modified	Added StarRocks to `supportedTableHeadDialects`
`table_cardinality.go`	Modified	Added StarRocks dialect support
`sqlutil.go`	Modified	Added StarRocks to dialect utilities
`resource_watermark.go`	Modified	Added StarRocks dialect support

Query Support Summary:

Query	Status	Notes
TableHead, TableColumns, TableCardinality	Full
ColumnCardinality, ColumnNullCount, ColumnTopK	Full
ColumnDescriptiveStatistics	Full	`percentile_approx()`, `stddev_samp()`
ColumnTimeRange, ColumnTimeGrain	Full	StarRocks-specific SQL
ColumnRugHistogram, ColumnNumericHistogram	Empty	Performance issues
ColumnTimeseries	Empty	TEMP TABLE not supported

3. Backend - Other (`runtime/`, `cli/`) - 2 files

File	Status	Changes
`runtime/drivers/olap.go`	Modified	StarRocks dialect registration
`cli/cmd/runtime/start.go`	Modified	StarRocks driver initialization

4. Frontend (`web-common/`, `web-local/`) - 8 files

File	Status	Changes
`web-common/src/features/sources/modal/yupSchemas.ts`	Modified	Database field optional (matches backend)
`web-common/src/features/sources/modal/constants.ts`	Modified	StarRocks connector constants
`web-common/src/features/sources/modal/icons.ts`	Modified	StarRocks icon import
`web-common/src/features/connectors/connectors-utils.ts`	Modified	StarRocks connector utilities
`web-common/src/features/connectors/connector-icon-mapping.ts`	Modified	StarRocks icon mapping
`web-common/src/components/icons/connectors/StarRocks.svelte`	Added	StarRocks icon component
`web-common/src/components/icons/connectors/StarRocksIcon.svelte`	Added	StarRocks icon variant
`web-local/src/routes/.../connector/starrocks/[name]/[database]/[schema]/[table]/+page.svelte`	Added	StarRocks table page
`web-local/src/routes/.../connector/starrocks/+page.ts`	Added	StarRocks connector page route

5. Documentation - 1 file

File	Status	Changes
`docs/docs/build/connectors/olap/starrocks.md`	Added	StarRocks OLAP connector documentation

StarRocks Naming Mapping

Rill API Parameter	StarRocks Concept	Example
`database`	Catalog	`default_catalog`, `iceberg_catalog`
`databaseSchema`	Database	`tpcds_100gb`, `my_database`
`table`	Table	`store_sales`, `customer`

Tested

Connection: default_catalog, external catalogs (Iceberg/Hive), DSN format
Table browsing: list tables, preview data, column metadata
Profiling: cardinality, null count, statistics, TopK, time range/grain
UI: connector modal, optional database field, table preview

Known Limitations

No Model Execution: Read-only OLAP connector only
No Histogram: Returns empty results due to performance issues
No Timeseries: Returns empty results (TEMP TABLE not supported)
MySQL Protocol: Uses StarRocks MySQL-compatible protocol (port 9030)
default_catalog on DSN : MySQL don't have Catalog

EdwardArchive · 2025-12-05T13:55:57Z

I just working on it find some edge case

- Add StarRocks support for ColumnRugHistogram with generate_series() - Add StarRocks support for ColumnNumericHistogram (FD & Diagnostic methods) - Add CTE-based ColumnTimeseries for StarRocks (no TEMP TABLE required) - Add helper functions: rangeNumbersEnd(), isNonNullFinite() for StarRocks - Handle reserved keywords (values, range) with backtick escaping

…alect

…ndling, connection timeouts, and error messages

EdwardArchive · 2025-12-07T13:15:04Z

@k-anshul
Hi, Can you help review this change so we can move it forward?

runtime/drivers/olap.go

runtime/queries/column_cardinality.go

runtime/queries/column_numeric_histogram.go

runtime/drivers/starrocks/olap.go

runtime/queries/column_time_range.go

runtime/metricsview/executor/executor_validate.go

runtime/drivers/starrocks/utils.go

EdwardArchive · 2025-12-08T07:45:25Z

Thank you for the Hard works!
I'll fixed it

k-anshul · 2025-12-08T09:16:33Z

Also tagging @ericpgreen2 for review on frontend changes.

- addressed multiple review comments - cleaned up import and minor issues

- Remove unused 'strings' import from starrocks.go - Fix undefined variable 't' in olap.go StarRocks case - Update date_trunc comment to reference validation location

…tarRocks dialect

…lorer - Fix build compilation errors - Ensure selected catalog (Database), schema (DatabaseSchema), and table are properly propagated when creating metrics from Data Explorer

EdwardArchive · 2025-12-09T01:43:10Z

Hi @k-anshul,

Thank you so much for taking the time to review my code — I really appreciate it, especially since it was likely during the weekend or early morning hours.

I've addressed all of your review comments. Please let me know if you have any questions or concerns.

Have a great day!

runtime/drivers/starrocks/information_schema.go

runtime/drivers/starrocks/olap.go

runtime/drivers/starrocks/starrocks.go

runtime/queries/column_desc_stats.go

runtime/queries/column_null_count.go

runtime/queries/column_numeric_histogram.go

runtime/metricsview/executor/executor_validate.go

k-anshul · 2025-12-09T06:45:31Z

Hey @EdwardArchive

Thanks for your work here. I added some more comments. Mostly should be minor.
Please also take a look at the failing checks.

EdwardArchive · 2025-12-09T10:19:02Z

Hey @EdwardArchive

Thanks for your work here. I added some more comments. Mostly should be minor. Please also take a look at the failing checks.

Hey @k-anshul
I'll take care of this. thanks again.

lovincyrus

For this pass, I'd proceed without the frontend since there are ongoing updates to how the form is rendered in #8467

We should avoid creating a separate AddStarRocksForm, as it is an anti-pattern that we do not want to maintain in the codebase.

web-common/src/features/sources/modal/AddStarRocksForm.svelte

lovincyrus

Nice job removing the StarRocks form! I see some remnants of StarRocks frontend integration.

We should limit this branch's scope to the StarRocks driver changes and exclude any frontend-related changes.

web-common/src/features/sources/modal/yupSchemas.ts

web-common/src/features/sources/modal/icons.ts

web-common/src/features/sources/errors/errors.ts

web-common/src/features/connectors/connectors-utils.ts

web-local/src/routes/(application)/(workspace)/connector/starrocks/+page.ts

...application)/(workspace)/connector/starrocks/[name]/[database]/[schema]/[table]/+page.svelte

… + Fix external catalog information schema check

EdwardArchive · 2025-12-18T10:39:49Z

@lovincyrus
Thanks a lot for your review!
I’ve updated the code as per your suggestions

web-common/src/features/sources/modal/AddDataFormManager.ts

web-common/src/features/sources/modal/constants.ts

web-common/src/features/sources/modal/AddDataForm.svelte

web-common/src/features/sources/modal/AddClickHouseForm.svelte

EdwardArchive · 2025-12-19T18:32:07Z

Sorry for the repeated review.
I've synced all changes to the main branch, excluding the icon-related frontend updates.
Thank you once again for your thorough feedback.

lovincyrus · 2025-12-22T08:37:38Z

@lovincyrus, given your ownership of the connector forms, can you please do the first frontend reviews of this code? For now, I've removed myself as reviewer and tagged you. Once the PR has your sign-off, then please request a review from me and I'll do a final review. Thank you!

This branch is now focused on supporting the StarRocks driver and adding some icons. Requesting the final +1 @ericpgreen2

ericpgreen2

Thanks Cyrus. I agree with the decision to remove the frontend changes from this branch, as they can be addressed in a separate dedicated PR. I'll approve the PR from the frontend perspective :)

EdwardArchive · 2025-12-30T04:30:38Z

Hi @k-anshul, sorry for commenting at the end of the year

It looks like the frontend changes have passed successfully in this PR.

Could you please guide me on what additional steps are needed to move this PR toward merge?

Thank you!

k-anshul · 2025-12-30T06:32:44Z

Hey @EdwardArchive

Can you fix the prettier checks? You can run npx prettier --check "**/*" --write at the repo root.

EdwardArchive · 2025-12-30T08:32:06Z

Oh.. thanks!

k-anshul · 2025-12-30T10:43:15Z

LGTM.

Thank you for your contribution @EdwardArchive

Already approved in comments above

EdwardArchive · 2026-01-10T06:40:00Z

Hi @k-anshul, @lovincyrus

Thanks for your hard work again! And happy new year!

I'd like to start working on the frontend portion for StarRocks integration.
Before I begin, I wanted to check with you:

Should I create a new issue to track the frontend work, or is it okay
to just open a follow-up PR directly (referencing this PR)?

Also, is #8467 at a point where I can proceed with the form integration?

Thanks!

k-anshul · 2026-01-12T04:35:31Z

Hey @EdwardArchive

Thank you for your interest in contributing to Rill.
Before we dive into frontend integration, I recommend prioritizing stability improvements for the driver.
I did some more testing and found following issues:

The varchar results are still being returned in binary. We discussed this earlier as well : Add StarRocks Driver Support #8390 (comment) May need to borrow changes from MySQL driver :

rill/runtime/drivers/mysql/olap.go

Line 224 in 34b9d21

type mysqlRows struct {
Some APIs are returning incorrect results. It might make sense to add unit tests for starrocks. You can take a reference from unit tests in runtime/queries for other drivers like clickhouse.

EdwardArchive · 2026-01-12T08:43:47Z

Hi @k-anshul,

Sure, I'll work for the stability improvements for the driver.
thanks for the testing.

EdwardArchive added 2 commits December 3, 2025 19:58

only olap connector(not incloud model)

fbde1a6

fix : message "not available for dialect 'starrocks'" & starrocks icon

ad513a0

EdwardArchive mentioned this pull request Dec 5, 2025

Add StarRocks Driver Support #8390

Closed

EdwardArchive added 4 commits December 5, 2025 23:07

feat(starrocks): add support for resolving timestamps in StarRocks di…

692f7c6

…alect

feat(starrocks): enhance StarRocks connector with improved context ha…

4c09870

…ndling, connection timeouts, and error messages

feat(starrocks): add StarRocks form and error handling to AddDataForm

c47dbd9

Update docs

0a3dbc5

k-anshul requested changes Dec 8, 2025

View reviewed changes

k-anshul requested a review from ericpgreen2 December 8, 2025 09:16

EdwardArchive added 9 commits December 8, 2025 21:22

fix: update code based on PR review feedback

ed2301b

- addressed multiple review comments - cleaned up import and minor issues

fix: apply fixes from PR comments

ffdce1b

fix: resolve build errors

948fa2a

- Remove unused 'strings' import from starrocks.go - Fix undefined variable 't' in olap.go StarRocks case - Update date_trunc comment to reference validation location

fix: update reserved keyword handling and SQL name sanitization for S…

4de5c64

…tarRocks dialect

fix: improve dry run handling in Query method and simplify SQL min query

2987351

fix: correct safeName call in Druid maxSQL query

003001e

fix: use equality check instead of Before for StarRocks time range bins

ae95f12

fix: resolve build errors and metrics context propagation in Data Exp…

436fa8c

…lorer - Fix build compilation errors - Ensure selected catalog (Database), schema (DatabaseSchema), and table are properly propagated when creating metrics from Data Explorer

change test code and fix on AddStarRocksFrom

b83c569

EdwardArchive requested a review from k-anshul December 9, 2025 01:43

k-anshul requested changes Dec 9, 2025

View reviewed changes

EdwardArchive added 2 commits December 9, 2025 19:53

Apply minor fixes from code review comments

f4cd71f

Refactor: removed unused code as suggested by Git CI

0d75a03

lovincyrus reviewed Dec 15, 2025

View reviewed changes

web-common/src/features/sources/modal/AddStarRocksForm.svelte Outdated Show resolved Hide resolved

[Fix] avoid creating a separate AddStarRocksForm

2c14c1c

EdwardArchive requested a review from lovincyrus December 16, 2025 12:09

lovincyrus requested changes Dec 18, 2025

View reviewed changes

[Enhancement] Limit StarRocks Frontend until integration is supported…

3a3bd35

… + Fix external catalog information schema check

EdwardArchive requested a review from lovincyrus December 18, 2025 10:34

lovincyrus previously requested changes Dec 19, 2025

View reviewed changes

EdwardArchive added 2 commits December 20, 2025 03:28

[Enhancement] Restrict changes to Backend components only (StarRocks)

4b37cbc

fix: format OLAP_DRIVERS_WITHOUT_MODELING array for consistency

65488dc

EdwardArchive requested a review from lovincyrus December 19, 2025 18:32

[Enhancement] Handle StarRocks icon in a more general way

78df8ec

EdwardArchive force-pushed the feat_starrocks_olap_connector branch from 17d5118 to 78df8ec Compare December 21, 2025 09:46

lovincyrus requested a review from ericpgreen2 December 22, 2025 08:36

ericpgreen2 approved these changes Dec 22, 2025

View reviewed changes

fix: Prettier checks

3a69ee6

k-anshul removed the request for review from lovincyrus December 30, 2025 10:43

k-anshul approved these changes Dec 30, 2025

View reviewed changes

k-anshul merged commit e46c9dd into rilldata:main Dec 30, 2025
10 of 13 checks passed

EdwardArchive mentioned this pull request Jan 12, 2026

fix: Stable StarRocks Driver and Add test code #8619

Open

8 tasks

[Feature] Add StarRocks Driver Support #8454

[Feature] Add StarRocks Driver Support #8454

Uh oh!

Conversation

EdwardArchive commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR: StarRocks OLAP Connector (Read-Only)

What is StarRocks

Summary

Changes

1. Backend - StarRocks Driver (runtime/drivers/starrocks/) - 5 files

2. Backend - Query Dialect Support (runtime/queries/) - 14 files

3. Backend - Other (runtime/, cli/) - 2 files

4. Frontend (web-common/, web-local/) - 8 files

5. Documentation - 1 file

StarRocks Naming Mapping

Tested

Known Limitations

Uh oh!

EdwardArchive commented Dec 5, 2025

Uh oh!

EdwardArchive commented Dec 7, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EdwardArchive commented Dec 8, 2025

Uh oh!

k-anshul commented Dec 8, 2025

Uh oh!

EdwardArchive commented Dec 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

k-anshul commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EdwardArchive commented Dec 9, 2025

Uh oh!

lovincyrus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lovincyrus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EdwardArchive commented Dec 18, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EdwardArchive commented Dec 19, 2025

Uh oh!

lovincyrus commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ericpgreen2 left a comment

Choose a reason for hiding this comment

EdwardArchive commented Dec 5, 2025 •

edited

Loading

1. Backend - StarRocks Driver (`runtime/drivers/starrocks/`) - 5 files

2. Backend - Query Dialect Support (`runtime/queries/`) - 14 files

3. Backend - Other (`runtime/`, `cli/`) - 2 files

4. Frontend (`web-common/`, `web-local/`) - 8 files

k-anshul commented Dec 9, 2025 •

edited

Loading

lovincyrus commented Dec 22, 2025 •

edited

Loading

k-anshul commented Jan 12, 2026 •

edited

Loading