feat(clickhouse-driver): add intelligent DDL handling for unsupported operations

[Greenystuff]

Check List

• Tests have been run in packages where changes made if available
• Linter has been run for changed code
• Tests for the changes have been added if not covered yet
• Docs have been added / updated if required

Issue Reference this PR resolves

No specific issue reference - this is a proactive enhancement to improve ClickHouse driver compatibility

Description of Changes Made

This PR adds intelligent DDL (Data Definition Language) handling to the ClickHouse driver to handle operations that are not natively supported by certain ClickHouse storage engines, particularly the Log engine.

Key Changes:

• Enhanced DDL detection with comprehensive keyword support (CREATE, ALTER, DROP, TRUNCATE, RENAME, ATTACH, DETACH, GRANT, REVOKE)
• Implemented storage engine compatibility layer for ALTER TABLE ADD COLUMN on Log engine
• Added specific ClickHouse error code handling with detailed error messages
• Comprehensive test coverage for DDL operations and mixed DDL/DML scenarios

Technical Implementation:

• DDL statements now use connection.command() instead of connection.query() to avoid JSON parsing errors
• For ALTER TABLE ADD COLUMN on Log engine: automatically recreates table with new schema, copies data, drops old table, renames new table
• Enhanced error messages include ClickHouse error codes and query context for better debugging
• Backward-compatible enhancement with no breaking changes

Testing:

• All existing tests pass
• New integration tests for DDL handling pass
• Mixed DDL/DML operations work correctly
• ALTER TABLE ADD COLUMN compatibility layer works as expected

Problem Solved:
The ClickHouse Log engine doesn't support ALTER TABLE ADD COLUMN operations, returning error code 48 (NOT_IMPLEMENTED). This PR implements a compatibility layer that automatically handles this limitation by recreating the table with the new schema, ensuring data integrity while maintaining the expected behavior.

[Greenystuff]

Hi there! 👋

I'm a FullStack developer with 10+ years of experience, currently working for a company that specializes in Business Intelligence using Cube.js and ClickHouse.

This PR addresses a real-world issue we encountered in production with Cube's pre-aggregation feature. We were getting this error:

Error: Query failed: SyntaxError: Unexpected end of JSON input; query id: 53b7c702-e773-496a-b5c0-a6f21b5377a5

The problem occurs when Cube tries to execute DDL operations (like CREATE TABLE or ALTER TABLE) through the ClickHouse driver. ClickHouse itself doesn't support using .query() for DDL statements - it requires .command() instead, and the driver was incorrectly trying to parse the response as JSON even though DDL statements don't return data.
Sources :
https://clickhouse.com/docs/integrations/javascript#query-method
https://clickhouse.com/docs/integrations/javascript#command-method

What this PR does:

• Intelligently detects DDL operations and uses .command() instead of .query() as required by ClickHouse
• Provides better error messages for ClickHouse-specific issues
• Implements a compatibility layer for storage engines that don't support certain DDL operations
• Maintains full backward compatibility

Why I'm contributing:
I believe in giving back to the open-source community, especially when we can solve real problems that others might face. This enhancement should help anyone using Cube.js with ClickHouse, particularly in production environments.

Testing:
Our unit tests confirmed that the current driver cannot handle DDL operations with .query() - ClickHouse simply doesn't support it. This PR fixes that fundamental incompatibility.

I'm open to any discussion about this implementation and happy to iterate based on feedback. Feel free to ask questions or suggest improvements!

P.S. This is my first PR ever, so please be gentle! But seriously, don't hesitate to question anything - I'm here to learn and improve. Looking forward to contributing more to this amazing project!

[Greenystuff]

I had to build it with a custom docker image to test It before asking review and I can tell you that okay now. With this fix there is no longer parsing error
I'm using this custom docker image and I can store pre-aggregates in my source database on Click House.
I have to let the dev mode true if I want that internals aggregation works. I don't know if I'm wrong with the usage but It works perfectly

[igorlukanin]

Hi @Greenystuff, thanks for the contribution!

This PR addresses a real-world issue we encountered in production with Cube's pre-aggregation feature. We were getting this error:

Error: Query failed: SyntaxError: Unexpected end of JSON input; query id: 53b7c702-e773-496a-b5c0-a6f21b5377a5

The problem occurs when Cube tries to execute DDL operations (like CREATE TABLE or ALTER TABLE) through the ClickHouse driver.

Sorry if I'm missing something here—but when exactly does Cube try to run ALTER TABLE ADD COLUMN query?

[Greenystuff]

Hi !

It's when the driver wants to store my pre-aggregations on ClickHouse in dev mode, I hade this error and not any table in ClicHouse.
I've read in some places that the support for pre-aggregation stored on ClickHouse is no longer maintained, which might justify your question ^^
I've recently made a lot of progress on using Cube and I'm currently doing my pre-aggregations on ClickHouse without any problems but I'm forced to stay in dev mode. Maybe I'm not using it correctly and I'm open to any help or corrections.