[Feature Request]: Simplify schema fetching in ClickHouseIO by leveraging Java Client v2's built-in getTableSchema

[BentsiLeviav]

What would you like to happen?

Description

The current ClickHouseIO implementation manually fetches table schemas by executing DESCRIBE TABLE queries and parsing the results. However, the ClickHouse Java Client v2 already provides this functionality through Client.getTableSchema(String table, String database).

Current Implementation

public static TableSchema getTableSchema(
    String clickHouseUrl, String database, String table, Properties properties) {
  // ... setup client
  String query = "DESCRIBE TABLE " + quoteIdentifier(table);
  try (Records records = client.queryRecords(query).get()) {
    for (GenericRecord record : records) {
      // Manually parse name, type, default_type, default_expression
      // Build TableSchema.Column objects
    }
  }
  // Return TableSchema
}

Java Client v2 Provides This

The Java Client v2 already has:

Client client = new Client.Builder()
    .addEndpoint(clickHouseUrl)
    .setUsername(user)
    .setPassword(password)
    .setDefaultDatabase(database)
    .build();

// Built-in method returns com.clickhouse.client.api.metadata.TableSchema
TableSchema schema = client.getTableSchema(table, database);

Proposed Solution

Leverage the client's built-in getTableSchema() method and convert the result to Beam's org.apache.beam.sdk.io.clickhouse.TableSchema:

public static TableSchema getTableSchema(
    String clickHouseUrl, String database, String table, Properties properties) {
  try (Client client = createClient(clickHouseUrl, database, properties)) {
    com.clickhouse.client.api.metadata.TableSchema chSchema = client.getTableSchema(table, database);
    return convertToBeamTableSchema(chSchema);
  }
}

Benefits

1. Less code: Remove manual DESCRIBE query execution and parsing logic
2. Better compatibility: Client handles version-specific differences (e.g., ClickHouse 25.x backticks in tuple types)
3. Reduced maintenance: Schema parsing logic maintained by ClickHouse team
4. Consistency: Use the same schema representation as other Java Client v2 operations

Considerations

• Need to map between com.clickhouse.client.api.metadata.TableSchema and Beam's org.apache.beam.sdk.io.clickhouse.TableSchema
• The tuple preprocessing logic may no longer be needed if the client handles it

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[BentsiLeviav]

.set-labels clickhouse,'new feature','awaiting triage', io,PS

[github-actions]

Label cannot be managed because it does not exist in the repo. Please check your spelling.