Update JDBC docs for V2 #3071
+273
−46
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,236 @@ | ||
| --- | ||
| sidebar_label: JDBC V2 | ||
| sidebar_position: 4 | ||
| keywords: [clickhouse, java, jdbc, driver, integrate] | ||
| description: ClickHouse JDBC driver V2 | ||
| slug: /en/integrations/java/jdbc-v2 | ||
| --- | ||
|
|
||
| import Tabs from '@theme/Tabs'; | ||
| import TabItem from '@theme/TabItem'; | ||
| import CodeBlock from '@theme/CodeBlock'; | ||
|
|
||
|
|
||
| # JDBC Driver | ||
|
|
||
| `clickhouse-jdbc` implements the standard JDBC interface using [client-v2](/docs/en/integrations/language-clients/java/client-v2.md). | ||
| We recommend using [client-v2](/docs/en/integrations/language-clients/java/client-v2.md) directly if performance/direct access is critical. | ||
|
|
||
| ## Changes from 0.7.x | ||
| In 0.8 we tried to make the driver more strictly follow the JDBC specification, so there are some removed features that may affect you: | ||
|
|
||
| | Old Feature | Notes | | ||
| |---------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | Transaction Support | Early versions of the driver only **simulated** transaction support, which could have unexpected results. | | ||
| | Response Column Renaming | ResultSets were mutable - for efficiency sake they're now read-only | | ||
| | Multi-Statement SQL | Multi-statement support was only **simulated**, now it strictly follows 1:1 | | ||
| | Named Parameters | Not part of the JDBC spec | | ||
| | Stream-based PreparedStatements | Early version of the driver allowed for non-jdbc usage of PreparedStatements - if you desire such options, we recommend looking at [Client-V2](/docs/en/integrations/language-clients/java/client-v2.md) and its [examples](https://github.com/ClickHouse/clickhouse-java/tree/main/examples/client-v2). | | ||
|
|
||
| ### Handling Dates, Times, and Timezones | ||
|
||
| java.sql.Date, java.sql.Time, and java.sql.Timestamp can complicate how Timezones are calculated - though they're of course supported, | ||
| you may want to consider using the [java.time](https://docs.oracle.com/javase/8/docs/api/java/time/package-summary.html) package for new code. [ZonedDateTime](https://docs.oracle.com/javase/8/docs/api/java/time/ZonedDateTime.html) and | ||
| [OffsetDateTime](https://docs.oracle.com/javase/8/docs/api/java/time/OffsetDateTime.html) are great replacements for java.sql.Timestamp, | ||
| and [LocalDateTime](https://docs.oracle.com/javase/8/docs/api/java/time/LocalDateTime.html) could be used for java.sql.Date and java.sql.Time (though the JDBC driver will assume local | ||
| timezone if no calendar is provided). | ||
|
|
||
| :::note | ||
| `Date` is stored without timezone, while `DateTime` is stored with timezone. This can lead to unexpected results if you're not careful. | ||
| ::: | ||
|
|
||
| ## Environment requirements | ||
|
|
||
| - [OpenJDK](https://openjdk.java.net) version >= 8 | ||
|
|
||
| ### Setup | ||
|
|
||
| <Tabs groupId="jdbc-base-dependencies"> | ||
| <TabItem value="maven" label="Maven" > | ||
|
|
||
| ```xml | ||
| <!-- https://mvnrepository.com/artifact/com.clickhouse/clickhouse-jdbc --> | ||
| <dependency> | ||
| <groupId>com.clickhouse</groupId> | ||
| <artifactId>clickhouse-jdbc</artifactId> | ||
| <version>0.8.0</version> | ||
| <classifier>shaded-all</classifier> | ||
| </dependency> | ||
| ``` | ||
|
|
||
| </TabItem> | ||
| <TabItem value="gradle-kt" label="Gradle (Kotlin)"> | ||
|
|
||
| ```kotlin | ||
| // https://mvnrepository.com/artifact/com.clickhouse/clickhouse-jdbc | ||
| implementation("com.clickhouse:clickhouse-jdbc:0.8.0:shaded-all") | ||
| ``` | ||
| </TabItem> | ||
| <TabItem value="gradle" label="Gradle"> | ||
|
|
||
| ```groovy | ||
| // https://mvnrepository.com/artifact/com.clickhouse/clickhouse-jdbc | ||
| implementation 'com.clickhouse:clickhouse-jdbc:0.8.0:shaded-all' | ||
| ``` | ||
|
|
||
| </TabItem> | ||
| </Tabs> | ||
|
|
||
| ## Configuration | ||
|
|
||
| **Driver Class**: `com.clickhouse.jdbc.ClickHouseDriver` | ||
|
|
||
| **URL Syntax**: `jdbc:(ch|clickhouse)[:<protocol>]://endpoint1[,endpoint2,...][/<database>][?param1=value1¶m2=value2][#tag1,tag2,...]`, for example: | ||
|
|
||
| - `jdbc:clickhouse:http://localhost:8123` | ||
| - `jdbc:clickhouse:https://localhost:8443?ssl=true` | ||
|
|
||
| **Connection Properties**: | ||
|
|
||
| Beyond standard JDBC properties, the driver supports the ClickHouse-specific properties offered by the underlying [client](/docs/en/integrations/language-clients/java/client-v2.md). | ||
| Where possible methods will return an `SQLFeatureNotSupportedException` if the feature is not supported. Other custom properties include: | ||
|
|
||
| | Property | Default | Description | | ||
| |----------------------------------|---------|----------------------------------------------------------------| | ||
| | `disable_frameworks_detection` | `true` | Disable frameworks detection for User-Agent | | ||
| | `jdbc_ignore_unsupported_values` | `false` | Suppresses `SQLFeatureNotSupportedException` | | ||
| | `clickhouse.jdbc.v1` | `false` | Use JDBC-V1 instead of JDBC-V2 | | ||
| | `default_query_settings` | `null` | Allows passing of default query settings with query operations | | ||
|
|
||
| ## Supported data types | ||
|
|
||
| JDBC Driver supports the same data formats as the underlying [client](/docs/en/integrations/language-clients/java/client-v2.md). | ||
|
|
||
| ## Creating Connection | ||
|
|
||
| ```java | ||
| String url = "jdbc:ch://my-server:8123/system"; | ||
|
|
||
| Properties properties = new Properties(); | ||
| DataSource dataSource = new DataSource(url, properties);//DataSource or DriverManager are the main entry points | ||
| try (Connection conn = dataSource.getConnection()) { | ||
| ... // do something with the connection | ||
| ``` | ||
|
|
||
| ## Supplying Credentials and Settings | ||
|
|
||
| ```java showLineNumbers | ||
| String url = "jdbc:ch://localhost:8123?jdbc_ignore_unsupported_values=true&socket_timeout=10"; | ||
|
|
||
| Properties info = new Properties(); | ||
| info.put("user", "default"); | ||
| info.put("password", "password"); | ||
| info.put("database", "some_db"); | ||
|
|
||
| //Creating a connection with DataSource | ||
| DataSource dataSource = new DataSource(url, info); | ||
| try (Connection conn = dataSource.getConnection()) { | ||
| ... // do something with the connection | ||
| } | ||
|
|
||
| //Alternate approach using the DriverManager | ||
| try (Connection conn = DriverManager.getConnection(url, info)) { | ||
| ... // do something with the connection | ||
| } | ||
| ``` | ||
|
|
||
| ## Simple Statement | ||
|
|
||
| ```java showLineNumbers | ||
|
|
||
| try (Connection conn = dataSource.getConnection(...); | ||
| Statement stmt = conn.createStatement()) { | ||
| ResultSet rs = stmt.executeQuery("select * from numbers(50000)"); | ||
| while(rs.next()) { | ||
| // ... | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Insert | ||
Paultagoras marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ```java showLineNumbers | ||
| try (PreparedStatement ps = conn.prepareStatement("INSERT INTO mytable VALUES (?, ?)")) { | ||
| ps.setString(1, "test"); // id | ||
| ps.setObject(2, LocalDateTime.now()); // timestamp | ||
| ps.addBatch(); | ||
| ... | ||
| ps.executeBatch(); // stream everything on-hand into ClickHouse | ||
| } | ||
| ``` | ||
|
|
||
Paultagoras marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ## Hikari | ||
|
|
||
| ```java showLineNumbers | ||
| // connection pooling won't help much in terms of performance, | ||
| // because the underlying implementation has its own pool. | ||
| // for example: HttpURLConnection has a pool for sockets | ||
| HikariConfig poolConfig = new HikariConfig(); | ||
| poolConfig.setConnectionTimeout(5000L); | ||
| poolConfig.setMaximumPoolSize(20); | ||
| poolConfig.setMaxLifetime(300_000L); | ||
| poolConfig.setDataSource(new ClickHouseDataSource(url, properties)); | ||
|
|
||
| try (HikariDataSource ds = new HikariDataSource(poolConfig); | ||
| Connection conn = ds.getConnection(); | ||
| Statement s = conn.createStatement(); | ||
| ResultSet rs = s.executeQuery("SELECT * FROM system.numbers LIMIT 3")) { | ||
| while (rs.next()) { | ||
| // handle row | ||
| log.info("Integer: {}, String: {}", rs.getInt(1), rs.getString(1));//Same column but different types | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## More Information | ||
Paultagoras marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| For more information, see our [GitHub repository](https://github.com/ClickHouse/clickhouse-java) and [Client-V2 documentation](/docs/en/integrations/language-clients/java/client-v2.md). | ||
|
|
||
|
|
||
| ## Troubleshooting | ||
| ### Logging | ||
| The driver uses [slf4j](https://www.slf4j.org/) for logging, and will use the first available implementation on the classpath. | ||
|
|
||
| ### Resolving JDBC Timeout on Large Inserts | ||
|
|
||
| When performing large inserts in ClickHouse with long execution times, you may encounter JDBC timeout errors like: | ||
|
|
||
| ```plaintext | ||
| Caused by: java.sql.SQLException: Read timed out, server myHostname [uri=https://hostname.aws.clickhouse.cloud:8443] | ||
| ``` | ||
| These errors can disrupt the data insertion process and affect system stability. To address this issue you may need to adjust a few timeout settings in the client's OS. | ||
|
|
||
| #### Mac OS | ||
|
|
||
| On Mac OS, the following settings can be adjusted to resolve the issue: | ||
|
|
||
| - `net.inet.tcp.keepidle`: 60000 | ||
| - `net.inet.tcp.keepintvl`: 45000 | ||
| - `net.inet.tcp.keepinit`: 45000 | ||
| - `net.inet.tcp.keepcnt`: 8 | ||
| - `net.inet.tcp.always_keepalive`: 1 | ||
|
|
||
| #### Linux | ||
|
|
||
| On Linux, the equivalent settings alone may not resolve the issue. Additional steps are required due to the differences in how Linux handles socket keep-alive settings. Follow these steps: | ||
|
|
||
| 1. Adjust the following Linux kernel parameters in `/etc/sysctl.conf` or a related configuration file: | ||
|
|
||
| - `net.inet.tcp.keepidle`: 60000 | ||
| - `net.inet.tcp.keepintvl`: 45000 | ||
| - `net.inet.tcp.keepinit`: 45000 | ||
| - `net.inet.tcp.keepcnt`: 8 | ||
| - `net.inet.tcp.always_keepalive`: 1 | ||
| - `net.ipv4.tcp_keepalive_intvl`: 75 | ||
| - `net.ipv4.tcp_keepalive_probes`: 9 | ||
| - `net.ipv4.tcp_keepalive_time`: 60 (You may consider lowering this value from the default 300 seconds) | ||
|
|
||
| 2. After modifying the kernel parameters, apply the changes by running the following command: | ||
|
|
||
| ```shell | ||
| sudo sysctl -p | ||
| ``` | ||
|
|
||
| After Setting those settings, you need to ensure that your client enables the Keep Alive option on the socket: | ||
|
|
||
| ```java | ||
| properties.setProperty("socket_keepalive", "true"); | ||
| ``` | ||
|
Member
There was a problem hiding this comment. I see a few issues with |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@laeg What about using the client versioning? There will be 2 pages:
We can manually add more pages if necessary and they always transparently mapped to JDBC release version
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah i would prefer if we aligned it more to release versions rather than internal code versions.
did we look into whether we could do something more elegant for versioning the docs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't manage to find a solution in 30min.