Optimize JDBC adaptor for MySQL #2801

brfrn169 · 2025-06-20T01:53:40Z

Description

When benchmarking the new release, I observed some performance degradation with the JDBC adapter for MySQL. After investigating the issue, I found the following:

Using Connection.setReadOnly(true) causes performance degradation.
Enabling cursor fetch mode in MySQL also results in performance degradation.

To address these issues, this PR introduces optimizations to the JDBC adapter for MySQL.

Related issues and/or PRs

N/A

Changes made

Added some inline comments. Please take a look for the details.

Checklist

The following is a best-effort checklist. If any items in this checklist are not applicable to this PR or are dependent on other, unmerged PRs, please still mark the checkboxes after you have read and understood each item.

I have commented my code, particularly in hard-to-understand areas.
I have updated the documentation to reflect the changes.
I have considered whether similar issues could occur in other products, components, or modules if this PR is for bug fixes.
Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
Tests (unit, integration, etc.) have been added for the changes.
My changes generate no new warnings.
Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

N/A

Release notes

N/A

Copilot

Pull Request Overview

This PR aims to optimize the JDBC adapter for MySQL by refining connection property management and disabling performance-degrading configurations such as setting MySQL connections to read-only mode. Key changes include:

Refactoring the getConnectionProperties API to accept a configuration parameter in multiple RdbEngine implementations.
Disabling setting the read-only flag on MySQL connections to address performance degradation.
Adjusting test cases to reflect changes in error handling and connection property configurations.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
core/src/test/java/com/scalar/db/transaction/jdbc/JdbcTransactionManagerTest.java	Changed RdbEngine strategy from MYSQL to POSTGRESQL and updated SQL error handling.
core/src/test/java/com/scalar/db/storage/jdbc/JdbcUtilsTest.java	Updated getConnectionProperties method signature usage in tests.
core/src/test/java/com/scalar/db/storage/jdbc/JdbcDatabaseTest.java	Changed RdbEngine strategy from MYSQL to POSTGRESQL and updated SQL error handling.
core/src/test/java/com/scalar/db/storage/jdbc/JdbcAdminTest.java	Adjusted read-only verification conditions to accommodate MySQL performance optimization.
core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineStrategy.java, RdbEngineSqlServer.java, RdbEngineMysql.java, RdbEngineMariaDB.java, RdbEngineDb2.java	Modified getConnectionProperties signatures to include JdbcConfig and introduced setConnectionToReadOnly for MySQL.
core/src/main/java/com/scalar/db/storage/jdbc/JdbcUtils.java	Updated loops to iterate using the new API signature.
core/src/main/java/com/scalar/db/storage/jdbc/JdbcConfig.java	Added a field and accessor for DatabaseConfig to support new API usage.

Comments suppressed due to low confidence (2)

core/src/test/java/com/scalar/db/transaction/jdbc/JdbcTransactionManagerTest.java:82

The test case now uses the POSTGRESQL engine instead of MYSQL, which might not accurately test the MySQL optimizations addressed by this PR. Please verify if this change is intentional.

            RdbEngine.createRdbEngineStrategy(RdbEngine.POSTGRESQL),

core/src/test/java/com/scalar/db/storage/jdbc/JdbcDatabaseTest.java:65

Switching from MYSQL to POSTGRESQL in this test case could lead to inconsistencies when validating the MySQL-specific performance improvements. Please confirm if this change aligns with the intended testing scenarios for MySQL optimizations.

            RdbEngine.createRdbEngineStrategy(RdbEngine.POSTGRESQL),

brfrn169 · 2025-06-20T01:55:16Z

core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineMysql.java

+  @Override
+  public void setConnectionToReadOnly(Connection connection, boolean readOnly) throws SQLException {
+    // Observed performance degradation when using read-only connections in MySQL. So we do not
+    // set the read-only mode for MySQL connections.
+  }


We don't call Connection.setReadOnly(true) for MySQL.

Just a question. We don't know the reason for the behavior?

Right, we don't know the exact reason. We just observed performance degradation when we call Connection.setReadOnly(true) with MySQL.

brfrn169 · 2025-06-20T01:56:25Z

core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineMysql.java

+  public Map<String, String> getConnectionProperties(JdbcConfig config) {
+    if (config.getDatabaseConfig().getScanFetchSize() == Integer.MIN_VALUE) {
+      // If the scan fetch size is set to Integer.MIN_VALUE, use the streaming mode.
+      return Collections.emptyMap();
+    }
+
+    // Otherwise, use the cursor fetch mode.
    return Collections.singletonMap("useCursorFetch", "true");
  }


If a user sets scalar.db.scan_fetch_size to Integer.MIN_VALUE, the streaming mode is used in MySQL. In that case, we should not set the connection property useCursorFetch=true.

Torch3333

LGTM, thank you!

feeblefakie

LGTM! Thank you!

feeblefakie · 2025-06-20T02:15:28Z

core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineMysql.java

+  @Override
+  public void setConnectionToReadOnly(Connection connection, boolean readOnly) throws SQLException {
+    // Observed performance degradation when using read-only connections in MySQL. So we do not
+    // set the read-only mode for MySQL connections.
+  }


Just a question. We don't know the reason for the behavior?

komamitsu · 2025-06-20T02:36:22Z

core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineMysql.java

-  public Map<String, String> getConnectionProperties() {
+  public Map<String, String> getConnectionProperties(JdbcConfig config) {
+    if (config.getDatabaseConfig().getScanFetchSize() == Integer.MIN_VALUE) {
+      // If the scan fetch size is set to Integer.MIN_VALUE, use the streaming mode.


[question] Is this behavior only for MySQL?

Don't need to add an integration test using scalar.db.scan_fetch_size = Integer.MIN_VALUE just in case ?

Okay, let me do that.

Added integration tests for it in bdb1309. Thanks!

komamitsu

LGTM, thank you!

Optimize JDBC adaptor for MySQL

3b493be

brfrn169 requested a review from Copilot June 20, 2025 01:53

brfrn169 self-assigned this Jun 20, 2025

brfrn169 added the improvement label Jun 20, 2025

Copilot AI reviewed Jun 20, 2025

View reviewed changes

brfrn169 commented Jun 20, 2025

View reviewed changes

brfrn169 requested review from Torch3333, feeblefakie and komamitsu June 20, 2025 01:58

Torch3333 approved these changes Jun 20, 2025

View reviewed changes

feeblefakie approved these changes Jun 20, 2025

View reviewed changes

komamitsu reviewed Jun 20, 2025

View reviewed changes

brfrn169 requested a review from komamitsu June 20, 2025 03:08

Add integration tests

bdb1309

brfrn169 force-pushed the optimize-jdbc-adopter-for-mysql branch from 0292941 to bdb1309 Compare June 20, 2025 03:26

komamitsu approved these changes Jun 20, 2025

View reviewed changes

Fix

ab5343b

brfrn169 merged commit f3969d9 into master Jun 20, 2025
56 checks passed

brfrn169 deleted the optimize-jdbc-adopter-for-mysql branch June 20, 2025 04:52

feeblefakie mentioned this pull request Jun 20, 2025

Backport to branch(3) : Optimize JDBC adaptor for MySQL #2804

Merged

brfrn169 added a commit that referenced this pull request Jun 20, 2025

Optimize JDBC adaptor for MySQL (#2801)

9a5b0e1

Optimize JDBC adaptor for MySQL #2801

Optimize JDBC adaptor for MySQL #2801

Uh oh!

Conversation

brfrn169 commented Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related issues and/or PRs

Changes made

Checklist

Additional notes (optional)

Release notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Torch3333 left a comment

Choose a reason for hiding this comment

Uh oh!

feeblefakie left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brfrn169 Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

komamitsu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

brfrn169 commented Jun 20, 2025 •

edited

Loading

brfrn169 Jun 20, 2025 •

edited

Loading