Skip to content

Conversation

@carsheng
Copy link
Contributor

@carsheng carsheng commented Nov 13, 2025

This change accomplishes Part 1 of removing Avatica from the datacloud-jdbc driver - specifically focusing on marshalling of column metadata. It removes Avatica dependencies from metadata conversion and introduces a custom SimpleResultSet implementation for database metadata queries which includes much of the JDBc boilerplate that was originally handled by Avatica.
QueryMetadataUtil now uses SimpleMetadataResultSet.of() instead of Avatica-based constructors, and type mappings were updated to use standard JDBCType instead of Avatica's SqlType enum.

@carsheng carsheng force-pushed the csheng/avatica-removal branch from 4bd32a0 to 40ea34f Compare November 13, 2025 00:45
@carsheng carsheng changed the title Csheng/avatica removal Avatical Removal Part1: remove Avatica dependency from metadata conversion flow Nov 14, 2025
immutableEntry("int4", JDBCType.INTEGER.toString()),
immutableEntry("oid", JDBCType.BIGINT.toString()),
immutableEntry("int8", JDBCType.BIGINT.toString()),
immutableEntry("float", JDBCType.DOUBLE.toString()),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkaufmann confirm whether FLOAT --> double conversion is correct?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty unclear where the names here are coming from as they don't seem to be SQL types. The only use case of this map seems to be for private static List<Object> constructColumnData(ResultSet resultSet) throws SQLException {

That method doesn't really make sense though as it seems to be only used for convering a ResultSet in the end to a result set. So I think this + constructColumnData can just be completely removed. Instead of all that complication instead this function

    public static ResultSet createColumnResultSet(
            String schemaPattern, String tableNamePattern, String columnNamePattern, Connection connection)
            throws SQLException {

        final List<Object> data;

        try (val statement = connection.createStatement()) {
            val getColumnsQuery = getColumnsQueryInner(schemaPattern, tableNamePattern, columnNamePattern);
            val resultSet = statement.executeQuery(getColumnsQuery);
            data = constructColumnData(resultSet);
        }

        return getMetadataResultSet(QueryDBMetadata.GET_COLUMNS, data);
    }

Should be adjusted so that it directly return the resultSet from statement.executeQuery(getColumnsQuery); and to enable that the query needs to be adjusted to directly return the columns with the expected names and types for GET_COLUMNS. That can be achieved by using CAST( ... AS <TYPE>) and as <COLUMN_NAME> in the top level select

Copy link
Contributor Author

@carsheng carsheng Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkaufmann i think we can remove some amount of the code through refactoring, but the base problem still exists (i.e. we still need to determine the correct type mapping).

The names which we're seeing in the above map (such as int2, float4...etc) are base hyper types (we see the same in Postgres) that are fetched by querying the internal db table pg_catalog.pg_attribute for column metadata. Agreed, they're not SQL types, but they are the way column types are stored in this internal db table, and so we need to have a mapping between such base db types and SQL types.

If we do away with the mapping, when running metadata.getColumns() the customer will see non-SQL types such as int4 which would break the JDBC contract

@codecov
Copy link

codecov bot commented Nov 20, 2025

Codecov Report

❌ Patch coverage is 69.88235% with 128 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.43%. Comparing base (b8c5eb9) to head (c28d02a).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
...datacloud/jdbc/core/resultset/SimpleResultSet.java 63.46% 20 Missing and 18 partials ⚠️
...e/datacloud/jdbc/core/SimpleMetadataResultSet.java 57.83% 20 Missing and 15 partials ⚠️
...core/resultset/ResultSetWithPositionalGetters.java 35.29% 22 Missing ⚠️
...force/datacloud/jdbc/core/metadata/ColumnType.java 24.00% 18 Missing and 1 partial ⚠️
...ud/jdbc/core/metadata/SimpleResultSetMetaData.java 81.57% 3 Missing and 4 partials ⚠️
...loud/jdbc/core/resultset/ForwardOnlyResultSet.java 75.00% 3 Missing and 1 partial ⚠️
...esforce/datacloud/jdbc/core/QueryMetadataUtil.java 90.47% 2 Missing ⚠️
...tacloud/jdbc/core/resultset/ReadOnlyResultSet.java 98.93% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #136      +/-   ##
============================================
- Coverage     83.47%   82.43%   -1.04%     
- Complexity     1271     1509     +238     
============================================
  Files           104      113       +9     
  Lines          3765     4214     +449     
  Branches        375      420      +45     
============================================
+ Hits           3143     3474     +331     
- Misses          462      535      +73     
- Partials        160      205      +45     
Components Coverage Δ
JDBC Core 83.47% <69.88%> (-2.09%) ⬇️
JDBC Main 40.69% <ø> (ø)
JDBC HTTP 91.09% <ø> (ø)
JDBC Utilities 66.07% <ø> (+5.10%) ⬆️
Spark Datasource ∅ <ø> (∅)
Files with missing lines Coverage Δ
.../datacloud/jdbc/core/resultset/ColumnAccessor.java 100.00% <100.00%> (ø)
...e/datacloud/jdbc/util/ArrowToColumnTypeMapper.java 62.26% <ø> (ø)
...com/salesforce/datacloud/jdbc/util/ArrowUtils.java 84.12% <ø> (ø)
...tacloud/jdbc/core/resultset/ReadOnlyResultSet.java 98.93% <98.93%> (ø)
...esforce/datacloud/jdbc/core/QueryMetadataUtil.java 91.48% <90.47%> (-0.15%) ⬇️
...loud/jdbc/core/resultset/ForwardOnlyResultSet.java 75.00% <75.00%> (ø)
...ud/jdbc/core/metadata/SimpleResultSetMetaData.java 81.57% <81.57%> (ø)
...force/datacloud/jdbc/core/metadata/ColumnType.java 68.96% <24.00%> (ø)
...core/resultset/ResultSetWithPositionalGetters.java 35.29% <35.29%> (ø)
...e/datacloud/jdbc/core/SimpleMetadataResultSet.java 57.83% <57.83%> (ø)
... and 1 more

... and 21 files with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@carsheng carsheng force-pushed the csheng/avatica-removal branch from 282e5d0 to d56d810 Compare November 24, 2025 22:37
@carsheng carsheng force-pushed the csheng/avatica-removal branch from 4716e63 to 8c07f84 Compare December 17, 2025 19:59
@carsheng carsheng force-pushed the csheng/avatica-removal branch from 8c07f84 to bbc22d8 Compare December 17, 2025 22:38
@carsheng carsheng force-pushed the csheng/avatica-removal branch from 36255fe to 023405d Compare December 18, 2025 00:01
@carsheng carsheng force-pushed the csheng/avatica-removal branch from 2d45c7c to c28d02a Compare December 18, 2025 02:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants