feat: add unknownLength connection property (#2286)

Adds an `unknownLength` connection property that can be used to configure the length that
the JDBC driver should return as the data type / column length when this is not known.
Spanner does not return the (maximum) length of a column in ResultSetMetadata. This means
that the JDBC driver does not know what the length is of the various columns in query results.

The value of `unknownLength` will be returned when the getPrecision and getColumnDisplaySize
methods of ResultSetMetaData are called.

This connection property aligns with the same connection parameter in the PostgreSQL JDBC driver:
https://jdbc.postgresql.org/documentation/use/#connection-parameters

Author: olavloite

documentation/connection_properties.md

@@ -13,11 +13,12 @@ The 'Context' value indicates whether the property can only be set when a connec
 | autocommit_dml_mode | Determines the transaction type that is used to execute DML statements when the connection is in auto-commit mode. | TRANSACTIONAL | TRANSACTIONAL, PARTITIONED_NON_ATOMIC, TRANSACTIONAL_WITH_FALLBACK_TO_PARTITIONED_NON_ATOMIC, null | USER |
 | autoconfigemulator | Automatically configure the connection to try to connect to the Cloud Spanner emulator (true/false). The instance and database in the connection string will automatically be created if these do not yet exist on the emulator. Add dialect=postgresql to the connection string to make sure that the database that is created uses the PostgreSQL dialect. | false | true, false | STARTUP |
 | autopartitionmode | Execute all queries on this connection as partitioned queries. Executing a query that cannot be partitioned will fail. Executing a query in a read/write transaction will also fail. | false | true, false | USER |
+| batch_dml_update_count | The update count that is returned for DML statements that are executed in an explicit DML batch. The default is -1 | -1 |  | USER |
 | channelprovider | The name of the channel provider class. The name must reference an implementation of ExternalChannelProvider. If this property is not set, the connection will use the default grpc channel provider. |  |  | STARTUP |
 | clientcertificate | Specifies the file path to the client certificate required for establishing an mTLS connection. |  |  | STARTUP |
 | clientkey | Specifies the file path to the client private key required for establishing an mTLS connection. |  |  | STARTUP |
 | connection_state_type | The type of connection state to use for this connection. Can only be set at start up. If no value is set, then the database dialect default will be used, which is NON_TRANSACTIONAL for GoogleSQL and TRANSACTIONAL for PostgreSQL. |  | TRANSACTIONAL, NON_TRANSACTIONAL | STARTUP |
-| credentials | The location of the credentials file to use for this connection. If neither this property or encoded credentials are set, the connection will use the default Google Cloud credentials for the runtime environment. |  |  | STARTUP |
+| credentials | The location of the credentials file to use for this connection. If neither this property or encoded credentials are set, the connection will use the default Google Cloud credentials for the runtime environment. WARNING: Using this property without proper validation can expose the application to security risks. It is intended for use with credentials from a trusted source only, as it could otherwise allow end-users to supply arbitrary credentials. For more information, seehttps://cloud.google.com/docs/authentication/client-libraries#external-credentials |  |  | STARTUP |
 | credentialsprovider | The class name of the com.google.api.gax.core.CredentialsProvider implementation that should be used to obtain credentials for connections. |  |  | STARTUP |
 | databaserole | Sets the database role to use for this connection. The default is privileges assigned to IAM role |  |  | STARTUP |
 | databoostenabled | Enable data boost for all partitioned queries that are executed by this connection. This setting is only used for partitioned queries and is ignored by all other statements. | false | true, false | USER |
@@ -31,8 +32,9 @@ The 'Context' value indicates whether the property can only be set when a connec
 | enabledirectaccess | Configure the connection to try to connect to Spanner using DirectPath (true/false). The client will try to connect to Spanner using a direct Google network connection. DirectPath will work only if the client is trying to establish a connection from a Google Cloud VM. Otherwise it will automatically fallback to the standard network path. NOTE: The default for this property is currently false, but this could be changed in the future. |  | true, false | STARTUP |
 | enableendtoendtracing | Enable end-to-end tracing (true/false) to generate traces for both the time that is spent in the client, as well as time that is spent in the Spanner server. Server side traces can only go to Google Cloud Trace, so to see end to end traces, the application should configure an exporter that exports the traces to Google Cloud Trace. | false | true, false | STARTUP |
 | enableextendedtracing | Include the SQL string in the OpenTelemetry traces that are generated by this connection. The SQL string is added as the standard OpenTelemetry attribute 'db.statement'. |  | true, false | STARTUP |
-| encodedcredentials | Base64-encoded credentials to use for this connection. If neither this property or a credentials location are set, the connection will use the default Google Cloud credentials for the runtime environment. |  |  | STARTUP |
+| encodedcredentials | Base64-encoded credentials to use for this connection. If neither this property or a credentials location are set, the connection will use the default Google Cloud credentials for the runtime environment. WARNING: Enabling this property without proper validation can expose the application to security risks. It is intended for use with credentials from a trusted source only, as it could otherwise allow end-users to supply arbitrary credentials. For more information, seehttps://cloud.google.com/docs/authentication/client-libraries#external-credentials |  |  | STARTUP |
 | endpoint | The endpoint that the JDBC driver should connect to. The default is the default Spanner production endpoint when autoConfigEmulator=false, and the default Spanner emulator endpoint (localhost:9010) when autoConfigEmulator=true. This property takes precedence over any host name at the start of the connection URL. |  |  | STARTUP |
+| grpc_interceptor_provider | The class name of a com.google.api.gax.grpc.GrpcInterceptorProvider implementation that should be used to provide interceptors for the underlying Spanner client. This is a guarded property that can only be set if the Java System Property ENABLE_GRPC_INTERCEPTOR_PROVIDER has been set to true. This property should only be set to true on systems where an untrusted user cannot modify the connection URL, as using this property will dynamically invoke the constructor of the class specified. This means that any user that can modify the connection URL, can also dynamically invoke code on the host where the application is running. |  |  | STARTUP |
 | isexperimentalhost | Set this value to true for communication with a Experimental Host. | false | true, false | STARTUP |
 | keeptransactionalive | Enabling this option will trigger the connection to keep read/write transactions alive by executing a SELECT 1 query once every 10 seconds if no other statements are being executed. This option should be used with caution, as it can keep transactions alive and hold on to locks longer than intended. This option should typically be used for CLI-type application that might wait for user input for a longer period of time. | false | true, false | USER |
 | lenient | Silently ignore unknown properties in the connection string/properties (true/false) | false | true, false | STARTUP |
@@ -53,11 +55,13 @@ The 'Context' value indicates whether the property can only be set when a connec
 | routetoleader | Should read/write transactions and partitioned DML be routed to leader region (true/false) | true | true, false | STARTUP |
 | rpcpriority | Sets the priority for all RPC invocations from this connection (HIGH/MEDIUM/LOW). The default is HIGH. |  | LOW, MEDIUM, HIGH, UNSPECIFIED, null | USER |
 | savepoint_support | Determines the behavior of the connection when savepoints are used. | FAIL_AFTER_ROLLBACK | ENABLED, FAIL_AFTER_ROLLBACK, DISABLED | USER |
+| statement_timeout | Adds a timeout to all statements executed on this connection. This property is only used when a statement timeout is specified. |  |  | USER |
 | tracing_prefix | The prefix that will be prepended to all OpenTelemetry traces that are generated by a Connection. | CloudSpanner |  | STARTUP |
 | trackconnectionleaks | Capture the call stack of the thread that created a connection. This will pre-create a LeakedConnectionException already when a connection is created. This can be disabled, for example if a monitoring system logs the pre-created exception. If disabled, the LeakedConnectionException will only be created when an actual connection leak is detected. The stack trace of the exception will in that case not contain the call stack of when the connection was created. | true | true, false | STARTUP |
 | tracksessionleaks | Capture the call stack of the thread that checked out a session of the session pool. This will pre-create a LeakedSessionException already when a session is checked out. This can be disabled, for example if a monitoring system logs the pre-created exception. If disabled, the LeakedSessionException will only be created when an actual session leak is detected. The stack trace of the exception will in that case not contain the call stack of when the session was checked out. | true | true, false | STARTUP |
 | transaction_timeout | Timeout for read/write transactions. |  |  | USER |
 | universedomain | Configure the connection to try to connect to Spanner using a different partner Google Universe than GDU (googleapis.com). | googleapis.com |  | STARTUP |
+| unknownlength | Spanner does not return the length of the selected columns in query results. When returning meta-data about these columns through functions like ResultSetMetaData.getColumnDisplaySize and ResultSetMetaData.getPrecision, we must provide a value. Various client tools and applications have different ideas about what they would like to see. This property specifies the length to return for types of unknown length. | 50 |  | USER |
 | useautosavepointsforemulator | Automatically creates savepoints for each statement in a read/write transaction when using the Emulator. This is no longer needed when using Emulator version 1.5.23 or higher. | false | true, false | STARTUP |
 | useplaintext | Use a plain text communication channel (i.e. non-TLS) for communicating with the server (true/false). Set this value to true for communication with the Cloud Spanner emulator. | false | true, false | STARTUP |
 | useragent | The custom user-agent property name to use when communicating with Cloud Spanner. This property is intended for internal library usage, and should not be set by applications. |  |  | STARTUP |

src/main/java/com/google/cloud/spanner/jdbc/JdbcConnection.java

@@ -29,6 +29,7 @@
 import com.google.cloud.spanner.connection.AutocommitDmlMode;
 import com.google.cloud.spanner.connection.Connection;
 import com.google.cloud.spanner.connection.ConnectionOptions;
+import com.google.cloud.spanner.connection.ConnectionProperties;
 import com.google.cloud.spanner.connection.SavepointSupport;
 import com.google.cloud.spanner.connection.TransactionMode;
 import com.google.common.annotations.VisibleForTesting;
@@ -236,6 +237,11 @@ public String getOptimizerVersion() throws SQLException {
     return getSpannerConnection().getOptimizerVersion();
   }
 
+  /** Returns the value that should be returned for column types with an unknown length. */
+  int getColumnTypeUnknownLength() {
+    return getSpannerConnection().getConnectionPropertyValue(ConnectionProperties.UNKNOWN_LENGTH);
+  }
+
   @Override
   public boolean isInTransaction() throws SQLException {
     checkClosed();

src/main/java/com/google/cloud/spanner/jdbc/JdbcResultSetMetaData.java

@@ -17,20 +17,16 @@
 package com.google.cloud.spanner.jdbc;
 
 import com.google.cloud.spanner.ResultSet;
+import com.google.cloud.spanner.connection.ConnectionProperties;
 import com.google.common.base.Preconditions;
+import java.sql.Connection;
 import java.sql.ResultSetMetaData;
 import java.sql.SQLException;
 import java.sql.Statement;
 import java.sql.Types;
 
 /** Implementation of {@link ResultSetMetaData} for Cloud Spanner */
 class JdbcResultSetMetaData extends AbstractJdbcWrapper implements ResultSetMetaData {
-  /**
-   * The default column display size for columns with a data type of variable size that is used when
-   * the actual column size is not known.
-   */
-  private static final int DEFAULT_COL_DISPLAY_SIZE_FOR_VARIABLE_LENGTH_COLS = 50;
-
   private final ResultSet spannerResultSet;
   private final Statement statement;
 
@@ -83,16 +79,15 @@ public boolean isSigned(int column) {
   }
 
   @Override
-  public int getColumnDisplaySize(int column) {
+  public int getColumnDisplaySize(int column) throws SQLException {
     int colType = getColumnType(column);
     switch (colType) {
       case Types.ARRAY:
-        return DEFAULT_COL_DISPLAY_SIZE_FOR_VARIABLE_LENGTH_COLS;
+        return getUnknownLength();
       case Types.BOOLEAN:
         return 5;
       case Types.BINARY:
-        int binaryLength = getPrecision(column);
-        return binaryLength == 0 ? DEFAULT_COL_DISPLAY_SIZE_FOR_VARIABLE_LENGTH_COLS : binaryLength;
+        return getPrecision(column);
       case Types.DATE:
         return 10;
       case Types.REAL:
@@ -105,8 +100,7 @@ public int getColumnDisplaySize(int column) {
       case Types.NUMERIC:
         return 14;
       case Types.NVARCHAR:
-        int length = getPrecision(column);
-        return length == 0 ? DEFAULT_COL_DISPLAY_SIZE_FOR_VARIABLE_LENGTH_COLS : length;
+        return getPrecision(column);
       case Types.TIMESTAMP:
         return 16;
       default:
@@ -130,7 +124,7 @@ public String getSchemaName(int column) throws SQLException {
   }
 
   @Override
-  public int getPrecision(int column) {
+  public int getPrecision(int column) throws SQLException {
     int colType = getColumnType(column);
     switch (colType) {
       case Types.BOOLEAN:
@@ -153,9 +147,20 @@ public int getPrecision(int column) {
         // For column types with variable size, such as text columns, we should return the length
         // in characters. We could try to fetch it from INFORMATION_SCHEMA, but that would mean
         // parsing the SQL statement client side in order to figure out which column it actually
-        // is. For now we just return the default column display size.
-        return DEFAULT_COL_DISPLAY_SIZE_FOR_VARIABLE_LENGTH_COLS;
+        // is. Instead, we return a configurable fixed length. This is also consistent with for
+        // example the PostgreSQL JDBC driver. See the 'unknownLength' connection property:
+        // https://jdbc.postgresql.org/documentation/use/#connection-parameters
+        return getUnknownLength();
+    }
+  }
+
+  private int getUnknownLength() throws SQLException {
+    Connection connection = statement.getConnection();
+    if (connection instanceof JdbcConnection) {
+      JdbcConnection jdbcConnection = (JdbcConnection) connection;
+      return jdbcConnection.getColumnTypeUnknownLength();
     }
+    return ConnectionProperties.UNKNOWN_LENGTH.getDefaultValue();
   }
 
   @Override

src/test/java/com/google/cloud/spanner/jdbc/JdbcResultSetMetaDataTest.java

@@ -362,7 +362,7 @@ public void testIsSigned() {
   }
 
   @Test
-  public void testGetColumnDisplaySize() {
+  public void testGetColumnDisplaySize() throws SQLException {
     for (int i = 1; i <= TEST_COLUMNS.size(); i++) {
       assertEquals(
           "Wrong column display size for " + TEST_COLUMNS.get(i - 1).type,
@@ -371,7 +371,7 @@ public void testGetColumnDisplaySize() {
     }
   }
 
-  private int getDefaultDisplaySize(Type type, int column) {
+  private int getDefaultDisplaySize(Type type, int column) throws SQLException {
     Preconditions.checkNotNull(type);
     switch (type.getCode()) {
       case BOOL:
@@ -425,7 +425,7 @@ public void testGetSchemaName() throws SQLException {
   }
 
   @Test
-  public void testGetPrecision() {
+  public void testGetPrecision() throws SQLException {
     for (int i = 1; i <= TEST_COLUMNS.size(); i++) {
       assertEquals(
           "Wrong precision for type " + TEST_COLUMNS.get(i - 1).type,