Skip to content

[Feat][Connector-v2][JDBC][Oracle] Add Testcontainers-based unit tests #10435

Open
zooo-code wants to merge 5 commits intoapache:devfrom
zooo-code:dev-oracle-testcontainers
Open

[Feat][Connector-v2][JDBC][Oracle] Add Testcontainers-based unit tests #10435
zooo-code wants to merge 5 commits intoapache:devfrom
zooo-code:dev-oracle-testcontainers

Conversation

@zooo-code
Copy link

@zooo-code zooo-code commented Feb 1, 2026

Purpose of this pull request

Add comprehensive Testcontainers-based unit tests for the Oracle JDBC connector to improve test coverage and enable CI execution.

This PR addresses issue #10212 by introducing Docker-based unit tests that can run in CI environments without requiring a local Oracle installation.


Does this PR introduce any user-facing change?

No.

This PR only adds unit tests for the Oracle connector. No functional changes to the connector itself.


How was this patch tested?

New Tests Added

  • AbstractOracleContainerTest: Base class providing Oracle container setup with ARM64 support
  • OracleCatalogContainerTest: 9 tests for catalog operations (database/table management, complex types, primary keys)
  • OracleDialectContainerTest: 15 tests for Oracle-specific SQL features (DUAL, ORA_HASH, identifier quoting, data sampling)

Test Environment

  • Uses gvenzl/oracle-free:23-slim Docker image for ARM64 / Apple Silicon compatibility
  • Configured with:
    • 1GB shared memory
    • 5-minute startup timeout
  • Disabled on Windows for CI compatibility

Test Results

  • Tests run: 570
  • Failures: 0
  • Errors: 0
  • Skipped: 0

All 570 tests in the connector-jdbc module pass successfully, including 24 new Oracle tests.

Local Testing

cd seatunnel-connectors-v2/connector-jdbc
mvn test -Dtest=OracleCatalogContainerTest
mvn test -Dtest=OracleDialectContainerTest

Check list

@DanielCarter-stack
Copy link

Issue 1: Test method lacks assertions

Location: OracleDialectContainerTest.java:172-183

@Test
public void testQueryNextChunkMax() throws Exception {
    TablePath tablePath = TablePath.of(DATABASE, SCHEMA, TEST_TABLE);
    JdbcSourceTable table =
            JdbcSourceTable.builder()
                    .tablePath(tablePath)
                    .useSelectCount(false)
                    .skipAnalyze(false)
                    .build();

    Object maxValue = dialect.queryNextChunkMax(connection, table, "ID", 10, 0);
}

Related Context:

  • Reference: DuckDBDialectTest.java:167-173 has complete assertions for the same method
  • Method under test: OracleDialect.queryNextChunkMax() (OracleDialect.java:263-310)

Problem Description:
This test method calls queryNextChunkMax() but does not perform any assertion verification on the return value. Referencing the same test in DuckDBDialectTest, the returned maximum value should be verified as expected. The current implementation will "pass" even if the method returns null or throws an exception (unless the exception directly causes failure).

Potential Risks:

  • Cannot verify the logic correctness of queryNextChunkMax()
  • Cannot catch regression issues (e.g., future modifications breaking this method)
  • False high test coverage (method is called but not verified)

Impact Scope:

  • Direct impact: OracleDialect.queryNextChunkMax() method
  • Indirect impact: Oracle data source reading functionality that depends on chunk sharding
  • Affected area: Oracle Connector's data sharding logic

Severity: MAJOR

Improvement Suggestions:

@Test
public void testQueryNextChunkMax() throws Exception {
    TablePath tablePath = TablePath.of(DATABASE, SCHEMA, TEST_TABLE);
    JdbcSourceTable table =
            JdbcSourceTable.builder()
                    .tablePath(tablePath)
                    .useSelectCount(false)
                    .skipAnalyze(false)
                    .build();

    // Test first chunk
    Object firstChunkMax = dialect.queryNextChunkMax(connection, table, "ID", 3, 1);
    Assertions.assertNotNull(firstChunkMax);
    Assertions.assertEquals(1, ((Number) firstChunkMax).intValue());
    
    // Test second chunk
    Object secondChunkMax = dialect.queryNextChunkMax(connection, table, "ID", 3, 1);
    Assertions.assertNotNull(secondChunkMax);
    // Should return the only ID value (1) since table only has one row
    Assertions.assertEquals(1, ((Number) secondChunkMax).intValue());
}

Rationale: Referencing the implementation of DuckDBDialectTest, the return value should be verified as expected. Since only one row of data (ID=1) was inserted in the test table, the chunk sharding should correctly handle this case.


Issue 2: Testcontainers version is outdated

Location: pom.xml:431-439

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>oracle-xe</artifactId>
    <version>${testcontainer.version}</version>
    <scope>test</scope>
</dependency>

Related Context:

  • Parent POM definition: testcontainer.version = 1.17.6 (pom.xml:141)
  • Other modules in the project: connector-databend-e2e uses 1.20.2, connector-hudi-e2e uses 1.19.1

Problem Description:
Testcontainers 1.17.6 was released in 2022 and is relatively old. Although this version works, there are the following issues:

  1. Lacks new features and bug fixes
  2. May have known security issues
  3. Inconsistent with versions used in other modules of the project (e.g., databend 1.20.2)

Potential Risks:

  • Possible unfixed bugs affecting test stability
  • Compatibility issues with newer Docker/Java versions
  • Security vulnerabilities (although test dependencies don't affect production, CI environment is still affected)

Impact Scope:

  • Direct impact: Oracle tests in the connector-jdbc module
  • Indirect impact: Test execution in CI environment
  • Affected area: Test dependencies of a single module

Severity: MINOR

Improvement Suggestions:
Consider upgrading Testcontainers to version 1.19.x or 1.20.x to align with other modules in the project. However, this requires:

  1. Upgrading the testcontainer.version property in the parent POM
  2. Verifying compatibility of all modules using Testcontainers
  3. Handling in a separate issue, should not block the current PR

Rationale: Although the version is old, it does not affect functionality. Suggested as a follow-up optimization item to be handled together when upgrading Testcontainers version globally.


Issue 3: Debug output should use logging framework

Location:

  • OracleCatalogContainerTest.java:52
  • OracleCatalogContainerTest.java:180
System.out.println("Available databases: " + databases);
System.out.println("Tables found: " + tables);

Related Context:

  • Production code uses Slf4j + Logback (see OracleDialect.java:39 uses @Slf4j)
  • Test code should also follow unified standards

Problem Description:
Test code uses System.out.println for debug output instead of using a logging framework. This practice in CI environments may lead to:

  1. Inconsistent log output format
  2. Unable to control output through log levels
  3. Confused output during concurrent testing

Potential Risks:

  • Reduced CI log readability
  • Unable to disable debug output through configuration
  • Inconsistent with project logging standards

Impact Scope:

  • Direct impact: Log output of OracleCatalogContainerTest
  • Indirect impact: CI log readability
  • Affected area: Test code quality

Severity: MINOR

Improvement Suggestions:

import lombok.extern.slf4j.Slf4j;

@Slf4j
public class OracleCatalogContainerTest extends AbstractOracleContainerTest {
    
    @Test
    public void testDatabaseExists() {
        List<String> databases = catalog.listDatabases();
        log.info("Available databases: {}", databases);
        // ...
    }
    
    @Test
    public void testListTables() throws SQLException {
        // ...
        log.info("Tables found: {}", tables);
        // ...
    }
}

Rationale: Using Slf4j maintains consistency with the project's logging framework, supports log level control, and avoids generating excessive noise in CI. For debug information, log.debug() should be used instead of log.info(), or removed after confirming test stability.


Issue 4: Windows platform disablement lacks documentation

Location: AbstractOracleContainerTest.java:40

@DisabledOnOs(OS.WINDOWS)
public abstract class AbstractOracleContainerTest {

Related Context:

  • PR description mentions "Disabled on Windows for CI compatibility"
  • Other test classes do not use this annotation (e.g., DuckDBDialectTest)

Problem Description:
The test uses @DisabledOnOs(OS.WINDOWS) to disable the Windows platform, but does not explain the reason in JavaDoc or code comments. This may lead to:

  1. Confusion among Windows developers about why the test is skipped
  2. Unclear whether it's a Docker Desktop for Windows limitation or other reasons

Potential Risks:

  • Windows developers may mistakenly think there's an issue with the test
  • Cannot track whether there are plans to support Windows in the future

Impact Scope:

  • Direct impact: Developers on Windows platform
  • Indirect impact: Test documentation completeness
  • Affected area: Cross-platform test support

Severity: MINOR

Improvement Suggestions:

/**
 * Base class for Oracle Testcontainers-based unit tests. Provides shared Oracle container setup and
 * connection management.
 * 
 * <p>Note: Tests are disabled on Windows due to Docker Desktop limitations with shared memory
 * configuration required by Oracle container. Use WSL2 or Linux/macOS for running these tests.
 */
@DisabledOnOs(OS.WINDOWS)
public abstract class AbstractOracleContainerTest {

Rationale: Documenting the reason for disablement in JavaDoc helps Windows developers understand the limitation and provides alternative solutions (WSL2).


@AfterAll
public static void stopContainer() throws SQLException {
if (catalog != null) {
catalog.close();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution. If an exception is thrown when closing the catalog, will the following be unable to close

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added try-catch blocks to ensure all resources are closed even if catalog.close() throws an exception.

+ quoteIdentifier(TEST_TABLE),
128);

Assertions.assertNotNull(preparedStatement);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the assertion here fails, preparedStatement cannot be closed

Copy link
Author

@zooo-code zooo-code Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LiJie20190102 I've changed it to use try-with-resources pattern to ensure the PreparedStatement is always closed. Fixed in the latest commit.

@zooo-code zooo-code force-pushed the dev-oracle-testcontainers branch from 3f94907 to 0c39ed5 Compare February 5, 2026 09:15
@zooo-code
Copy link
Author

I followed the pattern in #10252 PR, with a few adjustments to align with project conventions

  • Disabled tests on Windows due to missing Docker environment in CI
  • Renamed test classes to align with naming conventions (removed "Container")
  • Applied try-with-resources

@zooo-code zooo-code force-pushed the dev-oracle-testcontainers branch from adfb234 to c0fc604 Compare February 10, 2026 09:51
Copy link
Collaborator

@LiJie20190102 LiJie20190102 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution. Could you migrate the unit tests from the jdbc-e2e module to connector-jdbc later? Or could you submit them together this time

@zooo-code
Copy link
Author

zooo-code commented Feb 10, 2026

Thank you for your contribution. Could you migrate the unit tests from the jdbc-e2e module to connector-jdbc later? Or could you submit them together this time

Thanks for the approval! @LiJie20190102

I'd prefer to keep this PR focused on adding the Testcontainers-based tests, as it's already been through several review iterations and is ready to merge.

I can create a follow-up PR to migrate the existing unit tests from the jdbc-e2e module to connector-jdbc.

  • This PR can be merged quickly without additional scope.
  • The migration can be reviewed separately with proper attention.
  • We maintain clean commit history with focused changes.

However, if you prefer merging them together in this PR, I'm happy to work on the migration now. Please let me know!

* quoting, SQL generation, and data sampling.
*/
@DisabledOnOs(OS.WINDOWS)
public class OracleDialectTest extends AbstractOracleContainerTest {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add getInsertIntoStatement(), getUpdateStatement(), getDeleteStatement(), getRowExistsStatement(), getUpsertStatement() and other corresponding unit tests, and it is best to add FieldNamedPreparedStatement.prepareStatement().

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chl-wxp Thanks for the review!
I've added all the requested tests as suggested.
(10 new tests)

  1. DML Statements

    • Covered Insert, Update, Delete, RowExists, and Upsert logic.
  2. FieldNamedPreparedStatement

    • Covered parameter parsing, duplicate handling, and execution.

All tests passed locally.
Thanks!

@zooo-code zooo-code force-pushed the dev-oracle-testcontainers branch from c0fc604 to 374f3e9 Compare February 11, 2026 14:34
@zooo-code zooo-code requested a review from chl-wxp February 13, 2026 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants