diff --git a/docs/modules/data_warehouses/vertica-ce.md b/docs/modules/data_warehouses/vertica-ce.md new file mode 100644 index 00000000000..07fe843078a --- /dev/null +++ b/docs/modules/data_warehouses/vertica-ce.md @@ -0,0 +1,94 @@ +# Vertica-CE Module + +!!! note +This module is INCUBATING. While it is ready for use and operational in the current version of Testcontainers, it is possible that it may receive breaking changes in the future. See [our contributing guidelines](/contributing/#incubating-modules) for more information on our incubating modules policy. + +!!! note +We will need to get clarification on the license - the docker image provides an _Evaluation_ license which is a 'license to use Licensed Products for +internal evaluation and testing purposes only, and not for any development, production, distribution or +commercial purpose' and is only valid for a limited amount of time. It's common to see TestContainers used on both the developer's laptop and in CI/CD pipelines +so at first glance it seems like we can't use this license... but it seems likely that anyone who needs this module will already have an 'Express' license or better. +Given this I think it's likely that we can get a carve out but I can't guarantee it, or that their terms won't be too onerous for the project to accept. + +[Vertica Analytics Database](https://www.vertica.com/) is a [data warehouse](https://en.wikipedia.org/wiki/Data_warehouse) implemented with a using a [columnar database](https://en.wikipedia.org/wiki/Column-oriented_database). See the [Vertica](https://en.wikipedia.org/wiki/Vertica) Wikipedia article for more details. Vertica also supports SQL access for legacy applications, to simplify initialization of the database, etc. + +Data warehouses are usually cloud-native due to the amount of data they store but some vendors provide docker-based __Community Editions__ for use by developers and students who are primarily concerned in learning how to use the tool. + +At this time this module is tested like all other relational databases. The contribution of data warehouse specific tests would be welcome. + +## Adding this module to your project dependencies + +Add the following dependency to your `pom.xml`/`build.gradle` file: + +=== "Gradle" +```groovy +testImplementation "org.testcontainers:vertica-ce:{{latest_version}}" + +// Vertica JDBC driver +testRuntimeOnly "com.vertica.jdbc:vertica-jdbc:24.1.0-0" +``` +=== "Maven" +```xml + + org.testcontainers + vertica-ce + {{latest_version}} + test + + + + + com.vertica.jdbc + vertica-jdbc + 24.1.0-0 + } test + +``` +} +!!! hint +Adding this Testcontainers library JAR will not automatically add the Vertica driver JAR to your project. +You should ensure that your project has the Vertica driver as a dependency. + +## Licenses + +I mentioned this in the note above - reading the licenses literally we can't use the docker image +since the module is likely to be used in CI/CD pipelines that feed into production. However I think we +can make a strong argument for a carve out since the only people using these CI/CD pipelines will already +need to have an 'Express' or better license - the purpose of this module is to facilitate software testing, +not development in isolation from all other resources. We won't know until we ask, and we probably shouldn't +ask until we have enough done to demonstrate the value of the module to their customers. + +This may require a few iterations since we'll need a second (non-customer) carve out sufficiently broad for +any user to download and build the entire TestContainer repo. + +From [Additional License Authorizations for Vertica software products](https://www.microfocus.com/media/documentation/additional-license-authorizations-for-vertica-software-products-documentation.pdf): + +!!! block +Vertica Community Edition may be used for evaluation purposes only and the license for Vertica Community Edition is an +“Evaluation License” as such term is defined in the then current Micro Focus End User License Agreement (the “EULA”) +found at https://software.microfocus.com/en-us/legal/software-licensing. The Evaluation License for Vertica Community +Edition is governed by the terms of the EULA. + +From [Micro Focus End User License Agreement](https://www.microfocus.com/media/documentation/micro_focus_end_user_license_agreement.pdf) + +!!! block +. Evaluation Licenses. Except as specifically permitted in the applicable ALA, when Micro Focus and its +affiliates, respectively deliver and license the Licensed Products solely for evaluation, Customer +receives a non-transferable, non-sublicensable, non-exclusive license to use Licensed Products for +internal evaluation and testing purposes only, and not for any development, production, distribution or +commercial purpose (“Evaluation License”). The term of an Evaluation License will be 30 days +starting from the date Licensed Product is delivered (i.e., made available for download or physically +delivered) to Customer (“Evaluation Term”), unless Micro Focus authorizes a different period in +writing. The Licensed Product is provided “as is” and there are no warranties or obligations for Micro +Focus to provide support. The Evaluation License terminates at the end of the Evaluation Term, and +Customer is required to return, or, if Micro Focus so directs, delete and destroy, all copies of such +Licensed Product and provide Micro Focus with written confirmation of its compliance with this +provision within 30 days of the end of the Evaluation Term. The Evaluation License for any pre-release +or beta versions of Licensed Software (“Pre-Release Software”) shall be for a term of 90 days unless +Micro Focus authorizes a different period in writing. Customer agrees to promptly report to Micro Focus +all problems (including errors, failures, nonconforming results, and unexpected performances) and any +comments regarding the Pre-Release Software and to timely respond to all questionnaires submitted +by Micro Focus regarding the results of Customer’s testing of the Pre-Release Software. Micro Focus +may choose not to release a final version of the Pre-Release Software or, even if released, to alter +prices, features, specifications, capabilities, functions, release dates, general availability, or other +characteristics of the Pre-Release Software. diff --git a/modules/vertica-ce/README.md b/modules/vertica-ce/README.md new file mode 100644 index 00000000000..8ab316d45cb --- /dev/null +++ b/modules/vertica-ce/README.md @@ -0,0 +1,12 @@ +# Vertica-CE Module + +## Remaining tasks + +- [ ] Learn how to use `:jdbc-test` and `AbstractJDBCDriverTest` +- [ ] Write tests +- [ ] Finish documentation +- [ ] Submit PR + +## Open questions + +- [ ] Should there be a new 'Data Warehouse' category in the documentation? diff --git a/modules/vertica-ce/Vertica-OT.svg.png b/modules/vertica-ce/Vertica-OT.svg.png new file mode 100644 index 00000000000..6ff03d7f1eb Binary files /dev/null and b/modules/vertica-ce/Vertica-OT.svg.png differ diff --git a/modules/vertica-ce/additional-license-authorizations-for-vertica-software-products-documentation.pdf b/modules/vertica-ce/additional-license-authorizations-for-vertica-software-products-documentation.pdf new file mode 100644 index 00000000000..dd7cd3f801a Binary files /dev/null and b/modules/vertica-ce/additional-license-authorizations-for-vertica-software-products-documentation.pdf differ diff --git a/modules/vertica-ce/build.gradle b/modules/vertica-ce/build.gradle new file mode 100644 index 00000000000..00f8e357b12 --- /dev/null +++ b/modules/vertica-ce/build.gradle @@ -0,0 +1,13 @@ +description = "Testcontainers-Java :: JDBC :: Vertica-CE" + +dependencies { + annotationProcessor 'com.google.auto.service:auto-service:1.1.1' + compileOnly 'com.google.auto.service:auto-service:1.1.1' + + api project(':jdbc') + + testImplementation project(':jdbc-test') + testRuntimeOnly 'com.vertica.jdbc:vertica-jdbc:24.1.0-0' + + compileOnly 'org.jetbrains:annotations:24.0.1' +} diff --git a/modules/vertica-ce/micro_focus_end_user_license_agreement.pdf b/modules/vertica-ce/micro_focus_end_user_license_agreement.pdf new file mode 100644 index 00000000000..3407b9a0336 Binary files /dev/null and b/modules/vertica-ce/micro_focus_end_user_license_agreement.pdf differ diff --git a/modules/vertica-ce/src/main/java/org/testcontainers/containers/VerticaCEContainer.java b/modules/vertica-ce/src/main/java/org/testcontainers/containers/VerticaCEContainer.java new file mode 100644 index 00000000000..b3ea983c9d3 --- /dev/null +++ b/modules/vertica-ce/src/main/java/org/testcontainers/containers/VerticaCEContainer.java @@ -0,0 +1,315 @@ +package org.testcontainers.containers; + +import lombok.Getter; +import org.apache.commons.lang3.StringUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy; +import org.testcontainers.utility.DockerImageName; + +import java.time.Duration; + +/** + * TestContainer for Vertica CE databases + *

+ * todo - document vsql + *

+ *

+ * todo - add TLS support + *

+ *

+ * todo- "deploy: mode: global" ?? + *

+ * + * {@see https://www.vertica.com/} + * {@see https://docs.vertica.com/12.0.x/en/getting-started/introducing-vmart-example-db/} + * {@see https://www.microfocus.com/en-us/legal/software-licensing} + * + * @param this class + */ +@Getter +public class VerticaCEContainer> extends JdbcDatabaseContainer { + + private static final Logger LOG = LoggerFactory.getLogger(VerticaCEContainer.class); + + private static final String JDBC_SUBPROTOCOL = "vertica"; + + private static final String TEST_QUERY_STRING = "SELECT 1"; + + private static final String JDBC_DRIVER_CLASSNAME = "com.vertica.jdbc.Driver"; + + /** Short name of database container */ + public static final String NAME = "Vertica"; + + /** Default image name */ + public static final String IMAGE = "vertica/vertica-ce"; + + /** Default version */ + public static final String DEFAULT_TAG = "24.1.0-0"; + + /** Default full image name */ + public static final DockerImageName DEFAULT_IMAGE_NAME = DockerImageName.parse(IMAGE); + + /** Default Vertica database port */ + public static final Integer VERTICA_DATABASE_PORT = 5433; + + /** Default Vertica database and data warehouse ports */ + public static final Integer[] VERTICA_PORTS = { VERTICA_DATABASE_PORT, 5444 }; + + /** Docker image's default database name */ + public static final String DEFAULT_DATABASE = "vmart"; + + /** Docker image's default user name */ + public static final String DEFAULT_APP_DB_USER = "dbadmin"; + + /** Docker image's default timezone */ + public static final String DEFAULT_TZ = "Europe/Prague"; + + /** Docker image's default password */ + static final String DEFAULT_APP_DB_PASSWORD = "vertica"; + + private String databaseName = DEFAULT_DATABASE; + + private String username = DEFAULT_APP_DB_USER; + + private String password = DEFAULT_APP_DB_PASSWORD; + + private String tz = DEFAULT_TZ; + + private Integer loginTimeout; + + private String keyStorePath; + + private String keyStorePassword; + + private String trustStorePath; + + private String trustStorePassword; + + /** + * Default constructor + */ + public VerticaCEContainer() { + this(DEFAULT_IMAGE_NAME.withTag(DEFAULT_TAG)); + } + + /** + * Constructor taking image name + * + * @param dockerImageName image name + */ + public VerticaCEContainer(final DockerImageName dockerImageName) { + super(dockerImageName); + dockerImageName.assertCompatibleWith(DEFAULT_IMAGE_NAME); + this.waitStrategy = + new LogMessageWaitStrategy() + .withRegEx(".*Vertica is now running.*\\s") + .withStartupTimeout(Duration.ofMinutes(5)); + + for (Integer port : VERTICA_PORTS) { + this.addExposedPort(port); + } + } + + /** + * {@inheritDoc} + */ + @Override + protected void configure() { + configureDockerContainer(); + configureUrlParameters(); + } + + /** + * Configure docker container + */ + void configureDockerContainer() { + if (!DEFAULT_APP_DB_USER.equals(username)) { + addEnv("APP_DB_USER", username); + } + + if (!DEFAULT_APP_DB_PASSWORD.equals(password)) { + addEnv("APP_DB_PASSWORD", password); + } + + if (!DEFAULT_TZ.equals(tz)) { + addEnv("TZ", tz); + } + } + + /** + * Configure UrlParameters + */ + void configureUrlParameters() { + urlParameters.put("user", username); + if (StringUtils.isNotBlank(password)) { + urlParameters.put("password", password); + } + + if (loginTimeout != null) { + urlParameters.put("loginTimeout", Integer.toString(loginTimeout)); + } + + if (StringUtils.isNotBlank(keyStorePath) && StringUtils.isNotBlank(keyStorePassword)) { + urlParameters.put("KeyStorePath", keyStorePath); + urlParameters.put("KeyStorePassword", keyStorePassword); + } + + if (StringUtils.isNotBlank(trustStorePath) && StringUtils.isNotBlank(trustStorePassword)) { + urlParameters.put("TrustStorePath", trustStorePath); + urlParameters.put("TrustStorePassword", trustStorePassword); + } + } + + /** + * Get recommended driver classname + * + * @return recommended driver classname + */ + @Override + public String getDriverClassName() { + return JDBC_DRIVER_CLASSNAME; + } + + /** + * Get container's JDBC URL + * + * @return container's JDBC URL + */ + @Override + public String getJdbcUrl() { + final String additionalUrlParams = constructUrlParameters("?", "&"); + return String.format( + "jdbc:%s://%s:%s/%s%s", + JDBC_SUBPROTOCOL, + getHost(), + getMappedPort(VERTICA_DATABASE_PORT), + databaseName, + additionalUrlParams + ); + } + + /** + * Get test query string (for DataSources) + * + * @return test query string + */ + @Override + public String getTestQueryString() { + return TEST_QUERY_STRING; + } + + /** + * Specify custom timezone + * + * @param tz time zone (default "Europe/Prague") + * @return this object + */ + public SELF withTZ(final String tz) { + this.tz = tz; + return self(); + } + + /** + * Specify custom database name + * + * @param databaseName database name + * @return this object + */ + @Override + public SELF withDatabaseName(final String databaseName) { + this.databaseName = databaseName; + return self(); + } + + /** + * Specify custom database username + * + * @param username username + * @return this object + */ + @Override + public SELF withUsername(final String username) { + this.username = username; + return self(); + } + + /** + * Specify custom database user password + * + * @param password user password + * @return this object + */ + @Override + public SELF withPassword(final String password) { + this.password = password; + return self(); + } + + /** + * Specify login timeout + * + * @param loginTimeout timeout (in seconds?) + * @return this object + */ + public SELF withLoginTimeout(final Integer loginTimeout) { + this.loginTimeout = loginTimeout; + return self(); + } + + /** + * Specify keyStore path. Must be JKS file. + * + * TODO: verify file exists and is JKS file. + * + * @param keyStorePath path to keyStore file (JKS) + * @return this object + */ + public SELF withKeyStorePath(final String keyStorePath) { + this.keyStorePath = keyStorePath; + return self(); + } + + /** + * Specify keyStore password + * + * @param keyStorePassword keyStore password + * @return this object + */ + public SELF withKeyStorePassword(final String keyStorePassword) { + this.keyStorePassword = keyStorePassword; + return self(); + } + + /** + * Specify trustStore path. Must be JKS file. + * + * TODO: verify file exists and is JKS file. + * + * @param trustStorePath path to trustStore file (JKS) + * @return this object + */ + public SELF withTrustStorePath(final String trustStorePath) { + this.trustStorePath = trustStorePath; + return self(); + } + + /** + * Specify trustStore password + * + * @param trustStorePassword truststore password + * @return this object + */ + public SELF withTrustStorePassword(final String trustStorePassword) { + this.trustStorePassword = trustStorePassword; + return self(); + } + + /** + * {@inheritDoc} + */ + @Override + protected void waitUntilContainerStarted() { + getWaitStrategy().waitUntilReady(this); + } +} diff --git a/modules/vertica-ce/src/main/java/org/testcontainers/containers/VerticaCEContainerProvider.java b/modules/vertica-ce/src/main/java/org/testcontainers/containers/VerticaCEContainerProvider.java new file mode 100644 index 00000000000..e05ff67519a --- /dev/null +++ b/modules/vertica-ce/src/main/java/org/testcontainers/containers/VerticaCEContainerProvider.java @@ -0,0 +1,34 @@ +package org.testcontainers.containers; + +import org.testcontainers.utility.DockerImageName; + +/** + * Factory for Vertica-CE containers. + */ +@SuppressWarnings("rawtypes") +public class VerticaCEContainerProvider extends JdbcDatabaseContainerProvider { + + /** + * {@inheritDoc} + */ + @Override + public boolean supports(String databaseType) { + return databaseType.equals(VerticaCEContainer.NAME); + } + + /** + * {@inheritDoc} + */ + @Override + public JdbcDatabaseContainer newInstance() { + return newInstance(VerticaCEContainer.DEFAULT_TAG); + } + + /** + * {@inheritDoc} + */ + @Override + public JdbcDatabaseContainer newInstance(String tag) { + return new VerticaCEContainer(DockerImageName.parse(VerticaCEContainer.IMAGE).withTag(tag)); + } +} diff --git a/modules/vertica-ce/src/main/resources/META-INF/services/org.testcontainers.containers.JdbcDatabaseContainerProvider b/modules/vertica-ce/src/main/resources/META-INF/services/org.testcontainers.containers.JdbcDatabaseContainerProvider new file mode 100644 index 00000000000..68dbf82fbb1 --- /dev/null +++ b/modules/vertica-ce/src/main/resources/META-INF/services/org.testcontainers.containers.JdbcDatabaseContainerProvider @@ -0,0 +1 @@ +org.testcontainers.containers.VerticaCEContainerProvider diff --git a/modules/vertica-ce/src/test/java/org/testcontainers/containers/VerticaCETestImages.java b/modules/vertica-ce/src/test/java/org/testcontainers/containers/VerticaCETestImages.java new file mode 100644 index 00000000000..a7b75a4e4b7 --- /dev/null +++ b/modules/vertica-ce/src/test/java/org/testcontainers/containers/VerticaCETestImages.java @@ -0,0 +1,14 @@ +package org.testcontainers.containers; + +import org.testcontainers.utility.DockerImageName; + +/** + * Vertica docker images + *

+ * Vertica has three editions: Community, Express, and Premium, plus a few additional products. + *

+ * The only edition available at docker hub is the Community Edition. + */ +public interface VerticaCETestImages { + DockerImageName VERTICA_CE_TEST_IMAGE = DockerImageName.parse("vertica/vertica-ce:24.1.0-0"); +} diff --git a/modules/vertica-ce/src/test/java/org/testcontainers/containers/jdbc/VerticaJDBCDriverTest.java.sav b/modules/vertica-ce/src/test/java/org/testcontainers/containers/jdbc/VerticaJDBCDriverTest.java.sav new file mode 100644 index 00000000000..10cf11163b7 --- /dev/null +++ b/modules/vertica-ce/src/test/java/org/testcontainers/containers/jdbc/VerticaJDBCDriverTest.java.sav @@ -0,0 +1,26 @@ +package org.testcontainers.containers.jdbc; + +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.testcontainers.jdbc.AbstractJDBCDriverTest; + +import java.util.Arrays; +import java.util.EnumSet; + +// disabled until I can figure out how to use ":jdbc-test" + +@RunWith(Parameterized.class) +public class VerticaJDBCDriverTest extends AbstractJDBCDriverTest { + + @Parameterized.Parameters(name = "{index} - {0}") + public static Iterable data() { + return Arrays.asList( + new Object[][] { + { + "jdbc:tc:vertica/vertica-ce:23.3.0-0://hostname/", + EnumSet.of(Options.JDBCParams), + }, + } + ); + } +} diff --git a/modules/vertica-ce/src/test/java/org/testcontainers/containers/junit/vertica/SimpleVerticaCETest.java b/modules/vertica-ce/src/test/java/org/testcontainers/containers/junit/vertica/SimpleVerticaCETest.java new file mode 100644 index 00000000000..1d7e76def61 --- /dev/null +++ b/modules/vertica-ce/src/test/java/org/testcontainers/containers/junit/vertica/SimpleVerticaCETest.java @@ -0,0 +1,42 @@ +package org.testcontainers.containers.junit.vertica; + +import org.junit.Test; +import org.testcontainers.containers.VerticaCEContainer; +import org.testcontainers.containers.VerticaCETestImages; +import org.testcontainers.db.AbstractContainerDatabaseTest; + +import java.sql.ResultSet; +import java.sql.SQLException; + +import static org.assertj.core.api.Assertions.assertThat; + +public class SimpleVerticaCETest extends AbstractContainerDatabaseTest { + + // static { + // LogManager.getLogManager().getLogger("").setLevel(Level.OFF); + // } + + @Test + public void testSimple() throws SQLException { + try (VerticaCEContainer vertica = new VerticaCEContainer<>(VerticaCETestImages.VERTICA_CE_TEST_IMAGE)) { + vertica.start(); + + ResultSet resultSet = performQuery(vertica, "SELECT 1"); + int resultSetInt = resultSet.getInt(1); + assertThat(resultSetInt).as("A basic SELECT query succeeds").isEqualTo(1); + assertHasCorrectExposedAndLivenessCheckPorts(vertica); + } + } + + /** + * Vertica exposes both a database port (5433) and one or more data warehouse ports (5444, ...). We + * only care about the database port. + * + * @param vertica + */ + private void assertHasCorrectExposedAndLivenessCheckPorts(VerticaCEContainer vertica) { + assertThat(vertica.getExposedPorts()).contains(VerticaCEContainer.VERTICA_DATABASE_PORT); + assertThat(vertica.getLivenessCheckPortNumbers()) + .contains(vertica.getMappedPort(VerticaCEContainer.VERTICA_DATABASE_PORT)); + } +} diff --git a/modules/vertica-ce/src/test/resources/logback-test.xml b/modules/vertica-ce/src/test/resources/logback-test.xml new file mode 100644 index 00000000000..88610c1b0ef --- /dev/null +++ b/modules/vertica-ce/src/test/resources/logback-test.xml @@ -0,0 +1,17 @@ + + + + + + %d{HH:mm:ss.SSS} %-5level %logger - %msg%n + + + + + + + + + +