Could not find a valid Docker environment when using Gradle Shared Build Service

[debapgithub]

Hi,

I am new to testcontainers and setting up for a gradle service where I wanted to achieve the following in sequence.

When gradle build command will run : ./gradlew clean build

1. MySql Testcontainer should start
2. DB migrations should apply against the testcontainer DB
3. generateJooq task should execute against the testcontainer DB
4. Run the integration tests against testcontainer
5. At the end of the build process, testcotnainer should stop automatically.

I have tried starting MySql testcontainer as a gradle task and I was able to execute the following tasks properly except one issue due to which I have decided to switch to another approach where I can start the MySql testcontainer as a gradle Shared service so that it will automaticllay manage the testcontainer lifecycle.

Here is the code which will start the testcontainer as a gradle shared service and it worked fine as well. But next day when I tried running the gradle command, I started getting the below error.
Error

* What went wrong:
Execution failed for task ':startMySQLContainer'.
> Failed to create service 'mysqlContainerService'.
   > Could not create an instance of type MySQLContainerService.
      > Could not find a valid Docker environment. Please see logs and check configuration

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

Gradle Code

import org.testcontainers.containers.MySQLContainer;
import org.gradle.api.services.BuildService
import org.gradle.api.services.BuildServiceParameters

buildscript {
    repositories {
        maven {
            name repoName
            url repoUrl
            credentials() {
                username = repoUsername
                password = repoPassword
            }
        }
    }

    dependencies {
        classpath "org.testcontainers:testcontainers:${dependencyManagement.importedProperties['testcontainers.version']}"
        classpath "org.testcontainers:mysql:${dependencyManagement.importedProperties['testcontainers.version']}"
        classpath "com.mysql:mysql-connector-j:${dependencyManagement.importedProperties['mysql.version']}"
    }
}


// Here we register service for providing our database during the build.
gradle.sharedServices.registerIfAbsent('mysqlContainerService', MySQLContainerService) { spec ->
    spec.parameters.databaseSchemaName.set(project.ext.databaseSchemaName)
}

/**
 * Build service for providing database container.
 */
abstract class MySQLContainerService implements BuildService<MySQLContainerService.Params>, AutoCloseable {
    interface Params extends BuildServiceParameters {
        Property<String> getDatabaseSchemaName()
    }
    private final MySQLContainer mysqlContainer;

    @javax.inject.Inject
    MySQLContainerService(Params params) {
        // Services are initialized lazily, on first request to them, so we start container immediately.
        long startTime = System.currentTimeMillis()
        println("DB Schema Name: ${params.getDatabaseSchemaName().get()}")
        mysqlContainer = new MySQLContainer(MySQLContainer.IMAGE).withDatabaseName(params.getDatabaseSchemaName().get())
        mysqlContainer.start()
        long duration = System.currentTimeMillis() - startTime
        println("MySQL Testcontainer Started successfully with URL: ${mysqlContainer.getJdbcUrl()}. Total time taken: ${duration} ms")
    }

    String getJdbcUrl() {
        return mysqlContainer.getJdbcUrl()
    }

    String getUsername() {
        return mysqlContainer.getUsername()
    }

    String getPassword() {
        return mysqlContainer.getPassword()
    }

    String getDriverClassName() {
        return mysqlContainer.getDriverClassName()
    }


    @Override
    void close() {
        // Ensure to stop container in the end
        if (mysqlContainer != null) {
            try {
                mysqlContainer.stop()
                println("MySQL container stopped successfully.")
            } catch (Exception e) {
                println("Failed to stop MySQL container: ${e.message}")
            }
        } else {
            println("MySQL container is already null (was not started).")
        }
    }
}

tasks.register('startMySQLContainer') {
    doLast {
        gradle.sharedServices.getRegistrations().getByName('mysqlContainerService')
    }
}

// Ensure the integrationTest uses the same container and stops it after
tasks.named('integrationTest').configure {
    dependsOn 'startMySQLContainer'
    doFirst {
        def service = gradle.sharedServices.getRegistrations().getByName('mysqlContainerService').service.get()
        systemProperty 'spring.datasource.url', service.getJdbcUrl()
        systemProperty 'spring.datasource.username', service.getUsername()
        systemProperty 'spring.datasource.password', service.getPassword()
        systemProperty 'spring.datasource.driver-class-name', service.getDriverClassName()
    }

    doLast {
        systemProperties.remove('spring.datasource.url')
        systemProperties.remove('spring.datasource.username')
        systemProperties.remove('spring.datasource.password')
        systemProperties.remove('spring.datasource.driver-class-name')
    }
}

Not sure whats going wrong and how it worked first time when I did this code.

Thanks in advance!!

-Deba

[kiview]

Hey @debapgithub, can you please make sure your post is properly formatted? Would make reading the examples much easier. (edit: I just fixed the formatting in your original post)

TBH I think that is a level complexity for which I would I best need a reproducer project, to see if this is an issue inherent to your environment, or the Gradle setup.

Just to double check, the scenario is as follow:

• Using Testcontainers on your local machine with Docker Desktop using Gradle task so start the container works.
• Using Testcontainers on your local machine with Docker Desktop using Gradle shared service so start the container fails with Could not find a valid Docker environment

Is this correct?

[debapgithub]

@kiview

Sure I will do that.

1. Using Testcontainers on your local machine with Docker Desktop using Gradle task so start the container works. YES
2. Using Testcontainers on your local machine with Docker Desktop using Gradle shared service so start the container fails with Could not find a valid Docker environment : YES

[debapgithub]

Just adding something which I missed out at the beginning, I am seeing the below error before it says "Could not find a valid Docker environment. Please see logs and check configuration"

Can't instantiate a strategy from org.testcontainers.dockerclient.UnixSocketClientProviderStrategy (ClassNotFoundException). This probably means that cached configuration refers to a client provider class that is not available in this version of Testcontainers. Other strategies will be tried instead.
Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
As no valid configuration was found, execution cannot continue.
See https://java.testcontainers.org/on_failure.html for more details.

[kiview]

Oh wait, that is the reason actually:

Can't instantiate a strategy from org.testcontainers.dockerclient.UnixSocketClientProviderStrategy (ClassNotFoundException)

So when using the Gradle Shared Service, the JVM class loader seems to not be able to instantiate the class (which is part of Testcontainers). That seems to be like something to ask Gradle about, or open an issue with Gradle.

Hence I would suggest to not use Gradle Shared Services here to unblock the scenario.

[kiview]

Moved to an issue #9050.