Skip to content

[Bug]: Schema Registry cannot connect to ConfluentKafkaContainer #9631

New issue
New issue
Closed as not planned
Closed as not planned
[Bug]: Schema Registry cannot connect to ConfluentKafkaContainer#9631
Labels
type/bug
@dogruis

Description

@dogruis
dogruis
opened on Dec 13, 2024

Module

Core

Testcontainers version

1.20.4

Using the latest Testcontainers version?

Yes

Host OS

alpine

Host Arch

arm64

Docker version

4.35.1

What happened?

I am trying to update one of your class KafkaContainerDef because you flagged it as deprecated. I switched to ConfluentKafkaContainer but the schema registry fails to start because it cannot find connect to kafka. Here is the log dump and how I setup my containers

My schema registry setup

    public SchemaRegistry(Map<String, String> envSettings) {
        try (GenericContainer<?> genericContainer = new GenericContainer<>(IMAGE_NAME)
                .withExposedPorts(SCHEMA_REGISTRY_PORT)
                .withNetwork(Network.SHARED)
                .withLogConsumer(new Slf4jLogConsumer(log).withPrefix("KafkaSchemaRegistry"))
                .withEnv("SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS", "PLAINTEXT://0.0.0.0:9092")
                .withEnv(Map.of(
                        "SCHEMA_REGISTRY_HOST_NAME", "localhost",
                        "SCHEMA_REGISTRY_LISTENERS", "http://0.0.0.0:" + SCHEMA_REGISTRY_PORT,
                        "SCHEMA_REGISTRY_DEBUG", "true",
                        "SCHEMA_REGISTRY_KAFKASTORE_TIMEOUT_MS", "10000"))
                .withEnv(envSettings)) {
            this.container = genericContainer;
        }
    }

My kafka setup

container = new ConfluentKafkaContainer(IMAGE_NAME)
        .withNetwork(Network.SHARED)
        .withExposedPorts(9092)
        .withLogConsumer(new Slf4jLogConsumer(log).withPrefix("Apache Kafka"))
        .withEnv(envSettings);

When running tests it just fails because the adminclient cannot connect to kafka.

INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: [2024-12-13 10:17:24,677] INFO [AdminClient clientId=adminclient-1] Node -1 disconnected. (org.apache.kafka.clients.NetworkClient)
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: [2024-12-13 10:17:24,678] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (/127.0.0.1:9092) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)

I also tried to setup SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS but nothing works

    public String getBootstrapServersForContainer() {
        return getAddress("", this.container.getNetworkAliases().get(0), 9092);
    }

Relevant log output

INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: ===> User
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: ===> Configuring ...
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: ===> Running preflight checks ... 
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: ===> Check if Kafka is healthy ...
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: [2024-12-13 10:17:24,552] INFO AdminClientConfig values: 
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	auto.include.jmx.reporter = true
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	bootstrap.controllers = []
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	bootstrap.servers = [PLAINTEXT://0.0.0.0:9092]
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	client.dns.lookup = use_all_dns_ips
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	client.id = 
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	connections.max.idle.ms = 300000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	default.api.timeout.ms = 60000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	enable.metrics.push = true
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	metadata.max.age.ms = 300000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	metadata.recovery.strategy = none
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	metric.reporters = []
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	metrics.num.samples = 2
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	metrics.recording.level = INFO
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	metrics.sample.window.ms = 30000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	receive.buffer.bytes = 65536
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	reconnect.backoff.max.ms = 1000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	reconnect.backoff.ms = 50
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	request.timeout.ms = 30000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	retries = 2147483647
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	retry.backoff.max.ms = 1000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	retry.backoff.ms = 100
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.client.callback.handler.class = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.jaas.config = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.kerberos.kinit.cmd = /usr/bin/kinit
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.kerberos.min.time.before.relogin = 60000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.kerberos.service.name = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.kerberos.ticket.renew.jitter = 0.05
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.kerberos.ticket.renew.window.factor = 0.8
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.callback.handler.class = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.class = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.connect.timeout.ms = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.read.timeout.ms = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.refresh.buffer.seconds = 300
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.refresh.min.period.seconds = 60
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.refresh.window.factor = 0.8
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.refresh.window.jitter = 0.05
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.retry.backoff.max.ms = 10000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.login.retry.backoff.ms = 100
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.mechanism = GSSAPI
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.clock.skew.seconds = 30
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.expected.audience = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.expected.issuer = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.jwks.endpoint.url = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.scope.claim.name = scope
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.sub.claim.name = sub
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	sasl.oauthbearer.token.endpoint.url = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	security.protocol = PLAINTEXT
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	security.providers = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	send.buffer.bytes = 131072
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	socket.connection.setup.timeout.max.ms = 30000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	socket.connection.setup.timeout.ms = 10000
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.cipher.suites = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.endpoint.identification.algorithm = https
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.engine.factory.class = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.key.password = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.keymanager.algorithm = SunX509
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.keystore.certificate.chain = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.keystore.key = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.keystore.location = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.keystore.password = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.keystore.type = JKS
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.protocol = TLSv1.3
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.provider = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.secure.random.implementation = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.trustmanager.algorithm = PKIX
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.truststore.certificates = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.truststore.location = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.truststore.password = null
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: 	ssl.truststore.type = JKS
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT:  (org.apache.kafka.clients.admin.AdminClientConfig)
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: [2024-12-13 10:17:24,670] INFO Kafka version: 7.8.0-ccs (org.apache.kafka.common.utils.AppInfoParser)
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: [2024-12-13 10:17:24,670] INFO Kafka commitId: cc7168da1fddfcfd (org.apache.kafka.common.utils.AppInfoParser)
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: [2024-12-13 10:17:24,670] INFO Kafka startTimeMs: 1734085044668 (org.apache.kafka.common.utils.AppInfoParser)
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: [2024-12-13 10:17:24,677] INFO [AdminClient clientId=adminclient-1] Node -1 disconnected. (org.apache.kafka.clients.NetworkClient)
INFO  com.rbs.digital.messaging.library.test.docker.SchemaRegistry - [KafkaSchemaRegistry] STDOUT: [2024-12-13 10:17:24,678] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (/127.0.0.1:9092) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)

Additional Information

cp-kafka and cp-schema-registry images are 7.8.0

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions