How to configure kafka url in openmeter?

[mkschreder]

I checked the docker compose and I see that all services are configured to listen on localhost. This works when openmeter by default assumes that they are on localhost, but what about when openmeter is inside the dockercompose and needs to bootstrap with kafka at "kafka" hostname instead?

I haven't found any documentation abour how to configure this. Does anyone know?

openmeter             | %3|1757857223.489|FAIL|rdkafka#producer-1| [thrd:127.0.0.1:29092/bootstrap]: 127.0.0.1:29092/bootstrap: Connect to ipv4#127.0.0.1:29092 failed: Connection refused (after 0ms in state CONNECT)
openmeter             | %3|1757857223.540|FAIL|rdkafka#producer-1| [thrd:127.0.0.1:29092/bootstrap]: 127.0.0.1:29092/bootstrap: Connect to ipv4#127.0.0.1:29092 failed: Connection refused (after 0ms in state CONNECT, 1 identical error(s) suppressed)
openmeter             | %3|1757857253.571|FAIL|rdkafka#producer-1| [thrd:127.0.0.1:29092/bootstrap]: 127.0.0.1:29092/bootstrap: Connect to ipv4#127.0.0.1:29092 failed: Connection refused (after 0ms in state CONNECT, 582 identical error(s) suppressed)

What do I need to set in the environment to make this work?

I have tried all kinds of options that could make sense:

KAFKA_BROKER=kafka:9092
KAFKA_BROKERS=kafka:9092
KAFKA_BOOTSTRAP_SERVERS=kafka:9092
OPENMETER_EVENTS_KAFKA_BROKERS=kafka:9092
OPENMETER_SINK_KAFKA_BROKERS=kafka:9092

[turip]

The correct env var for broker is INGEST_KAFKA_BROKER (but we might change it to KAFKA_BROKER pretty soon, as the INGEST_ prefix is due to kafka being only used for ingested events in some of the early versions).

The full list of config options is available here:

openmeter/app/config/ingest.go

Lines 61 to 85 in cde8a86

	type KafkaConfiguration struct {
	Broker string
	SecurityProtocol string
	SaslMechanisms string
	SaslUsername string
	SaslPassword string
	StatsInterval pkgkafka.TimeDurationMilliSeconds
	// BrokerAddressFamily defines the IP address family to be used for network communication with Kafka cluster
	BrokerAddressFamily pkgkafka.BrokerAddressFamily
	// SocketKeepAliveEnable defines if TCP socket keep-alive is enabled to prevent closing idle connections
	// by Kafka brokers.
	SocketKeepAliveEnabled bool
	// TopicMetadataRefreshInterval defines how frequently the Kafka client needs to fetch metadata information
	// (brokers, topic, partitions, etc) from the Kafka cluster.
	// The 5 minutes default value is appropriate for mostly static Kafka clusters, but needs to be lowered
	// in case of large clusters where changes are more frequent.
	// This value must not be set to value lower than 10s.
	TopicMetadataRefreshInterval pkgkafka.TimeDurationMilliSeconds
	// Enable contexts for extensive debugging of librdkafka.
	// See: https://github.com/confluentinc/librdkafka/blob/master/INTRODUCTION.md#debug-contexts
	DebugContexts pkgkafka.DebugContexts
	}

[mkschreder]

Thanks that works. But what database setting am I missing now?

openmeter | {"time":"2025-09-15T15:02:39.95183009Z","level":"ERROR","msg":"failed to initialize database","deployment.environment.name":"unknown","service.name":"openmeter-backend","service.version":"main","telemetry.sdk.language":"go","telemetry.sdk.name":"opentelemetry","telemetry.sdk.version":"1.38.0","error":"failed to migrate db: querying server version failed to connect to user= database=: /tmp/.s.PGSQL.5432 (/tmp): dial error: dial unix /tmp/.s.PGSQL.5432: connect: no such file or directory"}

[mkschreder]

Got past that one. Had to use POSTGRES_URL instead of DATABASE _URL.

[mkschreder]

That worked, but