Is Zookeeper mandated for using Burrow?

[abdelrahman-a-ibrahim]

Hi Team,

I am trying to use Burrow for my Kafka cluster but I didn't provide Zookeeper configuration since I might switch to KRaft mode soon and thought it was not mandatory to use Zookeeper.

My setup failed and my pod on K8s couldn't start and entered a CrashLoopBackOff issue and found the following in the pod logs:

Reading configuration from /etc/burrow
panic: No Zookeeper servers specified [recovered]
        panic: No Zookeeper servers specified [recovered]
        panic: No Zookeeper servers specified

goroutine 1 [running]:
main.handleExit()
        /home/runner/work/Burrow/Burrow/main.go:63 +0xf8
panic({0xbeb8a0, 0xc000210a30})
        /opt/hostedtoolcache/go/1.20.1/x64/src/runtime/panic.go:884 +0x213
go.uber.org/zap/zapcore.CheckWriteAction.OnWrite(0x0?, 0xc0001001a0?, {0x0?, 0x0?, 0xc00021a700?})
        /home/runner/go/pkg/mod/go.uber.org/zap@v1.24.0/zapcore/entry.go:198 +0x65
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000200b60, {0x0, 0x0, 0x0})
        /home/runner/go/pkg/mod/go.uber.org/zap@v1.24.0/zapcore/entry.go:264 +0x3ec
go.uber.org/zap.(*Logger).Panic(0x0?, {0xd26f47?, 0x0?}, {0x0, 0x0, 0x0})
        /home/runner/go/pkg/mod/go.uber.org/zap@v1.24.0/logger.go:258 +0x59
github.com/linkedin/Burrow/core.configureCoordinators.func1()
        /home/runner/work/Burrow/Burrow/core/burrow.go:96 +0x56
panic({0xbeb8a0, 0xebb050})
        /opt/hostedtoolcache/go/1.20.1/x64/src/runtime/panic.go:884 +0x213
github.com/linkedin/Burrow/core/internal/zookeeper.(*Coordinator).Configure(0xc000232400)
        /home/runner/work/Burrow/Burrow/core/internal/zookeeper/coordinator.go:64 +0x26b
github.com/linkedin/Burrow/core.configureCoordinators(0xc0002203c0, {{0xec26e0, 0xc000232400}, {0xec26b0, 0xc000206c60}, {0xec2500, 0xc00021a660}, {0xec25c0, 0xc000206cf0}, {0xec25f0, ...}, ...})
        /home/runner/work/Burrow/Burrow/core/burrow.go:103 +0xb3
github.com/linkedin/Burrow/core.Start(0xc000220360?, 0xc00012bef0?)
        /home/runner/work/Burrow/Burrow/core/burrow.go:151 +0x3dc
main.main()
        /home/runner/work/Burrow/Burrow/main.go:114 +0x4d2

Does this mean I can't use Burrow without providing a Zookeeper config? What if I am using KRaft?

Burrow image used: ghcr.io/linkedin/burrow/burrow:v1.6.0

Configurations

    [logging]
    filename="logs/burrow.log"

    [client-profile.myclient]
    kafka-version="3.6.1"
    client-id="borrow-client"
    sasl="saslprofile"

    [sasl.saslprofile]
    username="myuser"
    password="mypassword"
    mechanism="SCRAM-SHA-512"

    [httpserver.mylistener]
    address=":8080"

    [cluster.myclustername]
    class-name="kafka"
    servers=[ "my.single.server" ]    
    client-profile="myclient"
    topic-refresh=60
    offset-refresh=30
    groups-reaper-refresh=30

    [consumer.myconsumers]
    class-name="kafka"
    cluster="myclustername"
    servers=[ "my.single.server" ]
    client-profile="myclient"
    offsets-topic="__consumer_offsets"
    start-latest=true
    group-allowlist=".*"

I will appreciate any help from the team or the community, TIA :")

[petern-sc]

duplicate #534

[trvrnrth]

#838 removes the zookeeper dependency when notifiers are not in use which appears to be your use case.