fix: allow automq.zone.cidr.blocks to be set via static server.properties #3154
Conversation
|
@manmohak07 The all dynamic configs which aren't added to DynamicBrokerConfig.AllDynamicConfigs() need to be configured in a bundle. It means the config cannot change alone. Maybe we can pass KafkaConfig into DefaultClientRackProvider and then obtain the static config of automq.zone.cidr.blocks from KafkaConfig. |
Static Config Loads on StartupSetup: # Add static config to server.properties
docker exec automq-single-server bash -c "echo 'automq.zone.cidr.blocks=static-zone-a@172.16.0.0/16<>static-zone-b@172.17.0.0/16' >> /opt/automq/kafka/config/kraft/server.properties"
# Verify config was added
docker exec automq-single-server bash -c "grep 'automq.zone.cidr.blocks' /opt/automq/kafka/config/kraft/server.properties"Output: Delete any existing dynamic config to test static only: docker exec automq-single-server bash -c "/opt/automq/kafka/bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-default --alter --delete-config 'automq.zone.cidr.blocks'"Output: Restart broker: docker restart automq-single-serverVerify static config was loaded: docker exec automq-single-server bash -c "grep -h 'Initialized with static zone CIDR blocks' /opt/automq/kafka/logs/server.log*"Output: Result: Static config successfully loaded from Dynamic Config Still WorksSet dynamic config: docker exec automq-single-server bash -c "/opt/automq/kafka/bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-default --alter --add-config 'automq.zone.cidr.blocks=dynamic-test@10.99.0.0/16'"Output: Verify dynamic config was applied: docker exec automq-single-server bash -c "grep 'apply new zone CIDR blocks dynamic-test' /opt/automq/kafka/logs/server.log*"Output: |
|
There was an unused import (import kafka.server.DynamicBrokerConfig;), I removed it, and fixed the order by running |
core/src/main/java/kafka/automq/zerozone/DefaultClientRackProvider.java
Outdated
Show resolved
Hide resolved
|
Thanks for implementing the fix. Would it be possible to backport this change to |
cc: @superhx |
Hi, the fix will be included in 1.6.4 |
Fixes #3115
Problem
The
automq.zone.cidr.blocksconfig option cannot be set via staticserver.properties. It can only be configured dynamically viakafka-configs.sh.This is a blocker for Kubernetes operators like Strimzi, which assemble config maps and mount them as static properties files. Users following the AutoMQ documentation expect to set this config in
spec.kafka.config, but it gets ignored on startup.Root Cause
In
DefaultClientRackProvider.java, the static initializer addsautomq.zone.cidr.blockstoDynamicBrokerConfig.AllDynamicConfigs():When a config is in
AllDynamicConfigs, Kafka treats it as "dynamic-only" and ignores it in staticserver.properties.Solution
I removed the line that adds the config to
AllDynamicConfigs. The config will still be reconfigurable (via theReconfigurableinterface) but can now also be set statically.This preserves the expected priority order (per
DynamicBrokerConfig.scalalines 570-580):DYNAMIC_BROKER_CONFIG(per-broker dynamic)DYNAMIC_DEFAULT_BROKER_CONFIG(cluster-wide dynamic)STATIC_BROKER_CONFIG(server.properties) - now worksDEFAULT_CONFIGDynamic config still takes priority over static config when set:
Testing
Verified:
Before Fix (static config ignored)
Add static config:
Check if loaded:
Static config was ignored (shows
null).After Fix (static config works)
Add static config:
Check logs:
Static config loaded on startup.
Dynamic config still works (and takes priority)
Dynamic config applied and overrides static.
Committer Checklist (excluded from commit message)