KAFKA-19719 --no-initial-controllers should not assume kraft.version=… #20624
+147
−131
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apache#20604) ``` commit ec37eb5 (HEAD -> KAFKA-19719-cherry-pick-41, origin/KAFKA-19719-cherry-pick-41) Author: Kevin Wu <[email protected]> Date: Thu Sep 25 11:56:16 2025 -0500 KAFKA-19719: --no-initial-controllers should not assume kraft.version=1 (apache#20551) Just because a controller node sets --no-initial-controllers flag does not mean it is necessarily running kraft.version=1. The more precise meaning is that the controller node being formatted does not know what kraft version the cluster should be in, and therefore it is only safe to assume kraft.version=0. Only by setting --standalone,--initial-controllers, or --no-initial-controllers AND not specifying the controller.quorum.voters static config, is it known kraft.version > 0. For example, it is a valid configuration (although confusing) to run a static quorum defined by controller.quorum.voters but have all the controllers format with --no-initial-controllers. In this case, specifying --no-initial-controllers alongside a metadata version that does not support kraft.version=1 causes formatting to fail, which is does not support kraft.version=1 causes formatting to fail, which is a regression. Additionally, the formatter should not check the kraft.version against the release version, since kraft.version does not actually depend on any release version. It should only check the kraft.version against the static voters config/format arguments. This PR also cleans up the integration test framework to match the semantics of formatting an actual cluster. Reviewers: TengYao Chi <[email protected]>, Kuan-Po Tseng <[email protected]>, Chia-Ping Tsai <[email protected]>, José Armando García Sancio <[email protected]> Conflicts: core/src/main/scala/kafka/tools/StorageTool.scala Minor conflicts. Keep changes from cherry-pick. core/src/test/java/kafka/server/ReconfigurableQuorumIntegrationTest.java Remove auto-join tests, since 4.1 does not support it. docs/ops.html Keep docs section from cherry-pick. metadata/src/test/java/org/apache/kafka/metadata/storage/FormatterTest.java Minor conflicts. Keep cherry-picked changes. test-common/test-common-runtime/src/main/java/org/apache/kafka/common/test/KafkaClusterTestKit.java Conflicts due to integration test framework changes. Keep new changes. commit 02d58b1 (upstream/4.1) ``` Reviewers: Chia-Ping Tsai <[email protected]> Conflicts: core/src/main/scala/kafka/tools/StorageTool.scala core/src/test/java/kafka/server/ReconfigurableQuorumIntegrationTest.java core/src/test/java/kafka/testkit/KafkaClusterTestKit.java docs/ops.html metadata/src/main/java/org/apache/kafka/metadata/storage/Formatter.java metadata/src/test/java/org/apache/kafka/metadata/storage/FormatterTest.java server-common/src/main/java/org/apache/kafka/server/common/KRaftVersion.java test-common/test-common-runtime/src/main/java/org/apache/kafka/common/test/TestKitNodes.java Minor conflicts mostly. The main difference is 3.9 does not have integration test support for KIP-853, so did not cherry-pick those changes for this branch.
|
@kevin-wu24 Could you please check the build error? |
chia7712
reviewed
Oct 6, 2025
There was a problem hiding this comment.
@kevin-wu24 thanks for this backport. Could you please fix following build error?
> Task :metadata:compileTestJava
/home/chia7712/project/kafka/metadata/src/test/java/org/apache/kafka/metadata/storage/FormatterTest.java:415: error: cannot find symbol
assertEquals(List.of(
^
symbol: method of(String,String)
location: interface List
/home/chia7712/project/kafka/metadata/src/test/java/org/apache/kafka/metadata/storage/FormatterTest.java:512: error: cannot find symbol
assertEquals(List.of(
^
symbol: method of(String,String)
location: interface List
2 errors
| return Collections.singletonMap( | ||
| MetadataVersion.FEATURE_NAME, MetadataVersion.IBP_3_9_IV0.featureLevel()); | ||
| } | ||
| return new HashMap<>(); |
|
Thanks @chia7712, the build errors are fixed. It looks like there are some test failures, but they don't appear to be related to my changes. Could you take a look and confirm? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…1 (#20604)
Reviewers: Chia-Ping Tsai [email protected]
Conflicts:
core/src/main/scala/kafka/tools/StorageTool.scala
core/src/test/java/kafka/server/ReconfigurableQuorumIntegrationTest.java
core/src/test/java/kafka/testkit/KafkaClusterTestKit.java
docs/ops.html
metadata/src/main/java/org/apache/kafka/metadata/storage/Formatter.java
metadata/src/test/java/org/apache/kafka/metadata/storage/FormatterTest.java
server-common/src/main/java/org/apache/kafka/server/common/KRaftVersion.java
test-common/test-common-runtime/src/main/java/org/apache/kafka/common/test/TestKitNodes.java
Minor conflicts mostly. The main difference is 3.9 does not have integration test support
for KIP-853, so did not cherry-pick those changes for this branch.