[close #421] [br-release-1.0] br: Split for smaller batch of keys (#422)

[pingyu]

Cherry pick of #422

What problem does this PR solve?

Issue Number: close #421

Problem Description: br: split regions failed

What is changed and how does it work?

• Separate keys to smaller batch with maximum number of keys of 1024.
• Add flags to specify split region max keys and allowed gRPC message size.
• Use split-region-max-keys = 4 for integration tests.
• Adjust region split threshold to cover condition of more regions in integration tests.

Code changes

• No.

Check List for Tests

This PR has been tested by at least one of the following methods:

• Integration test

Manually confirm the coverage of split region with max keys = 4 by logs of integration tests.

[2025/04/16 10:06:03.597 +08:00] [INFO] [split.go:134] ["get split keys for region"] [len=24] [region=354]
[2025/04/16 10:06:03.597 +08:00] [INFO] [split.go:157] ["split regions"] [region="{ID=354,startKey=endKey=72000000696E6465FF783A5F3030303030FF3030303030303036FF3430333237330000FD,epoch=\"conf_ver:1 version:126 \",peers=\"id:355 store_id:1 \"}"] [keys="{total=4,keys=\"[72000000696E6465783A5F30303030303030303030303032353436363638,72000000696E6465783A5F30303030303030303030303032373434393630,72000000696E6465783A5F30303030303030303030303032383939313632,72000000696E6465783A5F30303030303030303030303033303433353134]\"}"] [ranges="{total=31,ranges=\"[\\\"[72000000696E6465783A5F30303030303030303030303032353030303030, 72000000696E6465783A5F30303030303030303030303032353436363638)\\\",\\\"(skip 29)\\\",\\\"[72000000696E6465783A5F30303030303030303030303037333832383332, 72000000696E6465783A5F30303030303030303030303037353030303030)\\\"]\",totalFiles=31,totalKVs=5000000,totalBytes=260000000,totalSize=260000000}"]
[2025/04/16 10:06:03.741 +08:00] [INFO] [split.go:186] ["scattered regions"] [count=4]
[2025/04/16 10:06:03.742 +08:00] [INFO] [split.go:157] ["split regions"] [region="{ID=354,startKey=72000000696E6465FF783A5F3030303030FF3030303030303033FF3034333531340000FD,endKey=72000000696E6465FF783A5F3030303030FF3030303030303036FF3430333237330000FD,epoch=\"conf_ver:1 version:130 \",peers=\"id:355 store_id:1 \"}"] [keys="{total=4,keys=\"[72000000696E6465783A5F30303030303030303030303033323431343637,72000000696E6465783A5F30303030303030303030303033343137303839,72000000696E6465783A5F30303030303030303030303033353235313431,72000000696E6465783A5F30303030303030303030303033363935333836]\"}"] [ranges="{total=31,ranges=\"[\\\"[72000000696E6465783A5F30303030303030303030303032353030303030, 72000000696E6465783A5F30303030303030303030303032353436363638)\\\",\\\"(skip 29)\\\",\\\"[72000000696E6465783A5F30303030303030303030303037333832383332, 72000000696E6465783A5F30303030303030303030303037353030303030)\\\"]\",totalFiles=31,totalKVs=5000000,totalBytes=260000000,totalSize=260000000}"]
[2025/04/16 10:06:03.835 +08:00] [INFO] [split.go:186] ["scattered regions"] [count=4]
[2025/04/16 10:06:03.835 +08:00] [INFO] [split.go:157] ["split regions"] [region="{ID=354,startKey=72000000696E6465FF783A5F3030303030FF3030303030303033FF3639353338360000FD,endKey=72000000696E6465FF783A5F3030303030FF3030303030303036FF3430333237330000FD,epoch=\"conf_ver:1 version:134 \",peers=\"id:355 store_id:1 \"}"] [keys="{total=4,keys=\"[72000000696E6465783A5F30303030303030303030303033383836383534,72000000696E6465783A5F30303030303030303030303034303434383030,72000000696E6465783A5F30303030303030303030303034323037313136,72000000696E6465783A5F30303030303030303030303034333631333138]\"}"] [ranges="{total=31,ranges=\"[\\\"[72000000696E6465783A5F30303030303030303030303032353030303030, 72000000696E6465783A5F30303030303030303030303032353436363638)\\\",\\\"(skip 29)\\\",\\\"[72000000696E6465783A5F30303030303030303030303037333832383332, 72000000696E6465783A5F30303030303030303030303037353030303030)\\\"]\",totalFiles=31,totalKVs=5000000,totalBytes=260000000,totalSize=260000000}"]
[2025/04/16 10:06:03.935 +08:00] [INFO] [split.go:186] ["scattered regions"] [count=4]
[2025/04/16 10:06:03.935 +08:00] [INFO] [split.go:157] ["split regions"] [region="{ID=354,startKey=72000000696E6465FF783A5F3030303030FF3030303030303034FF3336313331380000FD,endKey=72000000696E6465FF783A5F3030303030FF3030303030303036FF3430333237330000FD,epoch=\"conf_ver:1 version:138 \",peers=\"id:355 store_id:1 \"}"] [keys="{total=4,keys=\"[72000000696E6465783A5F30303030303030303030303034353135353230,72000000696E6465783A5F30303030303030303030303034363635333032,72000000696E6465783A5F30303030303030303030303034383438393736,72000000696E6465783A5F30303030303030303030303034393733373036]\"}"] [ranges="{total=31,ranges=\"[\\\"[72000000696E6465783A5F30303030303030303030303032353030303030, 72000000696E6465783A5F30303030303030303030303032353436363638)\\\",\\\"(skip 29)\\\",\\\"[72000000696E6465783A5F30303030303030303030303037333832383332, 72000000696E6465783A5F30303030303030303030303037353030303030)\\\"]\",totalFiles=31,totalKVs=5000000,totalBytes=260000000,totalSize=260000000}"]
[2025/04/16 10:06:04.058 +08:00] [INFO] [split.go:186] ["scattered regions"] [count=4]
[2025/04/16 10:06:04.059 +08:00] [INFO] [split.go:157] ["split regions"] [region="{ID=354,startKey=72000000696E6465FF783A5F3030303030FF3030303030303034FF3937333730360000FD,endKey=72000000696E6465FF783A5F3030303030FF3030303030303036FF3430333237330000FD,epoch=\"conf_ver:1 version:142 \",peers=\"id:355 store_id:1 \"}"] [keys="{total=4,keys=\"[72000000696E6465783A5F30303030303030303030303035303833323534,72000000696E6465783A5F30303030303030303030303035333033313734,72000000696E6465783A5F30303030303030303030303035343630373834,72000000696E6465783A5F30303030303030303030303035363134393836]\"}"] [ranges="{total=31,ranges=\"[\\\"[72000000696E6465783A5F30303030303030303030303032353030303030, 72000000696E6465783A5F30303030303030303030303032353436363638)\\\",\\\"(skip 29)\\\",\\\"[72000000696E6465783A5F30303030303030303030303037333832383332, 72000000696E6465783A5F30303030303030303030303037353030303030)\\\"]\",totalFiles=31,totalKVs=5000000,totalBytes=260000000,totalSize=260000000}"]
[2025/04/16 10:06:04.137 +08:00] [INFO] [split.go:186] ["scattered regions"] [count=4]
[2025/04/16 10:06:04.137 +08:00] [INFO] [split.go:157] ["split regions"] [region="{ID=354,startKey=72000000696E6465FF783A5F3030303030FF3030303030303035FF3631343938360000FD,endKey=72000000696E6465FF783A5F3030303030FF3030303030303036FF3430333237330000FD,epoch=\"conf_ver:1 version:146 \",peers=\"id:355 store_id:1 \"}"] [keys="{total=4,keys=\"[72000000696E6465783A5F30303030303030303030303035373431313235,72000000696E6465783A5F30303030303030303030303035393130363031,72000000696E6465783A5F30303030303030303030303036303230303231,72000000696E6465783A5F30303030303030303030303036323030363233]\"}"] [ranges="{total=31,ranges=\"[\\\"[72000000696E6465783A5F30303030303030303030303032353030303030, 72000000696E6465783A5F30303030303030303030303032353436363638)\\\",\\\"(skip 29)\\\",\\\"[72000000696E6465783A5F30303030303030303030303037333832383332, 72000000696E6465783A5F30303030303030303030303037353030303030)\\\"]\",totalFiles=31,totalKVs=5000000,totalBytes=260000000,totalSize=260000000}"]
[2025/04/16 10:06:04.232 +08:00] [INFO] [split.go:186] ["scattered regions"] [count=4]

Side effects

• No side effects

Related changes

• No related changes

To reviewers

Please follow these principles to check this pull requests:

• Concentration. One pull request should only do one thing. No matter how small it is, the change does exactly one thing and gets it right. Don't mix other changes into it.
• Tests. A pull request should be test covered, whether the tests are unit tests, integration tests, or end-to-end tests. Tests should be sufficient, correct and don't slow down the CI pipeline largely.
• Functionality. The pull request should implement what the author intends to do and fit well in the existing code base, resolve a real problem for TiDB/TiKV users. To get the author's intention and the pull request design, you could follow the discussions in the corresponding GitHub issue or internal.tidb.io topic.
• Style. Code in the pull request should follow common programming style. (For Go, we have style checkers in CI). However, sometimes the existing code is inconsistent with the style guide, you should maintain consistency with the existing code or file a new issue to fix the existing code style first.
• Documentation. If a pull request changes how users build, test, interact with, or release code, you must check whether it also updates the related documentation such as READMEs and any generated reference docs. Similarly, if a pull request deletes or deprecates code, you must check whether or not the corresponding documentation should also be deleted.
• Performance. If you find the pull request may affect performance, you could ask the author to provide a benchmark result.

(The above text mainly refers to TiDB Development Guide. It's also highly recommended to read about Writing code review comments)

[ti-chi-bot]

@pingyu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-integration-test	017dfa6	link	true	/test pull-integration-test
pull-integration-kafka-test	017dfa6	link	true	/test pull-integration-kafka-test
pull-unit-test	017dfa6	link	true	/test pull-unit-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.