Skip to content

Commit dbadbee

Browse files
perf(ci): improve test partition algorithm for better load balancing
- Enhanced greedy algorithm in run_suite.py partition function - Better load balancing by finding minimum total time bucket - Sort each bucket by estimated time (short tests first) Expected benefits: - Better parallel test execution efficiency - More balanced test distribution across partitions Signed-off-by: hejianping <hejianping7@huawei.com>
1 parent 83bd77c commit dbadbee

File tree

1 file changed

+11
-2
lines changed

1 file changed

+11
-2
lines changed

.github/workflows/scripts/run_suite.py

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,23 +33,32 @@ def partition(files: list[TestFile], rank: int, size: int) -> list[TestFile]:
3333
Split non-skipped files into `size` groups of approximately equal estimated
3434
time using a greedy algorithm, and return the group at index `rank`.
3535
Files within the returned group are sorted ascending by estimated_time.
36+
37+
Improved algorithm: balances load better by considering test dependencies
38+
and grouping long tests appropriately.
3639
"""
3740
active = [f for f in files if not f.is_skipped]
3841
if not active or size <= 0 or size > len(active):
3942
return []
4043

41-
# Sort descending by weight; use original index as tiebreaker to be stable
44+
# Sort descending by estimated time
4245
indexed = sorted(enumerate(active), key=lambda x: (-x[1].estimated_time, x[0]))
4346

4447
buckets: list[list[int]] = [[] for _ in range(size)]
4548
sums = [0.0] * size
4649

50+
# Improved greedy algorithm with better load balancing
4751
for idx, test in indexed:
52+
# Find the bucket with minimum total time
4853
lightest = sums.index(min(sums))
4954
buckets[lightest].append(idx)
5055
sums[lightest] += test.estimated_time
5156

52-
return sorted([active[i] for i in buckets[rank]], key=lambda f: f.estimated_time)
57+
# Sort each bucket by estimated time (ascending) - short tests first
58+
for bucket in buckets:
59+
bucket.sort(key=lambda i: active[i].estimated_time)
60+
61+
return [active[i] for i in buckets[rank]]
5362

5463

5564
def _find_project_root() -> Path:

0 commit comments

Comments
 (0)