Visualize and adjust builds #863
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, the cron schedule is put together basically on vibes, which can cause a number of builds to start at the same time and back up the matrix. Add a script that generates a basic graph of the week and show how many builds are running at one point in time. The current output of this script shows that a large number of builds happen in the middle of the day UTC throughout the week, which we will want to spread out. --------------------------------------------------------------------------------------------------------------------------------------------- | | | | Sunday | 2 2 0 0 0 0 3 3 0 0 0 0 2 2 0 0 0 0 2 2 0 0 0 0 | | | | | Monday | 7 7 5 5 1 0 5 5 5 5 1 0 15 15 15 15 13 10 4 4 4 4 2 0 | | | | | Tuesday | 7 7 2 2 2 0 5 5 2 2 2 0 16 16 13 13 13 10 11 11 3 3 3 0 | | | | | Wednesday | 17 17 15 15 3 0 15 15 15 15 3 0 29 29 29 29 15 10 15 15 15 15 3 0 | | | | | Thursday | 7 7 2 2 2 0 5 5 2 2 2 0 16 16 13 13 13 10 11 11 3 3 3 0 | | | | | Friday | 7 7 5 5 1 0 5 5 5 5 1 0 15 15 15 15 13 10 4 4 4 4 2 0 | | | | | Saturday | 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | | | | --------------------------------------------------------------------------------------------------------------------------------------------- | | 0 3 6 9 12 15 18 21 | --------------------------------------------------------------------------------------------------------------------------------------------- Signed-off-by: Nathan Chancellor <[email protected]>
The matrix does not have a lot of flexibility around scheduling. Increase the granularity of the start times from six hours to three hours and every individual day as an option. Signed-off-by: Nathan Chancellor <[email protected]>
Rather than starting a large number of builds at once, such as the middle of the day during the week, spread the builds out across the entire week, including weekends. While nobody will be around to actively look at the reports on the weekend, they will get seen eventually and it is a better use of available resources. --------------------------------------------------------------------------------------------------------------------------------------------- | | | | Sunday | 3 3 3 7 5 3 5 3 0 5 5 3 8 5 5 9 4 4 8 4 4 6 2 2 | | | | | Monday | 11 5 2 6 6 2 4 4 1 4 4 1 11 11 10 11 11 11 5 4 3 4 2 1 | | | | | Tuesday | 5 5 1 4 4 1 3 3 1 4 4 1 11 10 10 11 11 11 5 4 3 6 4 3 | | | | | Wednesday | 7 4 1 4 4 3 5 3 1 1 1 0 10 10 10 11 11 11 5 4 3 5 3 2 | | | | | Thursday | 6 4 1 4 4 1 5 5 3 4 2 1 11 10 10 11 11 11 5 4 3 4 2 1 | | | | | Friday | 6 5 2 6 6 2 4 4 1 2 2 1 11 11 10 11 11 11 5 4 3 4 2 1 | | | | | Saturday | 4 4 3 6 4 3 3 3 0 3 3 3 6 3 3 6 3 3 5 2 2 6 4 4 | | | | --------------------------------------------------------------------------------------------------------------------------------------------- | | 0 3 6 9 12 15 18 21 | --------------------------------------------------------------------------------------------------------------------------------------------- Visualizing the builds also allows us to spread builds of the same tree with different toolchains out so that each tree has more coverage throughout the week, rather than all builds of the same tree running around the same time. Signed-off-by: Nathan Chancellor <[email protected]>
Signed-off-by: Nathan Chancellor <[email protected]>
nathanchance
requested review from
JustinStitt,
bwendling and
nickdesaulniers
as code owners
Signed-off-by: Justin Stitt <[email protected]>
Contributor
|
See downstream nathanchance#1 and let me know if you think its an improvement. |
Member
Author
|
Yeah I definitely think that is an improvement, I'll cherry-pick it into this PR, thanks a lot! |
Improve the visualize script
JustinStitt
approved these changes
Aug 14, 2025
Signed-off-by: Nathan Chancellor <[email protected]>
scripts/visualize-builds.py:169:23: SIM115 Use a context manager for opening files
|
167 | output_file = None
168 | if args.output:
169 | output_file = open(args.output, "w", newline="")
| ^^^^ SIM115
170 |
171 | try:
|
scripts/visualize-builds.py:176:9: PLR5501 [*] Use `elif` instead of `else` then `if`, to reduce indentation
|
174 | elif args.format == "pretty":
175 | output_pretty_table(days, output_file)
176 | / else: # regular table
177 | | if (
| |____________^ PLR5501
178 | output_file
179 | ): # evil hack so I don't have to rewrite visualize_data() prints
|
= help: Convert to `elif`
Found 2 errors.
************* Module visualize-builds
scripts/visualize-builds.py:169:22: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
scripts/visualize-builds.py:169:22: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
Signed-off-by: Nathan Chancellor <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a script,
visualize-builds.py, to help visualize the cron schedule of the matrix, with rough estimates for how long each tree builds based on currently available data. Using that script, we balance the builds to run more evenly throughout the week.Currently:
After this PR:
I plan to base the LLVM 21 schedule on this since I wanted to make sure we would not overload the matrix with that addition.