Set system test threads default based on CPU count#40785
Set system test threads default based on CPU count#40785thomashampson wants to merge 2 commits intomainfrom
Conversation
cailafinn
left a comment
There was a problem hiding this comment.
Looks good, can see that it's working for the GH-Actions node. Just a question about the calculation.
| BUILD_THREADS_SYSTEM_TESTS=3 | ||
| # Default to a 3/16 fraction of the build threads, e.g. 8 for a 32-core machine, | ||
| # to allow enough RAM for concurrently running system tests. | ||
| BUILD_THREADS_SYSTEM_TESTS=$(( BUILD_THREADS * 3 / 16 )) |
There was a problem hiding this comment.
Do you think we need to make sure this is an integer here? I'm just considering the STFC Cloud might change again and not have a nicely divisible number of cores.
There was a problem hiding this comment.
This will do integer division, so will take the floor. However, thinking about it more, maybe we should use RAM to determine the system test threads. E.g. our Windows runners currently have 16GB RAM and 24 cores. With the above calculation, that would mean 4 system test threads, but we are currently limiting to 2. What do you think?
There was a problem hiding this comment.
I think that's probably more sensible, since RAM is the bottleneck for the problem we're solving by restricting anyway.
Description of work
In Jenkins, we set the value of
BUILD_THREADS_SYSTEM_TESTSper machine to define how many system test to run in parallel. This adds logic to set a sensible default value of 3/16 multiplied by the core count.To test:
This is running on a github self-hosted runner here:
thomashampson#3
And here is the run with this change in it:
https://github.com/thomashampson/mantid_github_actions_test/actions/runs/21756642425/job/62768649276?pr=3
The runner has 32 cores, check that the number of system test threads is 6. This currently what we set it to for our Jenkins agents for 32-core machines.
Reviewer
Your comments will be used as part of the gatekeeper process. Comment clearly on what you have checked and tested during your review. Provide an audit trail for any changes requested.
As per the review guidelines:
mantid-developersormantid-contributorsteams, add a review commentrerun cito authorize/rerun the CIGatekeeper
As per the gatekeeping guidelines: