fix error message for MSQ TooManyInputFilesFault #18799
Merged
+21
−63
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes an issue with MSQ input file validation that results in creating a
TooManyInputFilesFaultfrom the number of an individual worker instead of totals across all workers. This lead to incorrect error messaging, since the error message advises on the minimum number of workers required to run the work order, so by only considering a single workers inputs it would advise to have at least the number of workers required to run that workers inputs, even though the number of actual workers might far exceed that.For example, the adjusted test in this PR has 8 workers to run 100k inputs, before the changes in this PR the error message would be something like:
The number of files is less than the 100k total for the job, and the "... increasing the number of workers to at least [2] ..." isn't helpful because we already have 8 workers.