Skip to content

Commit 41d5aae

Browse files
committed
[SPARK-26148][PYTHON][TESTS] Increases default parallelism in PySpark tests to speed up
## What changes were proposed in this pull request? This PR proposes to increase parallelism in PySpark tests to speed up from 4 to 8. It decreases the elapsed time from https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99163/consoleFull Tests passed in 1770 seconds to https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99186/testReport/ Tests passed in 1027 seconds ## How was this patch tested? Jenkins tests Closes apache#23111 from HyukjinKwon/parallelism. Authored-by: hyukjinkwon <[email protected]> Signed-off-by: hyukjinkwon <[email protected]>
1 parent eea4a03 commit 41d5aae

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

dev/run-tests.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -460,7 +460,7 @@ def parse_opts():
460460
prog="run-tests"
461461
)
462462
parser.add_option(
463-
"-p", "--parallelism", type="int", default=4,
463+
"-p", "--parallelism", type="int", default=8,
464464
help="The number of suites to test in parallel (default %default)"
465465
)
466466

0 commit comments

Comments
 (0)