Skip to content

Commit f90c39d

Browse files
committed
mutation: Use hasFixedSize in sizeInClosedRange
The benchmarks in `//tests/benchmarks` show that biasing the size of subsets of collections of primitives chosen by the mutator to be small results in much worse performance than a comparable unstructured fuzz test. Before this change, 11 out of 15 runs time out with no run limit, the other ones result in: ``` { "values": [ 11143, 28128, 581194, 4229980 ], "minimum": 11143, "maximum": 4229980, "average": 1212611.25, "median": 304661 } ``` After this change, all runs pass within a limit of 35,000 runs: ``` { "values": [ 887, 1557, 1889, 2557, 3023, 3346, 3517, 6075, 6613, 7991, 9578, 10850, 15583, 23638, 31046 ], "minimum": 887, "maximum": 31046, "average": 8543.333333333334, "median": 6075 } ``` ExperimentalMutatorComplexProtoFuzzer now takes more runs on Linux, but still less than on other platforms, which seems to indicate that the Linux seed just happened to be a lucky choice.
1 parent a9f5255 commit f90c39d

File tree

4 files changed

+9
-9
lines changed

4 files changed

+9
-9
lines changed

.github/workflows/run-all-tests.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ jobs:
4242
extra_bazel_args: "--jvmopt=-Djava.security.manager=allow"
4343
- os: ubuntu-20.04
4444
arch: "linux"
45-
bazel_args: "//launcher/android:jazzer_android"
45+
bazel_args: "//launcher/android:jazzer_android //tests/benchmarks"
4646
cache: "/home/runner/.cache/bazel-disk"
4747
- os: macos-12
4848
arch: "macos-x86_64"

src/main/java/com/code_intelligence/jazzer/mutation/engine/SeededPseudoRandom.java

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -252,7 +252,11 @@ private int closedRangeBiasedTowardsSmall(int upperInclusive) {
252252
@Override
253253
public int sizeInClosedRange(
254254
int lowerInclusive, int upperInclusive, boolean elementsHaveFixedSize) {
255-
return lowerInclusive + closedRangeBiasedTowardsSmall(upperInclusive - lowerInclusive);
255+
if (elementsHaveFixedSize) {
256+
return closedRange(lowerInclusive, upperInclusive);
257+
} else {
258+
return lowerInclusive + closedRangeBiasedTowardsSmall(upperInclusive - lowerInclusive);
259+
}
256260
}
257261

258262
private static double zipf_h(double x) {

tests/BUILD.bazel

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -523,13 +523,9 @@ java_fuzz_target_test(
523523
"--experimental_mutator",
524524
"--instrumentation_includes=com.example.**",
525525
"--custom_hook_includes=com.example.**",
526-
] + select({
527526
# Limit runs to catch regressions in mutator efficiency and speed up test runs.
528-
"@platforms//os:linux": ["-runs=400000"],
529-
# TODO: Investigate why this test takes far more runs on macOS, with Windows also being
530-
# significantly worse than Linux.
531-
"//conditions:default": ["-runs=1200000"],
532-
}),
527+
"-runs=1200000",
528+
],
533529
target_class = "com.example.ExperimentalMutatorComplexProtoFuzzer",
534530
verify_crash_reproducer = False,
535531
deps = [

tests/benchmarks/BUILD.bazel

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ fuzzer_benchmark(
3232
"--experimental_mutator",
3333
"-use_value_profile=1",
3434
],
35-
max_runs = 55000,
35+
max_runs = 35000,
3636
num_seeds = 15,
3737
target_class = "com.example.StructuredMutatorMazeFuzzer",
3838
deps = [

0 commit comments

Comments
 (0)