Skip to content

Conversation

@addisoncrump
Copy link
Contributor

Fixes #2019.

@addisoncrump addisoncrump marked this pull request as ready for review August 19, 2024 02:06
@addisoncrump
Copy link
Contributor Author

addisoncrump commented Aug 19, 2024

This PR also restores the mruby bug benchmark. Potentially I could bring back other benchmarks for these fuzzers.

@DonggeLiu, would you think it appropriate to run a fuzzbench run for these fuzzers + a working symbolic fuzzer + baselines (AFL++ and libafl at least) against mruby? After #2028 lands.

@DonggeLiu
Copy link
Contributor

Yep sure, I would love to see the results too. Thanks for doing this, @addisoncrump!

If you are interested in running them on more bug-based benchmarks, they (and their results) can be found on sbft23 branch and #2022. @Dammo3 kindly added some benchmarks recently.

Two minor things:

  1. We conventionally append the commit ID after the directory name of bug-based benchmarks, e.g., https://github.com/google/fuzzbench/tree/master/benchmarks/bloaty_fuzz_target_52948c

  2. How about changin the type to type: bug:

- mbedtls_fuzz_dtlsclient
- mbedtls_fuzz_dtlsclient_7c6b0e
- mruby_mruby_fuzzer
- mruby_mruby_fuzzer_8c8bbd
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DonggeLiu Maybe something worth noting: the CI will not fail if the benchmark doesn't exist!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, I did not know this, thanks!
I reckon this is low priority because it does not affect checking fuzzers or running experiments.
WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reckon it's low priority as well, but maybe something to keep in mind since it might inadvertently mark a fuzzer as "working" for a non-existent benchmark.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, created this: #2037

@renatahodovan
Copy link

@addisoncrump Thanks for working on this! Is it planned to merge it anytime soon?

@addisoncrump
Copy link
Contributor Author

This is ready to be merged.

@DonggeLiu DonggeLiu merged commit 915cd23 into google:master Sep 19, 2024
ardier pushed a commit to ardier/fuzzbench that referenced this pull request Nov 25, 2024
SPINO-11 pushed a commit to SPINO-11/green_fuzzing_evaluation that referenced this pull request Aug 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Grammar fuzzer build failed

3 participants