Skip to content

Commit 5b4a9f4

Browse files
fanquakefanquake
committed
Merge #18342: doc: Add fuzzing quickstart guides for libFuzzer and afl-fuzz
33dd764 doc: Add fuzzing quickstart guides for libFuzzer and afl-fuzz. Simplify instructions. (practicalswift) Pull request description: Add fuzzing quickstart guide to make it trivial to start fuzzing Bitcoin Core. Fuzzing is fun and having more people contributing coverage-increasing inputs to https://github.com/bitcoin-core/qa-assets would be awesome :) ACKs for top commit: MarcoFalke: ACK 33dd764 fanquake: ACK 33dd764 - ran through the quick start and process message instructions. macOS users might see issues with afl-fuzz. Tree-SHA512: f3ca972ce6ed0df8bb8177bdbb1e16d8a235941ffe4fa7b95ce9520b6454694ee26d2c545eac0b8b81856a77e26befda0922a9121a445dd936a0e9f9dd034160
2 parents 5236b2e + 33dd764 commit 5b4a9f4

File tree

1 file changed

+123
-126
lines changed

1 file changed

+123
-126
lines changed

‎doc/fuzzing.md

Lines changed: 123 additions & 126 deletions
Original file line numberDiff line numberDiff line change
@@ -1,125 +1,93 @@
1-
Fuzz-testing Bitcoin Core
2-
==========================
3-
4-
A special test harness in `src/test/fuzz/` is provided for each fuzz target to
5-
provide an easy entry point for fuzzers and the like. In this document we'll
6-
describe how to use it with AFL and libFuzzer.
7-
8-
## Preparing fuzzing
9-
10-
The fuzzer needs some inputs to work on, but the inputs or seeds can be used
11-
interchangeably between libFuzzer and AFL.
12-
13-
Extract the example seeds (or other starting inputs) into the inputs
14-
directory before starting fuzzing.
15-
16-
```
17-
git clone https://github.com/bitcoin-core/qa-assets
18-
export DIR_FUZZ_IN=$PWD/qa-assets/fuzz_seed_corpus
19-
```
20-
21-
AFL needs an input directory with examples, and an output directory where it
22-
will place examples that it found. These can be anywhere in the file system,
23-
we'll define environment variables to make it easy to reference them.
24-
25-
So, only for AFL you need to configure the outputs path:
26-
27-
```
28-
mkdir outputs
29-
export AFLOUT=$PWD/outputs
30-
```
31-
32-
libFuzzer will use the input directory as output directory.
33-
34-
## AFL
35-
36-
### Building AFL
37-
38-
It is recommended to always use the latest version of afl:
39-
```
40-
wget http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz
41-
tar -zxvf afl-latest.tgz
42-
cd afl-<version>
43-
make
44-
export AFLPATH=$PWD
45-
```
46-
47-
For macOS you may need to ignore x86 compilation checks when running `make`:
48-
`AFL_NO_X86=1 make`.
49-
50-
### Instrumentation
51-
52-
To build Bitcoin Core using AFL instrumentation (this assumes that the
53-
`AFLPATH` was set as above):
54-
```
55-
./configure --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-gcc CXX=${AFLPATH}/afl-g++
56-
export AFL_HARDEN=1
57-
make
58-
```
59-
60-
If you are using clang you will need to substitute `afl-gcc` with `afl-clang`
61-
and `afl-g++` with `afl-clang++`, so the first line above becomes:
62-
```
63-
./configure --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-clang CXX=${AFLPATH}/afl-clang++
64-
```
65-
66-
We disable ccache because we don't want to pollute the ccache with instrumented
67-
objects, and similarly don't want to use non-instrumented cached objects linked
68-
in.
69-
70-
The fuzzing can be sped up significantly (~200x) by using `afl-clang-fast` and
71-
`afl-clang-fast++` in place of `afl-gcc` and `afl-g++` when compiling. When
72-
compiling using `afl-clang-fast`/`afl-clang-fast++` the resulting
73-
binary will be instrumented in such a way that the AFL
74-
features "persistent mode" and "deferred forkserver" can be used. See
75-
https://github.com/google/AFL/tree/master/llvm_mode for details.
76-
77-
### Fuzzing
78-
79-
To start the actual fuzzing use:
80-
81-
```
82-
export FUZZ_TARGET=bech32 # Pick a fuzz_target
83-
mkdir ${AFLOUT}/${FUZZ_TARGET}
84-
$AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- src/test/fuzz/${FUZZ_TARGET}
85-
```
86-
87-
You may have to change a few kernel parameters to test optimally - `afl-fuzz`
88-
will print an error and suggestion if so.
89-
90-
On macOS you may need to set `AFL_NO_FORKSRV=1` to get the target to run.
91-
```
92-
export FUZZ_TARGET=bech32 # Pick a fuzz_target
93-
mkdir ${AFLOUT}/${FUZZ_TARGET}
94-
AFL_NO_FORKSRV=1 $AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- src/test/fuzz/${FUZZ_TARGET}
95-
```
96-
97-
## libFuzzer
98-
99-
A recent version of `clang`, the address/undefined sanitizers (ASan/UBSan) and
100-
libFuzzer is needed (all found in the `compiler-rt` runtime libraries package).
101-
102-
To build all fuzz targets with libFuzzer, run
103-
104-
```
105-
./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=clang CXX=clang++
106-
make
107-
```
108-
109-
See https://llvm.org/docs/LibFuzzer.html#running on how to run the libFuzzer
110-
instrumented executable.
111-
112-
Alternatively, you can run the script through the fuzzing test harness (only
113-
libFuzzer supported so far). You need to pass it the inputs directory and
114-
the specific test target you want to run.
115-
116-
```
117-
./test/fuzz/test_runner.py ${DIR_FUZZ_IN} bech32
118-
```
119-
120-
### macOS hints for libFuzzer
121-
122-
The default clang/llvm version supplied by Apple on macOS does not include
1+
# Fuzzing Bitcoin Core using libFuzzer
2+
3+
## Quickstart guide
4+
5+
To quickly get started fuzzing Bitcoin Core using [libFuzzer](https://llvm.org/docs/LibFuzzer.html):
6+
7+
```sh
8+
$ git clone https://github.com/bitcoin/bitcoin
9+
$ cd bitcoin/
10+
$ ./autogen.sh
11+
$ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
12+
# macOS users: If you have problem with this step then make sure to read "macOS hints for
13+
# libFuzzer" on https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md#macos-hints-for-libfuzzer
14+
$ make
15+
$ src/test/fuzz/process_message
16+
# abort fuzzing using ctrl-c
17+
```
18+
19+
## Fuzzing harnesses, fuzzing output and fuzzing corpora
20+
21+
[`process_message`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/process_message.cpp) is a fuzzing harness for the [`ProcessMessage(...)` function (`net_processing`)](https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp). The available fuzzing harnesses are found in [`src/test/fuzz/`](https://github.com/bitcoin/bitcoin/tree/master/src/test/fuzz).
22+
23+
The fuzzer will output `NEW` every time it has created a test input that covers new areas of the code under test. For more information on how to interpret the fuzzer output, see the [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html).
24+
25+
If you specify a corpus directory then any new coverage increasing inputs will be saved there:
26+
27+
```sh
28+
$ mkdir -p process_message-seeded-from-thin-air/
29+
$ src/test/fuzz/process_message process_message-seeded-from-thin-air/
30+
INFO: Seed: 840522292
31+
INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55e121ef9ab8, 0x55e121f613a6),
32+
INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55e121f613a8,0x55e1225da288),
33+
INFO: 0 files found in process_message-seeded-from-thin-air/
34+
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
35+
INFO: A corpus is not provided, starting from an empty corpus
36+
#2 INITED cov: 94 ft: 95 corp: 1/1b exec/s: 0 rss: 150Mb
37+
#3 NEW cov: 95 ft: 96 corp: 2/3b lim: 4 exec/s: 0 rss: 150Mb L: 2/2 MS: 1 InsertByte-
38+
#4 NEW cov: 96 ft: 98 corp: 3/7b lim: 4 exec/s: 0 rss: 150Mb L: 4/4 MS: 1 CrossOver-
39+
#21 NEW cov: 96 ft: 100 corp: 4/11b lim: 4 exec/s: 0 rss: 150Mb L: 4/4 MS: 2 ChangeBit-CrossOver-
40+
#324 NEW cov: 101 ft: 105 corp: 5/12b lim: 6 exec/s: 0 rss: 150Mb L: 6/6 MS: 5 CrossOver-ChangeBit-CopyPart-ChangeBit-ChangeBinInt-
41+
#1239 REDUCE cov: 102 ft: 106 corp: 6/24b lim: 14 exec/s: 0 rss: 150Mb L: 13/13 MS: 5 ChangeBit-CrossOver-EraseBytes-ChangeBit-InsertRepeatedBytes-
42+
#1272 REDUCE cov: 102 ft: 106 corp: 6/23b lim: 14 exec/s: 0 rss: 150Mb L: 12/12 MS: 3 ChangeBinInt-ChangeBit-EraseBytes-
43+
NEW_FUNC[1/677]: 0x55e11f456690 in std::_Function_base::~_Function_base() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/std_function.h:255
44+
NEW_FUNC[2/677]: 0x55e11f465800 in CDataStream::CDataStream(std::vector<unsigned char, std::allocator<unsigned char> > const&, int, int) src/./streams.h:248
45+
#2125 REDUCE cov: 4820 ft: 4867 corp: 7/29b lim: 21 exec/s: 0 rss: 155Mb L: 6/12 MS: 2 CopyPart-CMP- DE: "block"-
46+
NEW_FUNC[1/9]: 0x55e11f64d790 in std::_Rb_tree<uint256, std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > >, std::_Select1st<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >, std::less<uint256>, std::allocator<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > > >::~_Rb_tree() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_tree.h:972
47+
NEW_FUNC[2/9]: 0x55e11f64d870 in std::_Rb_tree<uint256, std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > >, std::_Select1st<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >, std::less<uint256>, std::allocator<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > > >::_M_erase(std::_Rb_tree_node<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >*) /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_tree.h:1875
48+
#2228 NEW cov: 4898 ft: 4971 corp: 8/35b lim: 21 exec/s: 0 rss: 156Mb L: 6/12 MS: 3 EraseBytes-CopyPart-PersAutoDict- DE: "block"-
49+
NEW_FUNC[1/5]: 0x55e11f46df70 in std::enable_if<__and_<std::allocator_traits<zero_after_free_allocator<char> >::__construct_helper<char, unsigned char const&>::type>::value, void>::type std::allocator_traits<zero_after_free_allocator<char> >::_S_construct<char, unsigned char const&>(zero_after_free_allocator<char>&, char*, unsigned char const&) /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/alloc_traits.h:243
50+
NEW_FUNC[2/5]: 0x55e11f477390 in std::vector<unsigned char, std::allocator<unsigned char> >::data() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_vector.h:1056
51+
#2456 NEW cov: 4933 ft: 5042 corp: 9/55b lim: 21 exec/s: 0 rss: 160Mb L: 20/20 MS: 3 ChangeByte-InsertRepeatedBytes-PersAutoDict- DE: "block"-
52+
#2467 NEW cov: 4933 ft: 5043 corp: 10/76b lim: 21 exec/s: 0 rss: 161Mb L: 21/21 MS: 1 InsertByte-
53+
#4215 NEW cov: 4941 ft: 5129 corp: 17/205b lim: 29 exec/s: 4215 rss: 350Mb L: 29/29 MS: 5 InsertByte-ChangeBit-CopyPart-InsertRepeatedBytes-CrossOver-
54+
#4567 REDUCE cov: 4941 ft: 5129 corp: 17/204b lim: 29 exec/s: 4567 rss: 404Mb L: 24/29 MS: 2 ChangeByte-EraseBytes-
55+
#6642 NEW cov: 4941 ft: 5138 corp: 18/244b lim: 43 exec/s: 2214 rss: 450Mb L: 43/43 MS: 3 CopyPart-CMP-CrossOver- DE: "verack"-
56+
# abort fuzzing using ctrl-c
57+
$ ls process_message-seeded-from-thin-air/
58+
349ac589fc66a09abc0b72bb4ae445a7a19e2cd8 4df479f1f421f2ea64b383cd4919a272604087a7
59+
a640312c98dcc55d6744730c33e41c5168c55f09 b135de16e4709558c0797c15f86046d31c5d86d7
60+
c000f7b41b05139de8b63f4cbf7d1ad4c6e2aa7f fc52cc00ec1eb1c08470e69f809ae4993fa70082
61+
$ cat --show-nonprinting process_message-seeded-from-thin-air/349ac589fc66a09abc0b72bb4ae445a7a19e2cd8
62+
block^@M-^?M-^?M-^?M-^?M-^?nM-^?M-^?
63+
```
64+
65+
In this case the fuzzer managed to create a `block` message which when passed to `ProcessMessage(...)` increased coverage.
66+
67+
The project's collection of seed corpora is found in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo.
68+
69+
To fuzz `process_message` using the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) seed corpus:
70+
71+
```sh
72+
$ git clone https://github.com/bitcoin-core/qa-assets
73+
$ src/test/fuzz/process_message qa-assets/fuzz_seed_corpus/process_message/
74+
INFO: Seed: 1346407872
75+
INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55d8a9004ab8, 0x55d8a906c3a6),
76+
INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55d8a906c3a8,0x55d8a96e5288),
77+
INFO: 991 files found in qa-assets/fuzz_seed_corpus/process_message/
78+
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
79+
INFO: seed corpus: files: 991 min: 1b max: 1858b total: 288291b rss: 150Mb
80+
#993 INITED cov: 7063 ft: 8236 corp: 25/3821b exec/s: 0 rss: 181Mb
81+
…
82+
```
83+
84+
If you find coverage increasing inputs when fuzzing you are highly encouraged to submit them for inclusion in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo.
85+
86+
Every single pull request submitted against the Bitcoin Core repo is automatically tested against all inputs in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. Contributing new coverage increasing inputs is an easy way to help make Bitcoin Core more robust.
87+
88+
## macOS hints for libFuzzer
89+
90+
The default Clang/LLVM version supplied by Apple on macOS does not include
12391
fuzzing libraries, so macOS users will need to install a full version, for
12492
example using `brew install llvm`.
12593
@@ -128,11 +96,40 @@ may need to run `./configure` with `--disable-asm` to avoid errors
12896
with certain assembly code from Bitcoin Core's code. See [developer notes on sanitizers](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#sanitizers)
12997
for more information.
13098
131-
You may also need to take care of giving the correct path for clang and
132-
clang++, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems
133-
clang does not come first in your path.
99+
You may also need to take care of giving the correct path for `clang` and
100+
`clang++`, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems
101+
`clang` does not come first in your path.
134102
135103
Full configure that was tested on macOS Catalina with `brew` installed `llvm`:
136-
```
104+
105+
```sh
137106
./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ --disable-asm
138107
```
108+
109+
Read the [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html) for more information. This [libFuzzer tutorial](https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md) might also be of interest.
110+
111+
# Fuzzing Bitcoin Core using american fuzzy lop (`afl-fuzz`)
112+
113+
## Quickstart guide
114+
115+
To quickly get started fuzzing Bitcoin Core using [`afl-fuzz`](https://github.com/google/afl):
116+
117+
```sh
118+
$ git clone https://github.com/bitcoin/bitcoin
119+
$ cd bitcoin/
120+
$ git clone https://github.com/google/afl
121+
$ make -C afl/
122+
$ make -C afl/llvm_mode/
123+
$ ./autogen.sh
124+
$ CC=$(pwd)/afl/afl-clang-fast CXX=$(pwd)/afl/afl-clang-fast++ ./configure --enable-fuzz
125+
$ make
126+
# For macOS you may need to ignore x86 compilation checks when running "make". If so,
127+
# try compiling using: AFL_NO_X86=1 make
128+
$ mkdir -p inputs/ outputs/
129+
$ echo A > inputs/thin-air-input
130+
$ afl/afl-fuzz -i inputs/ -o outputs/ -- src/test/fuzz/bech32
131+
# You may have to change a few kernel parameters to test optimally - afl-fuzz
132+
# will print an error and suggestion if so.
133+
```
134+
135+
Read the [`afl-fuzz` documentation](https://github.com/google/afl) for more information.

0 commit comments

Comments
 (0)