Profiling for proptest shrinking. See my accompanying blog post: Demystifying monads in Rust through property-based testing.
Use cargo run to run in dev mode with opt-level 1 (typical configuration for testing), or cargo run --release to run in release mode with opt-level 3. This will write out the raw data to results-opt-level-N.tsv, where N is the opt-level value.
Run ./plot-cdfs.gnu. You'll need gnuplot and imagemagick installed.
On my workstation (Ryzen 7950X running Linux 6.12 with Rust 1.84.1), with opt-level 1, the amount of time it takes is:
| Metric | Pairs (prop_map) |
Triples (prop_map) |
Pairs (prop_flat_map) |
Triples (prop_flat_map) |
|---|---|---|---|---|
| min | 11 µs | 48 µs | 3.85 ms | 8.95 ms |
| p50 | 1.70 ms | 2.52 ms | 8.52 ms | 181 ms |
| p75 | 3.74 ms | 5.77 ms | 10.04 ms | 307 ms |
| p90 | 5.25 ms | 8.41 ms | 11.76 ms | 435 ms |
| max | 7.00 ms | 10.55 ms | 15.53 ms | 1808 ms |
The number of iterations:
| Metric | Pairs (prop_map) |
Triples (prop_map) |
Pairs (prop_flat_map) |
Triples (prop_flat_map) |
|---|---|---|---|---|
| min | 48 | 93 | 1228 | 11223 |
| p50 | 215 | 306 | 6722 | 281016 |
| p75 | 270 | 354 | 9315 | 481996 |
| p90 | 310 | 410 | 10722 | 693358 |
| max | 387 | 530 | 12242 | 884729 |
A CDF of results:
For more discussion, see the Measuring the impact section in my blog post.