Add fuzzer for distribution parameters

[mstoeckl]

• Added a CHANGELOG.md entry

Summary

This adds code to fuzz the distribution samplers and identify inputs and random samples on which have unexpected behavior (panic, spin in an infinite loop, or produce output which is outside the expected range).

Motivation

Random samplers are sometimes used with parameters derived from external data, random processes, or unstable formulas (like 1/x) which can produce extreme or otherwise unusual values. I think it would make the rand_distr library more useful if it were to clearly indicate which distribution parameters are valid, and on those parameters always produce output that is at least contained in the expected range.

(Perfect sampling on unusual extreme inputs, while it would be nice to have, is both hard to achieve and hard to define; even just sampling from the [0,1) interval of floating point values can be done in many different ways.)

Also, ensuring the distribution samplers pass fuzzing (even if this involves fixing events that would only happen with probability < 2^{-256}) will in turn make it easier for users to use fuzzing to find issues in parts of their code that interact with rand_distr.

Details

This creates one large fuzzer script, using https://github.com/rust-fuzz/cargo-fuzz as backend, to handle all parameter types. While this makes finding some issue in the library easy (just run the fuzzer script and wait), there are enough easily reachable and hard to fix issues at the moment that testing a specific distribution requires modifying the script to return early on other types.

The fuzzer code is unfortunately defined as a separate from the code to implement the samplers. I'd prefer a system that defines fuzz targets as functions inside each file, instead of having them separate, but am not aware of any simple and usable ways to do this.

Questions:

How strictly should this check that outputs are in the expected range (and not NaN)? Some relatively central distributions (like Normal) may produce NaN if the input parameters are NaN. Also, the rand crate's Uniform distribution on a floating point values outputs value0_1 * self.scale + self.low, which may be affected by floating point rounding and fall 1ulp outside the target interval or overflow. There are ways to fix these, but I don't know whether they are worth the runtime cost of checking for NaN inputs and adjusting the sampling logic.

[benjamin-lieser]

Thanks, that looks really nice to have. What problems we consider worth addressing is a different question.