Set the default of numRuns via an environment variable FAST_CHECK_NUM_RUNS

[rix0rrr]

🚀 Feature Request

Can we add an environment variable to override the numRuns default of 100?

Motivation

After having written a new property test suite (potentially containing dozens of property tests) I'd like to "burn it in": let it run for a bit to make sure it's not generating any cases that cause problems. For example, this is something I can easily do with Rust's proptest.

What I'd do is run:

$ env PROPTEST_CASES=10000 cargo test
# (or whatever)

This gives me more confidence that my properties indeed hold than running it for 100 cases. I'd rather know just after having written some property tests, than having it find a case I missed later on during a CI run. This has happened twice to me already, and it's rather frustrating to have unrelated PR builds fail just because the proptest happened to hit a seed that exhibits a problem, that I should have found earlier by having my suite hum along for 30 minutes or so.

An environment variable would give me an easy way to do a burn-in test.

Pre-emptive FAQ: Yeah, I could script it myself:

let numRuns = 100;
if (process.env.FAST_CHECK_NUM_RUNS) {
  numRuns = Number(process.env.FAST_CHECK_NUM_RUNS);
}

test('...', () => {
  fc.assert(..., {
    numRuns,
  }
});

test('...', () => {
  fc.assert(..., {
    numRuns,
  }
});

test('...', () => {
  fc.assert(..., {
    numRuns,
  }
});

But

• a) that becomes annoyingly repetitive
• b) it makes it more annoying to copy/paste in and remove the { seed } example when I'm doing a repro on a specific test.

So an easily configurable default, preferably controlled from outside the literal test source file, would still be preferred imo.

Example

Please provide an example for how this feature would be used.

$ env FAST_CHECK_NUM_RUNS=10000 npx jest my.prop.test

[dubzzz]

Can easily be done outside of fast-check with an helper doing:

if (process.env.FAST_CHECK_NUM_RUNS) {
  fc.configureGlobal({ ...fc.readConfigureGlobal(), numRuns: Number(process.env.FAST_CHECK_NUM_RUNS) });
}

You can possibly put this code in the setup script of your test runner. For jest, have a look to jest.setup.js as documented in https://fast-check.dev/docs/tutorials/setting-up-your-test-environment/property-based-testing-with-jest/#recommended-setup.

At the moment, we don't plan to add such capabilities directly within fast-check. The library is supposed to work perfectly well against any runtime including the browser, as such it stays agnostic of the runtime.

[rix0rrr]

any runtime including the browser

I figured that might be a counterargument 😝. Your workaround will suffice, and at least this issue is now a place where this trick is documented. Cheers.

(Also once again: thanks for an amazing tool!)