Increasing test case value results in no tests at all

[kolmodin]

Using the flag -a you can change the number of test cases that should run for each test. The default is 100 tests.

$ dist/build/qc/qc 
Utils:
  refragment id: [OK, passed 100 tests]
...

Now with increased number of test cases;

 dist/build/qc/qc -a 1000
Utils:
  refragment id: [OK, passed 1000 tests]
...

However, if I use more than 1000 test cases, test-framework somehow decides to not run any tests at all.

$ dist/build/qc/qc -a 1001
Utils:
  refragment id: [Failed]
Arguments exhausted after 0 tests
...

As the comment notes, the arguments are exhausted so it's unable to execute any of the tests.

[Blaisorblade]

This is the same issue as batterseapower#34 — it seems to mention a different option, but --maximum-generated-tests is the long name of -a.

[Blaisorblade]

To sum up a few interesting things we figured there:

• increasing --maximum_unsuitable_generated_tests makes the test pass
• using test-framework 0.7 avoids the issue, but maybe just because unsuitable tests were not counted (it's not clear — see High values of topt_maximum_generated_tests do not work batterseapower/test-framework#34 (comment))
• @s9gf4ult bisected the change down to f03afed.