Interpretation of test p-values

[GregPlowman]

This is a query rather than an issue.

I see that in runtest.jl that p-values of individual RNG tests are compared to pval (which is initialized to 0.001).

@test RNGTest.smarsa_BirthdaySpacings(f, 1, 5000000, 0, 1073741824, 2, 1) > pval

I guess the purpose of runtests.jl is to test the framework and machinery of the RNGTest package, rather than verifying the statistical soundness of the RNGs themselves.

If I want to test a RNG, should I be checking that p-values are between [0.001, 0.999]?
In this case, should I expect failures to be 1 in 1000?

[andreasnoack]

I guess the purpose of runtests.jl is to test the framework and machinery of the RNGTest package, rather than verifying the statistical soundness of the RNGs themselves.

That is correct

If I want to test a RNG, should I be checking that p-values are between [0.001, 0.999]?

Yeah. I think that is the idea in U01 and I think it makes sense.

In this case, should I expect failures to be 1 in 1000?

Almost since [0.001, 0.999] would leave 0.1% in each end I guess it would be 2 in 1000.

[GregPlowman]

OK, thanks for clarifying.