Unit testing for pg code

[lahvak]

I am following up a discussion at JMM, where at one of the tables we talked about this.

It would be nice to have some sort of unit testing for pg code.

One way this could be done is having some way where one could put in some specific values of randomized parameters, and check that the answer (or some intermediate variables) are what they are supposed to be. That would really help with testing edge cases in complicated randomized problems.

Another way, which may be easier to do, would be randomized testing: generating large number of randomized versions of a problem and testing that some conditions are satisfied. That would help with fixing some issues where a problem fails for some specific combination of random values.

[drgrice1]

There are unit tests for PG code already. They run every time a pull request is made.

But that isn't what you are really asking for it seems. What you are asking for is a way to test a problem. This is not entirely possible. Not all problems expose a correct answer to validate against. I have written unit tests for some of my local problem repositories that test a single problem instantiation, but I'm many cases the correct answer has to be provided. The basis for that test is the t/pg_problems/problem_file.t test.