Technique for testing non-exported functions

[kostmo]

I'm exploring how to write tests for individual functions without having to expose public access to them. I'm thinking about this problem with respect to functions that have privileged access to internal state, which would break encapsulation and perhaps allow users to accidentally violate invariants if they were to be called in inappropriate contexts.

Here's a few links I came up with when googling:

• Testing internals without exposing them?
• Test non public functions
• What is the idiomatic way to test "hidden" module functions in a Cabal project

Most resources seem to suggest using a "social convention", with modules named *.Internal.

I wonder if, instead, there is a technical means to enforce the convention.

Take the function addRobot. Excepting for one usage here and another questionable usage in #1372, perhaps that function could be removed from the export list of the State module to discourage its use in the wrong context.

However, the addRobot function is also referenced outside of the game in the TestUtil module. Could we somehow still test this function while also removing it from the State module's export list?

A "Testable" smart constructor

Define a type in a module named TestFramework:

newtype Testable a = Testable a

makeTestable :: a -> Testable a

and only export the makeTestable function, not the Testable constructor.

In the State module, define:

addRobotTestable = makeTestable addRobot

Then export the addRobotTestable function, but not the addRobot function, from the State module.

The TestFramework module has exclusive access to the contents of the Testable newtype. However, tests for the addRobot function cannot be written in the TestFramework module, because referencing the addRobotTestable function exported from State would entail a circular dependency.

But instead, maybe the TestFramework module could also export a monad that facilitates testing...

A "Testing" monad

Could we define a Reader monad named Testing, and then use it to access members of Testable? Then maybe the only way to access the addRobot function outside of the State module would be to run the Testing monad. But that monad would "pollute" the context in which it is run, so the only way in practice to execute the addRobotTestable function would be inside some dedicated testing framework?

I have not written code to prove this concept.

[byorgey]

I like this idea. But maybe something slightly more lightweight would suffice:

• One module (call it Testable) defines Testable and makeTestable and exports everything.
• A second module (call it WrapTestable) just re-exports makeTestable.
• In normal code you can import WrapTestable and wrap things up for export as testable things.
• In the test suite we can import Testable and just pattern-match to extract the thing to be tested.

Of course, this still relies on a bit of social convention; it would still be technically possible to import Testable and unwrap a thing you are not supposed to be able to use. But that would be really obvious in a code review, and it's not something anyone is likely to do by accident.

[kostmo]

Actually, another viable technique could be to use hlint to whitelist where certain functions may be used.

[xsebek]

I actually like the Internal convention. It is really useful for testing and in libraries it allows developers to modify the behaviour in ways that were unexpected/unsupported without a promise of support.

Consider IO - I like how it wraps the unsafe state, but at the same time it would be a shame if developers that "know what they are doing" could not use GHC.IO directly.