Add some smoketests with NixOS test framework

[chreekat]

Help wanted: if somebody wants to propose a way of testing the modules, I am open to it.

https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests

[chreekat]

I've got the bones of a flake test now. But it doesn't actually test anything yet.

One serious difficulty here is that stackage writes to external systems. We would need to build up a test environment where we could run integrations tests.

[chreekat]

...aand it broke, because I changed strategy and don't expose a nixosConfiguration directly from this repo.

Maybe I should go back to doing that?

I really need to figure out my strategy here... I'll create another issue.

[eyeinsky]

Looking at this and the other issue about nixosConfiguration, and wondering what could be tested. I guess the tests as part of this repo would be integration tests, in the sense that the depending packages should have their own tests within the packages themselves and here the packages should be ran together somehow.

Most if not all the nixosModules defined in flake.nix require credentials, is there anything that be ran without them? as otherwise I guess there would need to be a local mock of S3 to use them (or maybe have intermediate files and then rewrite the modules/services to account for that).

I also found the stackage deployment diagram, which is great! I wonder if that diagram could be expressed in code and then included in this repo? (And then maybe even sync the names of nixosModules and services with the ones on the diagram.) Then this could inform what tests to add.

[chreekat]

Some ideas

• Integration tests of Stackage services are probably a longer-term goal.
• Shorter term, I simply want to ensure that the system configuration derivation can be built. (The right word might be "instantiated" rather than "built".) Otherwise, this repo is a heap of Nix code that never gets exercised locally. In the past, I've temporarily pointed the private deployment repo at my dirty worktree in order to test changes. That's lame for a lot of reasons. Mainly the fact that nobody else can do it.