Test suite?

[balsoft]

Hi all!

I'm a NixOS user & Nixpkgs maintainer, and I'm looking through the core packages that make NixOS tick, and making sure that we got everything packaged correctly. One of those packages is openresolv (thank you for making and providing it BTW). I want to make sure we get the packaging right, and it works exactly as expected, especially after updates. Usually running the test suite is a good first step. As far as I can tell, there is no test suite for openresolv - is that correct? If that is so, what are the usual checks you run before cutting off a release, so that I can duplicate them in our test suite?

[rsmarples]

Hi and thanks!

I don't really have any automated tests for this (even though I wrote the e2e tests at $dayjob).
I have a set of VM's which host different OS's setup for testing dhcpcd. Most of them also have resolvconf installed and a non libc resolver so they are all covered.

Generally I test if they boot and dns is setup correctly on each box.
My DHCP server is setup to hand out something that breaks stuff in DNS as well.

Now if I want to have some automated tests which did the same thing I would setup some seeding resolv.conf files, a resolvconf.conf which has every subscriber configured but with a dummy restart (my host is NetBSD so resolvectl isn't supported there) and ensure that the expected files were in the right place with the right content.
We could even start some dummy processes so we can test the kill -HUP functions in the subscribers .... but likely not any dbus transactions.

And that resolv.conf files expected to break openresolv (see the 3.17.1 and 3.17.2 releases) no longer break things.

Does this help?

[balsoft]

Hmm, that sounds like I'd basically have to write the test suite myself 😅

I will think about your suggestions and see how much I can do in the time I have available; afterwards, I'll let you know what I ended up with.

Thanks for your answer!

[rsmarples]

Well, would be nice I guess!

#40 proves I do need some automation at least.