Consider full switch to Pytest

[Lotram]

Disclaimer: I'm biased towards pytest, it's used in most python projects, and I think it has been the standard in python for quite some time.

That said, I truly think the codebase would benefit from migrating to pytest

• we already use pytest to run the tests anway, so it wouldn't add a dependency.
• Simpler assertions: No more self.assertEqual(...), a simple assert a = b is ok.
• we already have a few pytest fixtures, it's more powerful and (IMO) readable than setUp / teardown methods. I admit that this is the most biased argument I have, some people may disagree on fixtures being better than factories / setup functions.
• pytest does not require to use classes, simple functions are fine, which is enough for simple tests (we can still use classes for bigger ones)
• parametrize avoids a lot of redundancy
• a lot of plugins are available, if we ever need any (let's say to run tests in parallel, or to randomize test order for example)

[ryneeverett]

I think my fundamental question is, which of these benefits do we have to switch completely to pytest to take advantage of? For example, do we need to switch completely over to use the parallelization/randomization plugins?

[Lotram]

Regarding plugins, I guess it depends on their actual implementation, as well as unittest.TestCase. I don't know how well pytest fixtures work with TestCases.

"Switching completely to Pytest" means 2 things IMO:

1. simplifying assertions.
2. Removing unitest.TestCase inheritance. That does not mean we have to remove all setUp / tearDown methods immediately, or migrate everything to fixtures, but we can then use test functions instead of always classes, and implement setup as we prefer.

So IMO the code would end up more readable, and more flexible.

We can definitely do 1. without 2. at first.

[ryneeverett]

Personally, I find the readability of pytest assertions marginally better at best. If we wanted a pytest feature that depended on converting all the assertions and TestCase inheritance, I'd be supportive.

As far as I know, there's nothing stopping you from writing new tests in test functions rather than classes. So I don't see how rewriting the existing tests gives us any more flexibility.

My perspective is that there is a cost to intentionally get further locked into pytest, even if it has dominated the field for the last decade. It's a very small cost, but there has to be some concrete benefit to justify it.

[Lotram]

As far as I know, there's nothing stopping you from writing new tests in test functions rather than classes.

It would mean using regular assert instead of self.assertAnything. While I do prefer that, I also think convention consistency is important, so I think we should stick to only one or the other.

It's a very small cost, but there has to be some concrete benefit to justify it.

I agree, but for me, the (smallish) readability improvement + sticking with standard tools beats the (even smaller) migration cost. You seem to disagree with this, so let's keep unittest.

Regarding plugins, here are some of the plugins I like, if you want to have a look

• From the pytest-dev org:

  • randomly to randomize test order (there is also random-order )
  • xdist run tests on multiple CPUs (maybe overkill for this project)
  • mock I prefer it over unittest.mock decorators, and it adds some useful functions such as spy, but it does not really make sense to use it without migrating to fixtures first.

• Third party plugins:

  • sugar just nicer UI
  • socket to block all network calls (ensuring we're properly mocking any HTTP calls). There are other similar ones, but it's the only one I know that does not simply patch a specific package, such as requests or httpx. I think this one (or a similar one) would be useful here.