Test: Move from unittest to fully pytest-based?

[leofang]

It has come to my attention that while we use pytest in this project, most tests are still written in unittest. If it is possible I would suggest to move away from it in new test code. It is not only faster but also allows accessing to full pytest features. While pytest claims to be compatible with unittest, there are some subtleties, for example, pytest parametrization are known to not work with unittest TestCases, which I ran into in commit b5b3a0b:

$ pytest -k test_nvjitlink_jit_with_linkable_code_lto_dump_assembly
========================================================================== test session starts ===========================================================================
platform linux -- Python 3.13.5, pytest-8.4.1, pluggy-1.6.0
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /local/home/leof/dev/numba-cuda/testing
configfile: pytest.ini
plugins: benchmark-5.2.3, xdist-3.8.0
collected 1940 items / 1938 deselected / 2 selected                                                                                                                      

cudadrv/test_nvjitlink.py FF                                                                                                                                       [100%]

================================================================================ FAILURES ================================================================================
___________________________________________________ TestLinker.test_nvjitlink_jit_with_linkable_code_lto_dump_assembly ___________________________________________________

self = <unittest.case._Outcome object at 0x7fc622855010>
test_case = <numba.cuda.tests.cudadrv.test_nvjitlink.TestLinker testMethod=test_nvjitlink_jit_with_linkable_code_lto_dump_assembly>, subTest = False

    @contextlib.contextmanager
    def testPartExecutor(self, test_case, subTest=False):
        old_success = self.success
        self.success = True
        try:
>           yield

../../../miniforge3/envs/py313_cu129/lib/python3.13/unittest/case.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniforge3/envs/py313_cu129/lib/python3.13/unittest/case.py:651: in run
    self._callTestMethod(testMethod)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <numba.cuda.tests.cudadrv.test_nvjitlink.TestLinker testMethod=test_nvjitlink_jit_with_linkable_code_lto_dump_assembly>
method = <bound method TestLinker.test_nvjitlink_jit_with_linkable_code_lto_dump_assembly of <numba.cuda.tests.cudadrv.test_nvjitlink.TestLinker testMethod=test_nvjitlink_jit_with_linkable_code_lto_dump_assembly>>

    def _callTestMethod(self, method):
>       if method() is not None:
           ^^^^^^^^
E       TypeError: TestLinker.test_nvjitlink_jit_with_linkable_code_lto_dump_assembly() missing 1 required positional argument: 'files'

../../../miniforge3/envs/py313_cu129/lib/python3.13/unittest/case.py:606: TypeError
________________________________________________ TestLinker.test_nvjitlink_jit_with_linkable_code_lto_dump_assembly_warn _________________________________________________

self = <unittest.case._Outcome object at 0x7fc622674910>
test_case = <numba.cuda.tests.cudadrv.test_nvjitlink.TestLinker testMethod=test_nvjitlink_jit_with_linkable_code_lto_dump_assembly_warn>, subTest = False

    @contextlib.contextmanager
    def testPartExecutor(self, test_case, subTest=False):
        old_success = self.success
        self.success = True
        try:
>           yield

../../../miniforge3/envs/py313_cu129/lib/python3.13/unittest/case.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniforge3/envs/py313_cu129/lib/python3.13/unittest/case.py:651: in run
    self._callTestMethod(testMethod)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <numba.cuda.tests.cudadrv.test_nvjitlink.TestLinker testMethod=test_nvjitlink_jit_with_linkable_code_lto_dump_assembly_warn>
method = <bound method TestLinker.test_nvjitlink_jit_with_linkable_code_lto_dump_assembly_warn of <numba.cuda.tests.cudadrv.test_nvjitlink.TestLinker testMethod=test_nvjitlink_jit_with_linkable_code_lto_dump_assembly_warn>>

    def _callTestMethod(self, method):
>       if method() is not None:
           ^^^^^^^^
E       TypeError: TestLinker.test_nvjitlink_jit_with_linkable_code_lto_dump_assembly_warn() missing 1 required positional argument: 'files'

../../../miniforge3/envs/py313_cu129/lib/python3.13/unittest/case.py:606: TypeError
======================================================================== short test summary info =========================================================================
FAILED cudadrv/test_nvjitlink.py::TestLinker::test_nvjitlink_jit_with_linkable_code_lto_dump_assembly - TypeError: TestLinker.test_nvjitlink_jit_with_linkable_code_lto_dump_assembly() missing 1 required positional argument: 'files'
FAILED cudadrv/test_nvjitlink.py::TestLinker::test_nvjitlink_jit_with_linkable_code_lto_dump_assembly_warn - TypeError: TestLinker.test_nvjitlink_jit_with_linkable_code_lto_dump_assembly_warn() missing 1 required positional argument: 'files'
=================================================================== 2 failed, 1938 deselected in 0.90s ===================================================================

I would consider this low priority, though.

[gmarkall]

I'd like this too, the only thing that has prevented it so far has been getting the time to do it.

[AryanBagade]

@gmarkall if its available for external contributors, can i work on it.
Been contributing to dynamo would love to work here as well!

[gmarkall]

@AryanBagade I think that would be great - you're welcome to work on it! That said, I have not yet had chance to think about how the transition should take place with respect to converting test cases and the various skip decorators we have. Do you have any thoughts on the approach already?

Would it work to pick a test module or two and convert it/them from unittest to pytest, implementing any necessary fixtures and parametrizations as required? Then we could PR / review that and see whether the approach works more generally for the test suite before proceeding to converting more of it. What do you think?

[maifeeulasad]

@gmarkall I can work on this one. But I think I should start with a module or two. Care to suggest me some easy modules at first and see how it goes?

[gmarkall]

@maifeeulasad test_matmul.py is pretty basic - if you are able to convert that successfully, then I think you could follow on by working on any other short test file in the numba.cuda.tests.cudapy module.

[maifeeulasad]

@gmarkall Thanks! I have completed migration for test_matmul.py and raised an MR here #754

Open for upcoming review and taking more migration, waiting for your response.