-
Notifications
You must be signed in to change notification settings - Fork 0
Description
The unit testing library currently in use (Criterion) is great for a couple reasons:
- Written in C
- Tests can be written in pure C
- Binaries provided for a range of systems
- Each test run on a separate process, allowing crashes (e.g. segfaults) to be intercepted and reported
- Parametrised tests
However, as the project has progressed, some issues have started showing up:
- Tests cannot actually be run in parallel (probably, at least with our setup)
- The requirement for the use of the library's custom allocators when setting up parametrised tests has become quite painful
The first problem is not a large deal at the moment - all our tests are still small and run in hundredths of a second - but it might become a limiter eventually.
The second issue is the more immediate problem - it means that we need to be able to use the functions that allocate memory (basically all functions from allocators.c and constructors.c) with two different allocators, the normal malloc/free inside tests and the special cr_malloc/cr_free when setting up parametrised tests. Up until now, that has been done by a Python script which parsed the (originally one file) c file with Python clang bindings and wrote the relevant functions with all mallocs/frees replaced and names changed into a different file. Unfortunately, with the split of the allocators and constructors into two separate files and changes to the dense_to_tree hodlr function to accomodate OpenMP, this has become unsustainable.
Now, the simplest solution would be to change all the functions that allocate memory to accept a function pointer(s) to a malloc and/or free, but this has been determined at the moment undesirable from the API perspective (mainly because Fortran does not have an allocator in the C sense). The middle ground (currently implemented) is to use the preprocessor to define the interface for functions that allocate memory to accept these function pointers only when a macro (_TEST_HODLR) is defined - this works well enough for now, but it is quite hacky and it's probably unmaintainable in the long term.
Therefore, we should consider other options for unit testing libraries and whether they might be able to better serve our needs. From previous research, the only other still-maintained C testing library is Unity, which is unfortunately very minimal and doesn't have parametrised tests. Therefore, GoogleTest might be the best option (since it's fully featured), even if it forces us to write our tests in C++.