[F&E]: Use mock API returns in unit tests

[larshinueber]

Description:

The unit tests in the data modules (MPC, Horizons, ...) are somewhat notorious to fail due to connection errors, timeouts etc. This makes the unit test fail, even tough the underlying function might work correctly, which is especially annoying in CI (see e.g. https://dev.azure.com/tudat-team/feedstock-builds/_build/results?buildId=2505&view=logs&j=c8b8640e-759f-5af3-65b5-809c078c71b1&t=f5b821d2-45c4-5128-2768-5052952aa5c6&l=11176).

Since the specific contents (numerical values, meta data, ...) of the returned data typically does not matter in a unit test, it is common practice to "mock" the return data and bypass the API call. This avoids connection/timeout errors raising false test fails, while still testing the underlying functionality.

Suggestions for implementation:

In pytest, this can be achieved by "monkey-patching" returned objects: https://docs.pytest.org/en/latest/how-to/monkeypatch.html#monkeypatching-returned-objects-building-mock-classes

Additional context:

No response

[luigigisolfi]

Hi Lars!
Indeed, this is how I will implement the tests for discos and spacetrack (yeah, I'll do that sooner or later...).
Perhaps the only weak point I see here is: in presence of a server response change (for instance an update in the data standard provided, or simply a changed endpoint), the tests will keep testing the mock, so we would not notice until someone runs the "online" version of the functionality.

[larshinueber]

Indeed, this is how I will implement the tests for discos and spacetrack (yeah, I'll do that sooner or later...).

Awesome, that's good to hear :)

Perhaps the only weak point I see here is: in presence of a server response change (for instance an update in the data standard provided, or simply a changed endpoint), the tests will keep testing the mock, so we would not notice until someone runs the "online" version of the functionality.

That's a very good point! I had a look at how astroquery handles this: they basically mark all tests which request online data with @pytest.mark.remote_data (https://github.com/astropy/astroquery/blob/349589f6fa23c9ed9f8100b0b9d7df71767965c2/astroquery/jplhorizons/tests/test_jplhorizons_remote.py#L16), which allows them to exclude running online tests (or only run online tests) depending on the purpose of CI. I think this could be a good compromise, see also https://astroquery.readthedocs.io/en/latest/testing.html#running-only-the-remote-data-tests

[luigigisolfi]

That's a very good point! I had a look at how astroquery handles this: they basically mark all tests which request online data with @pytest.mark.remote_data (https://github.com/astropy/astroquery/blob/349589f6fa23c9ed9f8100b0b9d7df71767965c2/astroquery/jplhorizons/tests/test_jplhorizons_remote.py#L16), which allows them to exclude running online tests (or only run online tests) depending on the purpose of CI. I think this could be a good compromise, see also https://astroquery.readthedocs.io/en/latest/testing.html#running-only-the-remote-data-tests

very nice :)

[luigigisolfi]

@larshinueber , @DominicDirkx
Update on this: I am writing unit tests for spacetrack.py. Some run online, others run offline by mocking functions and data returns.

I created a conftest.py file (see https://docs.pytest.org/en/stable/reference/fixtures.html), automatically detected by pytest, that basically acts with a check on the decorators in the tests, and only runs the ones decorated with :

@pytest.mark.remote_data

if the --remote-data flag is given, and skips those if the flag is not given.

The conftest.py file is unique and it can therefore be used in any other tudatpy test with the same decorator logic.
A PR will follow in the next few days to showcase this.

[luigigisolfi]

Here's the conftest.py implementation i was talking about:

https://github.com/luigigisolfi/tudatpy/blob/d15d92c5ec27f0d1bb22ba980269d7e5b747b95e/tests/test_tudatpy/conftest.py

Reference PR: #533