testing with betamax?

[anarcat]

is there a way to test this module alongside betamax?

the idea would be that the cachecontrol module would hit its own cache if present, otherwise bounce back into betamax. I am already using betamax extensively in my project to "mock" HTTP requests (record and replay, really) and it's been quite useful. But when I pair it with cachecontrol, I don't get good results.

either:

1. i add cachecontrol as an "old_adapter" in the betamax session, which works in the sense that requests never hit the network, but then they never hit the cachecontrol adapter either, because the query gets cached in betamax. even if they would hit cachecontrol, they would need to hit the network first
2. I mount cachecontrol on top of a betamax session, in which case cachecontrol completely obliterates the betamax adapter and sends requests on the network (instead of tapping in betamax)

Those two possible scenarios are pretty much covered in this ifcase:

cache_adapter = cachecontrol.CacheControlAdapter(cache=FeedContentCacheStorage(self.db_path))
# assume we mount over http and https all at once so check
# only the latter
adapter = self._session.adapters.get('https://', None)
if hasattr(adapter, 'old_adapters'):
    # looks like a betamax session was setup, hook ourselves behind it
    logging.debug('appending cache adapter (%r) to existing betamax adapter (%r)', cache_adapter, adapter)
    adapter.old_adapters['http://'] = cache_adapter
    adapter.old_adapters['https://'] = cache_adapter
else:
    logging.debug('mounting cache adapter (%r)', cache_adapter)
    # override existing adapters to use the cache adapter instead
    self._session.mount('http://', cache_adapter)
    self._session.mount('https://', cache_adapter)

That's pretty much case 1 above, which falls back to case 2 if we don't detect a "betamax" adapter in place.

And of course, it doesn't work.

What I would need is something like betamax's "old_adapters" system. Instead of calling the parent class, CacheControlAdapter.send() should allow the caller to define a list of adapters to call instead. The way betamax does this is fairly simple: there's an old_adapters parameter that you can pass and that it uses for sending its requests..

I have looked into the cachecontrol test suite to figure out if there could be inspiration there, but it seems to "mock" an http adapter instead of delegating requests... Since I don't want to reimplement the entire thing myself (and I can't figure out how to hook betamax in there either), could we add a similar functionality here?

Thanks!

PS: it seems I can mock a parent class but that seems horribly dirty - I am not sure I want to go that far...

[kratsg]

Have you been able to figure this out? I ran into this situation.

Here's my hack for now (in testing setup):

    with betamax.Betamax(session).use_cassette('test_cache.test_cache_component'):
        session.mount(session.prefix_url, session.adapters.get('https://', None).old_adapters[session.prefix_url])
        response = session.get(...)

and it seems to work correctly, except it still calls HTTPAdapter instead of relying on requests.adapters.

[anarcat]

Have you been able to figure this out? I ran into this situation.

this is all i got:

https://gitlab.com/anarcat/feed2exec/-/blob/master/feed2exec/tests/test_feeds.py#L175
https://gitlab.com/anarcat/feed2exec/-/blob/master/feed2exec/controller.py#L84

[kratsg]

Note that i found out betamax never calls build_response so you never end up hitting cachecontrol from that. Which is somewhat frustrating.

[anarcat]

yep, that's what i found out as well.

[kratsg]

this is roughly how close I got

    def mock_factory(betamax_send):
        def wrap_betamax_send(self, request, **kwargs):
            resp = betamax_send(request, **kwargs)
            return session.adapters.get(client.prefix_url, None).build_response(
                request, resp
            )

        return wrap_betamax_send

    with betamax.Betamax(session).use_cassette('test_cache.test_cache_component'):
        mocker.patch(
            'betamax.adapter.BetamaxAdapter.send',
            new=mock_factory(sesssion.adapters.get('https://', None).send),
        )
        client.mount(
            session.prefix_url,
            session.adapters.get('https://', None).old_adapters[session.prefix_url],
        )

problem is that betamax seems to be doing a bunch of extra stuff that I'm unable to inject cachecontrol in correctly.