Emitting telemetry about query cache effectiveness

[bcoe]

Problem I'm trying to solve

I would like to be able to emit telemetry about QueryCache behaviour.

Specific questions that would be useful to answer:

• hits (a query that exists for a given key, and is newer than the given staleTime) / misses (how often does the query fall through cache and fetch upstream data):
  • Being able to answer this question helps someone building an application understand if the configured staleTime is actually having the intended effect of reducing calls for upstream data.
• hit latency, vs., miss latency:
  • It's interesting to know the actual latency cost associated with every lookup that results in fetching upstream data.

What I've Tried

• queryCache.subscribe: As far as I could tell this provides information about items being added to the cache, but does not provide information about cache hit successes.
• subclassing QueryCache and overriding getQueryData():
  • In experimenting, this seemed to work as expected:
    • On initial page load, cache hits would "miss".
    • When navigating on a SPA application, coming back to prior pages would "hit".
    • If I waited 5 minutes until GC happened, calls to getQueryData() would again miss.
  • Problems:
    • The logic for staleTime did not work as I would expect: despite a 60 second staleTime being configured, items continued to return from cache until they were deleted after 5 minutes by a separate operation.
    • It felt pretty fragile to subclass getQueryData().
    • From this discussion, it sounded like getQueryData() would fire more than intended?
      • 
      • Open Question: Assuming we want to understand for a long-lived user session, how often were we able to return values from cache, vs., an expensive fetch operation, what do we want?
        • To me, it seemed valuable to know any time a value was returned from cache that could have potentially resulted in fetching upstream data -- but perhaps there's a middle-ground here that gives an idea of cache utilization with a few less events?

Suggested Implementation

Something like a queryClient.subscribe that would allow an observer to see, for each query:

1. the latency
2. whether or not the results of the query were served from cache.

[TkDodo]

subclassing QueryCache and overriding getQueryData

we are not calling queryClient.getQueryData anywhere internally. It's a user-facing function to read from the cache imperatively.

The logic for staleTime did not work as I would expect: despite a 60 second staleTime being configured, items continued to return from cache until they were deleted after 5 minutes by a separate operation.

Data being marked as stale after staleTime does not mean the next time it is read, it will lead to an automatic fetch. This goes back to my tweet you've linked where I tried to explain that every time a component re-renders, it reads data from the cache. How else could const { data } = useQuery(...) work ? It reads from the cache and give data to you. If data is there, it's a cache it. If not, it will put the query in pending state and fetch data. If data exists and is considered stale, you'll still get it + a background refetch. That's stale-while-revalidate in a nutshell.

I guess the easy answer is No, we don't provide metrics about cache effectiveness. End of story. But I'm still trying to understand what you actually want to compare.

I guess you are not counting a re-render that reads data from the cache as a cache it? So what is a cache hit then? A situation where, if you wouldn't use react-query but useEffect + useState, you'd get a fetch, but with react-query, you get a cache hit?