Improve loader data prefetching

[richardscarrott]

Problem

Putting loader data in the browser's prefetch cache can easily cause your UI to become out of sync with the server because Chrome (and others?) aggressively cache for 5 minutes, ignoring all cache control headers other than 'no-store' -- even if you say "max-age=10" or "no-cache, must-revalidate" it'll keep it for 5 minutes.

This makes <Link prefetch /> unusable for any pages which cannot tolerate being stale for 5 minutes (i.e. any page which returns data that can change as the result of a mutation).

For example, imagine this common ecom scenario:

1. User visits product listing page
2. Remix prefetches product detail pages on "render" or "viewport".
3. Instead of navigating to the product detail page, user adds product to wishlist
4. Finally user navigates to product detail page

EXPECT: Product detail page to indicate it's in your wishlist
ACTUAL: Product detail page does not indicate it's in your wishlist because it rendered stale loader data from the prefetch cache 😬

It's therefore critical for loaders which cannot tolerate stale responses to return Cache-Control: no-store just in case somebody accidentally links to it with a prefetch link, making prefetching useless 😔

Possible Solutions

1. Given the prefetch cache doesn't have an API to invalidate the cache (AFAIK), I think it would be worth avoiding the browser prefetch cache for loader data and instead caching it in memory so that Remix can decide when to invalidate / revalidate the cache entry after a mutation.

2. Alternatively, it might be reasonable for Remix to re-prefetch after a mutation, as I believe browsers will use the latest prefetch, even if in-flight.

   • Incidentally, using prefetch on hover with prefetch="intent" effectively does this because you naturally hover over the link again in step 4.

[richardscarrott]

I pushed up a PR which demonstrates how 2. might be implemented #7903

[brophdawg11]

avoiding the browser prefetch cache for loader data and instead caching it in memory

Remix is pretty bullish on "using the platform" so I don't know that we want to forego the browser prefetch cache in favor of maintaining our own internal in-memory cache. I think this feels like a behavior of the prefetch cache that maybe can be explicitly called out in our docs so folks know in which scenarios it might not be appropriate to use the <Link prefetch> feature.

re-prefetch after a mutation

Thanks for taking a stab at this! My main concerns with this approach in #7903 would be introducing over-prefetching and whether it works cross browser. You may be able to implement this in user-land as well via <PrefetchPageLinks> for the cases where a normal prefetch isn't suitable.

[richardscarrott]

I think this feels like a behavior of the prefetch cache that maybe can be explicitly called out in our docs so folks know in which scenarios it might not be appropriate to use the feature.

I think that'd be a shame as it relegates prefetching to static sites, or at least read-only websites. Prefetching is such a huge win for perceived performance, I really think it'd be worth nailing it for dynamic data.

My main concerns with this approach in #7903 would be introducing over-prefetching and whether it works cross browser.

If your headers are setup appropriately for dynamic content (Cache-Control: no-cache, must-revalidate) -- i.e. only allow use of the prefetch cache -- then Remix does a ton of over-fetching currently because, depending on the prefetch behaviour it'll repeatedly re-fetch whenever you hover or scroll the link into view.

re: cross browser, I agree. Kinda why I'd personally lean towards an in-memory cache for prefetched loader data as then you have control, but does sound like a bit of a faff to implement.

If there isn't a nice way to solve the issue of stale data then perhaps it would at least be worth allowing links to only prefetch static content (JS, CSS etc.) ahead of time? e.g.

<Link prefetchAssets="viewport" />

[sergiodxa]

Given the prefetch cache doesn't have an API to invalidate the cache (AFAIK)

You can use the Vary header, combined with a dummy cookie that contains some random value (it could even be just Date.now()), then configure the Vary header to tell browser's cache to ignore the cache if the cookie changed, allowing you to invalidate the cache by just setting a new value of that cookie.