[Feature]: Make Loader Data Global Data

[Stancobridge]

What is the new or updated feature that you are suggesting?

Though I have not used much of Remix, from the documentation, I didn't see a way to access router loader data globally, I don't mean with the use of useFetcher

In react-query, every data is automatically global which you can access using the queryKey if this concept can be adopted in remix it will also be good, since most developers use react-query it will be a comfortable development context for the developers using react-query.

Considering the case:
In my route I have 3 components that use different data for each, so there is a need to make the request in each of the component not only in the main route, if I update data in One Component I want another component in the same route refetch it's data, use the returned response the update request or refetch its contents based on some conditions, I don't want any other component to refetch it's data.

Features I think that can be implemented to help with global server data are:

1. Each Loader can have an optional unique key(this can be the route since every route is unique and also the api route) that identifies the loader: export const loader = withKey('route-link', async (): LoaderFunction => {})
2. useFetcher can use this key to know which API route to fetch and return cache data if already fetch: useFetcher('route-link')
3. Cache Response can be configure when to cache and when on a global level or per route
4. useCacheData can be used to access cached data only: useCacheData('route-link')
5. I don't know if useFetcher can be used programmatically no only through d Form Component that it returns

Main Points:

1. Make Loader Global
2. And Also Maker useFetcher programmable

Why should this feature be included?

giving the developer ability to control the global data from one base point

[sergiodxa]

You can use the useMatches hook to access the data of every route currently active, and a few more things too, like the route handle export.

[Stancobridge]

You can use the useMatches hook to access the data of every route currently active, and a few more things too, like the route handle export.

from https://remix.run/docs/en/v1/api/remix#usematches returns the matched routes,

it has not nothing with fetching data in a component

[sergiodxa]

I understood you wanted to be able to get the data from the loaders.

If you want to fetch data from any loader you can use useFetcher, that will give you fetcher.load method you can call passing a URL and call the loader of that route, e.g. fetcher.loader("/my/api/endpoint").

If you want that data fetched there be global you can move it to root and store it inside a context provider. Another option is that you can still use RQ/SWR.

But remember, if you move the data fetching to the client with useFetcher or RQ/SWR you will lose the option to do SSR of that component since you are going to fetch data client-side only. And if you don't fetch data using the routes loader Remix will not be able to revalidate after a form submit (you will have to do it yourself, either manually or by triggering RQ/SWR revalidation), you will lose the performance optimizations Remix can do for you like prefetch on link hover, only fetch the data a child route needs instead of refetching data from parents (you may still get this because of the client-side cache tho), a more things I'm not remembering right now.

[nickjs]

@sergiodxa I understood the context of the original issue raised here same as you, and indeed, useMatches() works great for accessing data from other nested loaders. However, because of the name (and possibly the implementation itself?), eslint will complain about not being allowed to use a React hook inside a non-component function (i.e. loader).

I'd like to propose exporting either an alias or separate function (if the implementation needs to be altered to not rely on any other hooks) called getMatches() that can be safely invoked on the server inside a loader or action.

If you think that's a worthwhile endeavor I'm happy to open a PR to add that. I can also open a separate issue from this one if that would be helpful.

Thank you!

[sergiodxa]

@nickjs the thing with accessing data from a loader inside another loader is that they run at the same time so there's no data to access when the loader run.

For client-side navigation Remix will only call each loader as individual requests. So they will also run in parallel, and depending on your deployment provider they can even run in different hardwares making impossible to share data between them without using an external service like Redis.

Another things is that even if Remix is only calling a single nested loader the only way to have access to the parent loaders data would be if Remix sent the whole data in useMatches to every loader call, with a POST request it may be possible, with a GET request like the ones used for loaders it could be too much data to send in the URL and you can't use the body, and loaders can be changed to use POST because they will not be cacheable anymore.

And yes, you can't run useMatches inside a non-hook or non-component function because it's a React hook, it needs access to the internal context Remix has.

---

So, if you need the same data in two loaders, you need to query/fetch it in both loaders, if you only need the data in the React component of two routes which are rendered at the same time (parent and child) then you can return the data only from the parent route loader and use the useMatches to read the data on the child route component.

[nickjs]

@sergiodxa Well those are all excellent points. Sorry I didn't think of that sooner, I'm still fairly new to Remix. I'll have to rethink my loading strategy a bit I guess! I'll open up a new issue if I come up with any new ideas for data sharing, sorry for hijacking this issue!

[sergiodxa]

@nickjs don't worry! it's not obvious at the first, I have been using Remix for a year now and learnt all of this things.

[willin]

how to use root loader data in other pages' loader function?

[sergiodxa]

how to use root loader data in other pages' loader function?

You can’t because they run at the same time, in parallel, not in a serie, so when other loaders run the root loaders is also running.

[divmgl]

I'm glad we still have this issue open. I think I'll return to Remix once it has an elegant way of doing this.

Last I spoke with @ryanflorence I proposed allowing loaders to pool promises into an array as a second parameter in the loader function so that nested loaders can await on them. That way we're not issuing multiple requests for the same data for a single page.

I started working on an implementation for this but it was taking up too much time. This was time that I could spend just working on my project in a plain ol' CSR environment. It also didn't help that I had tons of trouble developing Remix on my machine (maybe that's better now with the new playground). I ended up just moving away from Remix and creating a plain ol' CSR app. This was also a big reason why I've stopped evangelizing Remix professionally. The projects I've worked on usually use Redux and/or some kind of global state heavily (react-query, swr, etc.)

Anyhow, this isn't as relevant in the client-side because you can access this data with useLoaderData and useMatch, but if you ever want to redirect or bring back data in nested loaders based on some kind of parent data, well, you can't. You'll need to fetch the data over and over again.

There's some ways of overcoming this. One of the things I tried doing was storing a hash in memory with keys and request promises. If the key didn't exist then the request needed to be made. If it does exist then just await on that promise. The hard part was finding out what requests belonged to each other. Maybe if requests had some kind of unique ID so that loaders could understand they're all part of the same request, that'd be nice. I'm not sure how to do this though.

Hopefully the creators of Remix will change their mind.

[sergiodxa]

Passing a list of promises from other loaders wouldn’t be reliable.

The only time all the loaders are guaranteed to run all in the same server is on the initial document request, after that child loaders may be called without the parent loaders, or if they ask run together they could even will have different requests, in some platforms they may even have different physical hardware.

If your code expects other loaders to run to get data from there then in those cases it will break.

What you want is to batch your requests/queries, there’s an example showing how to do it using the dataloader package and the express adapter, this way on the initial document request to only query the same endpoint once and the rest keep working.

Also, I think the reason you want this is because you are thinking of loaders are RQ calls you do client side, when you should think of them as an API endpoint server side, if you had to build an API for your app using let’s say plain Express, and two or more endpoint needed to fetch or query the same data you would have to do it on each endpoint, there works be no way to share the request between endpoints without caching them somewhere.

Remix loaders and actions are the same, they are automatically generated API endpoints for your routes.

[divmgl]

I'm going to repeat here what I've said on Discord. I understand how Remix works; it doesn't mean that the use cases are invalid.

and two or more endpoint needed to fetch or query the same data you would have to do it on each endpoint, there works be no way to share the request between endpoints without caching them somewhere.

...unless the endpoints were nested somehow or there was middleware, which is very common in backend systems. Almost every API framework I've ever used, even going back to the ASP.NET MVC days, has the concept of middleware.

Here's one use-case. Say I have this route:

/admin/profile

admin has an <Outlet />, so it'll render the profile route. In order to avoid regular users reaching the admin route at all, the admin loader fetches some kind of resource to verify the identity of the user. Let's say it uses the JWT token in the cookie provided in the request, and let's assume we don't store the entire user object in the JWT token. The admin loader gets the ID from the token, fetches the User resource in a data store, and checks the user's role. If the user is not an admin, the admin loader redirects. All's well.

However, the profile route now needs this same resource again if the user IS an admin. So we fetch the User resource again. Normally this wouldn't be a problem, but then the profile has yet another subroute:

/admin/profile/security

...which AGAIN has to fetch the user. At this point we've now fetched the same User resource three times.

I would love to just be able to get the User that was grabbed from admin the one time. A common pattern in other frameworks is to create middleware that will store this user in the context of the request and child routes can use it. Sure, the loaders in Remix were run in parallel 🙄 yes, I get it, I've been told this a thousand times, and I realize this is antithetical to the way Remix works.

But hear me out: if there was some kind of global state per request, this problem would be mitigated. We'd store a promise of some kind there (one of the loaders would certainly beat another loader into placing a promise there) and all of the loaders would be able to await on that promise. When that promise is finished, all of the other loaders that await on it would continue in parallel.

I think there's an argument to be made here for just having plain ol' middleware, but that's a battle for another time :) There's lots of reasons why I think middleware would be useful, but at least having some kind of global state per request would somewhat mitigate this particular issue.

[sergiodxa]

I think your problem is not the duplicated fetching, but the duplication of code.

Because even with middleware the data would have been read one per API endpoint. Following your example, if you have a /admin, /admin/profile and /admin/profile/security endpoints in a normal API built using middleware, the code to get the user would have been extracted away from the controllers of those endpoints, but for each endpoint the code to fetch the resource would have run individually.

Middleware would not solver duplicated fetching, only duplicated code. You could in Remix create a function to abstract again all of this code and return the user or a 401 or redirect response if not authenticated. You would of course still need to call it yourself on every loader and action, but this is, right now, the most similar thing you will have to middleware without using Express middleware or another server middleware implementation.

But again, middleware would not solve the fact that the User resource is being fetched three times, because they are three individual request so they can't share data. The only scenario where you could avoid the resource being fetched three times would have been on the initial document request where all loaders runs as part of the same HTTP request.

---

That aside, I think there's an issue or discussion here about adding a pre and post hooks functions server side so you could run code before your loaders/actions, which would enable a global middleware support, and you could check the URL yourself to know if you should run or not your code if you want to limit to to some routes.

I agree that it could be useful for code organization, I think the main issue of middleware would be that typing them is way harder.

[kiliman]

@divmgl I used to think the same as you. I even added a patch to Remix to include a X-Remix-Navigation-Id header to track the same logical request even if they are multiple parallel requests. But it really only works if you have a single server that is guaranteed to process all the requests. So at that point, it's simpler to use a server-side cache that you manage.

Anyway, the initial request can still benefit from a shared context cache. My example adds a cache object to the getLoadContext context. Then in my loaders, I call a helper to get a value from cache (and pass a Promise). The first caller will initialize the promise, and subsequent callers (in the same request) will await on that promise.

export function getFromCache(
  cache: any,
  key: string,
  getter: () => Promise<any>,
) {
  let promise = cache[key]
  if (!promise) {
    promise = getter()
    cache[key] = promise
  }
  return promise
}

// from your loader
// since cache is the same on document request, only one call to getUser will be
// made. The other loaders will await the same promise
const user = await getFromCache(context.cache, 'user', () => getUser(123))

https://github.com/kiliman/remix-playground

[davecranwell]

I'm struggling here too with racing parallel requests in a scenario where all the requests need to refresh a token for the other requests in the group to be able to succeed.

A shared server side state, through which the race winner could give a promise to the other requests to await a usable access token, would solve my problem neatly without involving retrying.

And yes, only a full page refresh would trigger all N requests at once, but the improved ergonomics of handling exactly that situation seems to be one of the key reasons to use remix in the first place.

[kiliman]

Have you tried the solution above? The context.cache is initialized via getLoadContext in the adapter. This way all loaders will get the same cache instance for the life of the request.

[davecranwell]

I have indeed been looking at your playground with great interest! But remix has changed a fair bit since then and as far as I can tell your solution still depends on your patch file, which no longer applies so easily to the latest version (the second half does, the first half I've so far been unsuccessful with). Perhaps I've got it wrong though, are the patches unnecessarily now?

[kiliman]

The patch was to add a header X-Remix-Navigation-Id. It was used to group multiple loader requests into a single "logical" request. Useful when applying a cache across parallel fetches. However, the getFromCache function above doesn't need the patch.

Just return a cache object from getLoadContext, then you'll have a shared cache that you can use across initial document request, since that's a single request.

// server.js
const getLoadContext = (req, res) => {
  // return shared cache object (`context.cache`)
  return { cache: {} }
}

[pmlk]

I have a very similar struggle to @davecranwell's, where different loaders may update different parts of a session cookie: the root loader may update the CSRF token while other loaders may update access tokens. This has been quite the learning experience of remix' ins and outs and continues to be one.

I think I can solve my problem if I can add some values / cache to the context available in the loaders, e.g. with getLoadContext. However, I fail to understand how exactly to do so with my setup.

My entry.server.tsx looks similar to the one from the docs with export default function handleRequest(...). I don't have an express-style createRequestHandler(...) anywhere as mentioned on the same docs page regarding loader context. I don't have a file analogous to the some-express-server.ts mentioned in the "loader context" section of the docs; I only have an entry.server.tsx which I assumed would serve the same purpose.

So, I don't really know where to put the getLoadContext. I've tried the following as a wild guess (inspired by how handleDataRequest works), but it didn't work:

// entry.server.tsx
export const getLoadContext = (req, res) => {
  return { foo: 'bar' };
};

Any help would be much appreciated. Can someone point me in the right direction?

[kiliman]

Currently you need to use another adapter (other than Remix App Server) to use getLoadContext. This is part of the server entry file (hands off the request to Remix).

I do have a PR that lets you create a server file, yet still use RAS. PR #4123.

In the meantime, you can eject from RAS to Express adapter using the following script: https://rmx.fyi/ejectras

[promisetochi]

@sergiodxa @kiliman

A use-case that is presently quite difficult to implement out of the box with Remix is that of an ecommerce/cms platform where multiple domains (e.g shop owner website) points to one remix app, the remix app takes the url, retrieves information about the store/website based on the url and passes that data through to the loaders.

Not sure what the right way to implement this use-case is in remix. Presently testing with an express middleware that does the initial data fetching and passes the data on through the Request object. But that makes the remix application unusable on platforms like vercel.

[promisetochi]

Also looking into vercel middleware as a possible alternative

[sergiodxa]

I think right now you need to go with Express middlewares or check if your deployment platform allows you to run code before Remix runs, for example in Cloudflare you own the exported Worker function so you can run anything before Remix receives the actual request.

There's an RFC now to add middlewares to React Router (and eventually to Remix) remix-run/react-router#9564 so you could do that before your loaders run.

Still, on a client-side transition if two or more loaders are requested your middleware will run once per loader because each loader will be an individual HTTP request.

[john-griffin]

It seems like useRouteLoaderData solves for this problem generally? See docs here https://reactrouter.com/en/main/hooks/use-route-loader-data and example here https://github.com/epicweb-dev/rocket-rental/blob/26b6d8c5f71fe867cb93df3610fec293b3c83db3/app/utils/misc.ts#L35

[seanpascoe]

no, the problem is getting loader data from a different route in another loader. useRouteLoaderData allows you to get loader data from within the component.

[sergiodxa]

useRouteLoaderData let's you get the data of a route in another other route, to get the current route loader data in a component you can use useLoaderData directly.