RFC: serialization of route loaders data

[Varixo]

What is it about?

Reducing the size of the initial rendered HTML, by being smarter at which data to serialize from the route loaders.

What's the motivation for this proposal?

Problems you are trying to solve:

Let's say we have the following route loader:

export const useProductDetails = routeLoader$(async (requestEvent) => {
  const res = await fetch(
    `https://.../products/${requestEvent.params.productId}`
  );
  const product = await res.json();
  return product as Product;
});

Currently we are able to tree shake (or "Data Shake" ™) route loaders data in some simple cases, for example if only a portion of that data is being used.

Example:

export default component$(() => {
  const data = useProductDetails();

  return (
    <>
      Product name: {data.value.product.name} // 👈 the rest of this data will get "data shaken" and not serialized
    </>
  );
});

But when the component has a condition which affects its rendering, there is no way for Qwik to know whether its data will be required by one of the children of that component (or grandchildren), so the optimizer marks a component as dynamic and all loaders data are serialized, even if the data are not required later.

export default component$(() => {
  const data = useProductDetails();
  const condition = useSignal(false);
  return (
    <>
      Product name: {data.value.product.name}
      <button onClick$={() => (condition.value = !condition.value)}>
        Toggle
      </button>
      {condition.value ? <Child /> : <div />}  // 👈 this is the line which marks this entire component as "dynamic"
    </> 
  );
});

Due to the {condition.value ? <Child /> : <div/>} condition, the component is marked as dynamic. This is because we don't know if the <Child /> component calls the useProductDetails(). In that case, we need the data for the <Child /> component.

Goals you are trying to achieve:

• Create a solution without serializing the route loader data
• Create a simple, easy to understand and to explain solution

Any other context or information you want to share:

---

Proposed Solution / Feature

What do you propose?

The loader data will never serialized from context. After SSR, the data will always be discarded.

If a child component uses the route loader again, we will lazy load the data as needed.
Additionally, provide an option serializationStrategy: 'never' | 'always' | 'auto' to routeLoader$ to serialize the state anyway if developers want it.

export interface CommonLoaderActionOptions {
  readonly id?: string;
  readonly validation?: DataValidator[];
+  readonly serializationStrategy?: SerializationStrategy;
}

Code examples

1. Default behavior - data are not serialized

export const useProductDetails = routeLoader$(async (requestEvent) => {
  const res = await fetch(
    `https://.../products/${requestEvent.params.productId}`
  );
  const product = await res.json();
  return product as Product;
});

export default component$(() => {
  const data = useProductDetails();
  const condition = useSignal(false);
  return (
    <>
      Product name: {data.value.product.name}
      <button onClick$={() => (condition.value = !condition.value)}>
        Toggle
      </button>
      {condition.value ? <Child /> : <div />}
    </>
  );
});

export const Child = component$(() => {
  const data = useProductDetails();
  return (
    <div>
      Foo
      {data.value.product.name}
    </div>
  );
});

In above example the useProductDetails data are not serialized.
If the Child component is rendered, the data are lazy loaded through the server.

2. Persist option - data are serialized

export const useProductDetails = routeLoader$(
  async (requestEvent) => {
    const res = await fetch(
      `https://.../products/${requestEvent.params.productId}`
    );
    const product = await res.json();
    return product as Product;
  },
  {
    serializationStrategy: 'always',
  }
);

export default component$(() => {
  const data = useProductDetails();
  const condition = useSignal(false);
  return (
    <>
      Product name: {data.value.product.name}
      <button onClick$={() => (condition.value = !condition.value)}>
        Toggle
      </button>
      {condition.value ? <Child /> : <div />}
    </>
  );
});

export const Child = component$(() => {
  const data = useProductDetails();
  return (
    <div>
      Foo
      {data.value.product.name}
    </div>
  );
});

In above example the useProductDetails data are serialized, because serializationStrategy: always option is passed.
If the Child component is rendered, the data are reused from state.

---

Links / References

No response

[ianlet]

Would it affect the "prefetching" strategy? If the data is lazy loaded, it will introduce an extra delay visible to the user while the data is fetched, which will affect negatively the user experience.

It would have to be tested/benchmarked, but my feeling is that the extra time it takes to serialize data that is unused is less affecting the UX than a whole new server call (extra delays on user interaction) to fetch the data when needed.

Also, should it be the other way around? By default it's serialized (like it is right now) when we can make sure data won't be needed, but you can pass an option (e.g. eager: false) to fetch the data later.

[Varixo]

It would work just like partial SPA. If you don't want that behavior you can pass the persist option

[Varixo]

We want to serialize less data, so we can't make the persist: true as default.
Basically serializing sometimes unnecessary few MB of data is a huge problem.

It is worth mentioning that this behavior only occurs if you have a component that uses route loader data and is rendered on the browser side

[wmertens]

👍 but maybe eager is a better name? Defaulting to false.

Also, we have the reverse problem with contexts, where we always skip them if we don't see them being used during SSR. Maybe there we should add eager to useContextProvider, default true.

These could be slightly breaking changes to add to v2.0

[Varixo]

good idea, changed

[DustinJSilk]

It feels like this will mean developers must always think about the potential UX interaction delays when using data from a routeLoader$

If we reverse this proposal to allow an option called ‘lazy: true’ then it’s clear the data will only be fetched when it’s needed and that it will potentially impact UX.

[Varixo]

I managed to add additional global configuration for route loaders:

   qwikRouter({
        // config
        defaultLoadersSerializationStrategy: 'always' // 'always' | 'never' | 'auto', default to 'never'
      })

Name is a subject to change

[DustinJSilk]

When this feature is turned on, does it mean the loaders are called every time a new piece of data is required? And so the server will have to fetch/build/process that large data each time a piece of it is needed?

I'm just wondering if an alternative solution could be to stream the unused data at the end of the document after the qwikloader.
This way, the page can boot while the large unused data streams in afterwards. This won't reduce the amount of data being returned, but it would improve the speed by reducing the data required to bootstrap and wouldn't result is loaders running multiple times when they're interacted with.

Is something like that possible and would it solve the problem?

[Varixo]

Loaders are not called again, only data are delivered on demand. If you don't like the solution you can set serialize to always or auto (this one in future probably).

The state is at the end of html so this is just serialization of data.

[DustinJSilk]

This makes sense thanks

[Varixo]

small correction - for now loader can be called again. I need to talk with the other core members how it should behave, so no promise

[wmertens]

Probably this is for a different RFC, but we need a mechanism where we can invalidate routeloader data, separate from browser cache.

q-data is only requested for SPA, which means we have full control. If we determine a loader should be updated, we can fetch it with no-cache. We can even decide to show the old data in the meantime, or to block on the load.

What we need on the client is:

1. specification of loader staleness

• including whether to show old data while it loads

2. list of loaders used by a route

For 1 we could use a simple timeout. We should also allow some function$ that returns a boolean?

For 2 we can create a nested object that represents the paths, including aliases and wildcards. At each level it lists the loaders. If there are no loaders from a certain path, the subtree is empty.

The loader configuration is provided as part of the leader result, because you only need to know it when you already loaded is data.

The loader config requires finding all loader exports in the routes at build time. We can do this by checking the display names of exported QRLs.

[Varixo]

The PR has been merged