Heavy object makes getServerSideProps slow in development

[yacklab]

Hello y'all,

I have a getServerSideProps method fetching a quite big object (12mb). But when I return this object the request becomes extremly slow even without building the component with this data.

The query of that object isn't the issue here. As you can see (I added the results in the code) the query itself is quite ok if I dont pass the object as prop.

export const getServerSideProps: GetServerSideProps = async function (params) {
  const res = await fetch(`${SERVER_DOMAIN}/api/list`);
  const data = await res.json();
  console.log(memorySizeOf(data)); // 12mb

  // this takes 25 to 40 sec to load with next dev
  // this takes 1.4 sec to load with next build && next start
  return { props: { data } };
  // this takes 1.55 sec to load with next dev
  // this takes 1.11 sec to load with next build && next start
  return { props: { ok: "ok" } };
};

export default function Home() {
  return <div>ok</div>;
}

[brianlovin]

getServerSideProps will run on every page request. In the case where you're sending 12mb over the wire, it does seem to make sense that it would be considerably slower than without that payload. Even if the component isn't consuming the prop, you've still told the server to send it to the client. Does that feel correct to you? If performance is a concern here, it seems like you could think about how to split that request up to fetch only the minimal amount of data needed for the page skeleton to load, and defer the rest of the load to the client to fetch after the user is interacting with the page.

[zackdotcomputer]

Yup - I agree with Brian here. Just to add that the issue is that in dev, even if you're not using the data, sending it as a prop to the component will cause it to get copied into the SSR result sent back to the client because dev isn't doing any optimizations to your code. Your production versions are the same and much faster because next build is able to recognize the pointless data transfer and optimize it away.

[brianlovin]

Your production versions are the same and much faster because next build is able to recognize the pointless data transfer and optimize it away.

I was actually wondering about this, if Next.js is actually able to optimize unused props and not send them over the wire 🤔

[zackdotcomputer]

Heh I was just about to try to spin up a lightweight reproduction here to inspect it and find out! That is TBH just my assumption based on the time stats @yacklab shared of his heavyweight call in dev vs production.

[zackdotcomputer]

Ok I've run out of time to explore this but quickly I'll share my learnings:

1. The getServerSideProps method is not optimized completely away. It still is getting called.
2. Optimization is entirely done by the webpack layer, which I am far from an expert in.
3. My current assumption (untested) is that the time delay might honestly just be memory copying times in JS to copy 12MB between different points in the heap as it flows up the stack from inside of the promise back to the template. My current guess, then, is that what webpack is doing is eliminating some series of copies there both using it's own efficiency magic and also by recognizing that the prop is not used at its destination and, so, can be discarded at somepoint.

[yacklab]

I will get rid of that heavy payload in the future, but I'm still messing around with my app so I don't really mind for now to just get all of the datas like this. I was just currious about what happened, I had the same kind of idea as @zackdotcomputer but I din't have time this afternoon to test, I will probably be able to do this tomorow.
To give you guys some hint,

• When my request is processed I have a huge spike in cpu activity (7% to 80%).
• The document sent to the browser is about 1.7mb with the production server as well as with the dev server
• if I display just a part of the data (a 10 char string) I have exactly the same behaviour as I had before
• if I alter my component (let's say I add a title) the HMR works fine
  This is were I was before leaving this afternoon. I will keep you guys updated.

Should I open an issue ?

[Timer]

getServerSideProps and getStaticProps are meant for kilobytes of data, not megabytes. Everything you return from this function is downloaded by your user on initial page visit.

Our development mode has a strict object shape verification process that takes some time to process the 12MB, so I'd say this is behaving as expected. It's not a bug, and is pointing out you should simplify your return structure.