Parallelized SSG Pages

[hassanbazzi]

Feature request

Is your feature request related to a problem? Please describe.

When building SSG pages, getStaticProps should be parallelized as much as possible instead of being synchronous. For example, we have 4000 pages coming from the same file on our website, and the build now takes ~14 minutes instead of 2 minutes.

Describe the solution you'd like

Parralelize all getStaticProps coming out of a single getStaticPaths call.

Describe alternatives you've considered

Turning off SSG, or somehow detecting if we're on master and only enabling it there (somehow).

[c0b41]

checkout this rfc #11552

[hassanbazzi]

Thank you @c0b41

[ijjk]

Hi, building SSG pages is parallelized currently. We create a worker for each CPU core you have available on your system and distribute rendering the SSG pages between these workers (related code).

You can also leverage the fallback feature which allows you to only prerender the most important pages and then render any additional ones after the build on-demand

[hassanbazzi]

So I have an Idea.

The biggest reason we have fallback:false is because we’re using SSG for SEO purposes, especially speed. Doing it on runtime would defeat the point.

What if, instead of running getStaticPaths before getStaticProps, we run a single path into getStaticProps (to make sure it works at build time).

Afterwards, we run the entire result of getStaticPaths as a form of cold hydration.

This would eliminate the crazy high build times on these large catalog sites, give you the SEO and SSG benefits, and introduce a more async build step for building these pages.

We can still report any failures with this back somewhere through error reporting.

The only issue I see in this is invalidating a build if errors happen in the cold hydration step, but there isn’t a good solution for that in the current fallback:true method anyways. In fact, at least you can pre-emptively react to it this way.

Thoughts?

Cc @rauchg

[timneutkens]

So I have an Idea.

The biggest reason we have fallback:false is because we’re using SSG for SEO purposes, especially speed. Doing it on runtime would defeat the point.

What if, instead of running getStaticPaths before getStaticProps, we run a single path into getStaticProps (to make sure it works at build time).

Afterwards, we run the entire result of getStaticPaths as a form of cold hydration.

This would eliminate the crazy high build times on these large catalog sites, give you the SEO and SSG benefits, and introduce a more async build step for building these pages.

We can still report any failures with this back somewhere through error reporting.

The only issue I see in this is invalidating a build if errors happen in the cold hydration step, but there isn’t a good solution for that in the current fallback:true method anyways. In fact, at least you can pre-emptively react to it this way.

Thoughts?

Cc @rauchg

It's unclear what you're trying to achieve exactly 🤔 Do you just want to prerender all pages? As @ijjk said we already parallelize to the maximum possible on the machine you run next build. We're going to introduce further optimizations on vercel.com though, but I'm not sure if you're currently on vercel or not.

[hassanbazzi]

Hey!

I'm re-opening this as I'm still using a custom solution that I think the rest of Vercel/Next's customers would benefit from.

Maybe I was too verbose, but basically, I want an async getStaticPaths method that gets called after the build is done to warm up the cache. Maybe a getColdPaths() extra method or something like that.

What I do now is that I have fallback:true enabled, and then I have a script that uses the same code as getStaticPaths(), but just visits every single page to get it built.

The benefit from this is that I have low build times (as opposed to our previous 30 minute build time where Vercel has to build thousands and thousands of pages), and I get the SEO benefits of pre-building all of my static pages.

[Timer]

Have you tried the new fallback: 'unstable_blocking' feature which will render the page on-demand instead of serving a static fallback for the first request?