What is "collecting page data" and how to turn it off?

[neongreen]

I am running next build for my app, but I don't have a database running or even an .env.production file on this machine.

I am not using statically generated pages so I expect that this shouldn't be a problem. All my pages are dynamic and so Next shouldn't have to run my backend to generate the frontend bundle.

However, I am getting errors like:

info  - Checking validity of types  
info  - Creating an optimized production build  
warn  - Compiled with warnings
info  - Collecting page data ......node_modules/@prisma/client/runtime/index.js:36171
            throw new PrismaClientInitializationError(error2.message, this.config.clientVersion, error2.error_code);
                  ^
PrismaClientInitializationError: error: Environment variable not found: DATABASE_URL.

What is happening? What is "collecting page data", can I see the pages that the data is being collected for, and how can I turn it off?

[ealexhaywood]

As far as I know "collecting page data" is a normal and necessary step of generating a production bundle with Next.

It appears the @prisma/client you're using still requires a DATABASE_URL environment variable to be set whether or not you're using a database. My guess is you'll need to add it to one of your .env.* files.

[neongreen]

I agree that @prisma/client needs DATABASE_URL but I don't use it in any of the pages — only in api routes, which shouldn't have any prerendering run on them.

[Ren-Logronio]

This makes containerizing next applications next to impossible, because you need to build the app when building the docker container..

[icyJoseph]

When you build a Next.js application, actual HTML is generated, so at build time, the build step runs functions such as getStaticProps, to pre-render pages, it also tries to figure out which pages can be build already, and does it. It is an optimisation that's welcomed.

If you don't want that to happen, use getServerSideProps on the page that's trying to collect build time data, and then you should be fine. However, at run time, DATABASE_URL and all things necessary to render that page, better be present!

This is actually a good thing Next.js does with getStaticProps, it sort of type checks and makes sure things are in place do that it can work well in production. getServerSideProps runs on demand only, if at runtime there's no DATABASE_URL, the page will fail to render.

[neongreen]

Agreed that ultimately it would be good to investigate if the imports are sane. (They might not be.)

However, I don't want to merely solve this by removing the underlying problem with imports, if it exists.

For the scope of this question I want to understand a specific thing about Next — what happens at the "collecting page data" stage, what does it do (does it try to prerender some pages? all pages? all files in pages/, including api routes? etc), whether it can be debugged (e.g. at least "what page is it looking at at the moment"), does it happen before the bundling or during it (or after?), and whether it can be customized/skipped. I went through the documentation and haven't found anything about it.

[icyJoseph]

I'd try, carefully, to add getInitialProps to _app, because that disables all optimisations and should, as far as I understand, disable any page data collection. Would be nice to see if that sorts the issue out. I mean carefully as in, have it as a noop:

function getInitialProps() {
    return {};  
}

CustomApp.getInitialProps = getInitialProps;

[neongreen]

I already have a non-noop getInitialProps there:

MyApp.getInitialProps = async (appContext: AppContext) => {
  let session: Session | null | undefined = undefined
  // getSession works both server-side and client-side but we want to avoid any calls to /api/auth/session
  // on page navigation, so we only call it server-side.
  if (typeof window === 'undefined')
    session = await getSession(appContext.ctx)
  const appProps = await App.getInitialProps(appContext)
  return { ...appProps, ...((session !== undefined) ? { session } : {}) }
}

export default MyApp

This should have had the same effect, right?

[ovidiuc]

@neongreen Are you by any chance applying something like this https://www.prisma.io/docs/support/help-articles/nextjs-prisma-client-dev-practices to instantiate Prisma and avoid the warn(prisma-client) There are already 10 instances of Prisma Client actively running. ? Had the same problem as your and I had to revert that method of instantiation to get rid of PrismaClientInitializationError error.

[neongreen]

@ovidiuc Yeah, I am using something similar to that method.

Have you found a way to instantiate Prisma properly while also avoiding the "10 instances" warning?

[eran-pinhas]

Same here, Executing api handlers and pages with getStaticProps on build time doesn't make much sense... Eventually I had to add some mock data for the build environment to get the build to pass

[FrancescoSaverioZuppichini]

100%

[paralin]

Can an option be added to skip this step?

[marcusstenbeck]

SKIP 👏 THE 👏 STEP 👏 SKIP 👏 THE 👏 STEP 👏

[fredrikburmester]

How do we skip this step, PLEASE!?

[alexey-sh]

@fredrikburmester yarn remove @types/react @types/react-dom next react react-dom

[ovidiuc]

@ovidiuc Yeah, I am using something similar to that method.

Have you found a way to instantiate Prisma properly while also avoiding the "10 instances" warning?

@neongreen
I'm replying outside of that thread to have my findings a bit more visible, Github interface collapses the newest discussion items.

My setup might not apply to you, I'm running next build from a Dockerfile that is executed by docker build within a Github Action runner.

The reason I was getting the DATABASE_URL was because I was setting the environment variable in my Github workflow file for the environment in which docker build was being ran, not for the docker build process itself. Found 2 solutions for this problem.

• Dynamically create the .env file in the github workflow (https://stackoverflow.com/a/63350136/4724998):

- name: Create env file
        run: |
          touch .env
          echo 'DATABASE_URL="${{ secrets.PRISMA_DATABASE_URL }}"' >> .env

I prefer this solution.

• or pass the the value as --build-arg :

DOCKER_BUILDKIT=1 docker build --build-arg DATABASE_URL=${{ secrets.PRISMA_DATABASE_URL }} -f Dockerfile.production .

and use it in Dockerfile to set the ENV:

ARG DATABASE_URL
ENV DATABASE_URL=$DATABASE_URL

For the "10 instances" warning, I've reverted back to the solution in the article I've mentioned, since I've fixed the environment variable issue:

// client.ts
import { PrismaClient } from '@prisma/client'

declare global {
  // allow global `var` declarations
  // eslint-disable-next-line no-var
  var prisma: PrismaClient | undefined
}

export const prisma =
  global.prisma ||
  new PrismaClient({
    log: ['query'],
  })

if (process.env.NODE_ENV !== 'production') global.prisma = prisma

[icyJoseph]

I hate to be repetitive, but that's just how modules work... this is not a Next.js issue.

Consider this CRA example: https://codesandbox.io/s/eloquent-grass-q7m9kq?file=/src/moduleA.js

Importing from moduleA causes the logs to happen, just like it makes it so that the 2+2 evaluates to 4.

I am not sure why is this so hard to understand, the top level of a module, is in a way, like the constructor of a class, as soon as you invoke the module, the top level code runs.

This is also observed in tests. If you import from testA into testB (a constant or helper), and run testB, then the entire testA suite runs too! Independently of the test runner!

Even simpler, consider these two files moduleA.js and index.js

// moduleA.js
const value = 2 + 2;

console.log("Hi from Module A");

module.exports = {value};

and

// index.js
const {value} = require("./moduleA");

console.log("Value:", value);

And guess what, running node index.js causes the console.log, from moduleA to print!

➜ node index.js
Hi from Module A
Value: 4

Does it start to make sense? It is important to get these concepts right!

[neongreen]

as soon as you invoke the module, the top level code runs.

My question is "why does Next run my code at all during the build".

[neongreen]

I hate to be repetitive, but that's just how modules work... this is not a Next.js issue.

Again, for full clarity: I understand that when you import from module A, top-level statements in module A will be executed. I'm not interested in changing my code so that it stops failing when executed. I want Next to not run any of my code in the first place.

[tim-trewartha]

@icyJoseph Thanks for the explanation, I felt like I was going crazy thinking getServerSideProps was running at build time. I am still unsure like @neongreen suggests if next should be running my code at all, but I suppose it is what it is. I fixed my issue with the solution from @tcharlat 🙌

[neongreen]

I suppose Next might be importing modules to figure out, for example, what they export..? Like if it exports getServerSideProps then treat this page in one way, otherwise in another way, etc.

In which case it makes sense that top-level stuff is run (it’s JavaScript’s fault that modules can’t be statically analyzed easily), but it doesn’t mean the issue is unfixable.

[tcharlat]

I have the same recurring issue that at build time, next is trying to render and start a lot of page lifecycles.
From _app.ts to pages with getServerSideProps, we have a lot of convolution to implement to avoid synchronous import of server-side logic setup kicking in.

Things like getting data from process.env to startup cache generation, render time fetch data, singleton initialisation, etc are triggering during build time and in our CI environment (the client env to be precise) it's not possible to let all this code execute when the only objective at this step is build a server ready to boot.

Given the nature of the application, we don't have page that can be statically rendered, we just need the server side rendering and chunk management from the nextjs solution, and this feature is causing a lot of failed builds and rewrites of imports to async as a workaround.

example :

const getServerSideProps = 
  async ({ req, params: { documentId, documentType, siteId } = {} }) => {
    const { delegation } = await import("@/services/delegation");
    const {
      getVatCat,
      getTermTimes,
      getIncotermCode,
      getIncotermPlace,
      getPaymentMode,
      getCountry,
      getTransportMode,
    } = await import("@/services/referentials");
    const { locale } = getLocaleFromRequest(req);

export function envThrow(key: string): string {
  if (process.env.hasOwnProperty(key)) {
    return process.env[key] as string;
  } else if (process.env.NEXT_PHASE !== "phase-production-build") {
    throw new Error(`missing critical environment variable : ${key}`);
  } else {
    return "0";
  }
}

One thing that is executed and that should maybe not, is getInitialProps of _app.ts.
Our most recurring problem is that it's executed and the weirdest part is the flakyness of the behaviour, as when build fail (1/5 roughly) it's generally passing at the next try.

Would anyone want that I try to setup a repo to illustrate the issue or it's not useful to take that time ?

[chrischandler25]

I'm having a similar issue to this. I work on an application to which NextJS doesn't seem at all well suited. There is no place for SSR in this app at all. I would like to remove NextJS, however for the immediate future this isn't a practical option.

In order to update react and other packages, I was forced to update from NextJS 8 to NextJS 10 (going further looked to be a big headache due to other dependencies). This appears to have completely changed the build process, which I think is now trying to build a static HTML file for each page. I now get a few "Collecting page data" output lines followed by an error (copied below) that gives me almost no detail. I've looked at the build JS file in question and to trace it back to any source code would be a big challenge. Ideally, I'd just revert the build process back to what it used to be which I don't think attempted any HTML generation.

Any advice on this would be hugely appreciated!

---

info - Collecting page data ...node:internal/process/promises:279
triggerUncaughtException(err, true /* fromPromise */);
^

TypeError: Cannot read properties of undefined (reading 'N5')
at Object.N5 (C:\dev\alvitas\UI.next\server\chunks\966.js:794:80)
at Object.1411 (C:\dev\alvitas\UI.next\server\chunks\966.js:5363:51)
at webpack_require (C:\dev\alvitas\UI.next\server\webpack-runtime.js:25:42)
at Object.6772 (C:\dev\alvitas\UI.next\server\chunks\966.js:7107:14)
at webpack_require (C:\dev\alvitas\UI.next\server\webpack-runtime.js:25:42)
at Object.8291 (C:\dev\alvitas\UI.next\server\chunks\966.js:9567:64)
at webpack_require (C:\dev\alvitas\UI.next\server\webpack-runtime.js:25:42)
at Object.2914 (C:\dev\alvitas\UI.next\server\chunks\966.js:8290:73)
at webpack_require (C:\dev\alvitas\UI.next\server\webpack-runtime.js:25:42)
at Object.5966 (C:\dev\alvitas\UI.next\server\chunks\966.js:8778:15) {
type: 'TypeError'
}

[mmahalwy]

Im running into a similar problem with using globalThis.File which becomes undefined

[mschipperheyn]

This stuff is extremely problematic during build scenarios because it assumes a working server with environment setup. I understand the value. But it should catch any errors during this process and warn instead of throw them.

[shredding]

I want to second this. It is the 2nd worst architectonic decision i've seen in my entire life. Rolling your own env parser by the same framework is on the 1st place.

[F1r3Hydr4nt]

Yes, could we perhaps be provided with a flag to disable this "prerendering" test during collecting page data which make use of getServerSideProps? Or demote the errors to warnings as mentioned above?

[jakekugel]

Here is an easy way to reproduce this using the with-mongodb example. This example doesn't use getStaticProps() anywhere:

1. npx create-next-app --example with-mongodb myapp
2. cd myapp
3. npm run build

This fails with:

- info Collecting page data ..Error: Invalid/Missing environment variable: "MONGODB_URI"

@icyJoseph it sounds like you wanted to see a reproducible example, would this give you what you need?

[rtritto]

Thanks, I fixed!

[01febAkanshaMittal]

@rtritto if possible can you please share the solution for ✓ Compiled successfully
#16 59.38 Collecting page data ...
#16 60.27 ReferenceError: document is not defined and Build error occurred
#16 60.28 Error: Failed to collect page data for /RichTextEditor
#16 60.28 at /app/node_modules/next/dist/build/utils.js:1171:15
#16 60.28 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
#16 60.28 type: 'Error'
#16 60.28 }
#16 60.36 error Command failed with exit code 1.

[rtritto]

I don't know, my problem was that the creation of mongodb connection was created when file is imported.
I fixed creating a function to create the mongodb connection at runtime.

[ianfebi01]

I don't know, my problem was that the creation of mongodb connection was created when file is imported.
I fixed creating a function to create the mongodb connection at runtime.

Can you share the function code?

[icyJoseph]

I don't know, my problem was that the creation of mongodb connection was created when file is imported.
I fixed creating a function to create the mongodb connection at runtime.

Can you share the function code?

#35534 (reply in thread)

[prashantchothani]

In Nextjs 13 on api under app directory (Route Handlers) also it fails with the same issue :

Error: Failed to collect page data for /api/#####/#####

[FESTUSNIX]

Any success fixing it?

[prashantchothani]

Yes

[johncalvinroberts]

How did you end up fixing it? @prashantchothani

[prashantchothani]

In the route, whatever method is used, GET or POST, it should fulfill the request. What I realised that one of them wasn't there and returned nothing. Don't keep any method to not return anything.

[johnpolacek]

I have an api route that I am setting up for a cron job. It is a post that requires a env var to be passed so that I can prevent it from running via publicly accessible url (need to know the secret). However, it fails on build because of this due to it throwing an error as it does not pass anything in the post request, let alone the secret.

Error: Failed to collect page data for /api/generate

I can change it to return a different response that isn't an error, but that seems dumb. I should be able to bypass this for an api route.

[mengxi-ream]

I met this problem too. When I have an edge function /api/generate, then when I build with this config

/** @type {import('next').NextConfig} */
const nextConfig = {
  output: "export",
};

module.exports = nextConfig;

I met the Error: Failed to collect page data for /api/generates

But if I remove output: "export", it can build correctly. But I want output: "export". Anyone know how to solve this?

[trautonen]

For now only workaround is to remove all module level process.env access and add export const dynamic = 'force-dynamic' to your routes. Not sure why @zoominGiladSher couldn't make this work, but at least I managed to get my api routes to access runtime only env vars this way.

In the build output it should show the λ symbol for dynamic routes which are not used in static generation.

[trautonen]

But as stated earlier, this is clearly a bug in the app router as these routes are not used as static even though next requires them to be static in build stage.

[mengxi-ream]

For me, even I delete the env and add export const dynamic = 'force-dynamic', it still got this error. But if I remove output: "export",, I can use env in my code. This

import { NextResponse, NextRequest } from "next/server";

export const dynamic = "force-dynamic"; // no caching

// export const runtime = "edge"; // 'nodejs' is the default
// const domain = process.env.PUBLIC_URL || process.env.LOCAL_URL;

export async function POST(req: NextRequest) {
  try {
    // for test, so I just wrote this simple return
    return new Response("Hello world", { status: 200 });
  } catch (error) {
    return error;
  }
}

If I turn on output: "export", I got Error: export const dynamic = "force-dynamic" on page "/api/diaries/generateDiary" cannot be used with "output: export".. Seems another issue with edge function and output: "export"

[trautonen]

Right. This could be that for some reason standalone and export also works differently. I'm building for standalone output.

[balsoft]

(I'd like to preface this with: I don't know what I'm doing, I'm not a frontend dev, I'm just trying to package an app built with next-js)

I think this disables the pre-rendering of pages:

next build --build-mode=experimental-compile

I'm yet not sure if the resulting app will work 100% correctly.

[noahgsolomon]

this worked! No clue where u found this but good job

[noahgsolomon]

INSHALLAH

[nick-kang]

next build --experimental-build-mode=compile works for me on [email protected]

[Abukodonosor]

And whole app stop working ... :(

[FrancescoSaverioZuppichini]

so how do I make it work with a route that have env vars at the top?

const myVar = process.env. ...

if (!myVar) {
  throw new Error("Must provide the env var");
}

export async function GET() {

}

Thanks

[trautonen]

You don't. 😞 Only way I figured how to do it is to load the env vars in the route function, GET() in this case. I have a function called staticConfig() that I use wherever I need the env vars. Seems ridiculous but couldn't figure any workaround.

[flodev]

this is very weird. it says in the manual that those env variables don't have to be there during build time. only NEXT_PUBLIC_ prefixed variables.
we have similar environment variable checks in our project and now we have to make our code unnecessary complicated and probably unperformant just because nextjs needs to "collect page data"

[is-it-ayush]

Just ran into this. In my situation, the place where the database is supposed to be does not really exist during the build process while the export const db = ... from src/db/db.ts is hard-wired to point to this non-existent (only during compile time not runtime) location. The entire application has dynamic pages and I still see Collecting page data... during the build process which I do not understand why! (even after reading the above responses 🙂; let me know if you do!) This ultimately crashes on Error: Failed to collect page data for /api/xyz.

A somewhat okay solution to this problem could've been next warning us that the page data does not exist & default that page to being dynamically served. Now just to workaround this, I'd have to create a mock db which is a bummer. 🙁

[trautonen]

But also on top of that bug you still need to do the dynamic import which is really annoying and potentially slow.

[is-it-ayush]

I think there's a bug in app router which thinks api routes are static

Interesting @trautonen! I'm leaning towards this. It'd be very nice if we could somehow print more verbose logging for next build to actually know what was causing this issue but I checked and there doesn't seem to exist a --verbose or --debug flag on the next cli.

I think I've managed to resolve such problem by setting export const dynamic = 'force-dynamic'

This would work as a workaround for the meantime but if it's a bug then I'd love to know more of what's the actual fix.

[icyJoseph]

I think there's a bug in app router which thinks api routes are static. I think I've managed to resolve such problem by setting
export const dynamic = 'force-dynamic' explicitly in every api route. I think that problem does not exist with pages.

@trautonen it's a feature... GET endpoints, given some conditions are automatically opted into static optimizations.

• https://nextjs.org/docs/app/building-your-application/routing/route-handlers#opting-out-of-caching

[icyJoseph]

Thank you @icyJoseph! I'm not sure "which" page ends up invoking the database since the /api/#### routes returned by the build error seem to oscillate between three specific /api/ routes (where all three throw the same db error). None of these three routes are called a single time across the entire application.

Are you using API route access while building the application? Like your server components try to reach for API handlers? That's not gonna fly.

• https://nextjs-faq.com/fetch-api-in-rsc

[is-it-ayush]

Are you using API route access while building the application? Like your server components try to reach for API handlers? That's not gonna fly.

No @icyJoseph! The three "route handlers" thrown by the build error are not called across the entire application. I double checked it. These route handlers are not supposed to be called in app's lifetime as they're external facing & not used by the application itself.

[iyxan23]

Although this is a pretty old discussion, I wanted to share how I addressed this issue:

I have a file that exports a Minio.Client like this:

import * as Minio from "minio";
import { env } from "@/env";

export const s3 = new Minio.Client({
  endPoint: env.S3_ENDPOINT,
  port: env.S3_PORT,
  useSSL: env.S3_USE_SSL,
  accessKey: env.S3_ACCESS_KEY,
  secretKey: env.S3_SECRET_KEY,
});

export const BUCKET_NAME = env.S3_BUCKET_NAME;

I have this file imported in several places; this works well using next dev. But it started to throw errors that the env variables aren't present during build.

What I did to fix this was removing every import { s3 } from "@/path/to/s3"; statements and replacing them with "dynamic imports": await import('@/path/to/s3') inside the function calls.

From what I could tell, node itself evaluates any top-level definitions of files when they are imported statically using the import ...; syntax in "NextJS entrypoint" files like route.ts, page.tsx, etc. And to mitigate that, we must import or instantiate them during runtime (which is inside the actual function calls)?

Am I in the right here? Would love for some clarification.

[icyJoseph]

From what I could tell, NextJS evaluates any top-level definitions of files when they are imported statically using the import ...; syntax

You mean, Node.js does that.

[iyxan23]

From what I could tell, NextJS evaluates any top-level definitions of files when they are imported statically using the import ...; syntax

You mean, Node.js does that.

Ah, that's true! Thanks for pointing that out 😄

[CarlosZiegler]

Got some error after updating to next 15 with output: "standalone" :(
Collecting page data ...ReferenceError: db is not defined

[fredrikburmester]

I get the same error.

[Abukodonosor]

[Abukodonosor]

import { supportFormAction } from '@/_actions/supportPage/supportFormAction';
import getTenantConfig from '@/lib/tenantConfig/tenantConfig';
import SupportPage from '@/page/support/support.page';

// There is also async function for generating the metadata

export async function generateMetadata() {
	return {
		title: 'xxxx',
		description: 'yyyy'
	};
}

export default async function SupportRoute() {
	return <SupportPage getTenant={getTenantConfig} supportForm={supportFormAction} />;
}

getTenantConfig uses

export default async function getTenantConfig(type: TenantType = TenantOrigin.Client): TenantConfigContext {
	const { serverRuntimeConfig, publicRuntimeConfig } = getConfig();
	const h = await headers();

and

const SupportPage = hocServerWrap(SupportPageController, SupportPageView);

export default SupportPage;

[fredrikburmester]

I still have this issue, please remove the Collecting page data during build...

[icyJoseph]

Are all of those dynamic routes? I guess yes?

[fredrikburmester]

Yes

[fredrikburmester]

Would you maybe want to continue this over email (or something) because I feel like this issue is getting bloated?

[icyJoseph]

I wonder if you could print the stack trace before those throw error statements.

What version of Next.js are you using?

I think you could try to contact me through LinkedIn. I'm on pappaledig though, so I am on and off from my phone.

[Abukodonosor]

‘’’
export var driver: Driver | null = null;
‘’’

Make the singleton instance of it.
I would also try accesing the env varibales trough next.config.ts

[nemanjam]

I shouldn't be required to connect to database at build time for pages that read database at request time.

[alexey-sh]

@icyJoseph yes it is the problem. Which is how nextjs works.

When a page is loaded the nextjs is trying to render it during build time. Render obviously requires connection to production database or so.

[nemanjam]

It's not, the problem is how modules are loaded. Which is how JavaScript works.

When the module is loaded the top level scope is executed. If you have statements/declarations that can't be worked during build time, or runtime for that matter, the module loading will crash.

Good point. So if I remove database instance from global scope and wrap it in factory function I should be able to build app without database connection, right?

[icyJoseph]

Yes, though, I know of one instance, where the codebase still had issues, but I couldn't get access to the repository to see it with my own eyes

[nemanjam]

I can confirm that app builds successfully without database connection when I remove database instance from global scope and wrap it with factory function, singleton.

[Abukodonosor]

As i mentiin above about the singleton.
I am glad that you solove the problem.

Bravo, cestitam!

[jalik]

I got the same error because I used to check required environment variables during runtime, like DATABASE_URL.
In a classic application it is very useful because the error triggers when the app is started and not later during an API call for example, so you cannot miss it.
In a server-less app, it is less useful as there is no app "starting".
I ended up removing the checks, so the build passes but I am not satisfied...