Custom Next.js Server Improvements

[cbratschi]

We got the custom server example (https://github.com/payloadcms/payload/tree/master/examples/custom-server) running in our setup but there are some usability issues which should be addressed.

Using "npm run dev" in a normal Next.js project is quite different to the integration used by the custom server example. For instance all console messages from Next.js are gone and editing files in src/app does not reload pages in the browser.

Next.js default implementation uses:

• https://github.com/vercel/next.js/tree/canary/packages/next/src/bin
• https://github.com/vercel/next.js/blob/canary/packages/next/src/lib/commands.ts
• https://github.com/vercel/next.js/blob/canary/packages/next/src/cli/next-dev.ts

Whereas the next() call uses:

• https://github.com/vercel/next.js/blob/2cf5d3a8aa325f8bece1094c9e566311e604e114/packages/next/src/server/lib/router-server.ts#L119

Which launches a new worker. I tried to modify some of the parameters but this did not improve anything.

Ideally I would like to keep all features from Next.js for development. I am wondering if there is a better way to integrate Payload using for instance Next.js' rewrite capabilities. Payload could run on a different port but still get the full server integration.

[jacobsfletch]

Hey @cbratschi, yup the HMR issue on the Custom Server Example is a known issue, check out my comment here: vercel/next.js#50461 (comment) (feel free to upvote 🙂). As soon as we can find a fix for this, we'll get it patched up. The Next.js Pages Router works as expected, though. If this is a pressing issue for you, then you could run them using different servers. Look to the the Auth Example for good reference of this.

[cbratschi]

Hi @jacobsfletch thanks, I upvoted the HRM issue.

I am seeing requests like /admin/main.030459b349f3d3f818b6.hot-update.json which are not handled by the Payload handler and get passed to Next.js. The referer is http://localhost:3000/admin/collections/pages/64d4aaa79674695644febb8c. This might be a Webpack request.

I am mainly missing the Next.js console output. Serving routes works fine. Next.js documentation states that some features are not available with custom servers. The documentation is not very clear about this.

My idea was to use Next.js as a router to the Payload Express server. For instance Next.js would use port 3000 and Payload port 3001. Next.js' config would redirect /admin, /cms (Payload APIs) and /media to the Payload port. Not yet sure if this is doable with Next.js' rewrite capabilities. Such a setup would minimize the Express routing overhead and enable all Next.js features. This could also be done with NGINX but this would be more difficult to setup on every developer machine.

[cbratschi]

I got my setup now running:

• Next.js runs on port 3000
• Payload runs on port 3001

This setup is more flexible because Payload can be moved to any server if needed and can be maintained separately. It supports all Next.js features without any restrictions. Instead of Next.js, NGINX could be used as reverse proxy for Payload.

next.config.js:

    /**
     * Change rewrite rules.
     *
     * See https://nextjs.org/docs/pages/api-reference/next-config-js/rewrites.
     *
     * Set environment variables:
     *
     *   - PAYLOAD_PATHS paths used by Payload e.g. /admin,/cms,/media
     *   - PAYLOAD_HOST host, usually localhost
     *   - PAYLOAD_PORT port (e.g. 3001)
     *
     * @returns
     */
    async rewrites() {
        if (!process.env.PAYLOAD_PATHS || !process.env.PAYLOAD_HOST || !process.env.PAYLOAD_PORT) {
            return {};
        }

        const paths = process.env.PAYLOAD_PATHS.split(',').map(path => path.trim());
        const rewrites = paths.map(path => {
            return {
                source: `${path}/:path*`,
                destination: `http://${process.env.PAYLOAD_HOST}:${process.env.PAYLOAD_PORT}/${path}/:path*`,
            };
        });

        //debug
        //console.dir(rewrites);

        return {
            beforeFiles: rewrites
        };
    }

.env example:

PAYLOAD_PATHS=/admin,/cms,/media

The Payload server optionally launches Next.js in dev mode and shows all process output:

/**
 * Payload server launcher.
 */

import { start } from './server-payload';
import { exec } from 'node:child_process';
import path from 'node:path';

start().catch(err => {
    console.error('Could not start server: ' + err.message);
    process.exit(1);
}).then(async () => {
    //optionally run Next.js (dev mode only)
    if (process.env.NODE_ENV !== 'production' && process.env.PAYLOAD_START_NEXTJS) {
        const prog = await exec('npm run next:dev', {
            cwd: path.join(__dirname, '../..')
        });

        //output
        prog.stdout?.on('data', data => {
            console.log(data);
        });

        //displays errors
        prog.stderr?.on('data', data => {
            console.error(data);
        });
    }
});

[cbratschi]

One big issue is still open: the SEO plugin does not work with Payload. It fails before Payload.init(). See payloadcms/plugin-seo#40 (comment). Looks like a problem with one of the plugin's imports.