Uploads/media returns 404 in production, but works in dev

[thefrana]

Hi,
I have very similar project like this one: https://github.com/payloadcms/remix-server

However, all uploads like /media/file.txt returns 404 not found in production. In development, they are returned successfully. Any idea why?

// server.ts - glues together Remix and Payload
import { createRequestHandler } from "@remix-run/express";
import { installGlobals } from "@remix-run/node";
import compression from "compression";
import dotenv from "dotenv";
import express from "express";
import fs from "fs";
import helmet from "helmet";
import morgan from "morgan";
import path from "path";
import payload from "payload";
import type { PayloadRequest } from "payload/types";
import type { RemixRequestContext } from "remix.env";

export type GetLoadContextFunction = (
  req: PayloadRequest,
  res: Express.Response
) => RemixRequestContext;

export type RequestHandler = (
  req: Express.Request,
  res: Express.Response,
  next: any
) => Promise<void>;

installGlobals();

// Loading environment variables, .env > .env.local
const config = dotenv.config();

if (config.error) {
  throw config.error;
}

const localEnvFilePath = path.resolve(process.cwd(), ".env.local");
if (fs.existsSync(localEnvFilePath)) {
  const localConfig = dotenv.config({
    path: localEnvFilePath,
    override: true,
  });
  if (localConfig.error) {
    throw localConfig.error;
  }
}

// Prepare environment variables
const MONGODB_URL = process.env.MONGODB_URL ?? "";
const PAYLOAD_SECRET = process.env.PAYLOAD_SECRET ?? "";
const ENVIRONMENT = process.env.NODE_ENV;

// During development this is fine. Conditionalize this for production as needed.
const BUILD_DIR = path.join(process.cwd(), "./dist/web");

const app = express();

// https://expressjs.com/en/advanced/best-practice-security.html#use-helmet
app.use(helmet({ contentSecurityPolicy: false }));

// https://expressjs.com/en/advanced/best-practice-security.html#reduce-fingerprinting
app.disable("x-powered-by");

// http://expressjs.com/en/advanced/best-practice-performance.html#use-gzip-compression
app.use(compression());

// https://expressjs.com/en/advanced/best-practice-performance.html#do-logging-correctly
app.use(morgan("tiny"));

// Remix fingerprints its assets so we can cache forever.
app.use(
  "/build",
  express.static("public/build", { immutable: true, maxAge: "1y" })
);

// Everything else (like favicon.ico) is cached for an hour. You may want to be
// more aggressive with this caching.
app.use(express.static("public", { maxAge: "1h" }));

// Initialize Payload CMS
payload
  .init({
    express: app,
    mongoURL: MONGODB_URL,
    secret: PAYLOAD_SECRET,
    onInit: () => {
      payload.logger.info(`Payload Admin URL: ${payload.getAdminURL()}`);
    },
  })
  .then(() => {
    app.use(payload.authenticate);

    app.all(
      "*",
      ENVIRONMENT === "development"
        ? (req, res, next) => {
            purgeRequireCache();

            return createRequestHandler({
              build: require(BUILD_DIR),
              mode: ENVIRONMENT,
              // TODO: This is not the best way to do this, but it works for now.
              // @ts-expect-error somehow this is not assignable to the RemixRequestContext
              getLoadContext(req, res) {
                return {
                  payload: req.payload,
                  user: req?.user,
                  res,
                };
              },
            })(req, res, next);
          }
        : createRequestHandler({
            build: require(BUILD_DIR),
            mode: ENVIRONMENT,
            // TODO: This is not the best way to do this, but it works for now.
            // @ts-expect-error somehow this is not assignable to the RemixRequestContext
            getLoadContext(req, res) {
              return {
                payload: req.payload,
                user: req?.user,
                res,
              };
            },
          })
    );

    const port = Number(process.env.PORT) || 3000;
    const hostname = process.env.HOST || "localhost";

    app.listen(port, hostname, () => {
      console.log(`Server listening on http://${hostname}:${port}`);
    });
  });

function purgeRequireCache() {
  // Purge require cache on requests for "server side HMR" this won't let
  // you have in-memory objects between requests in development,
  // alternatively you can set up nodemon/pm2-dev to restart the server on
  // file changes, but then you'll have to reconnect to databases/etc on each
  // change. We prefer the DX of this, so we've included it for you by default

  for (const key in require.cache) {
    if (key.startsWith(BUILD_DIR)) {
      delete require.cache[key];
    }
  }
}

[DanRibbens]

I'm not too familiar with Remix, but I'll give it a try anyways. I'm not seeing anything wrong in what you've shared.

In production I would see what the failing image requests look like. Is it all assets or just uploaded media?
Where are your images stored and what are you using for hosting? I'd like to rule out that you are not storing uploaded files to a host that uses ephemeral storage.

[thefrana]

Thank you for your answer. I am storing uploads in /files/media. Other assets are working correctly. Only uploads in production are not being served. Everything that goes to URL /media/file.txt returns 404. The response is handled by Remix, which is weird, because I would thought that Payload should handle them.

I can create new express.static that serves from /files/media. However, I would lost access capabilities of Payload and expose everything as public.

For the hosting, I am using standard Hetzner VM based on Ubuntu. It uses persistent storage.

[thefrana]

It somehow vanished. Maybe with some update.