Add a dataPathPrefix config option

[petewarman]

Describe the feature you'd like to request

I am currently not able to upgrade beyond 9.5.0 because assetPrefix no longer applies to _next/data requests (#15634).

A bit of context - my organisation splits traffic between multiple applications for various parts of our site. Incoming requests are routed to relevant applications by matching certain url patterns. The application I work on handles three different base paths - for the sake of simplicity let's call them /a, /b and /c, and we are currently using the assetPrefix config to add a path prefix to effectively add a namespace all our static assets and data requests (let's call that /app-prefix). So the routing logic that sits in front of my application can simply pass all requests that begin with /a, /b, /c or /app-prefix through to my application.

As of 9.5.1 the assetPrefix no longer gets applied to the data requests so those requests are not sent to our application and now return 404s instead.

We aren't able to use the newer basePath config option as that would also transform the /a, /b, /c paths to be /app-prefix/a, /app-prefix/b, /app-prefix/c. Which is not what we want.

Describe the solution you'd like

I'd like to propose a new config option (something like dataPathPrefix) to allow namespacing of _next/data requests from getServerSideProps calls.

Describe alternatives you've considered

The alternative would be to revert the changes introduced to assetPrefix in 9.5.1 but presumably that would break things for other people.

[petewarman]

@timneutkens thanks for converting this to a discussion and apologies if I got the process wrong here. It would be helpful if someone could give me some feedback so I know if the description above makes sense, or if I need to try and clarify it further? Also what is the best way of getting the ball rolling on this? Should I go ahead and have a stab at creating a pull request? Or wait for some feedback on the idea first?

[timneutkens]

Also what is the best way of getting the ball rolling on this? Should I go ahead and have a stab at creating a pull request?

We don't add features for one-off cases currently, so it's best to wait and see if there's interest from other people.

[borantula]

I don't believe it's a one-off case and it can even help to make the workarounds to the basePath after locale issue easier. (details in this RFC comments or quite some discussion topics around same issue)

[michaelschufi]

@timneutkens I don't think this is a one-off case. We and many other people in discussions have problems with the assumptions Next.js makes regarding routing. Especially if you have a slightly more complex setup that uses proxies etc.

[meotimdihia]

@timneutkens many people are upvoting the feature. I need it for getStaticsProps.

[brandonhuang]

Just wanted to voice my support here as well. Our organization has the exact same routing strategy that @petewarman described and we're currently running into the same issue where our requests to _next/data from getServerSideProps 404 with no easy workaround as we're unable to downgrade Next.js for internal reasons.

We've currently disabled client-side navigation due to this issue, losing performance optimizations that we'd get from prefetching. We've considered solving this problem by using getInitialProps over getServersideProps to avoid the additional call to _next/data , but it'll be a heavy refactor for us and it seems like the Next.js community is moving towards getServersideProps.

The additional configuration variable dataPathPrefix proposed above would seamlessly solve this problem for us. Happy to help out where possible @timneutkens.

[nsmith7989]

Piling on as well. This is definitely a feature that would help my team. From my digging there are a number of open discussions on this issue (including a solution I asked for).

#22404
#25883

I don't personally have an opinion about which would be best as they would all work for my use case.

@timneutkens I'm happy to contribute a PR or contribute in any other way to help out here. Which solution would be best?

[petewarman]

Those options work for me too - I just need a mechanism to identify which requests should be routed to my application

[PsyGik]

Upvoted!

My use-case is also similar where we have an external public URL, https://app.external and use Fastly to route requests to the appropriate server based on the path.

https://product.company.external/v2/custom/foobar => http://app.internal/product/custom/foobar

The routing was possible in Fastly with a custom VCL:

# redirect v2/custom/foobar => http://app.internal/product/custom/foobar
if (req.url.path ~ "^/v2/custom(/.*)?$") {
    call product_redirect;
    return(lookup);
  }

# Redirect Next.JS assets  
if (req.url ~ "^/_next(/?|/(.*))$") {
    call product_redirect;
    return(lookup);
  }

If you observe the second condition, we redirect _next/ and all subsequent calls to the internal server, but it doesn't work for _next/data because the url was _next/data/:buildId/v2/custom/${params}/*.json

So in this case, had to map /v2/custom/:params to /:product/custom/:params, where product and params are dynamic parameters for the Next.js application.

I have been able to solve this (re: workaround) with a custom server. I inject the product value to the request header using Fastly VCL before sending the request to Next.js.

server.get('/_next/data/:buildId/v2/custom/:path*', (req, res) => {
    const parsedUrl = parse(req.url, true);
    // => incoming url: /_next/data/:buildId/v2/custom/:path
    parsedUrl.pathname = parsedUrl.pathname.replace(
      'v2',
      req.headers['x-product']
    );
     // => outgoing url: /_next/data/:buildId/:product/custom/:path
    handle(req, res, parsedUrl);
  });
  // Let Next.js handle rest of the routes
  server.all('*', handle);

[boschni]

We are also running into this problem. It does not seem possible to run multiple Next.js applications on the same domain when the urls include a locale like /en/app-1 and /en/app-2 because both applications are using the same /_next prefix.

Related issues:

#22404
#21022
#21878
#25883
#25368
#26068
#25681
#22433
#22561

[yuuk]

Running into this problem too...

[borantula]

this is an issue for us as well at the moment. Our services are grouped under paths per service /service-a /service-b and also localised /en/service-a /fr/service-b and due to i18n structure of {locale}/{basePath} we cannot use NextJS basePath config. And to support /_next/data/development/en/service-a/page-b/123123.json structures we needed to add nginx locations configs matching this structure like /_next/data/development/[a-z-]*/service-a, which could have been solved better in the app level with {locale}/{basePath} support and dataPathPrefix

[petewarman]

For anyone else struggling with this problem - I came across this article which describes using a custom buildId to identify which calls to route to a given application https://arihantverma.com/posts/2021/07/18/workaround-nextjs-data-urls-same-domain-multiple-apps/

It's is a bit of a hacky work around though - nextjs needs a better solution to this problem.

[pranavmshinde]

I tried the above solution but its not working for me. can you help me out?

[Veightor]

Also running into issues with the exact same potential solution.

[sherifabdlnaby]

We're also affected by this and it is hindering our NextJS Adoption.

[lizhancheng]

expected to add this config

[alizeait]

Next.js uses the client side Router to get the _next/data/* data, namely the getDataHref method. Knowing this, we can monkey patch it to allow adding a namespace prefix to it, so it becomes /:namespace/_next/data/* instead of /_next/data/*, and then add a rewrite rule that rewrites traffic from /:namespace/_next/data/* to the root of the domain.

const useInterceptNextDataHref = ({
  router,
  namespace,
}: {
  router: Router;
  namespace: string;
}) => {
  useEffect(() => {
    if (router.pageLoader?.getDataHref) {
      const originalGetDataHref = router.pageLoader.getDataHref;
      router.pageLoader.getDataHref = function (...args: any[]) {
        const r = originalGetDataHref.call(router.pageLoader, ...args);
        return r && r.startsWith('/_next/data')
          ? `/${namespace}${r}`
          : r;
      };
    }
  }, [router, namespace]);
};

We can then use it inside of our _app for example:

import type { AppProps } from 'next/app'

function MyApp({ Component, pageProps, router }: AppProps) {
 useInterceptNextDataHref({
    router,
    namespace: 'my-app-namespace',
  });
  return <Component {...pageProps} />
}
export default MyApp

Then simply add a rewrite rule in next.config:

module.exports = {
  async rewrites() {
    return [
      {
       source: `/my-app-namespace/_next/data/:path*`,
       destination: '/_next/data/:path*'
      },
    ]
  },
}

[tkmcmaster]

Thank you. This workaround solves the issue for us.

[doptster]

seems not working in Next v12.1+

[haydnhkim]

@doptster I moved the code to beforeFiles, it works in Next v12.1+.

module.exports = {
  async rewrites() {
    return {
      beforeFiles: [
        {
          source: `/my-app-namespace/_next/data/:path*`,
          destination: '/_next/data/:path*'
         },
      ]
    }
  },
}

[vitorcosta039]

It worked for me, thanks! But I got an error in the console Error: Invariant: attempted to hard navigate to the same URL, has anyone had this same problem and managed to solve it?

[evanhameed99]

I have tried the above fix, it seems to be working fine expect for one thing. internal router navigation is causing pages to fully reload.
Any Idea how this can be fixed?

[tkmcmaster]

We're running into this issue as well. I don't know if adding a dataPathPrefix is the best solution. The issue is more that there isn't a way to add a prefix that applies to the whole server that doesn't move the route off the base on the local server. basePath changes the local server path, but it looks like there are multiple users that have our server routed to from a remote server based on a path. We've been using Deploy your NextJS Application on a different base path (i.e. not root)
for a while and the workarounds work until we tried to move to getServerSideProps from getInitialProps.

We almost need a change to basePath that lets us tell the server to do all the asPath changes and change route for all links, but doesn't actually change the local server to add the path. Something like:

module.exports = {
  basePath: {
    path: "/my-subpath",
    dontChangeLocalRoute: true
  }
}

where dontChangeLocalRoute?: boolean

[doptster]

Bump, spending much time in getStaticProps, turns out the basePath doesn't work well with assetPrefix.

[bobbyflowstate]

Just adding to the many voices here that would like something like a dataPathPrefix.

We're following this guide (provided by Vercel): https://vercel.com/guides/how-can-i-serve-multiple-projects-under-a-single-domain, but it's somewhat unusable because it doesn't work without the assetPrefix applying to all /_next/* paths. Unless we're missing something.

More Context

TL;DR: assetPrefix doesn't seem to be working with /_next/data requests. All /_next/data requests are trying to fetch data from https://official.com/_next/data/* when we expect it to fetch from https://webapp.vercel.app/_next/image* (because our assetPrefix = https://webapp.vercel.app).

The Setup

Rewrites on a separate domain
By following this guide exactly: https://vercel.com/guides/how-can-i-serve-multiple-projects-under-a-single-domain...

• We've deployed a project on vercel with just a single vercel.json file. That project deploys onto domain https://official.com
• That vercel.json file has several rewrite rules that point certain paths (e.g. /web/:match* to webapp.vercel.app/:match*)

assetPrefix
The above only works if we correctly set the assetPrefix value in our next.config.js file. Otherwise the rewrite project (official.com) will look for next static files under https://official.com/_next/* instead of https://webapp.vercel.app/_next/*.

So we've gone ahead and done that and it's working fine to fetch files under /_next/static . Unfortunately it's not working fine for /_next/data as many people here have stated.

We'll try the workaround for now that @alizeait has shared.

[Junting-Liu]

Hi @rajuashok , how did you handle the serve multiple project in a single domain finally ? I'm struggling with the same problem as you.

[owennicol]

Adding my support to this too 👍

[jontewks]

Same here as well, glad to have the workaround but was surprised to see it wasn't yet an option in the config.

[navdeep-singh-binmile]

Does anyone find a way to do this?

[leerob]

This is potentially solved by using basePath, depending on your use case, with the Pages Router.

It is fully solved when using the App Router (which did not exist when this was originally created), as _next/data is not used. Instead, it uses the same URL with a different header. This might work for some use cases as well, for folks looking to incrementally adopt the App Router.

[jontewks]

Agreed, same here. I was actually excited to see some activity on this since it seems such an important ask, only to find out it’s been closed without understanding the root issue we’ve had to hack around.

[leerob]

It’s worth mentioning that this issue is solved when using the App Router (which did not exist when this was originally created).

_next/data is not used. Instead, it uses the same URL with a different header. This might work for some use cases as well, for folks looking to incrementally adopt the App Router.

[rgrove]

@leerob That's great news! This will be a very welcome change once we're able to adopt the App Router.

This isn't a near-term solution for us, though. We're still waiting for some of the libraries we rely on to support the App Router before it will be feasible for us to begin adopting it incrementally, and even once we are able to begin, it won't be an immediate migration given the size of our app and our available resources and priorities.

In the meantime, reopening this discussion would at least make it easier for people to find the monkeypatch workaround, which is particularly valuable if (as I'm beginning to suspect) Next.js isn't interested in adding a dataPathPrefix option.

[leerob]

What specific libraries are you looking for support? Almost everything should work the same way as before, if you use client components, which isn't a bad thing and helpful for incremental adoption.

You can mark the entry point into your application (e.g. the page) as a client component with "use client", and then you don't need to add the directive to the children. Meaning all of those children components now function in a similar way to the Pages Router (prerendered on the server, hydrate on the client).

Related: reactwg/server-components#4

[rgrove]

Thank you. I do understand that client components are not a bad thing, and appreciate the educational resources you've been creating on this topic.

One example of something we rely on heavily is Apollo Client. Apollo currently offers an experimental package with App Router support, but there are caveats and issues that would make it a challenge to adopt right now. Even incremental adoption using client components and the experimental Apollo Client package would be challenging in our app given the complexity involved, so this isn't a trivial undertaking and will not be a fast migration even once we begin it.

We also need to be careful to remember that our job is to build a product for our customers, not to adopt new features of Next.js, so we must be pragmatic about where we invest our time.

But we've strayed pretty far from the topic of this discussion. Regardless of the existence of the App Router (which I look forward to adopting someday!), there are valid reasons why some Next.js apps will continue to need to use the Pages Router for at least a little while.

I hear you when you say that the App Router isn't affected by the data path problem being discussed here, and that there are ways people can adopt the App Router incrementally. However, the Pages Router is affected, and the Pages Router still exists, is still supported by Next.js, and is still used by many Next.js apps, some of which may not be able to immediately begin a migration to the App Router.

I don't entirely understand why you want to keep this discussion closed, but it seems like you feel strongly about it. Regardless, I hope we can agree that the data path workarounds discussed here may be valuable to users who have this use case and aren't able to adopt the App Router.

[mramadanahmed]

Here is my workaround to fix that, I hope it can help anyone facing the same issue.

Simply you can replace the path in the bundled javascript using webpack string-replace-loader loader as following:

const nextConfig = {
	assetPrefix: '/mypath',
	async rewrites() {
		return [{ source: '/mypath/_next/:path*', destination: '/_next/:path*' }];
	},
	webpack: (config) => {
		config.module.rules.push({
			test: /.*.js$/,
			loader: 'string-replace-loader',
			options: {
				search: '/_next/data',
				replace: '/mypath/_next/data',
			},
		});

		return config;
	},
};

[bhupendraptlk]

Thanks, I was facing an issue with the same assetPrefix not being part of _next/data files and this workaround worked for me. 🎉

[volos-sq]

Bump, the problem is still relevant.
I've solved it in the following way, hope it helps someone else.

1. Add a static prefix to the buildId:

// next.config.js or next.config.ts
import { execSync } from 'child_process';
...
assetPrefix: 'myapp',
generateBuildId: async () => {
    const gitHash = execSync('git rev-parse HEAD').toString().trim();
    return `_myapp_${gitHash}`;
},
...

As a result, Next.js will generate a /_next/data path like this:

/_next/data/_myapp_a14aa33b9ae0c98bc2962b8e480fa615b625470e/page.json

2. I have my own nginx server, so configure nginx to detect your app based on the static myapp marker:

location ~ ^/_next/(.*_myapp_[^/]+)/(.*)$ {
  alias /var/www/projects/SOME_PATH/out/_next/$1/$2;
}

This allows nginx to detect that the request belongs to this specific app and route it to the correct folder.

3. Serve public assets (optional, but useful) :

location ^~ /myapp {
  root /var/www/projects/SOME_PATH/out;
  index index.html;
  try_files $uri $uri/ /myapp/index.html;
}

Make sure to place all static files in public/myapp/, and then you can access them via https://yourdomain.com/myapp/path/to/file.pdf
In code, you can reference them like <Image src="/myapp/img.png" alt="..." width={300} height={200} />
It's for assetPrefix option