Allow proxying rewrites - for proxying requests via proxies like mitmproxy

[dwjohnston]

Goals

nb. the terminology here is unfortunately ambiguous - there are other discussions/issues with the keywords 'proxy' and 'rewrites' but AFAICT these relate to regular rewrites wherin the NextJS server acts as a proxy server. This discussion is about proxying those proxied requests via another proxy such as mitmproxy.

Where we might setup our NextJS server to rewrite relative requests to some API, eg:

//next.config.mjs

const nextConfig = {
    async rewrites() {
        return [
          {
            source: '/api/:path*',
            destination: 'http://jsonplaceholder.typicode.com/:path*',
            
          },
        ]
      },
};

The request here is to allow some mechanism to proxy these requests via some other proxy service (eg. mitmproxy)

In abstract:

//next.config.mjs

const nextConfig = {
    async rewrites() {
        return [
          {
            source: '/api/:path*',
            destination: 'https://jsonplaceholder.typicode.com/:path*',
            proxyVia: "https://localhost:8080"
            
          },
        ]
      },
};

Non-Goals

Existing issues relating to SSL certificates for rewritten requests is not the objective of this proposal.

Background

The motivation here is that we're wanting to analysis the requests our application is making, whether by the server or the client, and potentially be generating test data from the requests.

Where with a regular SPA it would be possible to generate a HAR file in the browser's network tab, in an application that has server side requests those server side requests will not be visible.

Therefore the approach is to proxy traffic via a tool like mitmproxy and have it record HAR files for us.

Server side fetch proxying can be configured with a snippet like this:

//next.config.mjs

import { setGlobalDispatcher, ProxyAgent } from "undici";

const dispatcher = new ProxyAgent({uri: new URL("https://127.0.0.1:8080").toString()});
setGlobalDispatcher(dispatcher);

And this will proxy all server side fetch requests (eg. made in getServerSideProps or in RSCs)

However, relative requests made by the browser, than are then rewritten/proxied by the NextJS application server will not be proxied with this technique.

Proposal

Allow some mechanism for allowing developers access to Next's rewrite/proxying mechanism - so that such proxying can be configured.

Related resources:

#19078 This is a good comprehensive discussion of rewrite behaviour. In this case it's people wanting to modify headers.

[dwjohnston]

For what it's worth - the best way I've found to do proxying, is set up seperate process as a proxy, and then direct your API calls to that, as if it were the upstream.

Run this,

var http = require('http');
var httpProxy = require('http-proxy');

const {HttpsProxyAgent} = require('https-proxy-agent');

const agent = new HttpsProxyAgent("https://127.0.0.1:8080") // your proxy service like mitmproxy

var proxy = httpProxy.createProxyServer({"agent": agent});

http.createServer(function(req, res) {
    proxy.web(req, res, {changeOrigin: true,  target: 'http://jsonplaceholder.typicode.com'});
}).listen(3001);

And then in your make your rewrites:

//next.config.mjs

const nextConfig = {
    async rewrites() {
        return [
          {
            source: '/api/:path*',
            destination: 'http://localhost:3001' // though probably refer to an environment variable
            
          },
        ]
      },
};

This way, whether your requests are nextjs rewrites, or server side calls, they all just go to the same place.