POC: Request through undici

[Corvince]

This is a proof of concept to explore HTTP/2 proxy support options. The ultimate goal is enabling full HTTP/2 proxy support, which presents a significant challenge: we currently proxy through HTTP/HTTPS requests, but can't detect HTTP/2 support beforehand. This would require attempting HTTP/2 first (using a different API), then falling back to regular requests - adding considerable complexity.

I explored using the Undici client instead, which offers:

• Unified API for HTTP/1.1, HTTPS, and HTTP/2
• Built-in features (connection pooling, redirects, etc.)
• Performance improvements
• Automatic protocol negotiation

To my surprise the code already passes most of the http and websocket tests (but struggling on lib tests).

This approach has significant API compatibility issues:

1. No custom agent support - HTTP/HTTPS agents are no longer used, breaking existing customizations.

2. Event model changes - The proxyReq event no longer exists, though similar functionality could be provided via interceptors. This is the most critical issue since many integrations depend on these events.

The latter point is where this approach falls together, because lots of things are built around these events. But the same would be somewhat true if we use a http2 client request, there is no API compatible way to handle both http2 requests.

So I am somewhat unsure how to proceed from here. First of all as I understand it http-proxy-3 should be a drop-in replacement for http-proxy, so any API changes should probably be forbidden at this point (especially given the rather low adaptation level currently). This leaves some option to consider

1. Just leave it as it is (supporting only incoming http2 requests), which probably isn't super future-proof
2. Create a new API for http2 requests (which then could also be used for http1/http1.1 used)
3. Try to recreate and/or emulate the proxyReq/proxyRes objects for http2 requests, which probably only works for simple things (e.g. returning an object with a setHeader method).

So I think the path forward depends on the general path forward and current use cases for this library, which I cannot evaluate.

[williamstein]

It seems to me that the best option is "Create a new API for http2 requests (which then could also be used for http1/http1.1 used)". This keeps things backward compatible -- which is a hard requirement for this project -- but still provides a path for http2 support.

[Corvince]

Probably makes sense. I started to go in this direction by adding "clientOptions" and "requestOptions", which are undici specific. If either of those are set, the new code path will trigger. The next step will be assessing the compatibility of the existing options with this new code path. Then I will be able to figure out when and how to warn for incompatible options . For example, setting dispatchOptions and a custom http agent will never be compatible, while followRedirects can, but simply has not been implemented so far.

[Corvince]

Staring at this stuff for a few more hours I realized that most of the options stuff happens outside of the "stream" function anyway. With the current modifications almost all tests pass, the exceptions being only

1. Tests depending on proxyReq event (1 direct tests one middleware test)
2. Tests depending on proxyRes event (2 tests)
3. A test that checks CamelCase of some headers (they are always lowercased with undici)
4. A test that checks if proxyReq is skipped when an "except" header is present. Except headers are not supported by undici.

So very promising so far. My next steps would be documentation on the new available options and their implications. And to provide some callback-based alternatives to proxyReq and proxyRes to call before sending the request and after receiving the response.