proposal: net/http: client connection API

[neild]

This issue is part of a project to move x/net/http2 into std: #67810

This is a proposal to add the ability to create client connections to net/http.

The http.Transport type is (as stated in the package documentation) "a low-level primitive for making HTTP and HTTPS requests". A Transport contains a pool of client connections, potentially including connections to many different servers. A Transport's connections can be HTTP/1, HTTP/2, or a mix of both. Transports do not expose individual connections to the user.

Most HTTP client users are well-served by the current Transport, which automatically manages creating connections, distributing requests among existing connections, handling connection lifetime, and so on. However, some users may wish to manage connections directly rather than using the Transport's pool. For example, users who want to define their own connection pool need a way to create individual connections.

I propose adding the ability to create client connections to net/http. The base API is:

// NewClientConn creates a new client connection to the given address.
//
// If scheme is "http", the connection is unencrypted.
// If scheme is "https", the connection uses TLS.
//
// The protocol used for the new connection is determined by the scheme,
// Transport.Protocols configuration field, and protocols supported by the
// server. See Transport.Protocols for more details.
//
// If Transport.Proxy is set and indicates that a request sent to the given
// address should use a proxy, the new connection uses that proxy.
//
// NewClientConn always creates a new connection,
// even if the Transport has an existing cached connection to the given host.
//
// The new connection is not added to the Transport's connection cache,
// and will not be used by [Transport.RoundTrip].
// It does not count against the MaxIdleConns and MaxConnsPerHost limits.
//
// The caller is responsible for closing the new connection.
func (t *Transport) NewClientConn(ctx context.Context, scheme, address string) (*ClientConn, error)

// A ClientConn is a client connection to an HTTP server.
//
// Unlike a [Transport], a ClientConn represents a single connection.
type ClientConn struct{}

// Close closes the connection.
// It does not wait for outstanding RoundTrip calls to complete.
func (cc *ClientConn) Close() error

// Err reports any fatal connection errors.
// It returns nil if the connection is usable.
// If it returns non-nil, the connection can no longer be used.
func (cc *ClientConn) Err() error

// RoundTrip implements the [RoundTripper] interface.
//
// The request is sent on the client connection, regardless of the URL being
// requested or any proxy settings.
//
// If the connection is at its concurrency limit,
// RoundTrip waits for the connection to become available before
// sending the request.
func (cc *ClientConn) RoundTrip(*Request) (*Response, error)

// RequestLimit is the total number of requests that may be sent
// on the ClientConn, including calls which have already been made.
// Up to RequestLimit - RequestsCompleted concurrent RoundTrip calls may be made
// without blocking.
//
// RequestLimit returns 0 if the connection is closed.
//
// Note that RequestLimit does not go down as requests are sent on the connection.
// Instead, it increases as previous requests complete.
//
// For example, an HTTP/1 connection will initially have a RequestLimit value of 1.
// After sending a request on the connection, RequestLimit will remain at 1.
// After the caller closes the response body, ending the request, RequestLimit will
// increase to 2, indicating that a new request may be sent on the connection.
//
// RequestLimit may decrease if an HTTP/2 server reduces the concurrency limit
// on a connection.
func (cc *ClientConn) RequestLimit() int64

// RequestsCompleted returns the total number of requests calls that have completed.
// A request is complete when closing the connection will have no effect on it,
// typically when the request has been fully sent and the response fully received;
// or when the request has encountered a fatal error.
func (cc *ClientConn) RequestsCompleted() int64

// SetStateHook arranges for f to be called whenever the state of the connection changes.
// At most one call to f is made at a time.
// If the connection's state has changed since it was created, f is called immediately
// in a separate goroutine.
// f may be called synchronously from RoundTrip or Response.Body.Close.
//
// If f is nil, no further calls will be made to f after SetStateHook returns.
//
// f is called when the result of RequestLimit, RequestsCompleted, or Err changes.
func (cc *ClientConn) SetStateHook(f func(*ClientConn))

Most of this design is straightforward, but some points deserve further discussion.

Why is NewClientConn a method of Transport?
An http.Transport combines two concerns: It contains a number of configuration fields that customize the client's behavior, and it manages a pool of connections. The Transport.NewClientConn method allows us to reuse the customization. For example, Transport.MaxResponseHeaderBytes will apply to new connections created in this manner.

What is the purpose of ClientConn.RequestLimit?
An HTTP connection may be used for many requests over its lifetime. HTTP/1 connections with keep-alive enabled (the default) can be used for only a single request at a time, but may be reused for many requests in serial. HTTP/2 and HTTP/3 connections support multiple concurrent requests. HTTP/2 connections have a server-defined concurrency limit which restricts the total number of concurrent requests. HTTP/3 connections have a server-defined limit on the total number of requests sent (including completed requests), which is typically increased by the server as requests complete.

A connection pool or other user of a client connection type generally needs to know how many concurrent requests may be sent on a connection. In addition, the user generally needs to know when existing requests complete and free up space for new requests.

In the above design, we expose a connection's concurrency limit as a (mostly) monotonically increasing RequestLimit value. So long as the total (not concurrent) number of RoundTrip calls made on a connection remains under RequestLimit, requests will be sent to the server without blocking. The user may determine whether another (possibly concurrent) RoundTrip call may be made by taking the difference of RequestLimit and the total number of RoundTrip calls made.

This may be simpler to understand with some examples:

An HTTP/1 connection with keep-alive can handle at most one request at a time.

• The connection's RequestLimit starts at 1.
• A request is sent.
• The user calls Response.Body.Close, ending the request. RequestLimit increases to 2, allowing another request to be sent.

An HTTP/2 connection to a server with a concurrency limit of 100.

• The connection's RequestLimit starts at the server-provided concurrency limit (100).
• Several requests are sent.
• The user calls Response.Body.Close, ending one request. RequestLimit increases to 101, indicating that another request may be sent.

An HTTP/3 connection to a server maintaining a concurrency limit of 100.

• The connection's RequestLimit starts at the server-provided concurrency limit (100).
• Several requests are sent.
• The user calls Response.Body.Close, ending one request. RequestLimit does not change, because the server controls when a concurrency slot becomes available.
• The server sends a MAX_STREAMS QUIC frame indicating that the client may open another stream. RequestLimit is set to the stream limit from the frame (101).

RequestLimit is mostly monotonically increasing: It starts at the connection concurrency limit and increases as requests complete. The one exception is that RequestLimit can decrease if an HTTP/2 server reduces its concurrency limit. (This is an unusual scenario.)

As an alternative to providing a (mostly) monotonically increasing RequestLimit, I considered providing a value indicating how many additional concurrent requests a connection can accept. (This is the approach taken by golang.org/x/net/http2.ClientConnState.) This approach is perhaps more intuitive, but makes it difficult to model HTTP/3 concurrency limits, requires providing an additional mechanism for tracking completed requests, and is highly prone to subtle race conditions.

What is the purpose of ClientConn.RequestsCompleted?
A user may want to wait for outstanding requests on a connection to complete before closing the connection. This can be done by waiting until RequestsCompleted equals the number of RoundTrip calls issued on the connection.

How do you create a connection using a specific protocol (HTTP/1, HTTP/2)?
The combination of scheme (http, https), Transport.Protocols configuration field, and negotiation between the client and server selects a protocol. For example, if the scheme is "https", Transport.Protocols contains HTTP2, and the TLS handshake negotiates an ALPN protocol of "h2", the connection will use HTTP/2. Or if the scheme is "http", Transport.Protocols contains UnencryptedHTTP2, and Transport.Protocols does not include HTTP1, the connection will use unencrypted HTTP/2.

What does this proposal have to do with golang.org/x/net/http2?
The golang.org/x/net/http2 package exposes a ClientConn type representing a single HTTP/2 client connection. New connections may be created with Transport.NewClientConn (note that this is an http2.Transport, not an http.Transport).

The http2 package also exposes a ClientConnPool interface which may be used to replace the client connection pool in an http2.Transport.

This proposal aims to replace the following elements of the golang.org/x/net/http2 package API:

• ClientConn
• ClientConnPool
• ClientConnState
• Transport.NewClientConn
• Transport.ConnPool

The net/http.Transport.NewClientConn function will replace the equivalent http2 package function, while adding the ability to create HTTP/1 connections. (And, perhaps, eventually HTTP/3 ones as well.)

The http2 package's client connection pool APIs are superfluous. The http2 package aims to permit the user to replace the connection pool contained within a Transport, but the simpler approach to creating a connection pool is for the user to implement a RoundTripper that handles connection pooling.

How will net/http create HTTP/2 connections?
The net/http package currently creates HTTP/2 connections by calling Transport.TLSNextProto["h2"], which returns a RoundTripper. The x/net/http2 package's implementation of this function currently returns a RoundTripper which manages its own pool of HTTP/2 connections. We need a way to create a single, non-pooled connection.

I intend to add a new key to TLSNextProto: "h2-clientconn". x/net/http2 will install a function under this key with creates and returns a RoundTripper backed by a single client connection. This RoundTripper will also implement several methods to support the net/http ClientConn API.

This is all largely internal to the net/http and x/net/http2 packages and will not be apparent to the majority of users.

[mateusz834]

What is the use of (*http.ClientConn).Err()? When would you expect it to be called? Between every request? I guess RoundTrip should also check this and return that error.

[neild]

I expect there to be, broadly speaking, two general use cases for ClientConn:

• A user, for whatever reason, wants to work with a single HTTP connection. They need the ability to create a connection and to send requests on it. They don't really care about sophisticated concurrency management.

  This sort of user will use Transport.NewClientConn and ClientConn.RoundTrip. They might check ClientConn.Err if RoundTrip returns an error, to distinguish a request-scoped error from the connection becoming unusable. Or they might just treat any error as fatal.

• A user wants to implement a connection pool. They need not only the ability to create connections, but to manage the number of concurrent requests made on a connection. They need to be able to identify when requests complete, what the server's concurrency limit is, etc.

  This user will register a state hook with ClientConn.SetStateHook and will check ClientConn.Err when the hook is called to see if the connection is still open.