dial tcp 127.0.0.1:10088: connect: can't assign requested address

[spacewander]

Description:

Describe the issue.

When I do a benchmark for ai-gateway with the default configuration, after a while, there will be lots of errors:

msg="failed to create MCP session" component=mcp-proxy backend=rust-upstream error="failed to send MCP initialize request: failed to send MCP notifications/initialized request: Post \"http://127.0.0.1:10088/mcp\": dial tcp 127.0.0.1:10088: connect: can't assign requested address"

My benchmark script is:

import http from 'k6/http';

export const options = {
  vus: 100, // Max virtual users (equivalent to 100 connections in wrk)
  duration: '30s', // Test duration
};

const url = 'http://localhost:1975/mcp';

const headers = {
  Accept: 'application/json, text/event-stream',
  'Content-Type': 'application/json',
};

const body = JSON.stringify({
  method: 'initialize',
  params: {
    protocolVersion: '2025-06-18',
    capabilities: {},
    clientInfo: {
      name: 'example-client',
      version: '1.0.0',
    },
  },
  jsonrpc: '2.0',
  id: 0,
});

export default function () {
  http.post(url, body, { headers });
}

The ai-gateway is started by aigw run --mcp-config envoy-ai-gateway/mcp-servers.json

The 10088 port is listened to by the internal MCP backend.
Is the :1975 to :10088 traffic using short-lived connections?

Any extra documentation required to understand the issue.

[Aditya7880900936]

Hi @mathetake
This looks like ephemeral port exhaustion caused by short-lived TCP connections
between the MCP proxy and the internal MCP backend (:10088).

Under sustained load, each request appears to create a new TCP connection
(mcp-proxy → backend), which quickly accumulates TIME_WAIT sockets on loopback.
Once the ephemeral port range is exhausted, new dials fail with:

dial tcp 127.0.0.1:10088: can't assign requested address

This suggests the MCP client / proxy path is not reusing connections
(no pooling / keepalive), or that MCP initialize triggers a fresh dial
per request instead of reusing a long-lived session/transport.

Question:

• Is the MCP backend connection intended to be long-lived and shared?
• Is initialize expected to open a new transport every time?

Happy to dig into where the dial happens and propose a fix if this
is unintended behavior.