Pass-through header forwarding with server-side exclusions

[prassanna-ravishankar]

EDIT: Updated from Security-by-default to pass-through-by-default

This PR enables pass-through by default forwarding of HTTP headers from applications to agents. Sensitive and hop by hop headers are excluded at the FastACP server boundary. Agents receive headers via the unified request object without additional setup.

What changed

• Forward all headers from request to agents by default
• Exclude sensitive and transport headers in FastACP using constants
  • Skips headers like x-agent-api-key, connection, content-length, transfer-encoding and common forwarded or sec prefixes
• Remove allowlist logic from the service
• Remove header_allowlist from agent config
• Keep a single request structure so agents read headers from params.request.headers

Why

This supports OAuth tokens, tenant context, tracing and role metadata with minimal configuration. Exclusions protect agent API key and noisy transport headers while keeping the model simple.

Tests

• Service forwards headers unchanged to agent RPC calls
• FastACP server does not expose x-agent-api-key and still passes custom headers

If an agent needs tighter rules it can filter inside its handler. This keeps the core path simple and fast.

[jasonyang101]

Instead of doing extra_headers and adding it to ACP, can you please do something like:

"request": {
  "headers": {
    ...
  }
}

for the ACP. This is because we likely may need to also pass in additional items as well to this and we shouldn't have it expand at the top level too far.