Gracefully reject mulitpart requests when proxying it to another server

[roma-glushko]

Hey everyone 👋

I'm proxing requests that comes to my Starlette API server to another downstream service via HTTPX like that:

client = httpx.AsyncClient(base_url=f"http://{hostname}/{path}")

downstream_request = client.build_request(
    request.method,
    path,
    content=request.stream(),
    # other params
)

response: HttpxResponse = await client.send(downstream_request, stream=True)

return StreamingResponse(
    response.aiter_raw(),
    status_code=response.status_code,
    background=BackgroundTask(response.aclose),
    headers=dict(response.headers),
)

When it comes to the multipart requests that upload files, it's possible that server can reject those requests if the attached file sizes are bigger then the allowed threshold. When that happens the downstream service responds with some 4XX error that was supposed to be returned back to the initial caller.

However it doesn't happen that way. Instead, HTTPX fails with the WriteError leaving it hard to distinguish whether the downstream service was unavailable or the file size was to big that that 4XX response get missed:

File "proxy.py", line 92, in proxy_request
    response: HttpxResponse = await client.send(downstream_request, stream=True)
  File "/.venv/lib64/python3.9/site-packages/httpx/_client.py", line 1620, in send
    response = await self._send_handling_auth(
  File "/.venv/lib64/python3.9/site-packages/httpx/_client.py", line 1648, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/.venv/lib64/python3.9/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/.venv/lib64/python3.9/site-packages/httpx/_client.py", line 1722, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/.venv/lib64/python3.9/site-packages/opentelemetry/instrumentation/httpx/__init__.py", line 403, in handle_async_request
    response = await self._transport.handle_async_request(
  File "/.venv/lib64/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/lib64/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/.venv/lib64/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.WriteError

Is there a way to gracefully reject such a request without swallowing the response of the downstream service?

Any ideas are appreciated 🙏

Thank you!