[Bug]: Connection to backing provider leaking when the source request cancels #9178
Description
What happened?
LiteLLM does not close the proxy connections to the backing LLM server, even if the requests are cancelled by the client.
I send requests with curl with very short timeout. curl cancels them, but LiteLLM proxy doesn't in turn cancel its own requests to the backing LLM provider.
Reproducer: https://github.com/aliok/litellm-connection-leak
Relevant log output
Are you a ML Ops Team?
No
What LiteLLM version are you on ?
v1.61.20-stable