Difference between model and per-request timeouts

[jamesleeht]

What's the difference between setting timeout per model vs timeout per request?
https://docs.litellm.ai/docs/proxy/timeout

Does the request timeout override the model timeout? Or does a request timeout span multiple fallbacks (e.g. if I set timeout for model to say 5 seconds, then i set request to 12 seconds, does that mean the router can only retry twice before being cancelled?)

[krrishdholakia]

Request timeout overrides model timeout (more dynamic)

[krrishdholakia]

It is run on each fallback - so it'll try each fallback for the request timeout value