Timeout for incoming requests in io queue. #673
Description
Problem
Currently, after whole request is read from client's socket it is put to io queue where it will wait until some io thread will pop it. The time that request is in io queue is not limited, so server node can start handling request when the client is desperate to wait a response, so it has already handled timeout error and probably resent the request or gone. But server node will spend time and resources on handling request and sending response. Situation becomes worse when server node is overloaded: it handles timeouted requests and delays handling still alive requests which will be timeouted when server node will reach them.
What can we do
We can introduce timeout that will limit:
- time it takes to read the requests from socket - it very depends on a client
- time that the request spent in the queue - it completely depends on the current server node load
In the first version timeout can be set via server node config, but a client should be able to disable this timeout via command flags.
If that will not be enough, in the second version we can allow a client to redefine this timeout with request, so request will contain field that will be used as timeout for this request.