Add parameters to control concurrency, max-concurrency and max-waiting_time. Only support in online mode.
Description:

• max-waiting-time: maximum wait time for establishing a connection upon service request. Default: -1 (indicates no wait time limit).
• max-concurrency: the actual number of connections established by the service, default 512.

If the number of requests is bigger than max-concurrency, the service will return error response 429 (Too Many Requests).

If the waiting time is bigger than max-waiting-time , the service will return error response 408 (Request Timeout).

Usage:

python -m fastdeploy.entrypoints.openai.api_server \
--model xxx
--max-concurrency 1024 \
--max-waiting-time 300