Config for High CPU tasks

[amarflybot]

Hi Wonderful folks, Did someone ran linkerd for single threaded CPU intensive tasks?

Use Case:
Machine learning inference serving (FastAPI based Python Web Server, using uvicorn with 1 worker).
Config: Request: 2CPU/4GB RAM, Limit: 4CPU/8GB RAM.
No. of Pods: 20
No. of Concurrent User: 20.
Note: on my local machine (mac M1), this takes 400ms,

Observation:
Response Time: 4 secs.
If I use the default kube-proxy the load balancing would not happen, hence I switched to linkerd.
Now the load distribution happens.
How can I reduce the response time?
Is there any config in linkerd-proxy to allow only 1 request at a time?

[adleong]

Hi @amarflybot, I'm not sure exactly what you're asking. Linkerd does not have configuration to limit request concurrency.