Request rate not reached.

[torebo]

Hi, is there anything from your side that would explain why a "rampTo 300" is not achieved. At max, 251 requests/vusers per second

Thank you
Best regards

[hassy]

Could be a couple of different reasons. Arrival rate is not the same thing as RPS. RPS is derived from a combination of:

• Arrival rate
• Number of requests in a scenario
• Response time from the server

For example, consider a phase of 10 seconds with 1 arrival/second, where each request takes 2 seconds to complete. The test will run for 12 seconds in total, and make 10 requests, so RPS will be reported as 0.83.

To increase RPS you may need to:

• Increase the arrival rate
• Use a loop to send multiple requests from each scenario (to avoid the overhead of every VU creating a new TCP connection)
• Scale out horizontally and run a distributed test on AWS Lambda/Fargate or Azure

We have a blog post that goes into more details of Artillery's workload modeling here: https://www.artillery.io/blog/load-testing-workload-models

[torebo]

Thank you @hassy!
Would it be possible to have a graph in Artillery Cloud showing the arrival rate, in order to compare the various response times metrics, the RPS, etc?
Assuming each vuser sends 1 and only 1 request, the number of vusers created shouldn't it be the same as the arrival number? If the arrival rate follows exacly the scenario definition, then why do we have such erratic curve for the vusers created?

Thank you for your answers
Best regards

[hassy]

Where are you running your tests from? You have an arrival rate of 300 specified in your second load phase - that's 300 new TCP connections created & torn down per second (plus TLS on top for HTTPS). It's likely that you're maxing out the CPU on the machine, and Artillery is unable to actually create that number of connections.

You have two options here:

1. Do you need to be testing how your service behaves with that number of new connections being established per second? If not, you can decrease the number of arrivals but send more requests from each scenario. This will reduce CPU usage and let you reach higher RPS.
2. Scale out your test horizontally on top of AWS Lambda/Fargate or Azure ACI. https://www.artillery.io/docs/load-testing-at-scale

[hassy]

& if all you need is to send the same request in a loop at constant RPS, something like Autocannon (https://github.com/mcollina/autocannon) may work better for you. Artillery is not designed for that type of use case.