Create a "universal" load generator

[minherz]

Description

Demo applications that Cloud Ops Sandbox uses (Online Boutique in version 0.9 and 0.10) provide their own load generators. It is not always useful because these generators can lack certain features that are useful for observability use cases. For example, customization of the load or an ability to program dynamic load.

To have a load generator that can be configured vs. multiple endpoints of the application can prove useful for Cloud Ops Sandbox users.

Technical requirements

Implement a stand-alone load generator using one of the existing solutions. While the language is not limited, it is preferred to keep all code in Golang. The following load generator implementations can be checked as references:

• locust
• jetty
• boomer

[archyufa]

@minherz During the POC we created the universal log generator for logs using Python, will be happy to share source code in case you need it.

[lvaylet]

Hi @minherz, I can work on this. Just wondering: what is wrong with the load generator included with the Online Boutique? It is able to send GET and POST requests to multiple points with complex scenarios, as configured in https://github.com/GoogleCloudPlatform/microservices-demo/blob/main/src/loadgenerator/locustfile.py:

Could you provide a specific example of a feature that is currently missing?

[minherz]

There is nothing wrong with LG in OB. the goal of this issue is to provide an alternative LG as a part of COS.
When I tried to use LG that comes with OB I found out that it does not expose UI interface. According to what I see in the code it runs a preset configuration of the load that cannot be changed. While it is good enough to show o11y data, it is not sufficient for simulating scenarios that are exposed under heavy load.

[lvaylet]

Got it. I agree the LG in OB does not seem to support custom load shapes for example. But then I wonder how to make a "universal" load generator with a large (infinite?) number of scenarios with a reasonable amount of time and energy on our side. Would it make more sense to let users spawn their own Locust container on their local machines, with their custom tasks? And document the whole thing with an example on our side.

[minherz]

Would it make more sense to let users spawn their own Locust container on their local machines, with their custom tasks? And document the whole thing with an example on our side.

This sounds like a great idea! I am unsure if we want to document the Locust load generator but we certainly can provide guidelines and to use Online Boutique as an example.