Seeking guidance on distributed load testing with Playwright

[danidiaz]

I intend to make use of the distributed load testing support in Artillery, in particular for Playwright-based load tests defined in TypeScript. I have a few questions:

• I gather that the tests will be run in my own infrastructure, correct? What Artillery is providing here is the coordinating command, plus storage of the reports in Artillery Cloud.

• There are several supported runtimes: AWS Lambda, AWS Fargate, Azure ACI. For Playwright tests, should AWS Lambda or AWS Fargate be preferred?

• My tests will perform file uploads, so I have some example files that must be bundled. I need to explicitly use includeFiles, correct? (btw that field is not explicitly mentioned in the TypeScript type declaration for Config)

• Is the configuration file that exports the Config value evaluated in the machine where run-lambda or run-fargate executes, or is it shipped and evaluated in each particular Lambda? I have a configuration file that reads environment variables using Node's process.env., so I wonder about what values will be available where.

[danidiaz]

Also

• In case the commnadsrun-lambda or run-fargate crash for whatever reason (or the executing machine itself crashes) what is the best practice to clean leftover AWS resources? It seems that the --tags <tags> flag could be useful here, to easily identify the resources.

[hassy]

Yes correct, the tests will execute on your own infrastructure. For Playwright tests AWS Fargate (or Azure ACI) are the officially supported versions. The config is loaded and evaluated on each AWS Fargate container, so any env var values it relies on have to be available there. The easiest way to do that is with the --env-file flag - https://www.artillery.io/docs/reference/cli/run-fargate#shared-flags

[danidiaz]

@hassy I see, thanks!

For Playwright tests AWS Fargate (or Azure ACI) are the officially supported versions.

Ah, so Lambda is not supported for that use case. I didn't see it mentioned in the documentation, perhaps here would be a good place to do so.

The easiest way to do that is with the --env-file flag

Is the --env-file flag actually supported for TypeScript-based configurations? This answer seems to recommend using process.env, which I'm not sure would work in the distributed case 🤔