-
Notifications
You must be signed in to change notification settings - Fork 364
Verify that benchmark tests meet expected performance thresholds #8574
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This comment was marked as off-topic.
This comment was marked as off-topic.
|
The WASM results are running much slower on the bots than locally. Locally, the benchmark tests pass and meet the expected thresholds, which are set to 16666 micros (16.6 ms is what is required for performant rendering on a 60 FPS device). On the CI the test is failing: @eyebrowsoffire @yjbanov any idea what would cause WASM to be significantly slower on the CI? The JS benchmarks seem to be consistent, or at least they are not exceeding the 16666 micros threshold for any of the metrics (on the CI and locally). |
|
Does the CI use the same hardware+OS combination as the local device you use for benchmarking? A lot depends on the hardware the benchmark runs on. As for why the JS is consistent and Wasm is not, I'm not sure. Someone explained to me in the past that V8 runs in different modes for different situations. For example, when you have DevTools open, V8 switches into a mode that supports debugging, which changes the performance characteristics of the code. |
|
@yjbanov looks like switching the action to run on |
Fixes #8583