You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[scout] Remove perf stats validation for Discover (#215130)
## Summary
With #212397 we added 2 tests for Discover app (loading `/app/discover`)
to track:
- js bundles loaded on page
- perf metrics like CPU time, Layout time and Script time fetched with
CDP Performance Domain API
While the first test for bundles _didn't report any failures_, second
test to validate Perf metrics fails periodically:
https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34729#0195a4de-6cd5-4d1e-be11-5d02be6de2b0
```
Error: CPU time (seconds) usage during page navigation should not exceed 1.5 seconds
expect(received).toBeLessThan(expected)
Expected: < 1.5
Received: 1.591343
```
https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34877
```
Error: Additional time spent executing JS scripts should not exceed 0.5 second
expect(received).toBeLessThan(expected)
Expected: < 0.5
Received: 0.601434
```
https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34899
```
Error: Total layout computation time should not exceed 0.06 second
expect(received).toBeLessThan(expected)
Expected: < 0.06
Received: 0.061723
```
https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34912#0195adb8-4536-42b7-ab4d-524535fdad9a
```
Error: Additional time spent executing JS scripts should not exceed 0.5 second
expect(received).toBeLessThan(expected)
Expected: < 0.5
Received: 0.561259
```
It was worth an experiment, but due to flakiness we decided to keep only
bundles limits validation for now and see if it is stable in the long
run.
If Data-Discovery team has interest in collecting Perf metrics without
strict validation in PRs, we can discuss the options. Alternatively we
can wait for Scout GA and you can deep dive into your own performance
testing with Playwright/CDP.
0 commit comments