-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Labels
enhancement: pending triagep2-to-be-discussedEnhancement under consideration (priority)Enhancement under consideration (priority)
Description
Clear and concise description of the problem
Sometimes tests are flaky and people use the retry option to avoid failing CI when that happens.
But you still want to know which tests are flaky to try and improve them.
Suggested solution
Collect flaky tests (reruns > 0) in the reporter and print a warning summary at the end of the run.
The reason for suggesting the warning summary at the end is that this is obviously less important than failures and shouldn't be intermittent with other errors or warnings in the output.
Ideally link to the test by line and if possible also link back to the execution in the log (no idea if the latter is possible)
Alternative
A custom reporter could be built to provide this
Additional context
discord discussion https://discord.com/channels/917386801235247114/1462789015895019724
Validations
- Follow our Code of Conduct
- Read the Contributing Guidelines.
- Read the docs.
- Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancement: pending triagep2-to-be-discussedEnhancement under consideration (priority)Enhancement under consideration (priority)
Type
Projects
Status
Approved