Commit 1670b50
committed
QLS tests: label the per-run number of actions
Both success, failing and total.
Initial results:
Actions total (100 in total):
55% NumActions "10 <= n < 100"
36% NumActions "1 <= n < 10"
7% NumActions "n == 0"
2% NumActions "100 <= n < 1000"
Actions that succeeded total (100 in total):
52% NumActions "10 <= n < 100"
38% NumActions "1 <= n < 10"
9% NumActions "n == 0"
1% NumActions "100 <= n < 1000"
Actions that failed total (100 in total):
68% NumActions "1 <= n < 10"
22% NumActions "n == 0"
10% NumActions "10 <= n < 100"
As we can see, there's a surprising number of runs with no actions at
all (7%) or no successful actions (9%).
While it's important to get coverage of failing actions, we don't need
lots of coverage since the reasons are always the same ones. There are
not a lot of complex interactions in these cases. So we're wasting run
actions on them.1 parent 356d988 commit 1670b50
1 file changed
+17
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1306 | 1306 | | |
1307 | 1307 | | |
1308 | 1308 | | |
1309 | | - | |
| 1309 | + | |
| 1310 | + | |
1310 | 1311 | | |
1311 | | - | |
| 1312 | + | |
| 1313 | + | |
1312 | 1314 | | |
1313 | 1315 | | |
1314 | 1316 | | |
| |||
1604 | 1606 | | |
1605 | 1607 | | |
1606 | 1608 | | |
| 1609 | + | |
| 1610 | + | |
1607 | 1611 | | |
1608 | 1612 | | |
1609 | 1613 | | |
| |||
1623 | 1627 | | |
1624 | 1628 | | |
1625 | 1629 | | |
| 1630 | + | |
1626 | 1631 | | |
1627 | 1632 | | |
1628 | 1633 | | |
| |||
1644 | 1649 | | |
1645 | 1650 | | |
1646 | 1651 | | |
| 1652 | + | |
| 1653 | + | |
| 1654 | + | |
| 1655 | + | |
| 1656 | + | |
| 1657 | + | |
| 1658 | + | |
| 1659 | + | |
| 1660 | + | |
| 1661 | + | |
1647 | 1662 | | |
1648 | 1663 | | |
1649 | 1664 | | |
| |||
0 commit comments