|
| 1 | +## RFC 190: Enable view=test mode in wpt.fyi |
| 2 | + |
| 3 | +## Summary |
| 4 | +This RFC proposes enabling a new "test" view mode. This mode introduces a new method for counting total and failing tests, focusing on a strict pass/fail metric based on subtest counts. It includes frontend modifications to facilitate this change. This is to support this [feature request](https://github.com/web-platform-tests/wpt.fyi/issues/3740). |
| 5 | + |
| 6 | +## Details |
| 7 | + |
| 8 | +### Key Changes |
| 9 | +Currently, the feature is behind a feature flag that is set to false by default in production. |
| 10 | +* **Frontend Changes:** |
| 11 | +No frontend changes. This is will only be reachable by a query parameter that the end user must construct. |
| 12 | + |
| 13 | +### Background about "test" View Mode Behavior |
| 14 | +* Test pass is only registered when the number of passed subtests equals the total number of subtests. |
| 15 | +* Percentages are not displayed, only raw counts at the directory level. |
| 16 | +* Individual test level display: |
| 17 | + - Shows "PASS" only when subtest passes equal total subtests. |
| 18 | + - Shows "FAIL" when `status == OK` but subtest passes do not equal total subtests. |
| 19 | + - Existing `STATUS_ABBREVIATIONS` mappings used for other statuses. |
| 20 | + - "FAIL" as a fallback for any unmapped statuses. |
| 21 | + |
| 22 | +### How To Test The Change |
| 23 | + |
| 24 | +The feature has been enabled in staging by default so users can try it out. |
| 25 | + |
| 26 | +- Go to any page on https://staging.wpt.fyi that shows test counts |
| 27 | +- Add the view=test query parameter to the URL |
| 28 | + |
| 29 | +Examples: |
| 30 | +- https://staging.wpt.fyi/results/?label=experimental&label=master&aligned&view=test |
| 31 | +- https://staging.wpt.fyi/results/compat?label=experimental&label=master&aligned&view=test |
| 32 | +- https://staging.wpt.fyi/results/webstorage?label=experimental&label=master&aligned&view=test |
| 33 | + |
| 34 | + |
| 35 | +### Risks |
| 36 | + |
| 37 | +1. This new approach has different strengths and weaknesses to the long-standing default subtest view on wpt.fyi. |
| 38 | + - Mitigation: The default will remain unchanged to support current workflows and existing links. |
0 commit comments