|
| 1 | +# RFC 149: Support reftest variants |
| 2 | + |
| 3 | +## Summary |
| 4 | + |
| 5 | +Currently, there is no support for [variants] for reftests. This RFC proposes |
| 6 | +that we should support reftest [variants]. |
| 7 | + |
| 8 | +## Details |
| 9 | + |
| 10 | +When writing reftests for anything that may have an argument the writer of the |
| 11 | +test is limmited to either creating one large test that contains several |
| 12 | +subtests within it, or creating several largely similar tests with minor |
| 13 | +differences. As an example, consider the `<bdo>` element and the test created |
| 14 | +in the [reftest tutorial]. To test both `dir="rtl"` and `dir="ltr"` for the |
| 15 | +`<bdo>` element, two separate tests or one less descriptive test with both |
| 16 | +variants must be created. |
| 17 | + |
| 18 | +The proposal is to change the reftest manifest to allow for reftests to specify |
| 19 | +variants via a `<meta name="variant" content="">` tag, as currently described in |
| 20 | +the [variants] section of the documentation. |
| 21 | + |
| 22 | +### Example |
| 23 | + |
| 24 | +Again consider the [reftest tutorial] example of the `<bdo>` element. With |
| 25 | +reftest variants, the test could look like the following: |
| 26 | + |
| 27 | +``` |
| 28 | +<!DOCTYPE html> |
| 29 | +<html class="reftest-wait"> |
| 30 | +<meta charset="utf-8"> |
| 31 | +<title>BDO element</title> |
| 32 | +<link rel="help" href="https://html.spec.whatwg.org/#the-bdo-element"> |
| 33 | +<meta name="assert" content="BDO element's DIR content attribute renders correctly."> |
| 34 | +<link rel="match" href="bdo-dir-ref.html"> |
| 35 | +<meta name="variant" content="?dir=rtl"> |
| 36 | +<meta name="variant" content="?dir=ltr"> |
| 37 | +
|
| 38 | +<p>Test passes if displayed correctly.</p> |
| 39 | +<bdo id="target">SAW</bdo> |
| 40 | +<script> |
| 41 | + onload = () => { |
| 42 | + if (location.search) { |
| 43 | + let bdoTag = document.getElementById("target"); |
| 44 | + let params = new URL(document.location).searchParams; |
| 45 | + bdoTag.dir = params.get("dir"); |
| 46 | + } |
| 47 | + document.documentElement.removeAttribute('class'); |
| 48 | + } |
| 49 | +</script> |
| 50 | +</html> |
| 51 | +``` |
| 52 | + |
| 53 | +With reftest variants, the test reference could look like the following: |
| 54 | + |
| 55 | +``` |
| 56 | +<!DOCTYPE html> |
| 57 | +<html class="reftest-wait"> |
| 58 | +<meta charset="utf-8"> |
| 59 | +<title>BDO element reference</title> |
| 60 | +
|
| 61 | +<p>Test passes if displayed correctly.</p> |
| 62 | +<p id="reference"></p> |
| 63 | +<script> |
| 64 | + onload = () => { |
| 65 | + function addReferenceText(text) { |
| 66 | + const newtext = document.createTextNode(text); |
| 67 | + const p = document.getElementById("reference"); |
| 68 | + p.appendChild(newtext); |
| 69 | + } |
| 70 | +
|
| 71 | + if (location.search) { |
| 72 | + let params = new URL(document.location).searchParams; |
| 73 | + let dir = params.get("dir"); |
| 74 | + if (dir == "ltr") { |
| 75 | + addReferenceText("SAW"); |
| 76 | + } else if (dir == "rtl") { |
| 77 | + addReferenceText("WAS"); |
| 78 | + } else { |
| 79 | + addReferenceText("Invalid `dir` argument"); |
| 80 | + } |
| 81 | + } else { |
| 82 | + addReferenceText("No variant provided"); |
| 83 | + } |
| 84 | + document.documentElement.removeAttribute('class'); |
| 85 | + } |
| 86 | +</script> |
| 87 | +</html> |
| 88 | +``` |
| 89 | + |
| 90 | +## Risks |
| 91 | + |
| 92 | +This change could encourage reftest authors to create more tests and test |
| 93 | +references that rely on [reftest-wait] and are generated by a script. It may be |
| 94 | +harder to evaluate the expected behavior of a test when viewing the reference |
| 95 | +for such tests. |
| 96 | + |
| 97 | +[reftest tutorial]: https://web-platform-tests.org/writing-tests/reftest-tutorial.html#writing-the-test-file |
| 98 | +[variants]: https://web-platform-tests.org/writing-tests/testharness.html#variants |
| 99 | +[reftest-wait]: https://web-platform-tests.org/writing-tests/reftests.html#controlling-when-comparison-occurs |
0 commit comments