Skip to content

Commit 12fbf7d

Browse files
jlp-craigmortenjlp-craigmorten
committed
test: update wpt suite
1 parent 2eff0fe commit 12fbf7d

File tree

5 files changed

+7784
-2237
lines changed

5 files changed

+7784
-2237
lines changed

‎README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -55,11 +55,11 @@ Internal modules responsible for constructing the Virtual Screen Reader accessib
5555

5656
The current status of the WPT coverage is:
5757

58-
| Passing | Failing | Skipped |
59-
| :-----: | :-----: | :-----: |
60-
| 411 | 128 | 338 |
58+
| Passing Test Suites | Failing Tests | Skipped Tests |
59+
| :-----------------: | :-----------: | :-----------: |
60+
| 395 | 86 | 338 |
6161

62-
The included tests, skipped tests, and expected failures can be found in the [WPT configuration file](./test/wpt-jsdom/to-run.yaml) with reasons as to skips and expected failures.
62+
The included tests, skipped tests, and expected failures can be found in the [WPT configuration file](./test/wpt-jsdom/to-run.yaml) with reasons as to skips and expected failures. "Tentative" tests are excluded as their behaviour is often not yet confirmed.
6363

6464
In addition to the W3C specifications [a11ysupport.io](https://a11ysupport.io/) has been used as a guide for test cases in the absence of anything formal for screen reader output. In future we hope to adopt test cases laid out by the [ARIA and Assistive Technologies (ARIA-AT) community group](https://github.com/w3c/aria-at).
6565

‎test/wpt‎

Submodule wpt updated 1492 files

‎test/wpt-jsdom/run-wpts.js‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,10 @@ describe("web-platform-tests", () => {
6161
for (const toRunDoc of toRunDocs) {
6262
describe(toRunDoc.DIR, () => {
6363
for (const testFilePath of possibleTestFilePaths) {
64-
if (testFilePath.startsWith(toRunDoc.DIR + "/")) {
64+
if (
65+
testFilePath.startsWith(toRunDoc.DIR + "/") &&
66+
!testFilePath.includes(".tentative.")
67+
) {
6568
const matchingPattern = expectationsInDoc(toRunDoc).find(
6669
(pattern) => {
6770
const matcher = minimatchers.get(toRunDoc.DIR + "/" + pattern);

‎test/wpt-jsdom/to-run.yaml‎

Lines changed: 5 additions & 89 deletions
Original file line numberDiff line numberDiff line change
@@ -139,17 +139,6 @@ name/comp_name_from_content.html:
139139
[fail, upstream issue with dom-accessibility-api package]
140140
"link name from content for each child (no space, inline)":
141141
[fail, upstream issue with dom-accessibility-api package]
142-
name/comp_name_from_heading.tentative.html:
143-
"alertdialog role, name from heading":
144-
[fail, upstream issue with dom-accessibility-api package]
145-
"article role, name from heading":
146-
[fail, upstream issue with dom-accessibility-api package]
147-
"dialog role, name from heading":
148-
[fail, upstream issue with dom-accessibility-api package]
149-
"native dialog element, name from heading":
150-
[fail, upstream issue with dom-accessibility-api package]
151-
"article role, name from DFS heading":
152-
[fail, upstream issue with dom-accessibility-api package]
153142
name/comp_text_node.html:
154143
"span[role=button] with text/element/text nodes, no space":
155144
[fail, upstream issue with dom-accessibility-api package]
@@ -287,13 +276,13 @@ aria-valuetext_value_changes-manual.html:
287276
targets accessibility APIs whose mappings have not been implemented in this test suite yet,
288277
]
289278
combobox-value-calculation-manual.html:
290-
[timeout, value taken from text content not implemented]
279+
[skip, value taken from text content not implemented]
291280
exclude_element_hidden_with_*:
292281
[
293282
skip,
294283
targets accessibility APIs whose mappings have not been implemented in this test suite yet,
295284
]
296-
form-manual.html: [timeout, attribute typo error in test - fix needed in WPT]
285+
form-manual.html: [skip, attribute typo error in test - fix needed in WPT]
297286
generic_roledescription_prohibited-manual.html:
298287
[
299288
skip,
@@ -345,7 +334,7 @@ DIR: dpub-aam
345334

346335
manual/*:
347336
[
348-
timeout,
337+
skip,
349338
targets accessibility APIs whose mappings have not been implemented in this test suite yet,
350339
]
351340

@@ -362,12 +351,9 @@ DIR: graphics-aria
362351
---
363352
DIR: html-aam
364353

365-
dir-role.tentative.html:
366-
"el-dir": [fail, upstream issue with html-aria package]
367354
roles-contextual.html:
368355
"el-aside-in-section-title": [fail, upstream issue with html-aria package]
369356
"el-img-no-name": [fail, upstream issue with html-aria package]
370-
"el-img-empty-alt-title": [fail, upstream issue with html-aria package]
371357
"el-section-title": [fail, upstream issue with html-aria package]
372358
"el-aside-in-section-aria-label-whitespace":
373359
[fail, upstream issue with html-aria package]
@@ -393,24 +379,6 @@ roles-contextual.html:
393379
[fail, upstream issue with html-aria package]
394380
"el-section-aria-labelledby-whitespace":
395381
[fail, upstream issue with html-aria package]
396-
roles-contextual.tentative.html:
397-
"el-footer": [fail, upstream issue with html-aria package]
398-
"el-footer-ancestormain": [fail, upstream issue with html-aria package]
399-
"el-header": [fail, upstream issue with html-aria package]
400-
"el-header-ancestormain": [fail, upstream issue with html-aria package]
401-
roles-dynamic-switch.tentative.window.html:
402-
"Connected <input type=checkbox switch>":
403-
[fail, upstream issue with html-aria package]
404-
"Connected <input type=checkbox switch>: adding switch attribute":
405-
[fail, upstream issue with html-aria package]
406-
"Connected <input type=checkbox switch>: removing switch attribute":
407-
[fail, upstream issue with html-aria package]
408-
"Connected <input type=checkbox switch>: removing type attribute":
409-
[fail, upstream issue with html-aria package]
410-
"Connected <input type=checkbox switch>: adding type attribute":
411-
[fail, upstream issue with html-aria package]
412-
roles.tentative.html:
413-
"el-input-checkbox-switch": [fail, upstream issue with html-aria package]
414382
table-roles.html:
415383
"el-th-in-row": [fail, upstream issue with html-aria package]
416384

@@ -419,65 +387,13 @@ DIR: svg-aam
419387

420388
name/comp_host_language_label.html:
421389
[fail, upstream issue with dom-accessibility-api package]
422-
role/role-img.tentative.html:
423-
"Label: el-image (label from image>title element)":
424-
[fail, upstream issue with dom-accessibility-api or html-aria package]
425-
"Role: el-image (empty label due to missing image>title element)":
426-
[fail, upstream issue with html-aria package]
427-
"Role: el-image (label from image>title element)":
428-
[fail, upstream issue with html-aria package]
429390

430391
---
431392
DIR: wai-aria
432393

433-
idlharness.window.html: [timeout, out of scope]
394+
idlharness.window.html: [skip, out of scope]
434395
manual/*:
435396
[
436-
timeout,
397+
skip,
437398
targets accessibility APIs whose mappings have not been implemented in this test suite yet,
438399
]
439-
role/grid-roles.tentative.html:
440-
"orphaned button with gridcell role outside the context of row":
441-
[fail, upstream issue with html-aria package]
442-
"orphaned row outside the context of table":
443-
[fail, upstream issue with html-aria package]
444-
"orphaned rowgroup outside the context of row":
445-
[fail, upstream issue with html-aria package]
446-
"orphaned div with gridcell role outside the context of row":
447-
[fail, upstream issue with html-aria package]
448-
"orphaned rowheader outside the context of row":
449-
[fail, upstream issue with html-aria package]
450-
"orphaned columnheader outside the context of row":
451-
[fail, upstream issue with html-aria package]
452-
role/list-roles.tentative.html:
453-
"orphan div with listitem role": [fail, upstream issue with html-aria package]
454-
"orphan p with listitem role": [fail, upstream issue with html-aria package]
455-
role/listbox-roles.tentative.html:
456-
"orphaned option outside the context of listbox":
457-
[fail, upstream issue with html-aria package]
458-
role/menu-roles.tentative.html:
459-
"orphan div with menuitem role": [fail, upstream issue with html-aria package]
460-
"orphan div with menuitemcheckbox role":
461-
[fail, upstream issue with html-aria package]
462-
"orphan div with menuitemradio role":
463-
[fail, upstream issue with html-aria package]
464-
"orphaned menuitem outside the context of menu/menubar":
465-
[fail, upstream issue with html-aria package]
466-
"orphaned menuitemradio outside the context of menu/menubar":
467-
[fail, upstream issue with html-aria package]
468-
"orphaned menuitemcheckbox outside the context of menu/menubar":
469-
[fail, upstream issue with html-aria package]
470-
"orphan button with menuitem role":
471-
[fail, upstream issue with html-aria package]
472-
"orphan button with menuitemcheckbox role":
473-
[fail, upstream issue with html-aria package]
474-
"orphan button with menuitemradio role":
475-
[fail, upstream issue with html-aria package]
476-
role/tab-roles.tentative.html:
477-
"orphan button with tab role": [fail, upstream issue with html-aria package]
478-
"orphan span with tab role": [fail, upstream issue with html-aria package]
479-
role/tree-roles.tentative.html:
480-
"orphaned treeitem outside the context of tree":
481-
[fail, upstream issue with html-aria package]
482-
"orphaned button with treeitem role outside tree context":
483-
[fail, upstream issue with html-aria package]

0 commit comments

Comments
 (0)