Skip to content

Commit 8e8b7b2

Browse files
Add a11y guide and tweak references (#5957)
* fix link * initial pass at guide * fix missing label * various text updates * cleanup and clarity * cleanup and clarity * cleanup and clarity * cleanup and clarity * remove quotes * lint fix * cleanup and clarity * Update docs/accessibility/get-started/introduction.mdx Co-authored-by: Jennifer Shehane <[email protected]> * Update docs/app/guides/accessibility-testing.mdx Co-authored-by: Jennifer Shehane <[email protected]> * Update docs/app/guides/accessibility-testing.mdx Co-authored-by: Jennifer Shehane <[email protected]> * Apply suggestions from code review Co-authored-by: Jennifer Shehane <[email protected]> * cleanup and clarity * Apply suggestions from code review Co-authored-by: Jennifer Shehane <[email protected]> * add managing performance section * add how it works section * update text and images * add test replay link * mention ARIA, other minor edits * tweaks * remove summary that isn't needed * lint fix * fix bad link --------- Co-authored-by: Jennifer Shehane <[email protected]>
1 parent 45ab00d commit 8e8b7b2

File tree

14 files changed

+260
-32
lines changed

14 files changed

+260
-32
lines changed

docs/accessibility/configuration/axe-core-configuration.mdx

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,23 @@
44

55
Configuration for Axe-Core® rules is available through your Account Executive. We we are happy to have a call with you to dial in your report config to make sure you are getting the most useful reports possible, and we find this onboarding very effective.
66

7-
Generally configuration of these rules isn't needed because your implementation of any policies about what should "fail a build" is handled using the [results-api.md](/accessibility/results-api), where you have full control over how to parse the results and what rules need to be reacted to. Keeping the results in Cypress Cloud broad helps you to still be able to see and understand all of the accessibility information, even if only a subset of the results would be considered blocking.
7+
In most cases, configuration of these rules as they run in Cypress Cloud isn't needed, because your implementation of any policies about what should "fail a build" is handled using the [Accessibility Results API](/accessibility/results-api), where you have full control over how to parse the results and what rules need to be reacted to. Keeping the results in Cypress Cloud broad helps you to still be able to see and understand all of the accessibility information, even if only a subset of the results would be considered blocking.
88

9-
**Note.** Certain Axe Core rules are off by default. Here are the rules that are currently either ignored by configuration, or for which results cannot be detected.
9+
**Note.** Certain Axe Core® rules are off by default. Here are the rules that are currently either ignored by configuration, or for which results cannot be detected.
1010

1111
- `Elements must meet minimum color contrast ratio thresholds`
12-
- This is off by default. This is both the slowest rule in the Axe Core ruleset, and the one that can have the highest percentage of false positives or incomplete checks.
12+
- This is off by default. This is both the slowest rule in the Axe Core® ruleset, and the one that can have the highest percentage of false positives or incomplete checks.
1313
- It does work perfectly in many projects, so we are happy to turn this on for you if requested, so that you can see the results.
1414
- `<video> or <audio> elements must not play automatically`
1515
- Ignored because Test Replay does not yet capture the metadata to check this
1616
- `Delayed refresh under 20 hours must not be used`
1717
- Ignored because Test Replay does not collect the metadata for this
1818

19-
### Component Testing
19+
## Component Testing
2020

2121
Cypress Accessibility works great with component testing. In addition the the rules that are off by default, "page-level" rules do not run for component tests, since a component is usually a fragment mounted in an empty page that would not be expected to have things like headings or a `main` element.
22+
23+
## Updates to the Axe Core® library
24+
25+
Updates to the Axe Core® library are released several times each year. While Cypress does not commit to updating our internal Axe Core® version within a specific timeframe, we will always wait at least 30 days before making an update.
26+
Since some updates introduce new rules or address bugs that change how issues are detected, this buffer is intended to give you time to make changes in your use of our [Results API](/accessibility/results-api) if needed. Axe Core® releases can be tracked [in GitHub](https://github.com/dequelabs/axe-core/releases).

docs/accessibility/core-concepts/accessibility-score.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@ sidebar_position: 30
1111

1212
Cypress produces a top-line percentage score that indicates a general sense of accessibility for a project. This overall score for a run is based on the **average score of** **all application snapshots** that were captured during the run, and is rooted in the ratio of passing and failing accessibility checks performed on each snapshot, weighted according to the severity of the issues.&#x20;
1313

14-
The severity levels presented in Cypress Cloud match the Axe Core `impact` level for each rule. This impact classification was developed by the creators of Axe, Deque Systems, to aid in prioritization of accessibility results, by highlighting rules that are likely to reveal the most substantial barriers (such as missing label text) so they can be addressed first.
14+
The severity levels presented in Cypress Cloud match the Axe Core® `impact` level for each rule. This impact classification was developed by the creators of Axe, Deque Systems, to aid in prioritization of accessibility results, by highlighting rules that are likely to reveal the most substantial barriers (such as missing label text) so they can be addressed first.
1515

16-
It's important to note that the Web Content Accessibility Guidelines (WCAG) standards do not have a concept of relative severity of the Success Criteria. For **compliance** purposes, all failures of the Success Criteria count and must be addressed. For **usability** purposes, however, the impact values provided by Axe Core provide excellent guidance about which issues to take up first to bring the most benefit to disabled users of your plaform as quickly as possible.
16+
It's important to note that the Web Content Accessibility Guidelines (WCAG) standards do not have a concept of relative severity of the Success Criteria. For **compliance** purposes, all failures of the Success Criteria count and must be addressed. For **usability** purposes, however, the impact values provided by Axe Core® provide excellent guidance about which issues to take up first to bring the most benefit to disabled users of your plaform as quickly as possible.
1717

1818
## Algorithm
1919

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
sidebar_label: 'How it works'
3+
title: 'How it works'
4+
description: Cypress Accessibility\'s checks are powered by Axe Core® from Deque Systems.
5+
sidebar_position: 40
6+
---
7+
8+
# How it works
9+
10+
The stages of the Cypress Accessibility process are as follows:
11+
12+
1. As your run is recorded and specs are uploaded to Cypress Cloud, accessibility checks begin, based on the incoming [Test Replay](/cloud/features/test-replay) data, on the Cypress Cloud servers.
13+
1. When your run completes, the final specs are processed for accessibility checks. All earlier specs have already entered processing and are likely completed by the end of the run.
14+
1. Cypress Cloud organizes all the the Views (pages and components) that were rendered during the run, merging and deduplicating across tests so that page-level results and live DOM snapshots can be grouped together.
15+
1. A combined "rule-level" report is created.
16+
1. The completed results are displayed in Cypress Cloud and are flagged in integrations such Slack as GitHub/Gitlab comments, and available CI via the Results API so that you can optionally fail your pipeline if your standards are not met.
17+
18+
## About Axe Core®
19+
20+
Axe Core® is an open source library for detecting accessibility violations. It is used widely in the industry and have over 1 billion total downloads on npm.
21+
The library may be familiar from other tools, such as Lighthouse, or tools used for code linting, mobile testing and more, and has also been used extensively in Cypress tests through open-source packages (discussed more our [accessibility testing guide](/app/guides/accessibility-testing)). This means that Cypress Accessibility's
22+
configuration can often be easily aligned with your existing accessibility standards across different teams and platforms.
23+
24+
Axe Core® checks for accessibility violations in your website's user interface, based on common accessibility standards. A violation detected by Axe usually indicates that some users with disabilities would have a problem using or perceiving a feature. An example of a violation would be an image without alternative text provided in the markup, which means the content is not perceivable to visually-impaired users. [Learn more about Axe.](https://github.com/dequelabs/axe-core)
25+
26+
While this process reaches a high standard for automated testing, it cannot test every possible aspect of compliance, especially for heavily custom interfaces or aspects requiring human judgment.
27+
28+
This means that a 100% axe score does not mean all possible accessibility errors have been ruled out. Deque reports that [Axe Core® can detect up to 57% of the issues that would appear in a manual accessibility audit](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/).
29+
30+
The value of this form of testing in Cypress Accessibility is to give you fast, reliable, easy-to-understand feedback about common accessibility mistakes that are found in most projects. Providing these results automatically as part of your test run means that you can find and fix these issues with minimal friction, shifting accessibility left in your software development lifecycle.
31+
32+
## Test Replay limitations
33+
34+
Because Cypress Accessibility uses data captured through the Test Replay protocol, it is subject to any [limitations of that protocol](https://docs.cypress.io/faq/questions/cloud-faq#Is-everything-captured-and-replayed-in-Test-Replay) related to data capture or browser support.

docs/accessibility/core-concepts/run-level-reports.mdx

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -54,20 +54,20 @@ An overall score is calculated based on all the Snapshots for a given View. The
5454
The count of elements in a view that failed a Rule, had an inconclusive check for a Rule, or were explicitly ignored by configuration.
5555

5656
- **Failed** elements had at least one accessibility Rule fail during the run. These are considered in your accessibility Score and must be fixed.&#x20;
57-
- **Inconclusive** elements had at least one check marked "incomplete" by Axe Core during the run. These represent checks that couldn't be completed for technical reasons, or items that need manual review to confirm if there is an accessibility issue. These do not affect your score.
57+
- **Inconclusive** elements had at least one check marked "incomplete" by Axe Core® during the run. These represent checks that couldn't be completed for technical reasons, or items that need manual review to confirm if there is an accessibility issue. These do not affect your score.
5858
- **Ignored** elements are those that had either failing or incomplete checks, but were ignored with [CSS selectors in your configuration](/accessibility/configuration/elementfilters). They do not affect your score. Cypress still processes these elements and makes the results available, so that you always know results were was ignored, since these elements are still present in your UI and users will encounter them.
5959

6060
#### Failed rules
6161

62-
Failed rules are broken down for each View with a mini chart displaying Axe Core severity levels, as well as a total count for failed rules. Both of these help you identify at a glance which pages or components have high numbers serious issues.
62+
Failed rules are broken down for each View with a mini chart displaying Axe Core® severity levels, as well as a total count for failed rules. Both of these help you identify at a glance which pages or components have high numbers serious issues.
6363

6464
#### Severity
6565

6666
In Cypress Accessibility, severity levels are mapped to Axe Core's `impact` level for each rule. It's important to note that these levels were created by Deque Systems, the creators of Axe Core, and not a part of the Web Content Accessibility Guidelines themselves. For more details on this, see our [accessibility score documentation](https://github.com/dequelabs/axe-core/blob/develop/doc/issue_impact.md).
6767

6868
## Rules
6969

70-
This section lists the outcome of every Axe Core rule that was enabled at the time of this run.
70+
This section lists the outcome of every Axe Core® rule that was enabled at the time of this run.
7171

7272
<DocsImage
7373
src="/img/accessibility/core-concepts/cypress-accessibility-rules-list.png"
@@ -81,11 +81,11 @@ Each Rule in the table has the following attributes:
8181
- An icon representing the overall status for the rule, which can be any one of these:
8282
- **Passed** -- every time this rule applied to an element, the element either passed that rule's checks or was intentionally ignored
8383
- **Failed** -- at least one failing element was detected in the run
84-
- **Inconclusive** -- no elements failed this rule, but at least one element was labelled "incomplete" by Axe Core and requires manual review
84+
- **Inconclusive** -- no elements failed this rule, but at least one element was labelled "incomplete" by Axe Core® and requires manual review
8585
- **Inapplicable** -- this rule was enabled during the run, but there were no elements detected that required the rule to run
8686
- **Ignored by config** -- this rule was ignored at the config level for the project and did not run
8787
- The description for the rule
88-
- The Axe Core severity level for the rule
88+
- The Axe Core® severity level for the rule
8989
- The counts of failed, inconclusive, and ignored elements for each rule
9090

9191
## Filtering
Lines changed: 19 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
sidebar_label: Introduction
33
title: 'Introduction'
4-
description: 'Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core by Deque Systems.'
4+
description: 'Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems.'
55
sidebar_position: 10
66
sidebar_custom_props: { 'new_label': true }
77
---
@@ -10,27 +10,35 @@ sidebar_custom_props: { 'new_label': true }
1010

1111
<AccessibilityAddon />
1212

13-
Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core by Deque Systems.
13+
## What is Cypress Accessibility?
14+
15+
Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems. This is the documentation page for that product. To learn about the general topic of accessibility testing using the Cypress App, see [our guide on this topic](/app/guides/accessibility-testing).
16+
17+
Cypress Accessibility generates a sortable, filterable report, to see scores and violation counts by page/component, or a combined rule-based report for the entire run. It also includes a drill-in view with live DOM snapshots highlighting each issue detected.
1418

1519
<DocsImage
1620
src="/img/accessibility/get-started/cypress-accessibility-overview.png"
1721
alt="A graphic showing the Cypress Accessibility dashboard including list of rules, views, and the user interface showing how devtools can be used"
1822
/>
1923

20-
## How it works
24+
## How are reports created?
2125

2226
Cypress Cloud generates an accessibility report for each unique state reached during your tests on a given Cypress run. This includes pages visited in end-to-end tests, as well as individual components rendered in component tests, and any states or pages that were encountered as a result of interactions performed during test execution.
2327

24-
These individual accessibility "snapshots" are grouped into page- or component-level reports (called "Views"), and also form the bases of a unified report covering the entire run. This gives you multiple ways to examine your report - by focusing on the most important parts of your application, or by
28+
All of these page or component states are detected using the same protocol that powers [Test Replay](/cloud/features/test-replay), meaning no additional code or setup is needed anywhere in your tests.
2529

26-
Reports are generated using the industry-standard open source [Axe Core](https://github.com/dequelabs/axe-core)® library created by Deque Systems, with a default configuration focused on the [Web Content Accessibility Guidelines (WCAG) 2.1](https://www.w3.org/WAI/standards-guidelines/wcag/).&#x20;
27-
28-
## Understanding what is tested
30+
These individual accessibility "snapshots" are grouped into page- or component-level reports (called "Views"), and also form the bases of a unified report covering the entire run. This gives you multiple ways to examine your report - by focusing on the most important parts of your application, or by
2931

30-
Axe Core checks for accessibility violations in your website's user interface, based on common accessibility standards. A violation detected by Axe usually indicates that some users with disabilities would have a problem using or perceiving a feature. An example of a violation would be an image without alternative text provided in the markup, which means the content is not perceivable to visually-impaired users. [Learn more about Axe.](https://github.com/dequelabs/axe-core)
32+
Reports are generated using the industry-standard open source [Axe Core®](https://github.com/dequelabs/axe-core) library created by Deque Systems, with a default configuration focused on the [Web Content Accessibility Guidelines (WCAG) 2.1](https://www.w3.org/WAI/standards-guidelines/wcag/).
3133

32-
Axe reaches a high standard for automated testing, but cannot test every possible aspect of compliance, especially for heavily custom interfaces or aspects requiring human judgment. Please note that a 100% axe score does not mean all possible accessibility errors have been ruled out. [Axe typically detects up to 57% of the issues that would appear in a manual accessibility audit.](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/)
34+
## Highlights
3335

34-
##### Trademark notice
36+
- All steps of all user flows tested in Cypress are captured automatically
37+
- Server-side execution means no disruption to existing test code or practices, and no performance impact from accessibility checks
38+
- Reports are generated at the page or component level, as well as a combined report showing the outcome of all accessibility rules checked in the run
39+
- Debuggable full-page DOM snapshots with highlights are provided for every violation
40+
- Most iframe and Shadow DOM uses cases are supported out-of-the-box
41+
- The [Results API](/accessibility/results-api) allows you to own how your CI pipeline reacts to the accessibility results of the run to implement a policy
42+
- "Ignored" elements are isolated and don't affect your score, but can still be reviewed
3543

36-
_Note: DEQUE, DEQUELABS, AXE®, and AXE-CORE® are trademarks of Deque Systems, Inc. in the US and other countries._
44+
_Note: DEQUE, DEQUELABS, AXE®, and AXE-CORE® are trademarks of Deque Systems, Inc. in the US and other countries. These terms are used throughout our documentation to refer to the accessibility testing library used by Cypress Accessibility._

docs/app/core-concepts/best-practices.mdx

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -220,16 +220,24 @@ If the answer is **no** because the text could be changed - then use
220220

221221
Cypress loves the Testing Library project. We use Testing Library internally,
222222
and our philosophy aligns closely with Testing Library's ethos and approach to
223-
writing tests. We strongly endorse their best practices.
223+
writing tests. We strongly endorse their best practices for situations where,
224+
as with `cy.contains()`, you want to fail a test if a specific piece of content or
225+
accessible role is not present.
224226

225227
You can use the
226228
[Cypress Testing Library](https://testing-library.com/docs/cypress-testing-library/intro/)
227229
package to use the familiar testing library methods (like `findByRole`,
228230
`findByLabelText`, etc...) to select elements in Cypress specs.
229231

230-
In particular, if you're looking for more resources to understand how we
232+
If you are coming from a React Testing Library background and looking for more resources to understand how we
231233
recommend you approach testing your components, look to: [Cypress Component Testing](/app/component-testing/get-started).
232234

235+
### Accessibility Testing
236+
237+
Selecting elements with data attributes, text content, or Testing Library locators can each have some different implications for accessibility,
238+
but none of these approaches is a "complete" accessibility test, and you will always need additional, accessibility-specific testing (including automated and manual tests) to confirm
239+
your application is working as expected for people with disabilities and the technology they use. See[our accessibility testing guide](/app/guides/accessibility-testing) for more details and comparisons of approaches.
240+
233241
## <Icon name="angle-right" /> Assigning Return Values
234242

235243
:::danger

0 commit comments

Comments
 (0)