You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* fix link
* initial pass at guide
* fix missing label
* various text updates
* cleanup and clarity
* cleanup and clarity
* cleanup and clarity
* cleanup and clarity
* remove quotes
* lint fix
* cleanup and clarity
* Update docs/accessibility/get-started/introduction.mdx
Co-authored-by: Jennifer Shehane <[email protected]>
* Update docs/app/guides/accessibility-testing.mdx
Co-authored-by: Jennifer Shehane <[email protected]>
* Update docs/app/guides/accessibility-testing.mdx
Co-authored-by: Jennifer Shehane <[email protected]>
* Apply suggestions from code review
Co-authored-by: Jennifer Shehane <[email protected]>
* cleanup and clarity
* Apply suggestions from code review
Co-authored-by: Jennifer Shehane <[email protected]>
* add managing performance section
* add how it works section
* update text and images
* add test replay link
* mention ARIA, other minor edits
* tweaks
* remove summary that isn't needed
* lint fix
* fix bad link
---------
Co-authored-by: Jennifer Shehane <[email protected]>
Copy file name to clipboardExpand all lines: docs/accessibility/configuration/axe-core-configuration.mdx
+9-4Lines changed: 9 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,18 +4,23 @@
4
4
5
5
Configuration for Axe-Core® rules is available through your Account Executive. We we are happy to have a call with you to dial in your report config to make sure you are getting the most useful reports possible, and we find this onboarding very effective.
6
6
7
-
Generally configuration of these rules isn't needed because your implementation of any policies about what should "fail a build" is handled using the [results-api.md](/accessibility/results-api), where you have full control over how to parse the results and what rules need to be reacted to. Keeping the results in Cypress Cloud broad helps you to still be able to see and understand all of the accessibility information, even if only a subset of the results would be considered blocking.
7
+
In most cases, configuration of these rules as they run in Cypress Cloud isn't needed, because your implementation of any policies about what should "fail a build" is handled using the [Accessibility Results API](/accessibility/results-api), where you have full control over how to parse the results and what rules need to be reacted to. Keeping the results in Cypress Cloud broad helps you to still be able to see and understand all of the accessibility information, even if only a subset of the results would be considered blocking.
8
8
9
-
**Note.** Certain Axe Core rules are off by default. Here are the rules that are currently either ignored by configuration, or for which results cannot be detected.
9
+
**Note.** Certain Axe Core® rules are off by default. Here are the rules that are currently either ignored by configuration, or for which results cannot be detected.
10
10
11
11
-`Elements must meet minimum color contrast ratio thresholds`
12
-
- This is off by default. This is both the slowest rule in the Axe Core ruleset, and the one that can have the highest percentage of false positives or incomplete checks.
12
+
- This is off by default. This is both the slowest rule in the Axe Core® ruleset, and the one that can have the highest percentage of false positives or incomplete checks.
13
13
- It does work perfectly in many projects, so we are happy to turn this on for you if requested, so that you can see the results.
14
14
-`<video> or <audio> elements must not play automatically`
15
15
- Ignored because Test Replay does not yet capture the metadata to check this
16
16
-`Delayed refresh under 20 hours must not be used`
17
17
- Ignored because Test Replay does not collect the metadata for this
18
18
19
-
###Component Testing
19
+
## Component Testing
20
20
21
21
Cypress Accessibility works great with component testing. In addition the the rules that are off by default, "page-level" rules do not run for component tests, since a component is usually a fragment mounted in an empty page that would not be expected to have things like headings or a `main` element.
22
+
23
+
## Updates to the Axe Core® library
24
+
25
+
Updates to the Axe Core® library are released several times each year. While Cypress does not commit to updating our internal Axe Core® version within a specific timeframe, we will always wait at least 30 days before making an update.
26
+
Since some updates introduce new rules or address bugs that change how issues are detected, this buffer is intended to give you time to make changes in your use of our [Results API](/accessibility/results-api) if needed. Axe Core® releases can be tracked [in GitHub](https://github.com/dequelabs/axe-core/releases).
Copy file name to clipboardExpand all lines: docs/accessibility/core-concepts/accessibility-score.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,9 +11,9 @@ sidebar_position: 30
11
11
12
12
Cypress produces a top-line percentage score that indicates a general sense of accessibility for a project. This overall score for a run is based on the **average score of****all application snapshots** that were captured during the run, and is rooted in the ratio of passing and failing accessibility checks performed on each snapshot, weighted according to the severity of the issues. 
13
13
14
-
The severity levels presented in Cypress Cloud match the Axe Core `impact` level for each rule. This impact classification was developed by the creators of Axe, Deque Systems, to aid in prioritization of accessibility results, by highlighting rules that are likely to reveal the most substantial barriers (such as missing label text) so they can be addressed first.
14
+
The severity levels presented in Cypress Cloud match the Axe Core®`impact` level for each rule. This impact classification was developed by the creators of Axe, Deque Systems, to aid in prioritization of accessibility results, by highlighting rules that are likely to reveal the most substantial barriers (such as missing label text) so they can be addressed first.
15
15
16
-
It's important to note that the Web Content Accessibility Guidelines (WCAG) standards do not have a concept of relative severity of the Success Criteria. For **compliance** purposes, all failures of the Success Criteria count and must be addressed. For **usability** purposes, however, the impact values provided by Axe Core provide excellent guidance about which issues to take up first to bring the most benefit to disabled users of your plaform as quickly as possible.
16
+
It's important to note that the Web Content Accessibility Guidelines (WCAG) standards do not have a concept of relative severity of the Success Criteria. For **compliance** purposes, all failures of the Success Criteria count and must be addressed. For **usability** purposes, however, the impact values provided by Axe Core® provide excellent guidance about which issues to take up first to bring the most benefit to disabled users of your plaform as quickly as possible.
description: Cypress Accessibility\'s checks are powered by Axe Core® from Deque Systems.
5
+
sidebar_position: 40
6
+
---
7
+
8
+
# How it works
9
+
10
+
The stages of the Cypress Accessibility process are as follows:
11
+
12
+
1. As your run is recorded and specs are uploaded to Cypress Cloud, accessibility checks begin, based on the incoming [Test Replay](/cloud/features/test-replay) data, on the Cypress Cloud servers.
13
+
1. When your run completes, the final specs are processed for accessibility checks. All earlier specs have already entered processing and are likely completed by the end of the run.
14
+
1. Cypress Cloud organizes all the the Views (pages and components) that were rendered during the run, merging and deduplicating across tests so that page-level results and live DOM snapshots can be grouped together.
15
+
1. A combined "rule-level" report is created.
16
+
1. The completed results are displayed in Cypress Cloud and are flagged in integrations such Slack as GitHub/Gitlab comments, and available CI via the Results API so that you can optionally fail your pipeline if your standards are not met.
17
+
18
+
## About Axe Core®
19
+
20
+
Axe Core® is an open source library for detecting accessibility violations. It is used widely in the industry and have over 1 billion total downloads on npm.
21
+
The library may be familiar from other tools, such as Lighthouse, or tools used for code linting, mobile testing and more, and has also been used extensively in Cypress tests through open-source packages (discussed more our [accessibility testing guide](/app/guides/accessibility-testing)). This means that Cypress Accessibility's
22
+
configuration can often be easily aligned with your existing accessibility standards across different teams and platforms.
23
+
24
+
Axe Core® checks for accessibility violations in your website's user interface, based on common accessibility standards. A violation detected by Axe usually indicates that some users with disabilities would have a problem using or perceiving a feature. An example of a violation would be an image without alternative text provided in the markup, which means the content is not perceivable to visually-impaired users. [Learn more about Axe.](https://github.com/dequelabs/axe-core)
25
+
26
+
While this process reaches a high standard for automated testing, it cannot test every possible aspect of compliance, especially for heavily custom interfaces or aspects requiring human judgment.
27
+
28
+
This means that a 100% axe score does not mean all possible accessibility errors have been ruled out. Deque reports that [Axe Core® can detect up to 57% of the issues that would appear in a manual accessibility audit](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/).
29
+
30
+
The value of this form of testing in Cypress Accessibility is to give you fast, reliable, easy-to-understand feedback about common accessibility mistakes that are found in most projects. Providing these results automatically as part of your test run means that you can find and fix these issues with minimal friction, shifting accessibility left in your software development lifecycle.
31
+
32
+
## Test Replay limitations
33
+
34
+
Because Cypress Accessibility uses data captured through the Test Replay protocol, it is subject to any [limitations of that protocol](https://docs.cypress.io/faq/questions/cloud-faq#Is-everything-captured-and-replayed-in-Test-Replay) related to data capture or browser support.
Copy file name to clipboardExpand all lines: docs/accessibility/core-concepts/run-level-reports.mdx
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -54,20 +54,20 @@ An overall score is calculated based on all the Snapshots for a given View. The
54
54
The count of elements in a view that failed a Rule, had an inconclusive check for a Rule, or were explicitly ignored by configuration.
55
55
56
56
-**Failed** elements had at least one accessibility Rule fail during the run. These are considered in your accessibility Score and must be fixed. 
57
-
-**Inconclusive** elements had at least one check marked "incomplete" by Axe Core during the run. These represent checks that couldn't be completed for technical reasons, or items that need manual review to confirm if there is an accessibility issue. These do not affect your score.
57
+
-**Inconclusive** elements had at least one check marked "incomplete" by Axe Core® during the run. These represent checks that couldn't be completed for technical reasons, or items that need manual review to confirm if there is an accessibility issue. These do not affect your score.
58
58
-**Ignored** elements are those that had either failing or incomplete checks, but were ignored with [CSS selectors in your configuration](/accessibility/configuration/elementfilters). They do not affect your score. Cypress still processes these elements and makes the results available, so that you always know results were was ignored, since these elements are still present in your UI and users will encounter them.
59
59
60
60
#### Failed rules
61
61
62
-
Failed rules are broken down for each View with a mini chart displaying Axe Core severity levels, as well as a total count for failed rules. Both of these help you identify at a glance which pages or components have high numbers serious issues.
62
+
Failed rules are broken down for each View with a mini chart displaying Axe Core® severity levels, as well as a total count for failed rules. Both of these help you identify at a glance which pages or components have high numbers serious issues.
63
63
64
64
#### Severity
65
65
66
66
In Cypress Accessibility, severity levels are mapped to Axe Core's `impact` level for each rule. It's important to note that these levels were created by Deque Systems, the creators of Axe Core, and not a part of the Web Content Accessibility Guidelines themselves. For more details on this, see our [accessibility score documentation](https://github.com/dequelabs/axe-core/blob/develop/doc/issue_impact.md).
67
67
68
68
## Rules
69
69
70
-
This section lists the outcome of every Axe Core rule that was enabled at the time of this run.
70
+
This section lists the outcome of every Axe Core® rule that was enabled at the time of this run.
description: 'Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core by Deque Systems.'
4
+
description: 'Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems.'
Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core by Deque Systems.
13
+
## What is Cypress Accessibility?
14
+
15
+
Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems. This is the documentation page for that product. To learn about the general topic of accessibility testing using the Cypress App, see [our guide on this topic](/app/guides/accessibility-testing).
16
+
17
+
Cypress Accessibility generates a sortable, filterable report, to see scores and violation counts by page/component, or a combined rule-based report for the entire run. It also includes a drill-in view with live DOM snapshots highlighting each issue detected.
alt="A graphic showing the Cypress Accessibility dashboard including list of rules, views, and the user interface showing how devtools can be used"
18
22
/>
19
23
20
-
## How it works
24
+
## How are reports created?
21
25
22
26
Cypress Cloud generates an accessibility report for each unique state reached during your tests on a given Cypress run. This includes pages visited in end-to-end tests, as well as individual components rendered in component tests, and any states or pages that were encountered as a result of interactions performed during test execution.
23
27
24
-
These individual accessibility "snapshots" are grouped into page- or component-level reports (called "Views"), and also form the bases of a unified report covering the entire run. This gives you multiple ways to examine your report - by focusing on the most important parts of your application, or by
28
+
All of these page or component states are detected using the same protocol that powers [Test Replay](/cloud/features/test-replay), meaning no additional code or setup is needed anywhere in your tests.
25
29
26
-
Reports are generated using the industry-standard open source [Axe Core](https://github.com/dequelabs/axe-core)® library created by Deque Systems, with a default configuration focused on the [Web Content Accessibility Guidelines (WCAG) 2.1](https://www.w3.org/WAI/standards-guidelines/wcag/). 
27
-
28
-
## Understanding what is tested
30
+
These individual accessibility "snapshots" are grouped into page- or component-level reports (called "Views"), and also form the bases of a unified report covering the entire run. This gives you multiple ways to examine your report - by focusing on the most important parts of your application, or by
29
31
30
-
Axe Core checks for accessibility violations in your website's user interface, based on common accessibility standards. A violation detected by Axe usually indicates that some users with disabilities would have a problem using or perceiving a feature. An example of a violation would be an image without alternative text provided in the markup, which means the content is not perceivable to visually-impaired users. [Learn more about Axe.](https://github.com/dequelabs/axe-core)
32
+
Reports are generated using the industry-standard open source [Axe Core®](https://github.com/dequelabs/axe-core) library created by Deque Systems, with a default configuration focused on the [Web Content Accessibility Guidelines (WCAG) 2.1](https://www.w3.org/WAI/standards-guidelines/wcag/).
31
33
32
-
Axe reaches a high standard for automated testing, but cannot test every possible aspect of compliance, especially for heavily custom interfaces or aspects requiring human judgment. Please note that a 100% axe score does not mean all possible accessibility errors have been ruled out. [Axe typically detects up to 57% of the issues that would appear in a manual accessibility audit.](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/)
34
+
## Highlights
33
35
34
-
##### Trademark notice
36
+
- All steps of all user flows tested in Cypress are captured automatically
37
+
- Server-side execution means no disruption to existing test code or practices, and no performance impact from accessibility checks
38
+
- Reports are generated at the page or component level, as well as a combined report showing the outcome of all accessibility rules checked in the run
39
+
- Debuggable full-page DOM snapshots with highlights are provided for every violation
40
+
- Most iframe and Shadow DOM uses cases are supported out-of-the-box
41
+
- The [Results API](/accessibility/results-api) allows you to own how your CI pipeline reacts to the accessibility results of the run to implement a policy
42
+
- "Ignored" elements are isolated and don't affect your score, but can still be reviewed
35
43
36
-
_Note: DEQUE, DEQUELABS, AXE®, and AXE-CORE® are trademarks of Deque Systems, Inc. in the US and other countries._
44
+
_Note: DEQUE, DEQUELABS, AXE®, and AXE-CORE® are trademarks of Deque Systems, Inc. in the US and other countries. These terms are used throughout our documentation to refer to the accessibility testing library used by Cypress Accessibility._
package to use the familiar testing library methods (like `findByRole`,
228
230
`findByLabelText`, etc...) to select elements in Cypress specs.
229
231
230
-
In particular, if you're looking for more resources to understand how we
232
+
If you are coming from a React Testing Library background and looking for more resources to understand how we
231
233
recommend you approach testing your components, look to: [Cypress Component Testing](/app/component-testing/get-started).
232
234
235
+
### Accessibility Testing
236
+
237
+
Selecting elements with data attributes, text content, or Testing Library locators can each have some different implications for accessibility,
238
+
but none of these approaches is a "complete" accessibility test, and you will always need additional, accessibility-specific testing (including automated and manual tests) to confirm
239
+
your application is working as expected for people with disabilities and the technology they use. See[our accessibility testing guide](/app/guides/accessibility-testing) for more details and comparisons of approaches.
0 commit comments