Skip to content
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
fd7e42c
fix link
marktnoonan Oct 18, 2024
2b467be
initial pass at guide
marktnoonan Oct 18, 2024
70ce39c
fix missing label
marktnoonan Oct 18, 2024
47c711a
various text updates
marktnoonan Oct 18, 2024
5152c28
cleanup and clarity
marktnoonan Oct 18, 2024
db9fe83
cleanup and clarity
marktnoonan Oct 18, 2024
a94c9d6
cleanup and clarity
marktnoonan Oct 18, 2024
ecad352
cleanup and clarity
marktnoonan Oct 18, 2024
8a11e6a
remove quotes
marktnoonan Oct 18, 2024
f1129a7
lint fix
marktnoonan Oct 18, 2024
19f9180
cleanup and clarity
marktnoonan Oct 18, 2024
8404787
Update docs/accessibility/get-started/introduction.mdx
marktnoonan Oct 18, 2024
f6e27f7
Update docs/app/guides/accessibility-testing.mdx
marktnoonan Oct 18, 2024
0cdd5f4
Update docs/app/guides/accessibility-testing.mdx
marktnoonan Oct 18, 2024
36225b4
Apply suggestions from code review
marktnoonan Oct 18, 2024
6621c14
cleanup and clarity
marktnoonan Oct 18, 2024
2106ca4
Apply suggestions from code review
marktnoonan Oct 18, 2024
32585c7
add managing performance section
marktnoonan Oct 18, 2024
2ded988
Merge branch 'a11y-ga-updates' of https://github.com/cypress-io/cypre…
marktnoonan Oct 18, 2024
43ddb22
add how it works section
marktnoonan Oct 19, 2024
df4dbf9
update text and images
marktnoonan Oct 19, 2024
d25fc6e
add test replay link
marktnoonan Oct 19, 2024
cb55822
mention ARIA, other minor edits
marktnoonan Oct 19, 2024
88e7923
tweaks
marktnoonan Oct 19, 2024
4f4369c
remove summary that isn't needed
marktnoonan Oct 19, 2024
e16af31
Merge branch 'ga-ui-cov-a11y' into a11y-ga-updates
marktnoonan Oct 19, 2024
9189618
lint fix
marktnoonan Oct 19, 2024
94a25eb
fix bad link
marktnoonan Oct 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/accessibility/configuration/axe-core-configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@

Configuration for Axe-Core® rules is available through your Account Executive. We we are happy to have a call with you to dial in your report config to make sure you are getting the most useful reports possible, and we find this onboarding very effective.

Generally configuration of these rules isn't needed because your implementation of any policies about what should "fail a build" is handled using the [results-api.md](/accessibility/results-api), where you have full control over how to parse the results and what rules need to be reacted to. Keeping the results in Cypress Cloud broad helps you to still be able to see and understand all of the accessibility information, even if only a subset of the results would be considered blocking.
In most cases, configuration of these rules as they run in Cypress Cloud isn't needed, because your implementation of any policies about what should "fail a build" is handled using the [Accessibility Results API](/accessibility/results-api), where you have full control over how to parse the results and what rules need to be reacted to. Keeping the results in Cypress Cloud broad helps you to still be able to see and understand all of the accessibility information, even if only a subset of the results would be considered blocking.

**Note.** Certain Axe Core rules are off by default. Here are the rules that are currently either ignored by configuration, or for which results cannot be detected.
**Note.** Certain Axe Core® rules are off by default. Here are the rules that are currently either ignored by configuration, or for which results cannot be detected.

- `Elements must meet minimum color contrast ratio thresholds`
- This is off by default. This is both the slowest rule in the Axe Core ruleset, and the one that can have the highest percentage of false positives or incomplete checks.
- This is off by default. This is both the slowest rule in the Axe Core® ruleset, and the one that can have the highest percentage of false positives or incomplete checks.
- It does work perfectly in many projects, so we are happy to turn this on for you if requested, so that you can see the results.
- `<video> or <audio> elements must not play automatically`
- Ignored because Test Replay does not yet capture the metadata to check this
Expand Down
4 changes: 2 additions & 2 deletions docs/accessibility/core-concepts/accessibility-score.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ sidebar_position: 30

Cypress produces a top-line percentage score that indicates a general sense of accessibility for a project. This overall score for a run is based on the **average score of** **all application snapshots** that were captured during the run, and is rooted in the ratio of passing and failing accessibility checks performed on each snapshot, weighted according to the severity of the issues.&#x20;

The severity levels presented in Cypress Cloud match the Axe Core `impact` level for each rule. This impact classification was developed by the creators of Axe, Deque Systems, to aid in prioritization of accessibility results, by highlighting rules that are likely to reveal the most substantial barriers (such as missing label text) so they can be addressed first.
The severity levels presented in Cypress Cloud match the Axe Core® `impact` level for each rule. This impact classification was developed by the creators of Axe, Deque Systems, to aid in prioritization of accessibility results, by highlighting rules that are likely to reveal the most substantial barriers (such as missing label text) so they can be addressed first.

It's important to note that the Web Content Accessibility Guidelines (WCAG) standards do not have a concept of relative severity of the Success Criteria. For **compliance** purposes, all failures of the Success Criteria count and must be addressed. For **usability** purposes, however, the impact values provided by Axe Core provide excellent guidance about which issues to take up first to bring the most benefit to disabled users of your plaform as quickly as possible.
It's important to note that the Web Content Accessibility Guidelines (WCAG) standards do not have a concept of relative severity of the Success Criteria. For **compliance** purposes, all failures of the Success Criteria count and must be addressed. For **usability** purposes, however, the impact values provided by Axe Core® provide excellent guidance about which issues to take up first to bring the most benefit to disabled users of your plaform as quickly as possible.

## Algorithm

Expand Down
10 changes: 5 additions & 5 deletions docs/accessibility/core-concepts/run-level-reports.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,20 +54,20 @@ An overall score is calculated based on all the Snapshots for a given View. The
The count of elements in a view that failed a Rule, had an inconclusive check for a Rule, or were explicitly ignored by configuration.

- **Failed** elements had at least one accessibility Rule fail during the run. These are considered in your accessibility Score and must be fixed.&#x20;
- **Inconclusive** elements had at least one check marked "incomplete" by Axe Core during the run. These represent checks that couldn't be completed for technical reasons, or items that need manual review to confirm if there is an accessibility issue. These do not affect your score.
- **Inconclusive** elements had at least one check marked "incomplete" by Axe Core® during the run. These represent checks that couldn't be completed for technical reasons, or items that need manual review to confirm if there is an accessibility issue. These do not affect your score.
- **Ignored** elements are those that had either failing or incomplete checks, but were ignored with [CSS selectors in your configuration](/accessibility/configuration/elementfilters). They do not affect your score. Cypress still processes these elements and makes the results available, so that you always know results were was ignored, since these elements are still present in your UI and users will encounter them.

#### Failed rules

Failed rules are broken down for each View with a mini chart displaying Axe Core severity levels, as well as a total count for failed rules. Both of these help you identify at a glance which pages or components have high numbers serious issues.
Failed rules are broken down for each View with a mini chart displaying Axe Core® severity levels, as well as a total count for failed rules. Both of these help you identify at a glance which pages or components have high numbers serious issues.

#### Severity

In Cypress Accessibility, severity levels are mapped to Axe Core's `impact` level for each rule. It's important to note that these levels were created by Deque Systems, the creators of Axe Core, and not a part of the Web Content Accessibility Guidelines themselves. For more details on this, see our [accessibility score documentation](https://github.com/dequelabs/axe-core/blob/develop/doc/issue_impact.md).

## Rules

This section lists the outcome of every Axe Core rule that was enabled at the time of this run.
This section lists the outcome of every Axe Core® rule that was enabled at the time of this run.

<DocsImage
src="/img/accessibility/core-concepts/cypress-accessibility-rules-list.png"
Expand All @@ -81,11 +81,11 @@ Each Rule in the table has the following attributes:
- An icon representing the overall status for the rule, which can be any one of these:
- **Passed** -- every time this rule applied to an element, the element either passed that rule's checks or was intentionally ignored
- **Failed** -- at least one failing element was detected in the run
- **Inconclusive** -- no elements failed this rule, but at least one element was labelled "incomplete" by Axe Core and requires manual review
- **Inconclusive** -- no elements failed this rule, but at least one element was labelled "incomplete" by Axe Core® and requires manual review
- **Inapplicable** -- this rule was enabled during the run, but there were no elements detected that required the rule to run
- **Ignored by config** -- this rule was ignored at the config level for the project and did not run
- The description for the rule
- The Axe Core severity level for the rule
- The Axe Core® severity level for the rule
- The counts of failed, inconclusive, and ignored elements for each rule

## Filtering
Expand Down
18 changes: 14 additions & 4 deletions docs/accessibility/get-started/introduction.mdx
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
---
sidebar_label: Introduction
title: 'Introduction'
description: 'Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core by Deque Systems.'
description: 'Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems.'
sidebar_position: 10
---

# Cypress Accessibility

<AccessibilityAddon />

Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core by Deque Systems.
Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems. This is the documentation page for that product. To learn about the general topic of accessibility testing using the Cypress App, see [our guide on this topic](/app/guides/accessibility-testing).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻


<DocsImage
src="/img/accessibility/get-started/cypress-accessibility-overview.png"
Expand All @@ -26,10 +26,20 @@ Reports are generated using the industry-standard open source [Axe Core](https:/

## Understanding what is tested

Axe Core checks for accessibility violations in your website's user interface, based on common accessibility standards. A violation detected by Axe usually indicates that some users with disabilities would have a problem using or perceiving a feature. An example of a violation would be an image without alternative text provided in the markup, which means the content is not perceivable to visually-impaired users. [Learn more about Axe.](https://github.com/dequelabs/axe-core)
Axe Core® checks for accessibility violations in your website's user interface, based on common accessibility standards. A violation detected by Axe usually indicates that some users with disabilities would have a problem using or perceiving a feature. An example of a violation would be an image without alternative text provided in the markup, which means the content is not perceivable to visually-impaired users. [Learn more about Axe.](https://github.com/dequelabs/axe-core)

Axe reaches a high standard for automated testing, but cannot test every possible aspect of compliance, especially for heavily custom interfaces or aspects requiring human judgment. Please note that a 100% axe score does not mean all possible accessibility errors have been ruled out. [Axe typically detects up to 57% of the issues that would appear in a manual accessibility audit.](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/)

##### Trademark notice
## Highlights

- All steps of all user flows tested in Cypress are captured automatically
- Server-side execution means no disruption to existing test code or practices, and no performance impact from accessibility checks
- Reports are generated at the page or component level, as well as a combined report showing the outcome of all accessibility rules checked in the run
- Debuggable full-page DOM snapshots with highlights are provided for every violation
- Iframes and Shadow DOM are supported out-of-the-box
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This got me thinking - I think it'd be good somewhere to note the relationship between this product and Test Replay. Noting some limitations for example - they have to be on v13+, they have to have Test Replay enabled. The same things that are not supported in Test Replay will not be supported in this product. This might be a good shared snippet. I can work on this if you don't have time btw. https://docs.cypress.io/faq/questions/cloud-faq#Is-everything-captured-and-replayed-in-Test-Replay

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point, feel free to add something about this. Also just helps the premise of how this works so readers can develop some intuition and expectations around it. When I'm through with other changes, I can come back to this if you haven't pushed an update yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to a new "how it works" section where I also moved the Axe Core description

- The Results API allows you to own how your CI pipeline reacts to the accessibility results of the run to implement a policy
- "Ignored" elements are isolated and don't affect your score, but can still be reviewed

### Trademark notice

_Note: DEQUE, DEQUELABS, AXE®, and AXE-CORE® are trademarks of Deque Systems, Inc. in the US and other countries._
6 changes: 6 additions & 0 deletions docs/app/core-concepts/best-practices.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,12 @@ package to use the familiar testing library methods (like `findByRole`,
In particular, if you're looking for more resources to understand how we
recommend you approach testing your components, look to: [Cypress Component Testing](/app/component-testing/get-started).

### Accessibility Testing

Selecting elements with data attributes, text content, or Testing Library locators can each have some different implications for accessibility,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

but none of these approaches is a "complete" test, and you will always need additional, accessibility-specific testing to confirm
your application is working as expected for people with disabilities. If accessibility testing is important you, [see our guide](/app/guides/accessibility-testing) for more details and comparisons of approaches.

## <Icon name="angle-right" /> Assigning Return Values

:::danger
Expand Down
13 changes: 6 additions & 7 deletions docs/app/get-started/why-cypress.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ of your test suite and the quality of your application.

- [Cypress App](/app/core-concepts/open-mode), a free,
[open source](https://github.com/cypress-io/cypress),
locally installed app.
locally installed app for writing and running tests.
- [Cypress Cloud](/cloud/get-started/introduction), a paid service for recording tests, surfacing test results, and providing test analytics.
- [UI Coverage](/ui-coverage/get-started/introduction), a paid add-on providing a visual overview of test coverage across every page and component of your app, offering clear insights into uncovered areas that everyone can understand.
- [Cypress Accessibility](/accessibility/get-started/introduction), a paid add-on providing accessibility checks to ensure your application is accessible to all users.
- [Cypress Accessibility](/accessibility/get-started/introduction), a paid add-on providing accessibility checks, which helps detect barriers for people with disabilities using your application.

Cypress is a robust solution that can improve the quality of your application.

Expand All @@ -43,8 +43,7 @@ Cypress is a robust solution that can improve the quality of your application.
CI Provider, [Cypress Cloud](/cloud/get-started/introduction) can record your test
runs surfacing exactly what happened during the test in [Test Replay](/cloud/features/test-replay).
You'll never have to wonder: _Why did this fail?_
- **Finally:** Add on [Cypress Accessibility](/accessibility/get-started/introduction) to ensure your application is
accessible to all users and [UI Coverage](/ui-coverage/get-started/introduction) to ensure you've tested
- **Finally:** Add on [Cypress Accessibility](/accessibility/get-started/introduction) to get continuous feedback on accessibility issues and regressions, and [UI Coverage](/ui-coverage/get-started/introduction) to ensure you've tested
every part of your application.

## Features
Expand Down Expand Up @@ -210,9 +209,9 @@ it('adds a todo', () => {

### Accessibility Testing

You can write Cypress tests to check the accessibility of your application.
When combined with [Cypress Accessibility](/accessibility/get-started/introduction), insights can be surfaced when
accessibility standards are not met through your testing - with no configuration required.
You can write Cypress tests to check the accessibility of your application, and use plugins to run broad accessibility scans.
When combined with [Cypress Accessibility](/accessibility/get-started/introduction) in Cypress Cloud, insights can be surfaced when specific
accessibility standards are not met through your testing - with no configuration required. See our [Accessibility Testing guide](/app/guides/accessibility-testing) for more details.

```js
it('adds todos', () => {
Expand Down
Loading