Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/accessibility/changelog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,11 @@ sidebar_position: 200

# Changelog

## Week of 3/04/2025
## Week of 3/10/2025

- The [Cypress Accessibility Results API](/accessibility/results-api) has been updated with additional details about accessibility scores and failed element counts at the run level. Also, view-level reporting has been added, breaking out accessibility report data by the pages and components covered in the report. These improvements support more advanced workflows based on the accessibility data in the API, such as alerting a specific team about violations in a part of the app they own, or implementing different standards for different parts of the application.

## Week of 3/03/2025

- We've upgraded our axe-core version to [4.10.2](https://docs.deque.com/axe-release-impact/4.10.2/en/release-notes).
- The axe-core version used to generate a report can now be viewed in the Run Properties tab.
Expand Down
2 changes: 2 additions & 0 deletions docs/accessibility/configuration/axe-core-configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,5 @@ Cypress Accessibility works great with component testing. In addition the rules

Updates to the Axe Core® library are released several times each year. While Cypress does not commit to updating our internal Axe Core® version within a specific timeframe, we will always wait at least 30 days before making an update.
Since some updates introduce new rules or address bugs that change how issues are detected, this buffer is intended to give you time to make changes in your use of our [Results API](/accessibility/results-api) if needed. Axe Core® releases can be tracked [in GitHub](https://github.com/dequelabs/axe-core/releases).

The specific Axe Core® version used by Cypress for a run can be seen in the Properties tab for that run in Cypress Cloud, and is also included in the Results API.
31 changes: 26 additions & 5 deletions docs/accessibility/get-started/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,29 @@ sidebar_custom_props: { 'new_label': true }

# Automated accessibility checks on every test

Instantly visualize, triage, and fix accessibility violations without any additional code or configuration.
Dive deep into each violation with live, fully-rendered DOM snapshots of your application as it appeared during your tests.
Track your team's progress over time with historical scores to monitor improvements and identify regressions.
Looking for a general guide for accessibility testing concepts in Cypress App? Go to the [accessibility testing guide](/app/guides/accessibility-testing).
Cypress Accessibility adds detailed accessibility checks and workflows in Cypress Cloud, with zero impact on test execution.

- Instantly visualize, triage, and fix accessibility violations without any additional code or configuration.
- Dive deep into each violation with live, fully-rendered DOM snapshots of your application as it appeared during your tests.
- Filter out the noise to explore only newly-introduced issues related to specific commits.
- Track your team's progress over time with historical scores to monitor improvements and identify regressions.

<Btn
label="Request trial ➜"
variant="indigo-dark"
className="!mb-2 mr-1"
href="https://www.cypress.io/accessibility?utm_medium=intro-cta&utm_source=docs.cypress.io&utm_content=Request%20trial"
/>
<Btn
label="See a demo"
icon="action-play-small"
className="!mb-2 mr-1"
href="https://www.loom.com/share/eedb1d18c22942d39a5ed2471ae72788?sid=e39f791e-c0b3-4efc-9770-cbfac403c3b9"
/>
<Btn
label=" Explore an example project"
icon="cypress-accessibility-outline"
className="!mb-2 mr-1"
href="https://on.cypress.io/rwa-accessibility-views?utm_source=docs.cypress.io&utm_medium=intro-cta&utm_content=Explore%20an%20example%20project"
/>

Expand All @@ -37,6 +42,8 @@ Looking for a general guide for accessibility testing concepts in Cypress App? G
noBorder={true}
/>

Looking for a general guide for accessibility testing concepts in Cypress App? Go to the [accessibility testing guide](/app/guides/accessibility-testing).

## Get Started

You don't need to make any changes to your code or your test setup. If you record test runs to the Cypress Cloud with Test Replay, you're ready to start using Cypress Accessibility.
Expand All @@ -63,6 +70,20 @@ From there, you can integrate with CI to set your own standards for handling the
</p>
</a>
</li>
<li class="card">
<a
href="/accessibility/guides/detect-changes"
aria-labelledby="card-title-8"
>
<Icon name="chart-line" />
<h3 id="card-title-8">Detect and manage changes</h3>
<p>
Compare reports in detail to review only _new_ accessibility issues, use
our API to react to runs that don't meet your standards, and spot trends
with analytics.
</p>
</a>
</li>
<li class="card">
<a
href="/accessibility/guides/maintain-accessibility"
Expand Down Expand Up @@ -156,7 +177,7 @@ From there, you can integrate with CI to set your own standards for handling the

Cypress Accessibility delivers detailed, actionable accessibility reports directly from your tests, powered by the industry-leading Axe Core® library by Deque Systems. These reports require no extra setup and are automatically generated for every unique state reached during your Cypress tests, whether in end-to-end or component testing.

- **Effortless setup**: Cypress Accessibility utilizes the same capture protocol as [Test Replay](/cloud/features/test-replay) (requires Cypress v13+), so no additional code or configuration is needed.
- **No setup**: Cypress Accessibility utilizes the same capture protocol as [Test Replay](/cloud/features/test-replay) (requires Cypress v13+), so no additional code or configuration is needed.
- **No test slowdown or flakiness**: Server-side execution ensures no impact on test performance or existing code.
- **Comprehensive reporting**: During test execution, unique states—defined by distinct DOM structures—are recorded for end-to-end views and components.
- **DOM snapshots**: Each detected violation is accompanied by a [full-page, inspectable DOM snapshot](/accessibility/core-concepts/inspecting-violation-details), highlighting the exact location and context of the violation.
Expand Down
7 changes: 7 additions & 0 deletions docs/accessibility/get-started/setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Get started with a free trial of Cypress Accessibility. You'll receive a persona

<Btn
label="Request trial ➜"
className="!mb-0"
variant="indigo-dark"
href="https://www.cypress.io/accessibility?utm_medium=get-started-page&utm_source=docs.cypress.io&utm_content=Request%20trial"
/>
Expand Down Expand Up @@ -52,3 +53,9 @@ To add or modify the configuration for your project, navigate to the **App Quali
src="/img/accessibility/configuration/cypress-accessibility-configuration-editor.png"
alt="The Cypress Cloud UI showing the configuration editor"
/>

## 5. Check out our detailed guides

Cypress Accessibility supports many different accessibility automation workflows that might not be obvious at first glance, such as production monitoring of large content-managed websites. There is also an API for programmatic access to the results from each run, and a report comparison feature in Branch Review that allows for reviewing only the new violations discovered in a certain run. The guides walk you through many of the scenarios and use cases supported by Cypress Accessibility in detail so you can see how you might apply the results in your own team.

Learn more about [how to maximize the impact of Cypress Accessibility](/accessibility/guides/introduction) for your organization.
106 changes: 106 additions & 0 deletions docs/accessibility/guides/detect-changes.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
---
sidebar_label: Detect and manage changes
title: 'Detect changes | Cypress Accessibility Documentation'
description: 'Monitor accessibility issues in production by running scheduled Cypress tests against live environments, capturing dynamic content changes, and generating automated reports for a comprehensive accessibility overview.'
sidebar_position: 70
---

<ProductHeading product="accessibility" />

# Detect and manage changes

Cypress Accessibility supports three main paths to observing and acting on changes in the accessibility report for your projects:

- Run comparisons with [Branch Review](/accessibility/core-concepts/comparing-reports)
- The [Results API](/accessibility/results-api) to fail builds or trigger alerts when results don't meet your standards
- [Analytics](/cloud/features/analytics/enterprise-reporting#Cypress-Accessibility) for high-level trend-spotting and analysis

In this guide we'll discuss how these work and some common situations where we might use them together.

## Branch Review

Branch Review helps you spot new issues quickly by comparing accessibility results from two Cypress runs.

- Pinpoint when and where problems have been introduced
- Understand and debug easily with fully rendered page snapshots
- Share links to specific new issues with all the context needed to dive in

### Branch Review use cases

#### Speed up code reviews and deployments

Use Branch review when reviewing a pull request made by another developer or before you create one of your own. You can compare the report from your changed branch with the equivalent report on the main branch. The comparison allows you to see any issues that your changes have improved or fixed completely, as well as net-new issues introduced, and any new instances of existing failures.

Since this reduces the amount of violations to review to just what was affected by the PR code changes, you won't spend any time looking for what's relevant -- and if there is no difference, you'll know your PR does not affect the accessibility score and is safe to merge.

See the main [Accessibility Branch Review docs](/accessibility/core-concepts/comparing-reports) for a video demo of this workflow.

#### Compare reports during local development (without waiting for CI)

You might expect that every Cypress Accessibility report requires a full standard Cypress test run in your Continuous Integration (CI) pipeline.

A common accessibility workflow is to [create small Cypress Cloud recordings from your local development environment](/accessibility/guides/local-development) to preview the effects of any code changes. This allows for fast analysis of just a single test or two, or as much as you need to have confidence your changes are working -- especially if you are actively remediating accessibility issues.

To get a high-quality comparison for a small set of tests, record the tests locally from your main branch, and then on the branch with changes. In Cypress Cloud, you will be able to compare these two runs knowing they cover exactly the same scope.

This completely sidesteps the usual "push, build, run tests, report" cycle you might be used to, and is great when you need fast feedback.

#### See what's changing between scheduled runs

Many teams have some Cypress projects that run on a schedule and target a QA, staging, or production environment. These runs produce Cypress Accessibility reports but are not connected to specific pull requests with code changes. Branch Review works perfectly in these situations, even if all runs are on the same branch.

Comparing reports on a regular schedule can let you see if accessibility issues have been introduced between this week and last week, for example, and then find the specific run where the changes happened. That "diff report" can can be forwarded to engineering or product teams as the basis of a ticket.

### Note

Comparing two runs in detail with Branch Review requires a good understanding of the runs themselves. The comparison is most useful if the runs are testing the same pages and workflows, and if each run fully passed all tests. This means that Cypress saw all the same states and variations of the application in both contexts. Your runs will likely never be 100% perfectly matching, but the closer they are, the better.

## Results API

Cypress Accessibility's [Results API](/accessibility/results-api) provides rich data related to the outcome of any accessibility report for a Cypress run. It supports adding a step in CI to block pull requests or trigger other side effects. To use the Results API, you'll need to [add a step in your CI process](/accessibility/results-api#Installation). This comes with a helper function which will automatically match your CI build with the related run in Cypress Cloud.

Once you receive the results, you can perform any action you need to. Below are some of the common ones customers have told us about, where the results are compared with expectations.

### Results API use cases

#### Fail a build based on new failed accessibility rules

The Results API provides a combined report with all the failing rules included. This means you can easily prepare a list of known, permitted failing rules to compare with the results from any run, and choose to fail only if a previously passing rule has started to fail. This is described in detail in the guide related to [blocking pull requests and setting policies](/accessibility/guides/block-pull-requests).

This concept is simple but powerful because it provides extremely high confidence that any failed build is related to a specific recent code, content, or testing change. Especially if there are a high number of existing failures, the actual number of failed elements detected can vary from run-to-run, especially if existing broken components are reused in new ways.

Rules that are fully passing do not have any of these problems, and for an extreme action like failing a pipeline, it's important to have high confidence that a problem is actually a new issues and can be traced to a specific build. With this confidence in place, the existing failed rules can be addressed one by one to get fully passing.

#### Fail a build based on deviation from a desired score

The Results API includes the specific failed element count and accessibility percentage score for the accessibility report. This makes it easy to set a score threshold and fail a build if the score falls below that expectation. This is especially useful when taking a monitoring approach, or wanting a broader, less specific standard than one based on specific rules.

Changes in percentage score are directly related to changes in the number of elements detected failing accessibility rules throughout the run. Alerting or failing a pipeline for scores going down is a great way to know when you need to jump into Branch Review to understand the underlying causes of the change in score.

#### Alert specific teams about failures they own

In addition to the consolidated accessibility report, accessibility failures are listed per View in the results API. A View is always related to a specific URL pattern or component test, which means in many projects it is possible to connect accessibility issues in the Results API with the team responsible for solving issues in that part of the application.

As an example, customers can use this functionality to trigger Slack webhooks that notify a particular channel of the issues in specific areas. The Results API provides direct links to the Views and to each failed rule within that View, making it easy to forward useful links to the correct places, or jump right into debugging from the CI output.

## Analytics

Analytics are located within the [Cypress Accessibility tab of Enterprise Reporting](/cloud/features/analytics/enterprise-reporting#Cypress-Accessibility) of Cypress Cloud. This provides an overview of accessibility trends across projects, and also supports filtering down to a single project and branch to see where scores have changed over time.

### Analytics use cases

#### Track progress and spot trends

This level of reporting is ideal for documenting progress and spotting unexpected trends. When you see a project's score dropping over the long term, you can explore further with Branch Review to understand what changes are driving the score. You can compare reports before and after any shifts in scores to determine which accessibility issues were introduced.

#### Set goals and minimum standards

Seeing all projects in one place gives a good sense of each project's relative level and helps inform what values you might want to use for comparison when implementing a policy using the Results API.

This also provides useful planning insights, because you can see your historical pace of improvements. If some projects are improving faster than others, you will be able to recognize this and understand how to replicate that success, or create realistic OKRs on a project basis.

## Combine approaches

As you can see above, there are times you may want to move between multiple systems to track down an issue. For example, if the Results API signals that a build on a feature branch has failed your accessibility standard, you might click through to the run itself, and then compare it with the main branch in Branch Review. That might be the end of the journey, but if the main branch is _also_ failing your policy, you can use high-level analytics to see when the score originally changed, and find the specific cause by comparing two runs from around that time. This helps detect where code was force merged, or when content served in the application had accessibility regression unrelated to code changes.

As you use these tools, remember to hit the "Submit Feedback" button any time you see something that could be improved to help you understand what's important and what's changing.
2 changes: 2 additions & 0 deletions docs/accessibility/guides/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Every project will be in a different state when you start to use Cypress Accessi
- Drive remediation and measurable improvement.
- Establish accessibility standards incrementally.
- Celebrate quick wins to build momentum.
- Compare runs to understand what's working and see only new issues.

**For well-maintained projects, we'll focus on:**

Expand All @@ -46,5 +47,6 @@ Passing automated checks is not the end goal—creating a truly usable, accessib
1. [Block pull requests and set policies](/accessibility/guides/block-pull-requests)
1. [Feedback during local development](/accessibility/guides/local-development)
1. [Production monitoring](/accessibility/guides/production-monitoring)
1. [Detect and manage changes](/accessibility/guides/detect-changes)
1. [Maximize coverage](/accessibility/guides/maximize-coverage)
1. [Accessibility automation principles](/accessibility/guides/accessibility-automation)
Loading