diff --git a/docs/accessibility/configuration/elementfilters.mdx b/docs/accessibility/configuration/elementfilters.mdx index f0954ebba0..3eef189eb8 100644 --- a/docs/accessibility/configuration/elementfilters.mdx +++ b/docs/accessibility/configuration/elementfilters.mdx @@ -11,8 +11,8 @@ sidebar_position: 20 :::info -Note: setting `elementFilters` impacts both Accessibility and UI Coverage -reports. Nesting this property under an `accessibility` or `uiCoverage` key is +**Note:** setting `elementFilters` impacts both Accessibility and UI Coverage +reports if set at the root of the configuration. Nesting this property under an `accessibility` or `uiCoverage` key is supported, if you need to split them up. ::: diff --git a/docs/ui-coverage/configuration/_category_.json b/docs/ui-coverage/configuration/_category_.json index ff6e7ca250..7f8cb5b6d7 100644 --- a/docs/ui-coverage/configuration/_category_.json +++ b/docs/ui-coverage/configuration/_category_.json @@ -1,4 +1,4 @@ { "label": "Configuration", - "position": 30 + "position": 40 } diff --git a/docs/ui-coverage/configuration/attributefilters.mdx b/docs/ui-coverage/configuration/attributefilters.mdx index 415e47248c..6fc8e4b238 100644 --- a/docs/ui-coverage/configuration/attributefilters.mdx +++ b/docs/ui-coverage/configuration/attributefilters.mdx @@ -8,13 +8,15 @@ sidebar_label: attributeFilters -UI Coverage has logic that automatically [identifies](/ui-coverage/core-concepts/element-identification) and [groups](/ui-coverage/core-concepts/element-grouping) elements based on their appearance and structure in the DOM. +UI Coverage [identifies](/ui-coverage/core-concepts/element-identification) and [groups](/ui-coverage/core-concepts/element-grouping) elements based on their attributes and structure in the DOM. However, some attributes may be auto-generated, dynamic, or unrepresentative, leading to inaccurate identification or grouping. The `attributeFilters` configuration property allows you to **exclude** specific attributes or patterns that should not be used for these purposes. -Sometimes, an element may have attributes that are auto-generated or otherwise not representative of the element that Cypress's UI Coverage algorithm uses for identification and grouping. This can cause the same element to be identified as multiple different elements, or multiple different elements to be identified as the same element. +By using `attributeFilters`, you can ensure UI Coverage selects more appropriate identifiers, leading to cleaner and more accurate coverage reports. -The `attributeFilters` configuration property allows users to specify patterns for attributes and their values that should **not** be used for identifying and grouping elements, thereby allowing UI Coverage to find more suitable identifiers for the impacted elements. +## Why use attribute filters? -For every attribute that an element has, the first `attributeFilters` rule for which the `attribute` property matches the attribute's name and the `value` property matches the attribute's value, the `include` value is used to determine whether or not the attribute will be used for element identification and grouping. Attributes that do not match any rules are included by default. +- **Handling library-specific attributes**: Attributes generated by libraries may not represent the element's purpose and should be ignored. +- **Improving grouping accuracy**: By filtering out irrelevant attributes, you ensure similar elements are grouped correctly. +- **Streamlining reports**: Eliminating noisy attributes reduces clutter in coverage reports, making them easier to interpret and act upon. ## Syntax @@ -32,49 +34,24 @@ For every attribute that an element has, the first `attributeFilters` rule for w } ``` -### attributeFilters - -_Optional._ Object\[] - -An array of objects used to determine whether or not an attribute will be used for element identification and grouping. _**Each object can have the following properties:**_ - -### attribute - -_Required._ String (Regex) - -Used to match the attribute names. - -### value - -_Optional._ String (Regex) - -_Default_: `.*` (matches any value). - -Used to match the attribute values. - -### include +### Options -_Optional._ Boolean - -_Default:_ `true` +For every attribute that an element has, the first `attributeFilters` rule for which the `attribute` property matches the attribute's name and the `value` property matches the attribute's value, the `include` value is used to determine whether or not the attribute will be used for element identification and grouping. Attributes that do not match any rules are included by default. -Whether or not a matched attribute should be used for element identification and grouping. +| Option | Required | Default | Description | +| ----------- | -------- | ------- | ---------------------------------------------------------------------- | +| `attribute` | Required | | A regex string to match attribute names | +| `value` | Optional | `.*` | A regex string to match attribute values | +| `include` | Optional | `true` | A boolean to specify whether the matched attribute should be included. | ## Examples ### Excluding common auto-generated id values -#### Config - ```json { "uiCoverage": { "attributeFilters": [ - { - "attribute": "id", - "value": "sizzle.*", - "include": false - }, { "attribute": "id", "value": ":r.*:", @@ -89,12 +66,12 @@ Whether or not a matched attribute should be used for element identification and ```xml - - + + ``` -#### Elements shown in UI +#### Elements shown in UI Coverage ``` [name="my-button"] @@ -105,8 +82,6 @@ Whether or not a matched attribute should be used for element identification and ### Excluding auto-generated attribute names -#### Config - ```json { "uiCoverage": { @@ -135,9 +110,38 @@ Whether or not a matched attribute should be used for element identification and ``` -#### Elements shown in UI +#### Elements shown in UI Coverage ``` [ng-include-me="my-button"] :nth-child(2) ``` + +### Ignoring dynamic attributes for accurate grouping + +```json +{ + "uiCoverage": { + "attributeFilters": [ + { + "attribute": "data-cy", + "value": "user-\\d+", + "include": false + } + ] + } +} +``` + +#### HTML + +```xml + + +``` + +#### Elements shown in UI Coverage + +``` +.user (2 instances) +``` diff --git a/docs/ui-coverage/configuration/elementfilters.mdx b/docs/ui-coverage/configuration/elementfilters.mdx index ee2a13261d..2cd66f37f5 100644 --- a/docs/ui-coverage/configuration/elementfilters.mdx +++ b/docs/ui-coverage/configuration/elementfilters.mdx @@ -8,15 +8,19 @@ sidebar_label: elementFilters +By default, every interactive and visible element is included in UI Coverage. The `elementFilters` property allows you to specify selectors for elements that should be excluded from these reports. This configuration is particularly useful for removing irrelevant or non-actionable elements to streamline coverage analysis. + +## Why use element filters? + :::info -Note: setting `elementFilters` impacts both Accessibility and UI Coverage -reports. Nesting this property under an `accessibility` or `uiCoverage` key is +**Note:** setting `elementFilters` impacts both Accessibility and UI Coverage +reports if set at the root of the configuration. Nesting this property under an `accessibility` or `uiCoverage` key is supported, if you need to split them up. ::: -By default, every interactive and visible element is included in UI Coverage. The `elementFilters` property allows you to specify selectors for elements that should be excluded from UI Coverage. - -For every element considered interactive and visible by UI Coverage, the first `elementFilters` rule for which the `selector` property matches the element is used to either include or exclude the element based on the `include` value. Elements that do not match any rules are included by default. +- **Excluding Irrelevant Elements**: Decorative icons, marketing banners, or tooltips that do not require testing can clutter reports. Use elementFilters to exclude them. +- **Ignoring 3rd Party Components**: Libraries or widgets that are not part of your application logic can be excluded from coverage reports. +- **Streamlining Reports**: Reducing noise by filtering out non-essential elements makes coverage reports more actionable. ## Syntax @@ -31,28 +35,14 @@ For every element considered interactive and visible by UI Coverage, the first ` } ``` -### elementFilters - -_Optional._ Object\[] - -An array of objects used to specify elements to exclude from UI Coverage. _**Each object can have the following properties:**_ - -### selector - -_Required._ String (CSS Selector) - -A CSS selector used to identify elements. +### Options -- Supports all standard CSS selector syntax including IDs, classes, attributes, and combinators -- The selector matches exactly what is specified - for example, `#container` matches the container element itself, while `#container *` matches all elements within the container +For every element considered interactive and visible by UI Coverage, the first `elementFilters` rule for which the selector property matches the element is used to either include or exclude the element based on the `include` value. Elements that do not match any rules are included by default. -### include - -_Optional._ Boolean - -_Default:_ `true` - -A boolean that represents whether or not a matched element should be included in UI Coverage. +| Option | Required | Default | Description | +| ---------- | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `selector` | Required | | A CSS selector to identify elements. Supports standard CSS selector syntax, including IDs, classes, attributes, and combinators. | +| `include` | Optional | `true` | A boolean indicating whether the matched elements should be included in UI Coverage. | ## Examples @@ -80,7 +70,7 @@ A boolean that represents whether or not a matched element should be included in ``` -#### Elements shown in UI +#### Elements shown in UI Coverage ``` #button-1 @@ -116,7 +106,7 @@ A boolean that represents whether or not a matched element should be included in ``` -#### Elements shown in UI +#### Elements shown in UI Coverage ``` #start @@ -124,7 +114,7 @@ A boolean that represents whether or not a matched element should be included in --- -### Including only elements in a container +### Including only elements in a specific container #### Config @@ -161,3 +151,67 @@ A boolean that represents whether or not a matched element should be included in ``` #name ``` + +--- + +### Excluding Elements by Attribute + +#### Config + +```json +{ + "elementFilters": [ + { + "selector": "[data-role='decorative']", + "include": false + } + ] +} +``` + +```xml + + + + +``` + +#### Elements shown in UI Coverage + +``` +[data-role="primary"] +``` + +### Excluding dynamic elements by pattern + +#### Config + +```json +{ + "elementFilters": [ + { + "selector": "[class^='auth']", + "include": false + } + ] +} +``` + +#### HTML + +```xml + + + + +``` + +#### Elements shown in UI Coverage + +``` +.cancel +``` diff --git a/docs/ui-coverage/configuration/elementgroups.mdx b/docs/ui-coverage/configuration/elementgroups.mdx index 3d2cc7800c..59454a9676 100644 --- a/docs/ui-coverage/configuration/elementgroups.mdx +++ b/docs/ui-coverage/configuration/elementgroups.mdx @@ -8,9 +8,14 @@ sidebar_label: elementGroups -UI Coverage has logic that automatically [groups](/ui-coverage/core-concepts/element-grouping) elements based on their appearance and structure in the DOM. +UI Coverage provides logic to automatically [group](/ui-coverage/core-concepts/element-grouping) elements based on their structure in the DOM. However, there are scenarios where you may want to customize these groups to better align with your application's functionality or testing requirements. The `elementGroups` configuration allows you to define custom logic for grouping elements, improving coverage clarity and simplifying analysis. -For every element considered by UI Coverage, the first `elementGroup` rule for which the `selector` property matches the element is used to group the element. Elements that do not match any rules are grouped by the default UI Coverage element grouping rules. +## Why use element groups? + +- **Improve Grouping Accuracy**: Ensure elements with shared attributes but different roles are correctly grouped, avoiding misclassification. +- **Simplify Test Coverage Reports**: Grouping similar elements, like navigation buttons or list items, reduces clutter in reports and provides a more concise view of test coverage. +- **Highlight Key Areas**: Use meaningful group names to draw attention to critical application areas, such as form controls. +- **Streamline Dynamic Elements**: Consolidate dynamic or repeated elements, such as items in a carousel or list, into a single logical group. ## Syntax @@ -27,23 +32,14 @@ For every element considered by UI Coverage, the first `elementGroup` rule for w } ``` -### elementGroups - -_Optional._ Object\[] - -An array of objects used to group elements within UI Coverage. _**Each object can have the following properties:**_ - -### selector +### Options -_Required._ String (CSS Selector) +For every element considered by UI Coverage, the first `elementGroup` rule for which the `selector` property matches the element is used to group the element. Elements that do not match any rules are grouped by the default UI Coverage [element grouping rules](/ui-coverage/core-concepts/element-grouping). -Used to match and group elements. - -### name - -_Optional._ String - -A human-readable name for the group, shown in the elements list. +| Option | Required | Default | Description | +| ---------- | -------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `selector` | Required | | A CSS selector to identify elements. Supports standard CSS selector syntax, including IDs, classes, attributes, and combinators. | +| `name` | Optional | `selector` | A human-readable name for the group, displayed in UI Coverage reports. | ## Examples @@ -117,7 +113,7 @@ A human-readable name for the group, shown in the elements list. --- -### Groups form controls by wrapped input +### Groups form controls with labels #### Config @@ -152,3 +148,71 @@ A human-readable name for the group, shown in the elements list. ``` Animal Option (4 instances) ``` + +### Groups dynamic elements + +#### Config + +```json +{ + "uiCoverage": { + "elementGroups": [ + { + "selector": "[id^='product']" + } + ] + } +} +``` + +#### HTML + +```xml + + + + + +``` + +#### Elements shown in UI + +``` +[id^='product'] (3 instances) +``` + +### Giving groups custom names + +Sometimes you may want to group elements by a common attribute but give the group a more descriptive name. In the following example, we group buttons with IDs starting with `listbox-button-` and name the group `Add Button`. + +#### Config + +```json +{ + "uiCoverage": { + "elementGroups": [ + { + "selector": "[id^='listbox-button-']", + "name": "Add Button" + } + ] + } +} +``` + +#### HTML + +```xml + + + + + + +``` + +#### Elements shown in UI + +``` +Add Button (4 instances) +``` diff --git a/docs/ui-coverage/configuration/elements.mdx b/docs/ui-coverage/configuration/elements.mdx index 716d50f5a2..f31da816e7 100644 --- a/docs/ui-coverage/configuration/elements.mdx +++ b/docs/ui-coverage/configuration/elements.mdx @@ -8,13 +8,15 @@ sidebar_label: elements -UI Coverage has logic that automatically [identifies](/ui-coverage/core-concepts/element-identification) elements based on their appearance and structure in the DOM. +UI Coverage provides automatic logic to [identify](/ui-coverage/core-concepts/element-identification) elements based on their DOM structure. However, in complex applications, elements may lack stable identifiers or have dynamic attributes that vary across snapshots, leading to incorrect identification. The `elements` configuration allows you to specify selectors to uniquely identify elements, ensuring consistency across snapshots and simplifying your coverage reports. -Elements may have different attributes and/or attribute values across different snapshots, which can cause the same element to be identified as multiple different elements by UI Coverage. The `elements` configuration property allows users to specify selectors to uniquely identify elements, even when they lack stable identifiers across snapshots. +The `elements` configuration is used as the element's identity if **only one element** per snapshot matches. If there are multiple matches in the same snapshot, this configuration is ignored and the default [identify](/ui-coverage/core-concepts/element-identification) strategy is used. -For every element considered by UI Coverage, the first `elements` rule for which the `selector` property matches the element is used to identify the element. Elements that do not match any rules are identified by the default UI Coverage element identification rules. +## Why use elements configuration? -If multiple elements on the same snapshot match the same rule, then that rule cannot uniquely identify both of them. In this case, the rule is not used to identify either element, and the subsequent rules, and then the default rules, are used. +- **Handle Dynamic Attributes**: Ensure elements are consistently identified across snapshots, even when attributes change. +- **Ensure Unique Identification**: Use custom selectors to uniquely identify elements across snapshots that lack unique attributes or have dynamic values, avoiding misclassification. +- **Simplify Debugging**: Assign human-readable names to elements to make reports more interpretable and debugging easier. ## Syntax @@ -31,27 +33,20 @@ If multiple elements on the same snapshot match the same rule, then that rule ca } ``` -### elements +### Options -_Optional._ Object\[] +For every element considered by UI Coverage, the first applicable rule, determined by a match with the `selector` property, is used for identification. Elements that do not match any rules are identified by the default UI Coverage [element identification rules](/ui-coverage/core-concepts/element-identification). -An array of objects used to specify selectors in order to uniquely identify elements for UI Coverage. _**Each object can have the following properties:**_ +If multiple elements within the same snapshot satisfy the same rule, the rule cannot uniquely identify these elements. In such cases, the rule is bypassed, and either subsequent rules or the default element identification logic are applied. -### selector - -_Required._ String (CSS Selector) - -Used to match and identify elements. - -#### name - -_Optional._ String - -A human-readable name for the element, shown in the elements list. +| Option | Required | Default | Description | +| ---------- | -------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `selector` | Required | | A CSS selector to identify elements. Supports standard CSS selector syntax, including IDs, classes, attributes, and combinators. | +| `name` | Optional | `selector` | A human-readable name for the element, displayed in UI Coverage reports. | ## Examples -### Identifies elements by selector +### Identify elements by dynamic selector across snapshots #### Config @@ -73,19 +68,19 @@ A human-readable name for the element, shown in the elements list.
- +
- +
``` -#### Elements shown in UI +#### Elements shown in UI Coverage ``` #my-form [id^="dropdown"] @@ -93,7 +88,7 @@ A human-readable name for the element, shown in the elements list. --- -### Does not identify elements when multiple match the selector +### Identify elements with human-readable names #### Config @@ -102,7 +97,8 @@ A human-readable name for the element, shown in the elements list. "uiCoverage": { "elements": [ { - "selector": "#my-form [id^='dropdown']" + "selector": "#ui-popover-button", + "name": "Help Popover" } ] } @@ -112,18 +108,13 @@ A human-readable name for the element, shown in the elements list. #### HTML ```xml - -
- - -
+ ``` -#### Elements shown in UI +#### Elements shown in UI Coverage ``` -#my-form #dropdown-1 -#my-form #dropdown-2 +Help Popover ``` diff --git a/docs/ui-coverage/configuration/overview.mdx b/docs/ui-coverage/configuration/overview.mdx index 405e8a00e6..541347badf 100644 --- a/docs/ui-coverage/configuration/overview.mdx +++ b/docs/ui-coverage/configuration/overview.mdx @@ -9,20 +9,25 @@ sidebar_position: 10 -Configuration allows you to customize and fine-tune UI Coverage in Cypress to suit specific needs and scenarios. While UI Coverage is designed to work seamlessly out of the box, there are instances where custom configuration may be necessary to address unique application structures, testing requirements, or edge cases. This section guides you through setting configuration to enhance the functionality and accuracy of UI Coverage. +Configuration enables you to customize and fine-tune UI Coverage in Cypress to suit the unique needs of your application. While UI Coverage is designed to work seamlessly out of the box, there are scenarios where fine-tuning may be necessary—such as dealing with dynamic attributes, filtering out irrelevant elements, or grouping related components. These guides explain how to configure UI Coverage effectively to improve accuracy and usability. **Note**: By default, setting configuration is limited to Admin users. At your request, this can be changed to allow setting config by all users. Reach out to your Cypress point-of-contact if you would like to change this. -## Setting Configuration +## Setting configuration -To add or modify the configuration for your project, navigate to the "App Quality" tab in your project settings. +To add or modify the configuration for your project: + +1. Navigate to the **App Quality** tab in your project settings on Cypress Cloud. +2. Use the configuration editor to add or edit confiuration in JSON format. -You can use the provided editor to write configuration in JSON format. A complete configuration with all available options looks as follows: +## Configuration options + +A complete configuration with all available options looks as follows: ```json { @@ -67,13 +72,29 @@ You can use the provided editor to write configuration in JSON format. A complet } ``` -Note that the root-level properties `elementFilters` and `viewFilters` apply to both UI Coverage and Accessibility. These properties can also be nested under `uiCoverage` or `accessibility` to apply to only that product, with nested configuration taking precedence over root-level configuration. The `views` property also applies to both products but cannot be nested. +## Configuration scope + +The root-level properties `elementFilters` and `viewFilters` apply to both UI Coverage and Accessibility. These properties can also be nested under `uiCoverage` or `accessibility` to apply to only that product, with nested configuration taking precedence over root-level configuration. + +The `views` property applies to UI Coverage and Accessibility but cannot be nested. ## Viewing Configuration for a Run -You can view configuration information for each run in the Properties tab, as shown below. This is the configuration set for the project at the start of the run. +You can review the configuration used during a specific run by checking the **Properties** tab, as shown below. This displays the configuration as it was applied at the start of the run. + +## Next steps + +Check out the following configuration guides. + +- [Attribute Filters](/ui-coverage/configuration/attributefilters) +- [Element Filters](/ui-coverage/configuration/elementfilters) +- [Element Groups](/ui-coverage/configuration/elementgroups) +- [Elements](/ui-coverage/configuration/elements) +- [Significant Attributes](/ui-coverage/configuration/significantattributes) +- [View Filters](/ui-coverage/configuration/viewfilters) +- [Views](/ui-coverage/configuration/views) diff --git a/docs/ui-coverage/configuration/significantattributes.mdx b/docs/ui-coverage/configuration/significantattributes.mdx index d6ada3bf26..b627a5183e 100644 --- a/docs/ui-coverage/configuration/significantattributes.mdx +++ b/docs/ui-coverage/configuration/significantattributes.mdx @@ -8,11 +8,13 @@ sidebar_label: significantAttributes -UI Coverage has logic that automatically [identifies](/ui-coverage/core-concepts/element-identification) and [groups](/ui-coverage/core-concepts/element-grouping) elements based on their appearance and structure in the DOM. +UI Coverage provides logic to automatically [identify](/ui-coverage/core-concepts/element-identification) and [group](/ui-coverage/core-concepts/element-grouping) elements based on their attributes and DOM structure. However, your application may rely on custom attributes for identification that are not prioritized by default in UI Coverage. The `significantAttributes` configuration allows you to specify these attributes, ensuring accurate identification and grouping. -Sometimes, your project may use custom attributes to identify elements that are not considered "significant" to the identification and grouping rules used by UI Coverage by default. This can cause UI Coverage to identify and group elements incorrectly, as it does not use the most meaningful attributes available. +## Why use significant attributes? -The attributes listed in `significantAttributes` are considered "significant" and are prioritized above the default significant attributes, but below `data-cy-ui-id` and `data-cy-ui-group`, for the purpose of identification and grouping. +- **Handle Custom Identifiers**: Prioritize custom attributes like `data-custom-id` that are essential for identifying elements in your application. +- **Avoid Misclassification**: In cases where elements share generic attributes like class or name, prioritizing more specific attributes prevents incorrect grouping. +- **Streamline Dynamic Elements**: Attributes that change dynamically can disrupt accurate identification. Specifying significant attributes ensures stability. ## Syntax @@ -26,11 +28,13 @@ The attributes listed in `significantAttributes` are considered "significant" an } ``` -### significantAttributes +### Options -_Optional._ String\[] +The attributes listed in `significantAttributes` are considered "significant" and are prioritized above the default significant attributes, but below `data-cy-ui-id` and `data-cy-ui-group`, for the purpose of identification and grouping. -An array of attribute names that should be considered "significant" for the purpose of identification and grouping in UI Coverage. +| Option | Required | Default | Description | +| ----------------------- | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------- | +| `significantAttributes` | Required | | An array of attribute names to prioritize for element identification and grouping.Supports any valid HTML attribute names. | ## Examples @@ -41,7 +45,64 @@ An array of attribute names that should be considered "significant" for the purp ```json { "uiCoverage": { - "significantAttributes": ["data-custom-id"] + "significantAttributes": ["data-role"] + } +} +``` + +#### HTML + +```xml +
+ + + +
+``` + +#### Elements shown in UI Coverage + +``` +[data-role="email"] +[data-role="interests"] (2 instances) +``` + +### Prioritize stable attributes over dynamic + +#### Config + +```json +{ + "uiCoverage": { + "significantAttributes": ["data-type"] + } +} +``` + +#### HTML + +```xml +
    +
  • Search
  • +
  • Filter
  • +
+``` + +#### Elements shown in UI Coverage + +``` +[data-type="search"] +[data-type="filter"] +``` + +### Define aria attributes as significant + +#### Config + +```json +{ + "uiCoverage": { + "significantAttributes": ["aria-label"] } } ``` @@ -49,16 +110,13 @@ An array of attribute names that should be considered "significant" for the purp #### HTML ```xml - - - - - + + ``` -#### Elements shown in UI +#### Elements shown in UI Coverage ``` -[data-custom-id="foo"] -[data-custom-id="bar"] (2 instances) +[aria-label="Close"] +[data-cy="submit"] ``` diff --git a/docs/ui-coverage/configuration/viewfilters.mdx b/docs/ui-coverage/configuration/viewfilters.mdx index e69a235728..929535c654 100644 --- a/docs/ui-coverage/configuration/viewfilters.mdx +++ b/docs/ui-coverage/configuration/viewfilters.mdx @@ -8,14 +8,18 @@ sidebar_label: viewFilters +By default, every URL visited within a test run is included in UI Coverage. However, not all URLs are relevant for your coverage analysis. The `viewFilters` property allows you to specify patterns for URLs that should be excluded, ensuring your coverage reports focus on meaningful parts of your application. Excluding a URL also excludes all links pointing to that URL. + +## Why use view filters? + :::info -Note: setting `viewFilters` impacts both Accessibility and UI Coverage reports. -This cannot be nested. +**Note:** setting `viewFilters` impacts both Accessibility and UI Coverage reports. This cannot be nested. ::: -By default, every URL that is visited within a run is included in UI Coverage. The `viewFilters` property allows you to specify URL patterns for URLs that should be excluded from UI Coverage. Excluding a URL also excludes all links to that URL. - -For every URL visited by Cypress in a run and every link element found, the first `viewFilters` rule for which the `pattern` property matches the URL is used to either include or exclude the URL based on the `include` value. URLs that do not match any rules are included by default. +- **Exclude Third-Party URLs**: If your application integrates with third-party services, you might want to exclude their URLs from coverage analysis. +- **Exclude Admin Pages**: URLs that are not part of the user-facing application, such as admin pages, could be excluded from coverage reports. +- **Reduce Noise**: Certain URLs visited during tests (e.g., error pages or redirects) may not represent meaningful user flows. Exclude these to clean up your reports. +- **Optimize Performance**: Filtering out irrelevant URLs reduces the amount of data processed, speeding up analysis and improving report readability. ## Syntax @@ -30,25 +34,14 @@ For every URL visited by Cypress in a run and every link element found, the firs } ``` -### viewFilters - -_Optional._ Object\[] - -An array of objects used to specify URLs to exclude from UI Coverage. _**Each object can have the following properties:**_ - -### pattern - -_Required._ String (URL Pattern) - -Used to match URLs. Uses [URL Pattern API](https://developer.mozilla.org/en-US/docs/Web/API/URL_Pattern_API) syntax. - -### include - -_Optional._ Boolean +### Options -_Default:_ `true` +For every URL visited and link element found, the first `viewFilters` rule for which the `pattern` matches the URL is used to either include or exclude the URL based on the `include` value. URLs that do not match any rules are included by default. -A boolean that represents whether or not a matched URL should be included in UI Coverage. +| Option | Required | Default | Description | +| --------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | +| `pattern` | Required | | A string that matches URLs using [URL Pattern API](https://developer.mozilla.org/en-US/docs/Web/API/URL_Pattern_API) syntax. | +| `include` | Optional | `true` | A boolean that determines whether matching URLs should be included in coverage. | ## Examples @@ -75,7 +68,7 @@ https://cypress.io/home https://cypress.io/about ``` -#### Views shown in UI +#### Views shown in UI Coverage ``` https://cypress.io/home @@ -119,3 +112,36 @@ https://cypress.io/login https://cypress.io/dashboards https://cypress.io/dashboards/* ``` + +### Excluding error pages + +#### Config + +```json +{ + "viewFilters": [ + { + "pattern": "http*://*/404", + "include": false + }, + { + "pattern": "http*://*/error/*", + "include": false + } + ] +} +``` + +#### Visited URLs + +``` +https://cypress.io/home +https://cypress.io/404 +https://cypress.io/error/500 +``` + +#### Views shown in UI Coverage + +``` +https://cypress.io/home +``` diff --git a/docs/ui-coverage/configuration/views.mdx b/docs/ui-coverage/configuration/views.mdx index 0c6112e83d..24d2b2d4ce 100644 --- a/docs/ui-coverage/configuration/views.mdx +++ b/docs/ui-coverage/configuration/views.mdx @@ -8,17 +8,20 @@ sidebar_label: views +UI Coverage automatically groups certain URL patterns to create [views](/ui-coverage/core-concepts/views). However, for URLs that are not automatically grouped (e.g., `/users/alice` and `/users/bob` are not automatically grouped into `/users/*`), the `views` property allows you to specify custom URL patterns to define views. This configuration enhances the clarity and organization of your coverage reports. + +The `groupBy` property of a view definition allows you to create multiple views with a single URL pattern, grouping URLs by the specified named parameter(s). + +## Why use views? + :::info -Note: setting views impacts both Accessibility and UI Coverage reports. +**Note:** setting `views` impacts both Accessibility and UI Coverage reports. Nesting this property under an `accessibility` or `uiCoverage` key is supported, if you need to split them up. ::: -UI Coverage automatically groups certain URL patterns to create [views](/ui-coverage/core-concepts/views). For URLs that are not automatically grouped (e.g. `/users/alice` and `/users/bob` are not automatically grouped into `/users/*`), the `views` property allows you to specify URL patterns that represent views. - -Each URL pattern provided defines a view that is made up of all URLs that match the pattern. The first pattern that a given URL matches is used as its view. If a URL doesn't match any of the patterns, it is grouped by the default UI Coverage grouping rules, if possible. - -The `groupBy` property of a view definition allows you to create multiple views with a single URL pattern, grouping URLs by the specified named parameter(s). +- **Group Dynamic URLs**: Group URLs with dynamic path parameters (e.g., `/users/alice` and `/users/bob`) that are not ids or uuids into a single view. +- **Organize by Query Parameters**: Create views based on query parameters to group URLs where query strings are important to the context of the page. ## Syntax @@ -35,23 +38,14 @@ The `groupBy` property of a view definition allows you to create multiple views } ``` -### views - -_Optional._ Object\[] - -An array of objects used to define views within UI Coverage. _**Each object can have the following properties:**_ - -### pattern - -_Required._ String (URL Pattern) - -A URL pattern that groups matching URLs into a single view. This utilizes the [URL Pattern API](https://developer.mozilla.org/en-US/docs/Web/API/URL_Pattern_API) syntax for matching. - -### groupBy +### Options -_Optional._ String\[] +The first pattern that a given URL matches is used as its view. If a URL doesn't match any of the patterns, it is grouped by the default [view grouping rules](/ui-coverage/core-concepts/views), if possible. -An array of named URL parameters used to further subdivide the URLs into multiple views within the same pattern. When `groupBy` is defined, each URL that matches the specified `pattern` is further analyzed to create separate views based on the specified parameters. This is particularly useful in dynamic applications where URLs represent different user interactions or data segments. +| Option | Required | Default | Description | +| --------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `pattern` | Required | | A URL pattern to group matching URLs into a single view. Uses [URL Pattern API](https://developer.mozilla.org/en-US/docs/Web/API/URL_Pattern_API) syntax. | +| `groupBy` | Optional | | An array of named URL parameters used to further subdivide the URLs into multiple views within the same pattern. This is useful for dynamic applications where URLs represent different user interactions or data segments. | ## Examples @@ -63,7 +57,7 @@ An array of named URL parameters used to further subdivide the URLs into multipl { "views": [ { - "pattern": "https://cypress.io/users/*" + "pattern": "https://www.my-app.com/users/*" } ] } @@ -72,16 +66,17 @@ An array of named URL parameters used to further subdivide the URLs into multipl #### Visited URLs ``` -https://cypress.io/users/alice -https://cypress.io/users/alice?foo=bar -https://cypress.io/users/bob -https://cypress.io/users/bob#baz +https://www.my-app.com/users/alice +https://www.my-app.com/users/bob +https://www.my-app.com/users/bob#settings +https://www.my-app.com/users?assigned=true ``` #### Views shown in UI ``` -https://cypress.io/users/* +www.my-app.com/users +www.my-app.com/users/* ``` --- @@ -94,7 +89,7 @@ https://cypress.io/users/* { "views": [ { - "pattern": "https://cypress.io/users/:name" + "pattern": "https://www.my-app.com/users/:name" } ] } @@ -103,21 +98,22 @@ https://cypress.io/users/* #### Visited URLs ``` -https://cypress.io/users/alice -https://cypress.io/users/alice?foo=bar -https://cypress.io/users/bob -https://cypress.io/users/bob#baz +https://www.my-app.com/users/alice +https://www.my-app.com/users/bob +https://www.my-app.com/users/bob#settings +https://www.my-app.com/users?assigned=true ``` #### Views shown in UI ``` -https://cypress.io/users/:name +www.my-app.com/users +www.my-app.com/users/:name ``` --- -### Grouping URLs by named path parameters +### Group URLs by named parameters #### Config @@ -125,7 +121,7 @@ https://cypress.io/users/:name { "views": [ { - "pattern": "https://cypress.io/analytics/:type/:id", + "pattern": "https://www.my-app.com/analytics/:type/:id", "groupBy": ["type"] } ] @@ -135,22 +131,22 @@ https://cypress.io/users/:name #### Visited URLs ``` -https://cypress.io/analytics/performance/foo -https://cypress.io/analytics/performance/bar -https://cypress.io/analytics/usage/foo -https://cypress.io/analytics/usage/bar +https://www.my-app.com/analytics/performance/amara +https://www.my-app.com/analytics/performance/harper +https://www.my-app.com/analytics/usage/amara +https://www.my-app.com/analytics/usage/harper ``` #### Views shown in UI ``` -https://cypress.io/analytics/performance/:id -https://cypress.io/analytics/usage/:id +www.my-app.com/analytics/performance/:id +www.my-app.com/analytics/usage/:id ``` --- -### Grouping URLs by named query parameters +### Group URLs by named query parameters #### Config @@ -158,8 +154,8 @@ https://cypress.io/analytics/usage/:id { "views": [ { - "pattern": "https://cypress.io/app?*name=:name{&*}?#*", - "groupBy": ["name"] + "pattern": "https://www.my-app.com/home?*status=:status{&*}?#*", + "groupBy": ["status"] } ] } @@ -168,22 +164,22 @@ https://cypress.io/analytics/usage/:id #### Visited URLs ``` -https://cypress.io/app?foo=1&name=hello -https://cypress.io/app?name=hello&bar=2 -https://cypress.io/app?foo=3&name=world&bar=4 -https://cypress.io/app?foo=5&name=world#baz +https://www.my-app.com/home?page=1&status=done +https://www.my-app.com/home?status=done&group=2 +https://www.my-app.com/home?tag=trip&status=new&group=4 +https://www.my-app.com/home?tag=trip&status=new#statusView ``` #### Views shown in UI ``` -https://cypress.io/app?name=hello -https://cypress.io/app?name=world +www.my-app.com/home?status=done +www.my-app.com/home?status=new ``` --- -### Grouping URLs by path parameters to ignore dynamic hosts +### Grouping URLs across subdomains #### Config @@ -191,7 +187,7 @@ https://cypress.io/app?name=world { "views": [ { - "pattern": "https://*.cypress.io/:path*", + "pattern": "https://*.my-app.com/:path*", "groupBy": ["path"] } ] @@ -201,16 +197,17 @@ https://cypress.io/app?name=world #### Visited URLs ``` -https://sub1.cypress.io/home -https://sub2.cypress.io/home -https://sub3.cypress.io/profile -https://sub4.cypress.io/profile/edit +https://staging1.my-app.com/home +https://staging2.my-app.com/home +https://www.my-app.com/home +https://staging1.my-app.com/profile +https://www.my-app.com/profile/edit ``` #### Views shown in UI ``` -https://*.cypress.io/home -https://*.cypress.io/profile -https://*.cypress.io/profile/edit +https://*.my-app.com/home +https://*.my-app.com/profile +https://*.my-app.com/profile/edit ``` diff --git a/docs/ui-coverage/core-concepts/element-grouping.mdx b/docs/ui-coverage/core-concepts/element-grouping.mdx index e2e7ccf204..0b00708f7c 100644 --- a/docs/ui-coverage/core-concepts/element-grouping.mdx +++ b/docs/ui-coverage/core-concepts/element-grouping.mdx @@ -9,15 +9,44 @@ sidebar_position: 20 -When multiple elements on a page have the same behavior, testing each element individually is not always required to ensure confidence and coverage. In many cases, interacting with just one of these elements is sufficient to verify their collective behavior. For example, options in a dropdown may be grouped if only one of the options should be tested in order to verify their behavior. +Testing every element on a page is not always necessary when multiple elements share the same behavior. UI Coverage simplifies this by grouping related elements, allowing interactions with any element in the group to count as testing the entire group. This reduces redundancy and ensures efficient and accurate coverage. For example, options in a dropdown menu can be grouped if interacting with one option verifies the behavior of the entire set. -UI Coverage groups elements that are related to one another through structural or behavioral hints. A group of elements is considered one element when it comes to scoring, and interacting with any element in the group will cause all elements in the group to be considered tested (e.g. clicking one treats all of them as having been clicked). +## How element grouping works + +UI Coverage groups elements using structural and behavioral hints, along with user-defined configurations. When grouped, interacting with one element is considered equivalent to testing all elements in the group. ## Grouping Rules -- Elements are grouped by [elementgroups](/ui-coverage/configuration/elementgroups) configuration if defined. -- `label` elements are grouped with the form elements with which they are associated. -- Elements within table rows are grouped with elements that share common attributes or are in the same position in other rows of the table. -- Elements that are repeated in the DOM and share common attributes. -- Elements that are repeated in the DOM and have parent elements that share common attributes. -- Links with similar href patterns are resolved to [views](/ui-coverage/configuration/views) and grouped by their corresponding view. +- **User-Defined Groups**: Elements are grouped by [Element Groups](/ui-coverage/configuration/elementgroups) configuration if specified. +- **Labels and Form Elements**: `label` elements are automatically grouped with the form elements they are associated with. +- **Table Row Grouping**: Elements within table rows are grouped with elements sharing common attributes or the same position across rows. +- **Repeated Elements**: Duplicate elements in the DOM with shared attributes are grouped. +- **Parent-Shared Attributes**: Duplicate elements with parents that share common attributes are grouped. +- **Dynamic Links**: Links with similar `href` patterns are resolved into [views](/ui-coverage/configuration/views) and grouped by their `href` pattern. + +### Example + +Consider a table where each row contains a delete button: + +```html + + + + + + + + + +
John Doe + +
Jane Smith + +
+``` + +In this case, UI Coverage groups the delete buttons together in accordance with our grouping rules. Interacting with one delete button is equivalent to testing all delete buttons in the table. + +## Customizing element grouping + +You can customize element grouping to suit your application's structure and behavior. Refer to the [Element Groups](/ui-coverage/configuration/elementgroups) configuration guide to learn how to define custom groups and group elements based on specific attributes or selectors. diff --git a/docs/ui-coverage/core-concepts/element-identification.mdx b/docs/ui-coverage/core-concepts/element-identification.mdx index dfbcb67b7f..a6af3ed229 100644 --- a/docs/ui-coverage/core-concepts/element-identification.mdx +++ b/docs/ui-coverage/core-concepts/element-identification.mdx @@ -9,11 +9,11 @@ sidebar_position: 20 -Elements are uniquely identified across views and snapshots by their HTML attributes, location, and other signals in the DOM. +UI Coverage uniquely identifies elements across views and snapshots using a combination of HTML attributes, location, and other signals in the DOM. Stable and unique identifiers are crucial for accurately tracking and de-duplicating elements throughout the various snapshots captured in your tests. -It is important for elements to have unique and stable identifiers so that UI Coverage can track and de-duplicate them throughout the many different snapshots captured in your tests. +## Significant attributes for identification -UI Coverage considers the following attributes to have special significance for the purposes of identification and grouping: +Certain attributes are prioritized by UI Coverage for element identification and grouping. These include: - `data-cy` - `data-test` @@ -24,4 +24,40 @@ UI Coverage considers the following attributes to have special significance for - `id` - `name` -Element identification can be configured via [attributefilters](/ui-coverage/configuration/attributefilters) and [significantattributes](/ui-coverage/configuration/significantattributes) configuration. +By using these attributes consistently in your application, you can improve the accuracy of UI Coverage tracking. + +## Configuring element identification + +### Attribute filters + +Exclude irrelevant or dynamically generated attributes to reduce noise in UI Coverage reports. For example: + +```json +{ + "uiCoverage": { + "attributeFilters": [ + { + "attribute": "id", + "value": ":r.*:", + "include": false + } + ] + } +} +``` + +Learn more about attribute filtering in the [Attribute Filters](/ui-coverage/configuration/attributefilters) guide. + +### Significant attributes + +Define custom significant attributes to prioritize specific selectors that align with your application’s structure. For example: + +```json +{ + "uiCoverage": { + "significantAttributes": ["data-custom-id"] + } +} +``` + +Refer to the [Significant Attributes](/ui-coverage/configuration/significantattributes) guide for detailed instructions. diff --git a/docs/ui-coverage/core-concepts/interactivity.mdx b/docs/ui-coverage/core-concepts/interactivity.mdx index f6f55a915e..33d4888ba6 100644 --- a/docs/ui-coverage/core-concepts/interactivity.mdx +++ b/docs/ui-coverage/core-concepts/interactivity.mdx @@ -11,15 +11,17 @@ sidebar_position: 10 ## Interactive Elements -UI Coverage uses a set of rules, based on HTML semantics, [WHATWG standards](https://html.spec.whatwg.org/dev/dom.html#interactive-content), as well as some additional rules defined by Cypress, to determine which elements are interactive. Some examples of elements that are considered interactive and are thereby included in UI Coverage are: +UI Coverage determines interactivity based on a combination of HTML semantics, [WHATWG standards](https://html.spec.whatwg.org/dev/dom.html#interactive-content), and Cypress-specific rules. Interactive elements included in UI Coverage are: -- **Elements with implicit interactive roles** such as elements with a tag of `a`, `button`, `input`, `select`, `textarea`, etc. -- **Elements with explicit interactive roles** such as elements with a `role` attribute set to `button`, `checkbox`, `radio`, `tab`, `textbox`, etc. -- **Elements with a `tabindex` attribute set to >= 0.** +- **Implicit interactive roles**: Elements with a tag of `a`, `button`, `input`, `select`, `textarea`, etc. +- **Explicit interactive roles**: Elements with a `role` attribute set to values like `button`, `checkbox`, `radio`, `tab`, `textbox`, etc. +- **Tab-navigable elements**: Elements with a `tabindex` attribute set to `>= 0`. + +These elements are tracked for interaction to provide actionable insights into test coverage. ## Interaction Commands -UI Coverage considers interactive elements "tested" if they are interacted with by a Cypress command. The commands that are considered interactions are: +Interactive elements are marked as "tested" when they are interacted with using specific Cypress commands. These include: - `blur` - `check` @@ -37,22 +39,24 @@ UI Coverage considers interactive elements "tested" if they are interacted with - `type` - `uncheck` +By ensuring these commands interact with the appropriate elements, UI Coverage accurately reflects your test coverage. + ## Untested Links -UI Coverage tracks links (`` elements) that haven't been visited during testing. For each untested link, you can see: +UI Coverage tracks `` elements (links) that are not visited during testing. For each untested link, detailed information is available: ### Referrers -The Views containing links to this untested destination. This helps you understand: +The **Referrers** section identifies views that contain links to the untested destination, helping you: -- Where these untested areas are referenced from -- The navigation paths that could lead to this untested area -- The context in which these links appear +- Pinpoint untested areas referenced from these links. +- Understand navigation paths leading to untested sections. +- Gain context on where these links appear within your application. ### URLs -Shows how similar URLs are grouped together. For example, if you have links to `/users/1`, `/users/2`, and `/users/3`, they'll be grouped as `/users/*`. The URLs section shows all the actual URLs that were grouped together. This is particularly useful for: +The **URLs** section groups similar patterns for dynamic routing. For instance, links to `/users/1`, `/users/2`, and `/users/3` are grouped as /users/\*. This grouping aids in: - Understanding the scope of dynamic routes in your application -- Seeing how many variations of a URL pattern exist -- Writing configuration rules to handle URL patterns appropriately +- Viewing the count and variations of a URL pattern. +- Writing configuration rules to handle these patterns effectively. diff --git a/docs/ui-coverage/core-concepts/views.mdx b/docs/ui-coverage/core-concepts/views.mdx index 015a49cbc3..6dd4a04245 100644 --- a/docs/ui-coverage/core-concepts/views.mdx +++ b/docs/ui-coverage/core-concepts/views.mdx @@ -9,30 +9,67 @@ sidebar_position: 100 -UI Coverage organizes all unique URLs across snapshots from end-to-end tests and mounted components from component tests into "Views". A View represents a distinct page or state in your application. +UI Coverage organizes unique URLs from snapshots in end-to-end tests and mounted components in component tests into Views. Each View represents a distinct page or state in your application, allowing you to track coverage across different parts of your application. -## View Creation & Grouping +## How views are created -By default, UI Coverage uses certain rules to generate a single View from dynamic URLs that represent the same page: +By default, UI Coverage applies a set of rules to generate a single View for dynamic URLs representing the same page. These rules ensure clarity by grouping similar URLs and reducing noise: -- All search parameters are removed -- Hash parameters are removed (unless they start with `/`, such as `#/admin`) -- If multiple URLs differ only by an integer or UUID in the same path parameter, that parameter is replaced with a wildcard (`*`) +- **Search Parameters**: All query parameters are removed (e.g., `/dashboard?tab=overview` becomes `/dashboard`). +- **Hash Parameters**: Fragment identifiers are removed unless they represent a subpath (e.g., `#/admin`). +- **Dynamic Segments**: URL segments that differ only by integers or UUIDs are replaced with a wildcard (`*`). -For example: +### Examples of views -- `/users/123/profile` and `/users/456/profile` would be grouped into `/users/*/profile` -- `/dashboard?tab=overview` and `/dashboard?tab=settings` would be grouped into `/dashboard` -- `/app#settings` and `/app#profile` would be grouped into `/app` -- `/app#/admin` and `/app#/settings` would remain separate Views +- `/users/123/profile` and `/users/456/profile` become `/users/*/profile` +- `/dashboard?tab=overview` and `/dashboard?tab=settings` become `/dashboard` +- `/app#settings` and `/app#profile` become `/app` +- `/app#/admin` and `/app#/settings` remain separate Views + +These rules help consolidate similar pages into manageable Views while maintaining specificity where needed. ## Using Views -In the UI Coverage interface, Views are listed with their coverage scores, helping you: +In the UI Coverage interface, Views are listed along with their coverage scores, providing insights into: + +- **Tested Areas**: Identify which parts of your application have sufficient test coverage. +- **Untested Areas**: Spot pages or components that lack testing and prioritize them. +- **Coverage Trends**: Track how coverage evolves across Views over time. +- **Critical Flows**: Focus testing efforts on key user journeys and ensure their completeness. + +## Customizing views + +You can tailor how URLs are grouped into Views using configuration options: + +### View grouping + +Define rules for grouping URLs into Views to align with your application's structure: + +```json +{ + "views": [ + { + "pattern": "https://cypress.io/users/*" + } + ] +} +``` + +To learn more about the configuration options, refer to the [Views](/ui-coverage/configuration/views) documentation. + +### Ignoring views + +Exclude irrelevant or low-priority URLs from being tracked as Views: -- See which areas of your application have been tested -- Identify untested pages and components -- Track changes in coverage across different parts of your application -- Focus testing efforts on critical paths and flows +```json +{ + "viewFilters": [ + { + "pattern": "https://auth.example.com/*", + "include": false + } + ] +} +``` -You can customize how URLs are grouped into Views using [views](/ui-coverage/configuration/views) and [viewFilters](/ui-coverage/configuration/viewfilters) configuration options. +To learn more about the configuration options, refer to the [View Filters](/ui-coverage/configuration/viewfilters) documentation. diff --git a/docs/ui-coverage/get-started/introduction.mdx b/docs/ui-coverage/get-started/introduction.mdx index eb110136c1..3e312a011f 100644 --- a/docs/ui-coverage/get-started/introduction.mdx +++ b/docs/ui-coverage/get-started/introduction.mdx @@ -40,18 +40,116 @@ From there, you can easily customize reports to fit your needs with flexible con [Get started with UI Coverage ➜](/ui-coverage/get-started/setup) -## How it Works - -:::caution +## Top Guides -UI Coverage generates reports using [Test Replay](/cloud/features/test-replay) data and requires Cypress v13+. + -::: - -UI Coverage creates a dynamic visual map of test coverage across every page of your application. Each interactive element is identified and highlighted as either tested or untested, providing a clear and actionable overview of your test coverage. +## How it Works -UI Coverage captures "snapshots" of every page visited by your Cypress tests (and every component mounted in Component Testing). Snapshots represent all of the unique states found across your test run, and these states are analyzed to identify all of the unique elements that make up your application. +UI Coverage provides an interactive, visual map of test coverage for your application, powered by Cypress [Test Replay](/cloud/features/test-replay) (requires Cypress v13+). These reports are automatically generated for every unique state reached during your Cypress tests, whether in end-to-end or component testing. -As a result, a UI Coverage score is generated. This score quantifies coverage across your application by measuring the ratio of the unique elements that have been tested to the total unique elements that are found across your application. +- **Effortless Setup**: No extra configuration is required. UI Coverage uses the same capture protocol as Test Replay, so no additional code or configuration is needed. +- **Dynamic Coverage Mapping**: Each interactive element is identified and highlighted as tested or untested, giving you a clear view of your test coverage across all pages and components. +- **DOM Snapshots**: Each tested and untested element is accompanied by a full-page, inspectable DOM snapshot, highlighting the exact location and context of element. +- **Comprehensive Scoring**: A UI Coverage score is calculated by comparing tested elements to the total interactable elements in your application. +- **Actionable Reports**: Sortable and filterable views provide insights into which areas are tested and which need improvement. +- **Flexible configuration**: Customize and fine-tune UI Coverage to suit specific needs and scenarios like ignoring views or elements or grouping similar elements together. +- **Configurable CI Integration**: The [Results API](/ui-coverage/results-api) allows you to programmatically control your CI pipeline's behavior based on UI Coverage scores. -Because Cypress UI Coverage uses data captured through Cypress Test Replay, it is subject to Test Replay limitations. Read [How Test Replay Works](/cloud/features/test-replay#How-Test-Replay-Works) to discover which data is captured and which browsers are supported. +Read more about how it works in the [Core Concepts](/ui-coverage/core-concepts/interactivity) section. diff --git a/docs/ui-coverage/guides/_category_.json b/docs/ui-coverage/guides/_category_.json new file mode 100644 index 0000000000..c9532523a7 --- /dev/null +++ b/docs/ui-coverage/guides/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Guides", + "position": 30 +} diff --git a/docs/ui-coverage/guides/address-coverage-gaps.mdx b/docs/ui-coverage/guides/address-coverage-gaps.mdx new file mode 100644 index 0000000000..62f201353a --- /dev/null +++ b/docs/ui-coverage/guides/address-coverage-gaps.mdx @@ -0,0 +1,127 @@ +--- +sidebar_label: Address coverage gaps +title: 'Address coverage gaps | Cypress UI Coverage Documentation' +description: 'Learn how to address test coverage gaps with Cypress UI Coverage to ensure quality of your application.' +sidebar_position: 20 +--- + +# Address coverage gaps + +After [identifying test coverage gaps](/ui-coverage/guides/identify-coverage-gaps) using Cypress UI Coverage, the next step is to address these gaps to ensure your application is comprehensively tested. This guide outlines best practices and strategies for improving coverage and closing the identified gaps effectively. + +## Prioritize Gaps + +Not all coverage gaps are equally critical. Use the information provided in the UI Coverage reports to prioritize testing efforts based on: + +- **Critical Views**: Focus on views or components that represent high-priority user journeys, such as checkout pages, login screens, or submission forms. +- **Untested Links**: Address testing pages that are not being visited by your current test suite. You won't get insight into the untested elements on these pages until you visit them. + +By prioritizing based on application context and business impact, you can address the most significant gaps first. + +## Enhance Test Coverage + +Once you've identified priority areas, create or update tests to cover these gaps. + +### Write Targeted Tests + +Focus on creating tests that interact with the specific untested elements or pages identified in the coverage reports. For example: + +```js +describe('Dashboard', () => { + it('Submits form on landing page', () => { + cy.visit('/request-trial') + + // Interact with previously untested elements + cy.get('[data-cy="email"]').type('test@email.com') + cy.contain('Request Trial').click() + // UI Coverage will now surface the coverage of the thank you page + cy.url().should('include', '/thank-you') + }) +}) +``` + +### Cover Untested Links + +Use the **Untested Links** section of the UI Coverage report to identify pages your tests haven't visited. Add navigation steps to your tests to include these pages: + +```js +describe('Cover Untested Links', () => { + it('Visits untested pages', () => { + const untestedLinks = ['/about-us', '/contact', '/pricing'] + + untestedLinks.forEach((link) => { + cy.visit(link) + // Perform basic checks to ensure the page loads correctly + cy.get('h1').should('exist') + // UI Coverage will now surface the coverage of these pages + }) + }) +}) +``` + +## Refine Tests + +### Ensure Element Visibility + +Some gaps occur because elements are hidden or not rendered during tests. Update your tests to reveal these elements: + +```js +cy.get('[data-cy="dropdown-toggle"]').click() // Reveal hidden elements +cy.get('[data-cy="dropdown-item"]').should('be.visible').click() +``` + +### Handle Dynamic Content + +If coverage gaps are caused by dynamic or conditional rendering, ensure your tests account for various application states: + +```js +// Login to render elements that only display after login +cy.get('[data-cy="login-button"]').click() +cy.get('[data-cy="user-profile"]') +``` + +## Optimize Configuration + +To maximize the effectiveness of UI Coverage, consider refining your configuration: + +- Element Filters: Exclude irrelevant elements (e.g., placeholders, ads) from coverage reports. +- Significant Attributes: Define custom attributes that accurately identify elements. +- Attribute Filters: Remove auto-generated attributes to prevent redundant element identification. + +Refer to the [Configuration Guide](/ui-coverage/configuration/overview) to learn how to customize UI Coverage to address these common needs: + +- **Filtering**: Exclude specific elements or views from coverage reports. + - [Element Filters](/ui-coverage/configuration/elementfilters): Exclude specific elements from coverage reports. + - [View Filters](/ui-coverage/configuration/viewfilters): Exclude specific views from coverage reports. +- **Grouping**: Group similar elements together for easier analysis. + - [Elements](/ui-coverage/configuration/elements): Specify selectors to uniquely identify elements, even when they lack stable identifiers across snapshots. + - [Element Grouping](/ui-coverage/configuration/elementgroups): Group similar elements together for easier analysis. + - [Views](/ui-coverage/configuration/views): Group views together based on defined URL patterns. +- **Defining Attribute Patterns**: Define patterns for identifying and grouping elements by attributes. + - [Attribute Filters](/ui-coverage/configuration/attributefilters): Specify patterns for attributes and their values that should not be used for identifying and grouping elements. + - [Significant Attributes](/ui-coverage/configuration/significantattributes): Define selectors to prioritize above the default attributes Cypress uses for the purpose of identification and grouping. + +## Iterate and Monitor + +### Review Coverage Reports + +After updating your tests, record them again to Cypress Cloud and review the new coverage reports. Verify that: + +- Untested elements and links have been addressed. +- Overall coverage score has improved. + +### Automate Coverage Enforcement + +Use the [Results API](/ui-coverage/results-api) to integrate coverage checks into your CI/CD pipeline. Set thresholds for coverage scores to enforce quality standards. This ensures your application maintains high test coverage over time. + +#Collaborate with Your Team + +Improving test coverage often requires collaboration. Share insights from the UI Coverage reports with your team to: + +- Prioritize testing efforts collectively. +- Align on critical areas that require attention. +- Distribute tasks for writing or updating tests. + +## Next Steps + +You can also leverage UI Coverage to reduce test duplication to optimize your test suite further. Learn how to [reduce test duplication](/ui-coverage/guides/reduce-test-duplication) with UI Coverage to streamline your testing process. diff --git a/docs/ui-coverage/guides/identify-coverage-gaps.mdx b/docs/ui-coverage/guides/identify-coverage-gaps.mdx new file mode 100644 index 0000000000..53d36b99ac --- /dev/null +++ b/docs/ui-coverage/guides/identify-coverage-gaps.mdx @@ -0,0 +1,127 @@ +--- +sidebar_label: Identify coverage gaps +title: 'Identify coverage gaps | Cypress UI Coverage Documentation' +description: 'Learn how to identify coverage gaps with Cypress UI Coverage to ensure quality of your application.' +sidebar_position: 10 +--- + +# Identify coverage gaps + +Understanding your application's test coverage is crucial for ensuring quality and reliability. Cypress's UI Coverage tool provides insights into which parts of your application are tested and highlights untested areas. This guide will help you get started with UI Coverage to identify and address coverage gaps effectively. + +## Run Tests + +To identify coverage gaps, you need to first run and record Cypress tests to the Cloud. If you're new to Cypress, refer to the [Cypress documentation](/app/end-to-end-testing/writing-your-first-end-to-end-test) to get started with writing tests. + +### Example: Automated Sitemap-Based Testing + +If your project lacks existing Cypress tests, a common approach is to understand test coverage from a sitemap or an array of target URLs. These URLs can be used to perform light interactions and capture the initial gaps in testing. Below is an example of how to automate this process by using a sitemap: + +```js +describe('UI Coverage Scan', () => { + it('Checks ui coverage with the sitemap.xml', () => { + cy.request('https:///sitemap.xml').then((response) => { + const xmlString = response.body + const parser = new DOMParser() + (loc) => loc.textContent + }) + Cypress._.each(URLs, (URL) => { + cy.visit(URL) + }) + }) +}) +``` + +## Review Coverage Reports + +Once your tests have recorded to Cypress Cloud, you can analyze the coverage reports to identify gaps. Click on the runs in your project in [Cypress Cloud](https://on.cypress.io/cloud) to access the UI Coverage reports. This report provides a visual representation of your application's test coverage, highlighting tested and untested elements. + +### Overall Score + +The first metric to review is the **overall coverage score**. This score is calculated by comparing the number of tested elements to the total number of [interactive elements](/ui-coverage/core-concepts/interactivity) in your application. A higher score indicates better coverage, while a lower score indicates areas that need additional testing. The score will display on the runs page and within individual runs. + + + +### Views + +Within a run's **UI Coverage** tab, you'll find a **Views** section. Views represent different pages or components of your application. Each view in the list displays the URL or component path, the number of snapshots, the number of untested and tested elements in that view, and the coverage score. + + + +#### View Drilldown + +Clicking into a view shows a detailed breakdown of the tested and untested elements within that view. You can use this information to inspect the DOM snapshot of the element using your browser's developer tools and understand the context of any coverage gaps. The view includes: + +- **Untested Elements**: A list of interactive elements that were not interacted with during the test run. +- **Tested Elements**: A list of interactive elements that were interacted with during the test run. +- **DOM Snapshot**: A full-page, inspectable DOM snapshot of the view as it appeared during the test run. Tested elements highlight as green, while untested elements highlight as red. +- **Snapshot Navigation**: Navigate between snapshots to see the state of the view at different points during the test run. +- **Snapshot Coverage Score**: The coverage score for the specific snapshot based on the number of tested elements. +- **Test Replay**: A link to the Test Replay for the specific snapshot. +- **URL** and **Viewport size**: Metadata for the view. + + + +### Untested links + +In the **UI Coverage** tab, the **Untested links** section lists all the links not interacted with during the test run. This can help you identify pages of your application that are not being visited and tested. Use this to identify unvisited pages and prioritize testing. + + + +#### Referrer Drilldown + +Clicking a referrer link redirects you to the referrer's view, where the untested link is highlighted in red. + + + +### Untested elements + +The **Untested Elements** section in the **UI Coverage** tab lists all interactive elements not interacted with during the test run. This can help you identify specific elements that are not being tested across views. Use this information to prioritize testing for these elements. + + + +#### View Drilldown + +Clicking into an untested element's view shows a detailed breakdown of the element, including the element's selector, the number of times the element was interacted with, and the views without interactions. You can use this information to inspect the DOM snapshot of the element using your browser's developer tools and understand the context of any coverage gaps. + + + +## Configure UI Coverage + +While UI Coverage is designed to work seamlessly out of the box, there are instances where custom configuration may be necessary to address unique application structures, testing requirements, or edge cases. Refer to the [Configuration Guide](/ui-coverage/configuration/overview) to learn how to customize UI Coverage to address these common needs: + +- **Filtering**: Exclude specific elements or views from coverage reports. + - [Element Filters](/ui-coverage/configuration/elementfilters): Exclude specific elements from coverage reports. + - [View Filters](/ui-coverage/configuration/viewfilters): Exclude specific views from coverage reports. +- **Grouping**: Group similar elements together for easier analysis. + - [Elements](/ui-coverage/configuration/elements): Specify selectors to uniquely identify elements, even when they lack stable identifiers across snapshots. + - [Element Grouping](/ui-coverage/configuration/elementgroups): Group similar elements together for easier analysis. + - [Views](/ui-coverage/configuration/views): Group views together based on defined URL patterns. +- **Defining Attribute Patterns**: Define patterns for identifying and grouping elements by attributes. + - [Attribute Filters](/ui-coverage/configuration/attributefilters): Specify patterns for attributes and their values that should not be used for identifying and grouping elements. + - [Significant Attributes](/ui-coverage/configuration/significantattributes): Define selectors to prioritize above the default attributes Cypress uses for the purpose of identification and grouping. + +## Next Steps + +By leveraging these tools and techniques, you can effectively identify test coverage gaps. Next, read our guide on [addressing coverage gaps](/ui-coverage/guides/address-coverage-gaps) to ensure a robust and reliable application. diff --git a/docs/ui-coverage/guides/ignore-elements.mdx b/docs/ui-coverage/guides/ignore-elements.mdx new file mode 100644 index 0000000000..7212a46189 --- /dev/null +++ b/docs/ui-coverage/guides/ignore-elements.mdx @@ -0,0 +1,58 @@ +--- +sidebar_label: Ignore elements +title: 'Ignore elements | Cypress UI Coverage Documentation' +description: 'Learn how to exclude irrelevant elements from your UI Coverage reports to focus on meaningful insights.' +sidebar_position: 50 +--- + +# Ignore elements + +Not all elements in your application are relevant to your test coverage. Cypress UI Coverage allows you to exclude specific elements from coverage reports, helping you focus on meaningful insights and avoid unnecessary noise. This guide explains why and how to ignore elements in your UI Coverage reports. + +## Why ignore elements? + +Ignoring elements can be beneficial in the following scenarios: + +- **Transitional Elements**: Exclude elements that are still in a loading state or undergoing changes, as they may not represent the final state of the element. +- **Third-Party Widgets**: Exclude elements controlled by external libraries or third-party integrations. + +By ignoring irrelevant elements, you can maintain clean and actionable coverage metrics. + +## Identify Elements to Ignore + +After recording your tests to Cypress Cloud, review the UI Coverage reports: + +1. Navigate to the **UI Coverage** tab in your test run. +1. Look for elements that consistently appear but don't require testing. +1. Note down the selectors, attributes, or patterns for these elements. + +## Configure Ignored Elements + +[Element Filters](/ui-coverage/configuration/elementfilters) in the **App Quality** configuration are used to exclude elements based on their selectors or attributes. To add or modify the configuration for your project, navigate to the **App Quality** tab in your project settings and add an **elementFilters** configuration. + +```json +{ + "elementFilters": [ + { + "selector": "[data-external*]", + "include": false + }, + { + "selector": "iframe[title='Login']", + "include": false + }, + { + "selector": ".rdrDateRangePicker, .rdrDateRangePicker *", + "include": false + } + ] +} +``` + +To learn more about the configuration options, refer to the [Element Filters](/ui-coverage/configuration/elementfilters) documentation. + +## Validate Ignored Elements + +After updating the configuration, record your tests again and review the UI Coverage report. The ignored elements should no longer appear in the coverage reports, streamlining the data and focusing on the critical areas of your application. + +If new unnecessary elements appear in future reports, update your filters accordingly to keep reports clean and actionable. diff --git a/docs/ui-coverage/guides/ignore-views-and-links.mdx b/docs/ui-coverage/guides/ignore-views-and-links.mdx new file mode 100644 index 0000000000..02a6986a81 --- /dev/null +++ b/docs/ui-coverage/guides/ignore-views-and-links.mdx @@ -0,0 +1,79 @@ +--- +sidebar_label: Ignore views and links +title: 'Ignore views and links | Cypress UI Coverage Documentation' +description: 'Learn how to exclude irrelevant views and links from your UI Coverage reports to focus on meaningful insights.' +sidebar_position: 40 +--- + +# Ignore views and links + +Cypress UI Coverage provides detailed insights into test coverage, but not all views or links in your application are relevant to your test suite. By ignoring certain URLs, you can focus on meaningful coverage and streamline your reports. This guide explains why and how to ignore views and links in your UI Coverage reports. + +## Why Ignore Views and Links? + +Ignoring views and links can help in the following scenarios: + +- **Third-Party Pages**: Exclude external URLs like OAuth login pages or embedded content that you don't control or need to test. +- **Non-Critical Pages**: Remove irrelevant views, such as informational pages (e.g., FAQs) or admin-only sections. +- **Test Links**: Avoid testing URLs only generated for testing purposes or dynamically generated URLs that don't impact application functionality. +- **Focus on Key Areas**: Streamline coverage reports to prioritize high-impact areas and avoid clutter. + +By ignoring unnecessary views or links, you can maintain clear and actionable coverage metrics. + +## Identify Views and Links to Ignore + +After recording your tests to Cypress Cloud, review the UI Coverage reports: + +1. Navigate to the **UI Coverage** tab in your test run. +1. Look for views or links that consistently appear but don't require testing. +1. Note down the URLs, paths, or patterns for these views and links. + +### Common Candidates for Exclusion + +- Third-party authentication pages (e.g., https://auth.example.com). +- Redirects or intermediate URLs (e.g., /redirect). +- Informational pages (e.g., /terms). + +## Configure Ignored Views and Links + +[View Filters](/ui-coverage/configuration/viewfilters) in the **App Quality** configuration are used to exclude views and links based on their URLs or patterns. To add or modify the configuration for your project, navigate to the **App Quality** tab in your project settings and add a **viewFilters** configuration. + +Some examples of common view filter configurations are shown below: + +### Exclude a single URL + +```json +{ + "viewFilters": [ + { + "pattern": "https://auth.example.com/*", + "include": false + } + ] +} +``` + +### Exclude all third party URLs + +```json +{ + "viewFilters": [ + { + "pattern": "http://localhost:3000/*", + "include": true + }, + { + "pattern": "*", + "include": false + } + ] +} +``` + +To learn more about the configuration options, refer to the [View Filters](/ui-coverage/configuration/viewfilters) documentation. + +## Validate Ignored Views and Links + +After updating the configuration, record your tests again and review the UI Coverage report. The ignored views and links should no longer appear in the coverage reports, streamlining the data and focusing on the critical areas of your application. + +If new unnecessary views or links appear in future reports, update your filters accordingly to keep reports clean and actionable. diff --git a/docs/ui-coverage/guides/monitor-changes.mdx b/docs/ui-coverage/guides/monitor-changes.mdx new file mode 100644 index 0000000000..609ea9a1b1 --- /dev/null +++ b/docs/ui-coverage/guides/monitor-changes.mdx @@ -0,0 +1,14 @@ +--- +sidebar_label: Monitor changes +title: 'Monitor changes | Cypress UI Coverage Documentation' +description: 'Learn how to monitor changes to your UI Coverage scores over time to ensure proactive quality assurance.' +sidebar_position: 70 +--- + +# Monitor changes + +Monitoring changes to your UI Coverage scores over time ensures that regressions are identified and addressed before they are merged. Cypress UI Coverage provides tools to track these changes, enabling proactive quality assurance in your development workflow. + +## Automate monitoring with the Results API + +The UI Coverage [Results API](/ui-coverage/results-api) allows you to programmatically fetch UI Coverage data for integration into your CI/CD pipeline. diff --git a/docs/ui-coverage/guides/reduce-noise.mdx b/docs/ui-coverage/guides/reduce-noise.mdx new file mode 100644 index 0000000000..6f8c66caf0 --- /dev/null +++ b/docs/ui-coverage/guides/reduce-noise.mdx @@ -0,0 +1,118 @@ +--- +sidebar_label: Reduce noise +title: 'Reduce noise | Cypress UI Coverage Documentation' +description: 'Learn how to exclude irrelevant elements from your UI Coverage reports to focus on meaningful insights.' +sidebar_position: 60 +--- + +# Reduce noise + +A clean and focused UI Coverage report enables you to make informed decisions about your testing strategy. Cypress UI Coverage provides tools to reduce noise in your reports by properly grouping elements, special attribute handling, and grouping views that share common URL patterns. This guide explains how and why to create a streamlined report. + +## Group views by URL patterns + +Cypress attempts to groups views with similar URL patterns (like when a URL contains a user ID) to provide a more consolidated view of coverage metrics. + +Customozing views with similar URL patterns allows you to consolidate coverage metrics for pages you know are related. To add or modify the configuration for your project, navigate to the **App Quality** tab in your project settings and add an **viewGroups** configuration. + +### Example: Grouping user profiles + +``` +https://cypress.io/users/alice +https://cypress.io/users/alice?foo=bar +https://cypress.io/users/bob +https://cypress.io/users/bob#baz +``` + +```json +{ + "views": [ + { + "pattern": "https://cypress.io/users/*" + } + ] +} +``` + +To learn more about the configuration options, refer to the [Views](/ui-coverage/configuration/views) documentation. + +## Group similar elements + +Grouping elements improves clarity by reducing repetitive entries for elements with the same functionality. This is especially useful for elements like buttons, links, or form fields that appear across multiple views. By grouping similar elements, you can focus on the overall coverage of a specific element type rather than individual instances. + +### Example: Grouping dynamic IDs + +In the example below, the UI Coverage report shows multiple buttons that represent the same button due to a dynamically generated ID. By grouping these buttons, you can reduce noise and see the total coverage for the buttons with similar functionality in your application. + +```html + +``` + +To add or modify the configuration for your project, navigate to the **App Quality** tab in your project settings and add an **elementGroups** configuration under **uiCoverage**. + +```json +{ + "uiCoverage": { + "elementGroups": [ + { + "selector": "nav [id^=nav-button]" + } + ] + } +} +``` + +Now instead of 3 separate elements in the UI Coverage report, the report will show a single entry for all buttons with the `nav-button` ID prefix, simplifying the view and providing a clearer picture of the coverage. + +``` +nav [id^=nav-button] (3 instances) +``` + +To learn more about the configuration options, refer to the [Element Groups](/ui-coverage/configuration/elementgroups) documentation. + +## Configure attribute handling + +Attribute configuration ensures that Cypress surfaces element attributes that are most meaningful to your project. + +### Define significant attributes + +Specify which attributes Cypress should prioritize when identifying elements. To add or modify the configuration for your project, navigate to the **App Quality** tab in your project settings and add an **significantAttributes** configuration under **uiCoverage**. + +```json +{ + "uiCoverage": { + "significantAttributes": ["data-custom-id"] + } +} +``` + +To learn more about the configuration options, refer to the [Significant Attributes](/ui-coverage/configuration/significantattributes) documentation. + +### Use Attribute Filters + +Exclude auto-generated or irrelevant attributes that add unnecessary noise. To add or modify the configuration for your project, navigate to the **App Quality** tab in your project settings and add an **attributeFilters** configuration under **uiCoverage**. + +```json +{ + "uiCoverage": { + "attributeFilters": [ + { + "attribute": "id", + "value": "sizzle.*", + "include": false + }, + { + "attribute": "ng-.*", + "value": ".*", + "include": false + } + ] + } +} +``` + +To learn more about the configuration options, refer to the [Attribute Filters](/ui-coverage/configuration/attributefilters) documentation. diff --git a/docs/ui-coverage/guides/reduce-test-duplication.mdx b/docs/ui-coverage/guides/reduce-test-duplication.mdx new file mode 100644 index 0000000000..1088b7cf14 --- /dev/null +++ b/docs/ui-coverage/guides/reduce-test-duplication.mdx @@ -0,0 +1,79 @@ +--- +sidebar_label: Reduce test duplication +title: 'Reduce test duplication | Cypress UI Coverage Documentation' +description: 'Optimize your test suite by identifying and consolidating duplicate tests with Cypress UI Coverage.' +sidebar_position: 30 +--- + +# Reduce test duplication + +Efficient test suites not only maximize coverage but also avoid redundant tests that slow down development and CI pipelines. Cypress UI Coverage provides insights into areas where elements have been tested multiple times, helping you optimize your test strategy and reduce duplication. This guide explains how to identify and address test duplication for streamlined and effective testing. + +## Identify & Consolidate Duplicate Tests + +UI Coverage reports in Cypress Cloud highlight elements that have been tested multiple times across a run. To identify duplication: + +1. Navigate to the **Tested elements** section within the **UI Coverage** tab in your test run. +1. Locate tested elements that have a large number of interactions. +1. Review the Snapshots within the view to identify duplicate tests that interact with the tested element. + + + +### Common Signs of Duplication + +- **Repeated Interactions**: Elements interacted with in multiple snapshots or across different tests. +- **Overlapping Tests**: Multiple tests cover the same workflows or user journeys. +- **Excessive Setup Steps**: Repeated setup steps across tests that interact with the same elements. + +### Example: Welcome Screen Duplication + +In the example below, the launchpad within Cypress App shows that the **Continue** button on the Welcome Screen has been interacted with **184 times** in **327 snapshots**. This indicates that many tests interact with this element. Clicking through to some of the Test Replay's of the snapshots however reveal that most of our tests are not concerned with the behavior of the Welcome Screen - we're just passing through to get to our other tests. This highlights an opportunity to reduce test duplication. + + + +Now we can focus on consolidating tests and optimizing our test suite to avoid unnecessary duplication. + +```js +// This example is simplified for demonstration purposes +// In a real-world scenario, you would change the properties that are checked +// to hide or display the welcome screen for first-time users +Cypress.Commands.add('skipWelcome', () => { + cy.setCookie('welcome', 'dismissed') +}) + +it('shows welcome page', () => { + cy.visit('/') + cy.contains('Welcome') + cy.get('[data-cy="continue"]').click() + cy.contains('Projects') +}) + +it('shows projects page', () => { + cy.skipWelcome() + cy.visit('/') + cy.contains('Projects') +}) +``` + +After updating our tests and recording a new run, we can visit the **Tested elements** section to see the impact of our changes. The **Continue** button now appears in fewer snapshots and less interactions, indicating that we've successfully reduced duplication. + + + +## Monitor and Prevent Duplication + +### Document Your Testing Strategy + +Maintain a clear testing strategy that outlines: + +- The scope and purpose of each test suite. +- Which workflows and components each test covers. +- Guidelines for avoiding overlap in new tests. diff --git a/docs/ui-coverage/results-api.mdx b/docs/ui-coverage/results-api.mdx index e4469f3a2f..95c912feec 100644 --- a/docs/ui-coverage/results-api.mdx +++ b/docs/ui-coverage/results-api.mdx @@ -9,13 +9,11 @@ sidebar_position: 100 -The `@cypress/extract-cloud-results` module provides the `getUICoverageResults` utility which enables you to programmatically fetch your run's UI Coverage results in a CI environment. It determines the Cypress run created for the given CI workflow and will return the UI Coverage results associated with that run. The results will be returned once the Cypress run has finished and the UI Coverage report has been processed. - -This allows you to review the results within CI and to determine if the results are acceptable or need to be addressed before code changes can merge. +The `@cypress/extract-cloud-results` module provides the `getUICoverageResults` utility to programmatically fetch UI Coverage results for a run in a CI environment. This allows you to determine if test coverage meets your requirements before merging code changes. ## Supported CI Providers -Fetching UI Coverage results for a run supports fetching results for the following CI providers. Please see the docs below for information on general setup. +The utility supports the following CI providers. Refer to the linked guides for setup details: - [Azure](/app/continuous-integration/overview#Azure-Pipelines) (requires Cypress v13.13.1) - [CircleCI ](/app/continuous-integration/circleci) @@ -23,7 +21,7 @@ Fetching UI Coverage results for a run supports fetching results for the followi - [GitLab](/app/continuous-integration/gitlab-ci) - [Jenkins](/app/continuous-integration/overview#Jenkins) -Please reach out to Cypress Support to request support for a different provider. +For other CI providers, contact Cypress Support to request support. ## Installation @@ -43,15 +41,17 @@ If you check this in as a dependency, your installation will fail when we update ## Usage -### **1. Get the Results** +### **1. Fetch UI Coverage Results** -Write a script using the `getUICoverageResults` utility to retrieve the results and perform one or more assertions to verify if the changes are acceptable. This script will be executed in CI. +Write a script to fetch UI Coverage results and assert test coverage criteria. This script will be executed in CI. ```javascript title="scripts/verifyUICoverageResults.js" +const { getUICoverageResults } = require('@cypress/extract-cloud-results') + getUICoverageResults({ - projectId: '...', // optional if set from env - recordKey: '...', // optional if set from env - runTags: [process.env.RUN_TAGS], // required if recording multiple runs + projectId: process.env.CYPRESS_PROJECT_ID, // Optional if set from env + recordKey: process.env.CYPRESS_RECORD_KEY, // Optional if set from env + runTags: [process.env.RUN_TAGS], // Required if recording multiple runs }).then((results) => { const { runNumber, uiCoverageReportUrl, summary, views } = results @@ -60,139 +60,55 @@ getUICoverageResults({ ) console.log(`See full report at ${uiCoverageReportUrl}.`) - // verify project coverage + // Verify overall coverage if (summary.coverage < 80) { throw new Error( - `Project coverage is ${summary.coverage}, which does not meet the minimum coverage of 80%.` + `Project coverage is ${summary.coverage}, below the minimum threshold of 80%.` ) } const criticalViews = [/login/, /checkout/] - // verify critical view coverage - const criticalViewResults = views.forEach((view) => { + // Verify critical view coverage + views.forEach((view) => { const { displayName, coverage, uiCoverageReportUrl } = view - const isCriticalView = criticalViews.some((rx) => rx.test(displayName)) - if (!isCriticalView) return - - if (coverage < 95) { + if ( + criticalViews.some((pattern) => pattern.test(displayName)) && + coverage < 95 + ) { throw new Error( - `The critical view ${displayName} has the coverage of ${coverage}, which does not meet the critical view coverage of 95%. See report at ${uiCoverageReportUrl}.` + `Critical view "${displayName}" coverage is ${coverage}%, below the required 95%. See: ${uiCoverageReportUrl}` ) } }) - console.log('Your UI Coverage looks great!') + console.log('UI Coverage is above minimum thresholds.') }) ``` -#### `getUICoverageResults` Arguments +#### `getUICoverageResults` arguments -`getUICoverageResults` uses the following attributes to identify the Cypress run and return the UI Coverage results: +`getUICoverageResults` accpets the following arguments: ```javascript getUICoverageResults({ // The Cypress project ID. // Optional if the CYPRESS_PROJECT_ID env is set - // Can be explicitly passed to override the env var projectId: string - // The project's record key. // Optional if the CYPRESS_RECORD_KEY env is set - // Can be explicitly passed to override the env var recordKey: string - // The run tags associated with the run. // Required IF you are recording multiple Cypress runs from a single CI build. // Pass the run tags you used when recording in each run - // See below for more information runTags: string[] }) ``` -#### Result Types - -The UI Coverage results for the run are returned as an object containing the following data: - -```javascript -{ - // The run number of the identified build. - runNumber: number - - // The run url for the identified build. - runUrl: 'https://cloud.cypress.io/projects/:project_id/runs/:run_number' - - // The status of the identified build. - runStatus: 'passed' | 'failed' | 'errored' | 'timedOut' | 'cancelled' | 'noTests' - - // The url that links to UI Coverage report for the identified build. - uiCoverageReportUrl: 'https://cloud.cypress.io/[...]' - - summary: { - // Indicates whether a complete UI Coverage report was generated. - // For example, if a run was cancelled and the report expected to run - // for 20 specs, but only 10 ran, this would result in a partial report. - isPartialReport: boolean - - // The report coverage from 0-100 with 2 decimal precision (e.g 92.45). - coverage: float - - // The number of views tested and analyzed. - viewCount: number - - // The number of interactive elements that were tested. - testedElementsCount:number - - // The number of interactive elements that were not tested. - untestedElementsCount: number - } - - // The list of tested views and the coverage of each page. - views: [{ - // The sanatized URL pattern shown in the report. - displayName: string - - // The view coverage from 0-100 with 2 decimal precision (e.g 92.45). - coverage: float - - // The number of interactive elements that were tested on this view. - testedElementsCount:number - - // The number of interactive elements that were not tested on this view. - untestedElementsCount: number - - // The url that links the report for this view. - uiCoverageReportUrl: 'https://cloud.cypress.io/[...]' - }] -} -``` - -### **2. Add to CI Workflow** - -In your CI workflow that runs your Cypress tests, - -1. Update your install job to install the `@cypress/extract-cloud-results` module. -2. Pass in the necessary arguments to `getUICoverageResults`. -3. Add a new step to the job that runs your Cypress tests to verify the UI Coverage results. - -:::info - -If you record multiple runs in a single CI build, you must record these runs using the `--tag` parameter and then call `getUICoverageResults` with the `runTags` argument for each run. - -This is necessary to identify each unique run and return a corresponding set of results. The tags are how each run is uniquely identified. - -**Example** - -- Let's imagine that within a single CI build you call `cypress run --record` multiple times because you're running one set of tests against a `staging` environment, followed by a `production` environment. -- In this scenario, you pass a different `--tag` to each cypress run - - `cypress run --record --tag staging` - - `cypress run --record --tag production` -- When calling `getUICoverageResults` you would then pass these same tags to get the unique set of results for each run - - `getUICoverageResults({ runTags: ['staging']})` - - `getUICoverageResults({ runTags: ['production']})` +## 2. Integrate into CI workflow -::: +Add steps to your CI configuration to install the module, fetch results, and verify coverage: @@ -342,3 +258,73 @@ workflows: + +#### Result Details + +The `getUICoverageResults` utility returns the following data: + +```javascript +{ + // The run number of the identified build. + runNumber: number + // The run url for the identified build. + runUrl: 'https://cloud.cypress.io/projects/:project_id/runs/:run_number' + // The status of the identified build. + runStatus: 'passed' | 'failed' | 'errored' | 'timedOut' | 'cancelled' | 'noTests' + // The url that links to UI Coverage report for the identified build. + uiCoverageReportUrl: 'https://cloud.cypress.io/[...]' + summary: { + // Indicates whether a complete UI Coverage report was generated. + // For example, if a run was cancelled and the report expected to run + // for 20 specs, but only 10 ran, this would result in a partial report. + isPartialReport: boolean + // The report coverage from 0-100 with 2 decimal precision (e.g 92.45). + coverage: float + // The number of views tested and analyzed. + viewCount: number + // The number of interactive elements that were tested. + testedElementsCount:number + // The number of interactive elements that were not tested. + untestedElementsCount: number + } + // The list of tested views and the coverage of each page. + views: [{ + // The sanatized URL pattern shown in the report. + displayName: string + // The view coverage from 0-100 with 2 decimal precision (e.g 92.45). + coverage: float + // The number of interactive elements that were tested on this view. + testedElementsCount:number + // The number of interactive elements that were not tested on this view. + untestedElementsCount: number + // The url that links the report for this view. + uiCoverageReportUrl: 'https://cloud.cypress.io/[...]' + }] +} +``` + +### **2. Add to CI Workflow** + +In your CI workflow that runs your Cypress tests, + +1. Update your install job to install the `@cypress/extract-cloud-results` module. +2. Pass in the necessary arguments to `getUICoverageResults`. +3. Add a new step to the job that runs your Cypress tests to verify the UI Coverage results. + +:::info + +If you record multiple runs in a single CI build, you must record these runs using the `--tag` parameter and then call `getUICoverageResults` with the `runTags` argument for each run. + +This is necessary to identify each unique run and return a corresponding set of results. The tags are how each run is uniquely identified. + +**Example** + +- Let's imagine that within a single CI build you call `cypress run --record` multiple times because you're running one set of tests against a `staging` environment, followed by a `production` environment. +- In this scenario, you pass a different `--tag` to each cypress run + - `cypress run --record --tag staging` + - `cypress run --record --tag production` +- When calling `getUICoverageResults` you would then pass these same tags to get the unique set of results for each run + - `getUICoverageResults({ runTags: ['staging']})` + - `getUICoverageResults({ runTags: ['production']})` + +::: diff --git a/docs/ui-coverage/troubleshooting.mdx b/docs/ui-coverage/troubleshooting.mdx index ef1955ecc4..60de0fa2fa 100644 --- a/docs/ui-coverage/troubleshooting.mdx +++ b/docs/ui-coverage/troubleshooting.mdx @@ -9,77 +9,57 @@ sidebar_position: 150 -## A single element is showing up as multiple different elements +## A single element appears as multiple elements -If a single element is being identified as multiple different elements by UI Coverage and therefore shows up as many different elements, then you likely have a problem -with [Element Identification](/ui-coverage/core-concepts/element-identification). +When a single element is identified as multiple distinct elements in UI Coverage, it may indicate an issue with [Element Identification](/ui-coverage/core-concepts/element-identification). This often occurs due to: -If elements have auto-generated or otherwise unrepresentative attributes that are used as identifiers, then that attribute may have one value in one snapshot and a different value in another snapshot, which leads to UI Coverage considering those two different elements. +- Auto-generated or dynamic attributes (e.g., UUIDs) that differ across snapshots. +- Library-specific identifiers, such as id attributes generated by frameworks. -Some common examples of this include: +### Solutions -- Library-specific identifiers, such as `id` attributes with values that are library-generated and are not representative of the element -- Identifiers that include dynamic data, such as UUID's - -To resolve this problem, you can do one of the following: - -1. Add a unique identifier to the interactive element, such as a representative `data-cy` or `data-test` value -2. Ignore auto-generated or dynamic attributes using [attributeFilters](/ui-coverage/configuration/attributefilters) configuration -3. Manually identify the element using [elements](/ui-coverage/configuration/elements) configuration - -## Multiple different elements are being considered the same element - -If multiple different elements are being identified as the same element by UI Coverage and therefore only show up as one element, then you likely have a problem with [Element Identification](/ui-coverage/core-concepts/element-identification). +- **Add Unique Identifiers**: Assign a stable identifier like `data-cy` or `data-test` to the element. +- **Ignore Dynamic Attributes**: Use [`attributeFilters`](/ui-coverage/configuration/attributefilters) to exclude attributes that change across tests. +- **Manually Configure Identification**: Use [elements](/ui-coverage/configuration/elements) configuration to explicitly define how the element should be identified. -If elements do not have accurate identifiers, such as unique `data-cy` or `data-test` values, then UI Coverage may treat different elements as a single element due to the similarity of their DOM structure and attributes. +## Multiple different elements are identified as the same -Some common examples of this include: +If different elements are being grouped as one in UI Coverage, this could be due to insufficient or non-specific identifiers, such as: -- Missing `data-*` or `id` values that would otherwise identify the elements -- Generic identifiers, such as `data-test="button"`, used on many different elements -- The use of identifiers that are not used by UI Coverage by default, such as `aria-label` +- Missing unique `data-*` or `id` attributes. +- Generic identifiers like `data-test="button"` shared across multiple elements. +- Use of attributes not prioritized by UI Coverage, like `aria-label`. -To resolve this problem, you can do one of the following: +### Solutions -1. Add a unique identifier to the interactive elements, such as representative `data-cy` or `data-test` values -2. Ignore generic and unrepresentative attributes using [attributeFilters](/ui-coverage/configuration/attributefilters) configuration -3. Update the attributes used by UI Coverage for identification using - [significantAttributes](/ui-coverage/configuration/significantattributes) configuration -4. Manually identify the elements using [elements](/ui-coverage/configuration/elements) configuration +- **Add unique identifiers**: Assign distinct identifiers like `data-cy` or `data-test` values to each element. +- **Filter generic attributes**: Exclude unrepresentative attributes using [`attributeFilters`](/ui-coverage/configuration/attributefilters) configuration. +- **Customize attribute use**: Update [`significantAttributes`](/ui-coverage/configuration/significantattributes) configuration to prioritize relevant identifiers. +- **Manually Identify Elements**: Use [elements](/ui-coverage/configuration/elements) configuration to specify how elements should be identified. -## Elements that perform the same action are not being grouped together but should be +## Similar elements are not grouped together -If elements that perform the same action are not being grouped together, then you likely have a problem -with [Element Grouping](/ui-coverage/core-concepts/element-grouping). +If elements performing the same action are not [grouped](/ui-coverage/core-concepts/element-grouping), it may be due to: -If repeated elements have unique attributes that are used as identifiers, then UI Coverage may not group the elements together due to their lack of similarity. - -Some common examples of this include: - -- Library-specific identifiers, such as `id` attributes with values that are library-generated and are not representative of the element +- Library-specific identifiers, like dynamic `id` values, are not representative of the element and prevent grouping. - Identifiers that include dynamic data, such as UUID's -To resolve this problem, you can do one of the following: - -1. Add a shared identifier to the repeated elements, such as the same `data-cy` or `data-test` value -2. Ignore auto-generated or dynamic attributes using [attributeFilters](/ui-coverage/configuration/attributefilters) configuration -3. Manually group the elements using [elementGroups](/ui-coverage/configuration/elementgroups) configuration - -## Elements that perform different actions are being grouped together but should not be +### Solutions -If elements that perform different actions are being incorrectly grouped together, then you likely have a problem with [Element Grouping](/ui-coverage/core-concepts/element-grouping). +- **Use Shared Identifiers**: Apply consistent `data-cy` or `data-test` values across similar elements. +- **Ignore Dynamic Attributes**: Configure [`attributeFilters`](/ui-coverage/configuration/attributefilters) to exclude varying attributes. +- **Manually Group Elements**: Use [`elementGroups`](/ui-coverage/configuration/elementgroups) configuration to define custom groupings. -If repeated elements have shared attributes, such as the same `data-cy` or `data-test` values, then UI Coverage may group the elements together due to the similarity of their DOM structure and attributes. +## Different elements are incorrectly grouped together -Some common examples of this include: +If unrelated elements are [grouped](/ui-coverage/core-concepts/element-grouping), it may be due to overly generic identifiers or shared attributes. Some common examples of this include: -- Missing `data-*` or `id` values that would otherwise identify the elements -- Generic identifiers, such as `data-test="button"`, used on many different elements -- The use of identifiers that are not used by UI Coverage by default, such as `aria-label` +- Missing unique attributes that could distinguish the elements. +- Generic values like `data-test="button"` shared across different elements. -To resolve this problem, you can do one of the following: +### Solutions -1. Add a unique identifier to the interactive elements, such as representative `data-cy` or `data-test` values -2. Ignore generic and unrepresentative attributes using [attributeFilters](/ui-coverage/configuration/attributefilters) configuration -3. Update the attributes used by UI Coverage for identification using [significantAttributes](/ui-coverage/configuration/significantattributes) configuration -4. Manually group the elements into their own groups using [elementGroups](/ui-coverage/configuration/elementgroups) configuration +- **Add Distinct Identifiers**: Use unique `data-cy` or `data-test` values for each element. +- **Filter Shared Attributes**: Exclude generic identifiers with [`attributeFilters`](/ui-coverage/configuration/attributefilters) configuration. +- **Update Identification Rules**: Adjust [`significantAttributes`](/ui-coverage/configuration/significantattributes) configuration to improve differentiation. +- **Manually Separate Groups**: Define individual groups using [`elementGroups`](/ui-coverage/configuration/elementgroups) configuration. diff --git a/src/theme/MDXComponents.js b/src/theme/MDXComponents.js index 98830e7ded..3c7220ff79 100644 --- a/src/theme/MDXComponents.js +++ b/src/theme/MDXComponents.js @@ -88,6 +88,11 @@ import { faBookOpenReader, faUserShield, faCircleHalfStroke, + faCheckDouble, + faLinkSlash, + faListCheck, + faClipboardCheck, + faFilter, } from '@fortawesome/free-solid-svg-icons' library.add( @@ -139,6 +144,11 @@ library.add( faBookOpenReader, faUserShield, faCircleHalfStroke, + faCheckDouble, + faLinkSlash, + faListCheck, + faClipboardCheck, + faFilter, ) export default { diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-list-of-views.png b/static/img/ui-coverage/guides/cypress-ui-coverage-list-of-views.png new file mode 100644 index 0000000000..b3115571c5 Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-list-of-views.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-runs-list-with-uicov-score.png b/static/img/ui-coverage/guides/cypress-ui-coverage-runs-list-with-uicov-score.png new file mode 100644 index 0000000000..717da6607c Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-runs-list-with-uicov-score.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-test-duplication-example-after.png b/static/img/ui-coverage/guides/cypress-ui-coverage-test-duplication-example-after.png new file mode 100644 index 0000000000..ebc97c860d Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-test-duplication-example-after.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-test-duplication-welcome-screen-example.png b/static/img/ui-coverage/guides/cypress-ui-coverage-test-duplication-welcome-screen-example.png new file mode 100644 index 0000000000..0d4e70b27f Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-test-duplication-welcome-screen-example.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-tested-elements-with-high-interactions.png b/static/img/ui-coverage/guides/cypress-ui-coverage-tested-elements-with-high-interactions.png new file mode 100644 index 0000000000..e7c18ad0b0 Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-tested-elements-with-high-interactions.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-untested-elements-expanded.png b/static/img/ui-coverage/guides/cypress-ui-coverage-untested-elements-expanded.png new file mode 100644 index 0000000000..3103c661c8 Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-untested-elements-expanded.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-untested-elements-view-drilldown.png b/static/img/ui-coverage/guides/cypress-ui-coverage-untested-elements-view-drilldown.png new file mode 100644 index 0000000000..600719fd70 Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-untested-elements-view-drilldown.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-untested-link-view.png b/static/img/ui-coverage/guides/cypress-ui-coverage-untested-link-view.png new file mode 100644 index 0000000000..f833772845 Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-untested-link-view.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-untested-links.png b/static/img/ui-coverage/guides/cypress-ui-coverage-untested-links.png new file mode 100644 index 0000000000..cd2ee11420 Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-untested-links.png differ diff --git a/static/img/ui-coverage/guides/cypress-ui-coverage-view-drilldown.png b/static/img/ui-coverage/guides/cypress-ui-coverage-view-drilldown.png new file mode 100644 index 0000000000..0c80bfac7f Binary files /dev/null and b/static/img/ui-coverage/guides/cypress-ui-coverage-view-drilldown.png differ