Skip to content

Commit a39d358

Browse files
davethepunkyonedavethepunkyone
authored
Add functionality to execute Axe against multiple URLs (#32)
<!-- markdownlint-disable-next-line first-line-heading --> ## Description <!-- Describe your changes in detail. --> This functionality adds the ability to execute Axe against multiple URLs using a list, rather than just one URL at a time. ## Context <!-- Why is this change required? What problem does it solve? --> Allows for easier, generalised accessibility testing. ## Type of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply. --> - [ ] Refactoring (non-breaking change) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would change existing functionality) - [ ] Bug fix (non-breaking change which fixes an issue) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. --> - [ ] I am familiar with the [contributing guidelines](../docs/CONTRIBUTING.md) - [x] I have followed the code style of the project - [ ] I have added tests to cover my changes - [x] I have updated the documentation accordingly - [ ] This PR is a result of pair or mob programming --- ## Sensitive Information Declaration To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including [PII (Personal Identifiable Information) / PID (Personal Identifiable Data)](https://digital.nhs.uk/data-and-information/keeping-data-safe-and-benefitting-the-public) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter. - [x] I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.
1 parent bb7da0d commit a39d358

File tree

3 files changed

+137
-20
lines changed

3 files changed

+137
-20
lines changed

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ To utilise the blueprint code, you will need to have the following installed:
3434

3535
- [Python](https://www.python.org/downloads/) 3.12 or greater
3636

37+
> NOTE: There are currently known issues with Python 3.13 and Playwright, so if you encounter issues running this project whilst using Python 3.13 it is recommended to downgrade to Python 3.12 in the interim.
38+
3739
Whilst not required to get started, you may also want to [configure a Python virtual environment for your project](https://docs.python.org/3/library/venv.html) before proceeding with
3840
the configuration. If you are using an IDE such as Visual Studio Code or PyCharm, you will normally be prompted to do this automatically.
3941

‎docs/utility-guides/Axe.md‎

Lines changed: 92 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,22 @@
33
The Axe utility provided by this blueprint allows for the scanning of pages by using [axe-core](https://github.com/dequelabs/axe-core), a JavaScript
44
library used for scanning for accessibility issues and providing guidance on how to resolve these issues.
55

6+
## Table of Contents
7+
8+
- [Utility Guide: Axe](#utility-guide-axe)
9+
- [Table of Contents](#table-of-contents)
10+
- [Using the Axe class](#using-the-axe-class)
11+
- [.run(): Single page scan](#run-single-page-scan)
12+
- [Required arguments](#required-arguments)
13+
- [Optional arguments](#optional-arguments)
14+
- [Returns](#returns)
15+
- [Example usage](#example-usage)
16+
- [.run\_list(): Multiple page scan](#run_list-multiple-page-scan)
17+
- [Required arguments](#required-arguments-1)
18+
- [Optional arguments](#optional-arguments-1)
19+
- [Returns](#returns-1)
20+
- [Example usage](#example-usage-1)
21+
622
## Using the Axe class
723

824
You can initialise the Axe class by using the following code in your test file:
@@ -12,6 +28,8 @@ You can initialise the Axe class by using the following code in your test file:
1228
This Axe module has been designed as a static class, so you do not need to instantiate it when you want to run a scan on a page you have navigated to
1329
using Playwright.
1430

31+
## .run(): Single page scan
32+
1533
To conduct a scan, you can just use the following once the page you want to check is at the right location:
1634

1735
Axe.run(page)
@@ -20,37 +38,93 @@ This will inject the axe-core code into the page and then execute the axe.run()
2038

2139
By default, the `Axe.run(page)` command will do the following:
2240

23-
* Scan the page passed in to the WCAG 2.2 AA standard (which is the current expectation for NHS services outlined in the [NHS Service Manual](https://service-manual.nhs.uk/accessibility/what-all-NHS-services-need-to-do))
24-
* Generate a HTML and JSON report with the findings in the `axe-reports` directory, regardless of if any violations are found
25-
* Any steps after the `Axe.run()` command will continue to execute, and it will not cause the test in progress to fail (it runs a passive scan of the page)
26-
* Will return the full response from axe-core as a dict object if the call is set to a variable, e.g. `axe_results = Axe.run(page)` will populate `axe_results` to interact with as required
41+
- Scan the page passed in to the WCAG 2.2 AA standard (which is the current expectation for NHS services outlined in the [NHS Service Manual](https://service-manual.nhs.uk/accessibility/what-all-NHS-services-need-to-do))
42+
- Generate a HTML and JSON report with the findings in the `axe-reports` directory, regardless of if any violations are found
43+
- Any steps after the `Axe.run()` command will continue to execute, and it will not cause the test in progress to fail (it runs a passive scan of the page)
44+
- Will return the full response from axe-core as a dict object if the call is set to a variable, e.g. `axe_results = Axe.run(page)` will populate `axe_results` to interact with as required
2745

28-
## Required arguments
46+
### Required arguments
2947

3048
The following are required for `Axe.run()`:
3149

32-
|Argument|Format|Description|
33-
|--------|------|-----------|
34-
|page|playwright.sync_api.Page|A Playwright Page on the page to be checked.|
50+
| Argument | Format | Description |
51+
| -------- | ------------------------ | -------------------------------------------- |
52+
| page | playwright.sync_api.Page | A Playwright Page on the page to be checked. |
3553

36-
## Optional arguments
54+
### Optional arguments
3755

3856
The `Axe.run(page)` has the following optional arguments that can be passed in:
3957

40-
|Argument|Format|Supported Values|Default Value|Description|
41-
|--------|------|----------------|-------------|-----------|
42-
|`ruleset` |`list[str]`|Any provided by [axe-core](https://www.deque.com/axe/core-documentation/api-documentation/)|`['wcag2a', 'wcag21a', 'wcag2aa', 'wcag21aa', 'wcag22a', 'wcag22aa', 'best-practice']`|The tags that axe-core uses to filter specific checks. Defaulted to rules used for the WCAG 2.2 AA standard.|
43-
|`filename`|`str`|A string valid for a filename (e.g. `test_report`)||If provided, HTML and JSON reports will save with the filename provided. If not provided (default), the URL of the page under test will be used as the filename.|
44-
|`report_on_violation_only`|`bool`|`True`, `False`|`False`|If True, HTML and JSON reports will only be generated if at least one violation is found.|
45-
|`strict_mode`|`bool`|`True`, `False`|`False`|If True, when a violation is found an AxeAccessibilityException is raised, causing a test failure.|
46-
|`html_report_generated`|`bool`|`True`, `False`|`True`|If True, a HTML report will be generated summarising the axe-core findings.|
47-
|`json_report_generated`|`bool`|`True`, `False`|`True`|If True, a JSON report will be generated with the full axe-core findings.|
58+
| Argument | Format | Supported Values | Default Value | Description |
59+
| -------------------------- | ----------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
60+
| `ruleset` | `list[str]` | Any provided by [axe-core](https://www.deque.com/axe/core-documentation/api-documentation/) | `['wcag2a', 'wcag21a', 'wcag2aa', 'wcag21aa', 'wcag22a', 'wcag22aa', 'best-practice']` | The tags that axe-core uses to filter specific checks. Defaulted to rules used for the WCAG 2.2 AA standard. |
61+
| `filename` | `str` | A string valid for a filename (e.g. `test_report`) | | If provided, HTML and JSON reports will save with the filename provided. If not provided (default), the URL of the page under test will be used as the filename. |
62+
| `report_on_violation_only` | `bool` | `True`, `False` | `False` | If True, HTML and JSON reports will only be generated if at least one violation is found. |
63+
| `strict_mode` | `bool` | `True`, `False` | `False` | If True, when a violation is found an AxeAccessibilityException is raised, causing a test failure. |
64+
| `html_report_generated` | `bool` | `True`, `False` | `True` | If True, a HTML report will be generated summarising the axe-core findings. |
65+
| `json_report_generated` | `bool` | `True`, `False` | `True` | If True, a JSON report will be generated with the full axe-core findings. |
66+
67+
### Returns
4868

49-
## Example usage
69+
This function can be used independently, but when set to a variable returns a `dict` with the axe-core results.
70+
71+
### Example usage
5072

5173
from utils.axe import Axe
5274
from playwright.sync_api import Page
5375

5476
def test_axe_example(page: Page) -> None:
5577
page.goto("https://github.com/nhs-england-tools/playwright-python-blueprint")
5678
Axe.run(page)
79+
80+
## .run_list(): Multiple page scan
81+
82+
To scan multiple URLs within your application, you can use the following method:
83+
84+
Axe.run_list(page, page_list)
85+
86+
This runs the `Axe.run()` function noted above against each URL provided in the `page_list` argument, and will generate reports as required.
87+
88+
### Required arguments
89+
90+
The following are required for `Axe.run_list()`:
91+
92+
| Argument | Format | Description |
93+
| --------- | ------------------------ | ------------------------------------------------------------------ |
94+
| page | playwright.sync_api.Page | A Playwright Page object to drive navigation to each page to test. |
95+
| page_list | list[str] | A list of URLs to execute against |
96+
97+
> NOTE: It is heavily recommended that when using the `run_list` command, that you set a `--base-url` either via the pytest.ini file or by passing in the value when using the `pytest` command in the command line. By doing this, the list you pass in will not need to contain the base URL value and therefore make any scanning transferrable between environments.
98+
99+
### Optional arguments
100+
101+
The `Axe.run_list(page, page_list)` function has the following optional arguments that can be passed in:
102+
103+
| Argument | Format | Supported Values | Default Value | Description |
104+
| -------------------------- | ----------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
105+
| `use_list_for_filename` | `bool` | `True`, `False` | `True` | If True, the filename will be derived from the value provided in the list. If False, the full URL will be used. |
106+
| `ruleset` | `list[str]` | Any provided by [axe-core](https://www.deque.com/axe/core-documentation/api-documentation/) | `['wcag2a', 'wcag21a', 'wcag2aa', 'wcag21aa', 'wcag22a', 'wcag22aa', 'best-practice']` | The tags that axe-core uses to filter specific checks. Defaulted to rules used for the WCAG 2.2 AA standard. |
107+
| `report_on_violation_only` | `bool` | `True`, `False` | `False` | If True, HTML and JSON reports will only be generated if at least one violation is found. |
108+
| `strict_mode` | `bool` | `True`, `False` | `False` | If True, when a violation is found an AxeAccessibilityException is raised, causing a test failure. |
109+
| `html_report_generated` | `bool` | `True`, `False` | `True` | If True, a HTML report will be generated summarising the axe-core findings. |
110+
| `json_report_generated` | `bool` | `True`, `False` | `True` | If True, a JSON report will be generated with the full axe-core findings. |
111+
112+
### Returns
113+
114+
This function can be used independently, but when set to a variable returns a `dict` with the axe-core results for all pages scanned (using the URL value in the list provided as the key).
115+
116+
### Example usage
117+
118+
When using the following command: `pytest --base-url https://www.github.com`:
119+
120+
from utils.axe import Axe
121+
from playwright.sync_api import Page
122+
123+
def test_accessibility(page: Page) -> None:
124+
# A list of URLs to loop through
125+
urls_to_check = [
126+
"nhs-england-tools/playwright-python-blueprint",
127+
"nhs-england-tools/playwright-python-blueprint/wiki"
128+
]
129+
130+
Axe.run_list(page, urls_to_check)

‎utils/axe.py‎

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88

99
AXE_PATH = Path(__file__).parent / "resources" / "axe.js"
1010
PATH_FOR_REPORT = Path(__file__).parent.parent / "axe-reports"
11+
DEFAULT_WCAG_RULESET = ['wcag2a', 'wcag21a', 'wcag2aa', 'wcag21aa', 'wcag22a', 'wcag22aa', 'best-practice']
1112

1213

1314
class Axe:
@@ -19,7 +20,7 @@ class Axe:
1920
@staticmethod
2021
def run(page: Page,
2122
filename: str = "",
22-
ruleset: list = ['wcag2a', 'wcag21a', 'wcag2aa', 'wcag21aa', 'wcag22a', 'wcag22aa', 'best-practice'],
23+
ruleset: list[str] = DEFAULT_WCAG_RULESET,
2324
report_on_violation_only: bool = False,
2425
strict_mode: bool = False,
2526
html_report_generated: bool = True,
@@ -58,7 +59,47 @@ def run(page: Page,
5859
return response
5960

6061
@staticmethod
61-
def _build_run_command(ruleset: list) -> str:
62+
def run_list(page: Page,
63+
page_list: list[str],
64+
use_list_for_filename: bool = True,
65+
ruleset: list = DEFAULT_WCAG_RULESET,
66+
report_on_violation_only: bool = False,
67+
strict_mode: bool = False,
68+
html_report_generated: bool = True,
69+
json_report_generated: bool = True) -> dict:
70+
"""
71+
This runs axe-core against a list of pages provided.
72+
73+
NOTE: It is recommended to set a --base-url value when running Playwright using this functionality, so you only need to pass in a partial URL within the page_list.
74+
75+
Args:
76+
page (playwright.sync_api.Page): The page object to execute axe-core against.
77+
page_list (list[playwright.sync_api.Page): A list of URLs to execute against.
78+
use_list_for_filename (bool): If true, based filenames off the list provided. If false, use the full URL under test for the filename.
79+
ruleset (list[str]): [Optional] If provided, a list of strings to denote the ruleset tags axe-core should use. If not provided, defaults to the WCAG 2.2 AA standard (uses tags: 'wcag2a', 'wcag21a', 'wcag2aa', 'wcag21aa', 'wcag22a', 'wcag22aa', 'best-practice').
80+
report_on_violation_only (bool): [Optional] If true, only generates an Axe report if a violation is detected. If false (default), always generate a report.
81+
strict_mode (bool): [Optional] If true, raise an exception if a violation is detected. If false (default), proceed with test execution.
82+
html_report_generated (bool): [Optional] If true (default), generates a html report for the page scanned. If false, no html report is generated.
83+
json_report_generated (bool): [Optional] If true (default), generates a json report for the page scanned. If false, no json report is generated.
84+
"""
85+
results = {}
86+
for selected_page in page_list:
87+
page.goto(selected_page)
88+
filename = Axe._modify_filename_for_report(selected_page) if use_list_for_filename else ""
89+
results[selected_page] = Axe.run(
90+
page,
91+
filename=filename,
92+
ruleset=ruleset,
93+
report_on_violation_only=report_on_violation_only,
94+
strict_mode=strict_mode,
95+
html_report_generated=html_report_generated,
96+
json_report_generated=json_report_generated
97+
)
98+
return results
99+
100+
101+
@staticmethod
102+
def _build_run_command(ruleset: list[str]) -> str:
62103
return "run({runOnly: { type: 'tag', values: " + str(ruleset) + " }})"
63104

64105
@staticmethod

0 commit comments

Comments
 (0)