You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/playwright-testing/playwright-testing-reporting-with-sharding.md
+62-47Lines changed: 62 additions & 47 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,13 +10,13 @@ ms.date: 05/06/2024
10
10
ms.author: vanshsingh
11
11
---
12
12
13
-
# Use Microsoft Playwright Testing Reporting with Playwright sharding (preview)
13
+
# Use Microsoft Playwright Testing Reporting preview with Playwright sharding
14
14
15
15
In this article, you learn how to use the Microsoft Playwright Testing service's reporting feature with test runs that use [Playwright's sharding features](https://playwright.dev/docs/test-sharding).
16
16
17
17
Playwright's sharding enables you to split your test suite to run across multiple machines simultaneously. This feature helps running tests in parallel.
18
18
19
-
You can use Playwright Testing's reporting feature to get a consolidated report of a test run with sharding. You need to make sure you set the variable `PLAYWRIGHT_SERVICE_RUN_ID` so that it remains same across all shards.
19
+
You can use Playwright Testing's reporting feature to get a consolidated report of a test run with sharding. Ensure the `runId` of the test run is same across all shards.
20
20
21
21
> [!IMPORTANT]
22
22
> Microsoft Playwright Testing is currently in preview. For legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability, see the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
@@ -27,65 +27,80 @@ You can use Playwright Testing's reporting feature to get a consolidated report
27
27
28
28
* Set up continuous end-to-end testing. Complete the [Quickstart: Set up continuous end-to-end testing with Microsoft Playwright Testing Preview](./quickstart-automate-end-to-end-testing.md) to set up continuous integration (CI) pipeline.
29
29
30
+
## Update playwright service configuration file
31
+
32
+
Add or update `runId` to `playwright.service.config.ts` file in your setup.
33
+
34
+
```typescript
35
+
36
+
exportdefaultdefineConfig(
37
+
config,
38
+
getServiceConfig(config, {
39
+
runId: process.env.PLAYWRIGHT_SERVICE_RUN_ID, //Set runId for the test run
40
+
}),
41
+
);
42
+
43
+
```
44
+
You can use `PLAYWRIGHT_SERVICE_RUN_ID` variable in your setup to ensure the `runId` remains same across all shards.
30
45
31
46
## Set up variables
32
47
33
-
The `PLAYWRIGHT_SERVICE_RUN_ID` variable is an identifier that is used by Playwright Testing service to distinguish between test runs. The results from multiple runs with same `RUN_ID` are reported to the same run on the Playwright portal.
48
+
The `runId` setting is used as an identifier by Playwright Testing service to distinguish between test runs. The results from multiple runs with same `runId` are reported to the same run on the Playwright portal.
34
49
35
-
By default, a test run that uses reporting feature automatically generates a unique `RUN_ID` unless you explicitly set the value yourself. If the value of the variable remains same across runs, the results are reported together in the same run on the Playwright portal.
50
+
By default, a test run that uses reporting feature automatically generates a unique `runId` unless you explicitly set the value yourself. If the value of the variable remains same across runs, the results are reported together in the same run on the Playwright portal.
36
51
37
52
> [!Tip]
38
53
> If you use the cloud-hosted browsers provided by Microsoft Playwright Testing service to run your tests, you might have already set this variable. To avoid overwrites, make sure you set it only once.
39
54
40
55
41
-
While using sharding, make sure the same `RUN_ID` is set across all the shards for the results to be reported together.
56
+
While using sharding, make sure the same `runId` is set across all the shards for the results to be reported together. Use the variable `PLAYWRIGHT_SERVICE_RUN_ID` and set the value same across all shards.
42
57
43
58
Here's an example of how you can set it in your pipeline via GitHub Actions.
44
59
45
60
```yml
46
-
name: Playwright Tests
47
-
on:
48
-
push:
49
-
branches: [ main, master ]
50
-
pull_request:
51
-
branches: [ main, master ]
52
-
workflow_dispatch:
53
-
strategy:
54
-
fail-fast: false
55
-
matrix:
56
-
shardIndex: [1, 2, 3, 4]
57
-
shardTotal: [4]
58
-
permissions:
59
-
id-token: write
60
-
contents: read
61
-
jobs:
62
-
test:
63
-
timeout-minutes: 60
64
-
runs-on: ubuntu-latest
65
-
steps:
66
-
- uses: actions/checkout@v3
67
-
- uses: actions/setup-node@v3
68
-
with:
69
-
node-version: 18
70
-
# This step is to sign-in to Azure to run tests from GitHub Action workflow.
71
-
# You can choose how set up Authentication to Azure from GitHub Actions, this is one example.
72
-
- name: Login to Azure with AzPowershell (enableAzPSSession true)
PLAYWRIGHT_SERVICE_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}-${{ github.sha } #This Run_ID will be unique and will remain same across all shards
90
-
run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
101
+
env:
102
+
# Regional endpoint for Microsoft Playwright Testing
PLAYWRIGHT_SERVICE_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}-${{ github.sha } #This Run_ID will be unique and will remain same across all shards
105
+
run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
0 commit comments