Skip to content

Commit f63b75a

Browse files
authored
Merge pull request #292409 from vvs11/vvs11-bug-fixes-ga
update ci article to add authentication information
2 parents a61ba4c + 06c0eac commit f63b75a

File tree

2 files changed

+202
-104
lines changed

2 files changed

+202
-104
lines changed

articles/playwright-testing/playwright-testing-reporting-with-sharding.md

Lines changed: 62 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,13 @@ ms.date: 05/06/2024
1010
ms.author: vanshsingh
1111
---
1212

13-
# Use Microsoft Playwright Testing Reporting with Playwright sharding (preview)
13+
# Use Microsoft Playwright Testing Reporting preview with Playwright sharding
1414

1515
In this article, you learn how to use the Microsoft Playwright Testing service's reporting feature with test runs that use [Playwright's sharding features](https://playwright.dev/docs/test-sharding).
1616

1717
Playwright's sharding enables you to split your test suite to run across multiple machines simultaneously. This feature helps running tests in parallel.
1818

19-
You can use Playwright Testing's reporting feature to get a consolidated report of a test run with sharding. You need to make sure you set the variable `PLAYWRIGHT_SERVICE_RUN_ID` so that it remains same across all shards.
19+
You can use Playwright Testing's reporting feature to get a consolidated report of a test run with sharding. Ensure the `runId` of the test run is same across all shards.
2020

2121
> [!IMPORTANT]
2222
> Microsoft Playwright Testing is currently in preview. For legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability, see the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
@@ -27,65 +27,80 @@ You can use Playwright Testing's reporting feature to get a consolidated report
2727

2828
* Set up continuous end-to-end testing. Complete the [Quickstart: Set up continuous end-to-end testing with Microsoft Playwright Testing Preview](./quickstart-automate-end-to-end-testing.md) to set up continuous integration (CI) pipeline.
2929

30+
## Update playwright service configuration file
31+
32+
Add or update `runId` to `playwright.service.config.ts` file in your setup.
33+
34+
```typescript
35+
36+
export default defineConfig(
37+
config,
38+
getServiceConfig(config, {
39+
runId: process.env.PLAYWRIGHT_SERVICE_RUN_ID, //Set runId for the test run
40+
}),
41+
);
42+
43+
```
44+
You can use `PLAYWRIGHT_SERVICE_RUN_ID` variable in your setup to ensure the `runId` remains same across all shards.
3045

3146
## Set up variables
3247

33-
The `PLAYWRIGHT_SERVICE_RUN_ID` variable is an identifier that is used by Playwright Testing service to distinguish between test runs. The results from multiple runs with same `RUN_ID` are reported to the same run on the Playwright portal.
48+
The `runId` setting is used as an identifier by Playwright Testing service to distinguish between test runs. The results from multiple runs with same `runId` are reported to the same run on the Playwright portal.
3449

35-
By default, a test run that uses reporting feature automatically generates a unique `RUN_ID` unless you explicitly set the value yourself. If the value of the variable remains same across runs, the results are reported together in the same run on the Playwright portal.
50+
By default, a test run that uses reporting feature automatically generates a unique `runId` unless you explicitly set the value yourself. If the value of the variable remains same across runs, the results are reported together in the same run on the Playwright portal.
3651

3752
> [!Tip]
3853
> If you use the cloud-hosted browsers provided by Microsoft Playwright Testing service to run your tests, you might have already set this variable. To avoid overwrites, make sure you set it only once.
3954
4055

41-
While using sharding, make sure the same `RUN_ID` is set across all the shards for the results to be reported together.
56+
While using sharding, make sure the same `runId` is set across all the shards for the results to be reported together. Use the variable `PLAYWRIGHT_SERVICE_RUN_ID` and set the value same across all shards.
4257

4358
Here's an example of how you can set it in your pipeline via GitHub Actions.
4459

4560
```yml
46-
name: Playwright Tests
47-
on:
48-
push:
49-
branches: [ main, master ]
50-
pull_request:
51-
branches: [ main, master ]
52-
workflow_dispatch:
53-
strategy:
54-
fail-fast: false
55-
matrix:
56-
shardIndex: [1, 2, 3, 4]
57-
shardTotal: [4]
58-
permissions:
59-
id-token: write
60-
contents: read
61-
jobs:
62-
test:
63-
timeout-minutes: 60
64-
runs-on: ubuntu-latest
65-
steps:
66-
- uses: actions/checkout@v3
67-
- uses: actions/setup-node@v3
68-
with:
69-
node-version: 18
70-
# This step is to sign-in to Azure to run tests from GitHub Action workflow.
71-
# You can choose how set up Authentication to Azure from GitHub Actions, this is one example.
72-
- name: Login to Azure with AzPowershell (enableAzPSSession true)
73-
uses: azure/login@v2
74-
with:
75-
client-id: ${{ secrets.AZURE_CLIENT_ID }}
76-
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
77-
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
78-
enable-AzPSSession: true
79-
80-
- name: Install dependencies
61+
name: Playwright Tests (Microsoft Playwright Testing)
62+
on:
63+
push:
64+
branches: [ main, master ]
65+
pull_request:
66+
branches: [ main, master ]
67+
workflow_dispatch:
68+
strategy:
69+
fail-fast: false
70+
matrix:
71+
shardIndex: [1, 2, 3, 4]
72+
shardTotal: [4]
73+
permissions: # Required when using Microsoft Entra ID to authenticate
74+
id-token: write
75+
contents: read
76+
jobs:
77+
test:
78+
timeout-minutes: 60
79+
runs-on: ubuntu-latest
80+
steps:
81+
- uses: actions/checkout@v3
82+
- uses: actions/setup-node@v3
83+
with:
84+
node-version: 18
85+
# This step is to sign-in to Azure to run tests from GitHub Action workflow.
86+
# You can choose how set up Authentication to Azure from GitHub Actions, this is one example.
87+
- name: Login to Azure with AzPowershell (enableAzPSSession true)
88+
uses: azure/login@v2
89+
with:
90+
client-id: ${{ secrets.AZURE_CLIENT_ID }} # Required when using Microsoft Entra ID to authenticate
91+
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
92+
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
93+
enable-AzPSSession: true
94+
95+
- name: Install dependencies
8196
working-directory: path/to/playwright/folder # update accordingly
82-
run: npm ci
97+
run: npm ci
8398

84-
- name: Run Playwright tests
99+
- name: Run Playwright tests
85100
working-directory: path/to/playwright/folder # update accordingly
86-
env:
87-
# Regional endpoint for Microsoft Playwright Testing
88-
PLAYWRIGHT_SERVICE_URL: ${{ secrets.PLAYWRIGHT_SERVICE_URL }}
89-
PLAYWRIGHT_SERVICE_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}-${{ github.sha } #This Run_ID will be unique and will remain same across all shards
90-
run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
101+
env:
102+
# Regional endpoint for Microsoft Playwright Testing
103+
PLAYWRIGHT_SERVICE_URL: ${{ secrets.PLAYWRIGHT_SERVICE_URL }}
104+
PLAYWRIGHT_SERVICE_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}-${{ github.sha } #This Run_ID will be unique and will remain same across all shards
105+
run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
91106
```

0 commit comments

Comments
 (0)