You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Quickstart: Set up continuous end-to-end testing with Microsoft Playwright Testing Preview
@@ -57,6 +59,7 @@ To get the service endpoint URL and store it as a CI workflow secret, perform th
57
59
| ----------- | ------------ |
58
60
|*PLAYWRIGHT_SERVICE_URL*| Paste the endpoint URL you copied previously. |
59
61
62
+
::: zone pivot="playwright-test-runner"
60
63
## Add service configuration file
61
64
62
65
If you haven't configured your Playwright tests yet for running them on cloud-hosted browsers, add a service configuration file to your repository. In the next step, you then specify this service configuration file on the Playwright CLI.
@@ -73,7 +76,7 @@ If you haven't configured your Playwright tests yet for running them on cloud-ho
73
76
- Accelerate build pipelines by running tests in parallel using cloud-hosted browsers.
74
77
- Simplify troubleshooting with easy access to test results and artifacts published to the service.
75
78
76
-
However, you can choose to use either of these features or both. See [How to use service features](./how-to-use-service-features.md#manage-features-while-running-tests) and update the service configuration file as per your requirement.
79
+
However, you can choose to use either of these features or both. See [How to use service features](./how-to-use-service-features.md#manage-features-while-running-tests) and update the service configuration file as per your requirements.
77
80
78
81
3. Save and commit the file to your source code repository.
79
82
@@ -97,9 +100,40 @@ In the `playwright.config.ts` file of your project, make sure you are collecting
97
100
screenshot:'on'
98
101
},
99
102
```
103
+
::: zone-end
104
+
105
+
::: zone pivot="nunit-test-runner"
106
+
## Set up service configuration
107
+
108
+
1. Create a new file `PlaywrightServiceSetup.cs` in the root directory of your project. This file facilitates authentication of your client with the service.
Enable artifacts such as screenshot, videos and traces to be captured by Playwright.
127
+
- For screenshots, see [capture screenshots](https://playwright.dev/dotnet/docs/screenshots#introduction)
128
+
- For videos, see [record videos for your tests](https://playwright.dev/dotnet/docs/videos#introduction)
129
+
- For traces, see [recording a trace](https://playwright.dev/dotnet/docs/trace-viewer-intro#recording-a-trace)
130
+
131
+
Once you collect these artifacts, make sure you attach them to the test path. For more information, see our [sample](https://aka.ms/mpt/nunit-sample)
132
+
::: zone-end
100
133
101
134
## Update the workflow definition
102
135
136
+
::: zone pivot="playwright-test-runner"
103
137
Update the CI workflow definition to run your Playwright tests with the Playwright CLI. Pass the [service configuration file](#add-service-configuration-file) as an input parameter for the Playwright CLI. You configure your environment by specifying environment variables.
104
138
105
139
1. Open the CI workflow definition
@@ -114,6 +148,20 @@ Update the CI workflow definition to run your Playwright tests with the Playwrig
114
148
115
149
# This step is to sign-in to Azure to run tests from GitHub Action workflow.
116
150
# Choose how to set up authentication to Azure from GitHub Actions. This is one example.
151
+
on:
152
+
push:
153
+
branches: [ main, master ]
154
+
pull_request:
155
+
branches: [ main, master ]
156
+
permissions: # Required when using Micosoft Entra ID to authenticate
157
+
id-token: write
158
+
contents: read
159
+
jobs:
160
+
test:
161
+
timeout-minutes: 60
162
+
runs-on: ubuntu-latest
163
+
steps:
164
+
- uses: actions/checkout@v4
117
165
- name: Login to Azure with AzPowershell (enableAzPSSession true)
118
166
uses: azure/login@v2
119
167
with:
@@ -177,23 +225,140 @@ Update the CI workflow definition to run your Playwright tests with the Playwrig
177
225
```
178
226
179
227
---
228
+
::: zone-end
229
+
230
+
::: zone pivot="nunit-test-runner"
231
+
232
+
Update the CI workflow definition to run your Playwright tests with the Playwright NUnit CLI. Pass the `.runsettings` file as an input parameter for the Playwright CLI. You configure your environment by specifying environment variables.
233
+
234
+
1. Open the CI workflow definition
235
+
236
+
1. Add the following steps to run your Playwright tests in Microsoft Playwright Testing.
237
+
238
+
The following steps describe the workflow changes for GitHub Actions or Azure Pipelines. Similarly, you can run your Playwright tests by using the Playwright CLI in other CI platforms.
239
+
240
+
# [GitHub Actions](#tab/github)
241
+
242
+
```yml
243
+
on:
244
+
push:
245
+
branches: [ main, master ]
246
+
pull_request:
247
+
branches: [ main, master ]
248
+
permissions: # Required when using AuthType as EntraId
249
+
id-token: write
250
+
contents: read
251
+
jobs:
252
+
test:
253
+
timeout-minutes: 60
254
+
runs-on: ubuntu-latest
255
+
steps:
256
+
- uses: actions/checkout@v4
257
+
# This step is to sign-in to Azure to run tests from GitHub Action workflow.
258
+
# Choose how to set up authentication to Azure from GitHub Actions. This is one example.
259
+
260
+
- name: Login to Azure with AzPowershell (enableAzPSSession true)
When the CI workflow is triggered, your Playwright tests run in your Microsoft Playwright Testing workspace on cloud-hosted browsers, across 20 parallel workers.
345
+
When the CI workflow is triggered, your Playwright tests run in your Microsoft Playwright Testing workspace on cloud-hosted browsers, across 20 parallel workers. The results and artifacts collected are published to the service and can be viewed on service portal.
184
346
185
347
> [!NOTE]
186
348
> Reporting feature is enabled by default for existing workspaces. This is being rolled out in stages and will take a few days. To avoid failures, confirm that `Rich diagnostics using reporting` setting is ON for your workspace before proceeding. See, [Enable reporting for workspace](./how-to-use-service-features.md#manage-feature-for-the-workspace).
187
349
188
350
> [!CAUTION]
189
-
> With Microsoft Playwright Testing, you get charged based on the number of total test minutes and test result published. If you're a first-time user or [getting started with a free trial](./how-to-try-playwright-testing-free.md), you might start with running a single test at scale instead of your full test suite to avoid exhausting your free test minutes and test results.
351
+
> With Microsoft Playwright Testing, you get charged based on the number of total test minutes consumed and test results published. If you're a first-time user or [getting started with a free trial](./how-to-try-playwright-testing-free.md), you might start with running a single test at scale instead of your full test suite to avoid exhausting your free test minutes and test results.
190
352
>
191
353
> After you validate that the test runs successfully, you can gradually increase the test load by running more tests with the service.
192
354
>
355
+
::: zone pivot="playwright-test-runner"
356
+
193
357
> You can run a single test with the service by using the following command-line:
194
358
>
195
359
> ```npx playwright test {name-of-file.spec.ts} --config=playwright.service.config.ts```
196
360
361
+
::: zone-end
197
362
## View test runs and results in the Playwright portal
198
363
199
364
You can now troubleshoot the CI pipeline in the Playwright portal,
Copy file name to clipboardExpand all lines: articles/playwright-testing/quickstart-generate-rich-reports-for-tests.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -208,7 +208,7 @@ You can generate an access token from your Playwright Testing workspace and use
208
208
> [!CAUTION]
209
209
> We strongly recommend using Microsoft Entra ID for authentication to the service. If you are using access tokens, see [How to Manage Access Tokens](./how-to-manage-access-tokens.md)
210
210
211
-
## Enable artifacts in Playwright configuration
211
+
## Enable artifacts in your Playwright setup
212
212
::: zone pivot="playwright-test-runner"
213
213
214
214
In the `playwright.config.ts` file of your project, make sure you are collecting all the required artifacts.
Copy file name to clipboardExpand all lines: articles/playwright-testing/quickstart-run-end-to-end-tests.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -146,7 +146,7 @@ You can generate an access token from your Playwright Testing workspace and use
146
146
> We strongly recommend using Microsoft Entra ID for authentication to the service. If you are using access tokens, see [How to Manage Access Tokens](./how-to-manage-access-tokens.md)
147
147
148
148
149
-
## Enable artifacts in Playwright configuration
149
+
## Enable artifacts in your Playwright setup
150
150
::: zone pivot="playwright-test-runner"
151
151
152
152
In the `playwright.config.ts` file of your project, make sure you are collecting all the required artifacts.
0 commit comments