You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-**Description**: ThissettingallowsyoutosetauniqueIDforeverytestruntodistinguishthemintheserviceportal.UsingthesamerunIdformultipletestrunsresultsinerror. Ifyoudon't set it, the service package will generate a unique ID every time you trigger a test run.
240
240
-**Example**:
241
241
```xml
242
242
<Parameter name="RunId" value="sample-run-id1" />
@@ -258,7 +258,7 @@ Here's version of the `.runsettings` file with all the available options:
# Quickstart: Troubleshoot tests with Microsoft Playwright Testing Preview
@@ -44,12 +46,15 @@ To get started with publishing test results on Playwright Testing service, first
44
46
When the workspace creation finishes, you're redirected to the setup guide.
45
47
46
48
## Install Microsoft Playwright Testing package
49
+
::: zone pivot="playwright-test-runner"
47
50
48
51
To use the service, install the Microsoft Playwright Testing package.
49
52
50
53
```npm
51
54
npm init @azure/microsoft-playwright-testing
52
55
```
56
+
> [!NOTE]
57
+
> Make sure your project uses `@playwright/test` version 1.47 or above.
53
58
54
59
This command generates `playwright.service.config.ts` file which serves to:
55
60
@@ -75,21 +80,64 @@ export default defineConfig(
75
80
```
76
81
Setting the value as `false` ensures that cloud-hosted browsers aren't used to run the tests. The tests run on your local machine but the results and artifacts are published on the service.
77
82
83
+
::: zone-end
84
+
85
+
::: zone pivot="nunit-test-runner"
86
+
87
+
To use the service, install Microsoft Playwright Testing package.
> If you wish to accelerate your test run using cloud-hosted browser, you can set `useCloudHostedBrowsers` as true. This will run your tests on the service managed browsers.
80
126
81
127
## Configure the service region endpoint
82
128
83
129
In your setup, you have to provide the region-specific service endpoint. The endpoint depends on the Azure region you selected when creating the workspace.
84
130
85
-
To get the service endpoint URL:
131
+
To get the service endpoint URL, perform the following steps:
86
132
87
133
1. In **Add region endpoint in your setup**, copy the region endpoint for your workspace.
88
134
89
-
The endpoint URL matches the Azure region that you selected when creating the workspace.
135
+
The endpoint URL matches the Azure region that you selected when creating the workspace. Make sure this URL is available in `PLAYWRIGHT_SERVICE_URL` environment variable.
90
136
91
137
:::image type="content" source="./media/quickstart-run-end-to-end-tests/playwright-testing-region-endpoint.png" alt-text="Screenshot that shows how to copy the workspace region endpoint in the Playwright Testing portal." lightbox="./media/quickstart-run-end-to-end-tests/playwright-testing-region-endpoint.png":::
92
138
139
+
::: zone pivot="playwright-test-runner"
140
+
93
141
## Set up your environment
94
142
95
143
To set up your environment, you have to configure the `PLAYWRIGHT_SERVICE_URL` environment variable with the value you obtained in the previous steps.
@@ -110,6 +158,26 @@ We recommend that you use the `dotenv` module to manage your environment. With `
110
158
111
159
Make sure to replace the `{MY-REGION-ENDPOINT}` text placeholder with the value you copied earlier.
112
160
161
+
::: zone-end
162
+
163
+
::: zone pivot="nunit-test-runner"
164
+
## Set up service configuration
165
+
166
+
Create a file `PlaywrightServiceSetup.cs`in the root directory with the following content.
167
+
168
+
```csharp
169
+
using Azure.Developer.MicrosoftPlaywrightTesting.NUnit;
170
+
171
+
namespace PlaywrightTests; // Remember to change this as per your project namespace
172
+
173
+
[SetUpFixture]
174
+
public class PlaywrightServiceSetup : PlaywrightServiceNUnit {};
175
+
```
176
+
177
+
> [!NOTE]
178
+
> Make sure your project uses `Microsoft.Playwright.NUnit` version 1.47 or above.
179
+
180
+
::: zone-end
113
181
114
182
## Set up authentication
115
183
@@ -141,17 +209,32 @@ You can generate an access token from your Playwright Testing workspace and use
141
209
> We strongly recommend using Microsoft Entra ID for authentication to the service. If you are using access tokens, see [How to Manage Access Tokens](./how-to-manage-access-tokens.md)
142
210
143
211
## Enable artifacts in Playwright configuration
144
-
In the `playwright.config.ts` file of your project, make sure you're collecting all the required artifacts.
212
+
::: zone pivot="playwright-test-runner"
213
+
214
+
In the `playwright.config.ts` file of your project, make sure you are collecting all the required artifacts.
145
215
```typescript
146
216
use: {
147
217
trace: 'on-first-retry',
148
218
video:'retain-on-failure',
149
219
screenshot:'on'
150
-
},
220
+
}
151
221
```
222
+
::: zone-end
223
+
224
+
::: zone pivot="nunit-test-runner"
225
+
226
+
Enable artifacts such as screenshot, videos and traces to be captured by Playwright.
227
+
- For screenshots, see [capture screenshots](https://playwright.dev/dotnet/docs/screenshots#introduction)
228
+
- For videos, see [record videos for your tests](https://playwright.dev/dotnet/docs/videos#introduction)
229
+
- For traces, see [recording a trace](https://playwright.dev/dotnet/docs/trace-viewer-intro#recording-a-trace)
230
+
231
+
Once you collect these artifacts, make sure you attach them to the test path. For more information and example, see [sample configuration for NUnit](https://aka.ms/mpt/nunit-sample)
232
+
233
+
::: zone-end
152
234
153
235
## Run your tests and publish results on Microsoft Playwright Testing
154
236
237
+
::: zone pivot="playwright-test-runner"
155
238
You've now prepared the configuration for publishing test results and artifacts with Microsoft Playwright Testing. Run tests using the newly created `playwright.service.config.ts` file and publish test results and artifacts to the service.
156
239
157
240
```bash
@@ -168,6 +251,37 @@ Running 6 test using 2 worker
168
251
169
252
Test report: https://playwright.microsoft.com/workspaces/<workspace-id>/runs/<run-id>
170
253
```
254
+
::: zone-end
255
+
256
+
::: zone pivot="nunit-test-runner"
257
+
258
+
You've now prepared the configuration for publishing test results and artifacts with Microsoft Playwright Testing. Run tests using the `.runsettings` file and publish test results and artifacts to the service.
259
+
260
+
```bash
261
+
dotnet test --settings:.runsettings
262
+
```
263
+
264
+
The settings for your test run is defined in `.runsettings` file. See [how to use service package options](./how-to-use-service-config-file.md#options-in-runsettings-file)
265
+
266
+
> [!NOTE]
267
+
> For the Reporting feature of Microsoft Playwright Testing, you get charged based on the number test results published. If you're a first-time user or [getting started with a free trial](./how-to-try-playwright-testing-free.md), you might start with publishing single test result instead of your full test suite to avoid exhausting your free trial limits.
268
+
269
+
After the test run completes, you can view the test status in the terminal.
270
+
271
+
```output
272
+
Starting test execution, please wait...
273
+
274
+
Initializing reporting for this test run. You can view the results at: https://playwright.microsoft.com/workspaces/<workspace-id>/runs/<run-id>
275
+
276
+
A total of 100 test files matched the specified pattern.
277
+
278
+
Test Report: https://playwright.microsoft.com/workspaces/<workspace-id>/runs/<run-id>
Copy file name to clipboardExpand all lines: articles/playwright-testing/quickstart-run-end-to-end-tests.md
+14-1Lines changed: 14 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -165,6 +165,9 @@ Enable artifacts such as screenshot, videos and traces to be captured by Playwri
165
165
- For screenshots, see [capture screenshots](https://playwright.dev/dotnet/docs/screenshots#introduction)
166
166
- For videos, see [record videos for your tests](https://playwright.dev/dotnet/docs/videos#introduction)
167
167
- For traces, see [recording a trace](https://playwright.dev/dotnet/docs/trace-viewer-intro#recording-a-trace)
168
+
169
+
Once you collect these artifacts, make sure you attach them to the test path. For more information, see our [sample](https://aka.ms/mpt/nunit-sample)
170
+
168
171
::: zone-end
169
172
170
173
## Run your tests at scale and troubleshoot easily with Microsoft Playwright Testing
@@ -307,10 +310,14 @@ To run your Playwright test suite in Visual Studio Code with Microsoft Playwrigh
307
310
Run Playwright tests against browsers managed by the service and see the results in the unified portal using the configuration you created above.
308
311
309
312
```bash
310
-
dotnet test --logger "microsoft-playwright-testing" -- NUnit.NumberOfTestWorkers=20
313
+
dotnet test --settings:.runsettings --logger "microsoft-playwright-testing" -- NUnit.NumberOfTestWorkers=20
311
314
```
315
+
316
+
The settings for your test run is defined in `.runsettings` file. See [how to use service package options](./how-to-use-service-config-file.md#options-in-runsettings-file)
317
+
312
318
After the test run completes, you can view the test status in the terminal.
313
319
320
+
314
321
```output
315
322
Starting test execution, please wait...
316
323
@@ -345,7 +352,13 @@ Once your tests are running smoothly with the service, experiment with varying t
345
352
346
353
With Microsoft Playwright Testing, you can run with up to 50 parallel workers. Several factors influence the best configuration for your project, such as the CPU, memory, and network resources of your client machine, the target application's load-handling capacity, and the type of actions carried out in your tests.
347
354
355
+
::: zone pivot="playwright-test-runner"
348
356
You can specify the number of parallel workers on the Playwright CLI command-line, or configure the `workers` property in the Playwright service configuration file.
357
+
::: zone-end
358
+
359
+
::: zone pivot="nunit-test-runner"
360
+
You can specify the number of parallel workers on the Playwright CLI command-line, or configure the `NumberOfTestWorkers` property in the `.runsettings` file.
361
+
::: zone-end
349
362
350
363
Learn more about how to [determine the optimal configuration for optimizing test suite completion](./concept-determine-optimal-configuration.md).
0 commit comments