You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
>WithMicrosoftPlaywrightTesting, yougetchargedbasedonthenumberoftotaltestminutes. Ifyou're a first-time user or [getting started with a free trial](./how-to-try-playwright-testing-free.md), you might start with running a single test at scale instead of your full test suite to avoid exhausting your free test minutes.
@@ -81,7 +81,7 @@ We recommend that you use the `dotenv` module to manage your environment. With `
81
81
> [!CAUTION]
82
82
> Make sure that you don't add the `.env` file to your source code repository to avoid leaking your access token value.
83
83
84
-
## Add Microsoft Playwright Testing configuration
84
+
## Add a service configuration file
85
85
86
86
To run your Playwright tests in your Microsoft Playwright Testing workspace, you need to add a service configuration file alongside your Playwright configuration file. The service configuration file references the environment variables to get the workspace endpoint and your access token.
87
87
@@ -112,7 +112,7 @@ To add the service configuration to your project:
112
112
// },
113
113
workers: 20,
114
114
115
-
// Enable screenshot testing and configure directory with expectations.
115
+
// Enable screenshot testing and configure directory with expectations.
@@ -142,60 +142,135 @@ To add the service configuration to your project:
142
142
143
143
You've now prepared the configuration for running your Playwright tests in the cloud with Microsoft Playwright Testing. You can either use the Playwright CLI to run your tests, or use the [Playwright Test Visual Studio Code extension](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).
144
144
145
-
Perform the following steps to run your Playwright tests with Microsoft Playwright Testing.
145
+
### Run a single test at scale
146
+
147
+
With Microsoft Playwright Testing, you get charged based on the number of total test minutes. If you're a first-time user or [getting started with a free trial](./how-to-try-playwright-testing-free.md), you might start with running a single test at scale instead of your full test suite to avoid exhausting your free test minutes.
148
+
149
+
After you validate that the test runs successfully, you can gradually increase the test load by running more tests with the service.
150
+
151
+
Perform the following steps to run a single Playwright test with Microsoft Playwright Testing:
146
152
147
153
# [Playwright CLI](#tab/playwrightcli)
148
154
149
-
When you use the Playwright CLI to run your tests, specify the service configuration file in the command-line to connect to use remote browsers.
155
+
To use the Playwright CLI to run your tests with Microsoft Playwright Testing, pass the service configuration file as a command-line parameter.
150
156
151
-
Open a terminal window and enter the following command to run your Playwright tests on remote browsers in your workspace:
157
+
1. Open a terminal window.
152
158
153
-
```bash
154
-
npx playwright test --config=playwright.service.config.ts --workers=20
155
-
```
156
-
157
-
Depending on the size of your test suite, this command runs your tests on up to 20 parallel workers.
159
+
1. Enter the following command to run your Playwright test on remote browsers in your workspace:
158
160
159
-
You should see a similar output when the tests complete:
161
+
Replace the `{name-of-file.spec.ts}` text placeholder with the name of your test specification file.
160
162
161
-
```output
162
-
Running 6 tests using 6 workers
163
-
6 passed (18.2s)
163
+
```bash
164
+
npx playwright test {name-of-file.spec.ts} --config=playwright.service.config.ts
165
+
```
164
166
165
-
To open last HTML report run:
167
+
After the test completes, you can view the test status in the terminal.
166
168
169
+
```output
170
+
Running 1 test using 1 worker
171
+
1 passed (2.2s)
172
+
173
+
To open last HTML report run:
174
+
167
175
npx playwright show-report
168
-
```
176
+
```
169
177
170
178
# [Visual Studio Code](#tab/vscode)
171
179
172
-
To run your Playwrights tests in Visual Studio Code with Microsoft Playwright Testing:
180
+
To run a single Playwright testin Visual Studio Code with Microsoft Playwright Testing, selectthe service configuration file in the **Test Explorer** view. Then selectand run the test from the list of tests.
173
181
174
182
1. Install the [Playwright Test Visual Studio Code extension](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).
175
183
176
184
1. Open the **Test Explorer** view in the activity bar.
177
185
178
-
The test explorer automatically detects your Playwright tests and the service configuration.
186
+
The test explorer automatically detects your Playwright tests and the service configurationin your project.
179
187
180
188
:::image type="content" source="./media/quickstart-run-end-to-end-tests/visual-studio-code-test-explorer.png" alt-text="Screenshot that shows the Test Explorer view in Visual Studio Code, which lists the Playwright tests." lightbox="./media/quickstart-run-end-to-end-tests/visual-studio-code-test-explorer.png":::
181
189
182
-
1. Select a service profile to run your tests with Microsoft Playwright Testing.
190
+
1. Select **Select Default Profile**, and thenselectyourdefault projects from the service configuration file.
183
191
184
192
Notice that the service run profiles are coming from the `playwright.service.config.ts` file you added previously.
185
193
186
-
Optionally, select **Select Default Profile**, and then select your default projects. By setting a default profile, you can automatically run your services with the service, or run multiple Playwright projects simultaneously.
194
+
By setting a default profile, you can automatically run your tests with the service, or run multiple Playwright projects simultaneously.
187
195
188
196
:::image type="content" source="./media/quickstart-run-end-to-end-tests/visual-studio-code-choose-run-profile.png" alt-text="Screenshot that shows the menu to choose a run profile for your tests, highlighting the projects from the service configuration file." lightbox="./media/quickstart-run-end-to-end-tests/visual-studio-code-choose-run-profile.png":::
189
197
198
+
1. From the list of tests, selectthe**Run test** button next to a test to run it.
199
+
200
+
The test runs on the projects you selected in the default profile. If you selected one or more projects from the service configuration, the test runs on remote browsers in your workspace.
201
+
202
+
:::image type="content" source="./media/quickstart-run-end-to-end-tests/visual-studio-code-run-test.png" alt-text="Screenshot that shows how to run a single test in Visual Studio Code." lightbox="./media/quickstart-run-end-to-end-tests/visual-studio-code-run-test.png":::
203
+
190
204
> [!TIP]
191
-
> You can still debug your test code when you run your tests on remote browsers.
205
+
> You can still debug your test code when you run your tests on remote browsers by using the **Debug test** button.
192
206
193
207
1. You can view the test results directly in Visual Studio Code.
194
208
195
209
:::image type="content" source="./media/quickstart-run-end-to-end-tests/visual-studio-code-test-results.png" alt-text="Screenshot that shows the Playwright test results in Visual Studio Code." lightbox="./media/quickstart-run-end-to-end-tests/visual-studio-code-test-results.png":::
196
210
197
211
---
198
212
213
+
You can now run multiple tests with the service, or run your entire test suite on remote browsers.
214
+
215
+
> [!CAUTION]
216
+
> Depending on the size of your test suite, you might incur additional charges for the test minutes beyond your allotted free test minutes.
217
+
218
+
### Run a full test suite at scale
219
+
220
+
Now that you've validated that you can run a single test with Microsoft Playwright Testing, you can run a full Playwright test suite at scale.
221
+
222
+
Perform the following steps to run a full Playwright test suite with Microsoft Playwright Testing:
223
+
224
+
# [Playwright CLI](#tab/playwrightcli)
225
+
226
+
When you run multiple Playwright tests or a full test suite with Microsoft Playwright Testing, you can optionally specify the number of parallel workers as a command-line parameter.
227
+
228
+
1. Open a terminal window.
229
+
230
+
1. Enter the following command to run your Playwright test suite on remote browsers in your workspace:
231
+
232
+
```bash
233
+
npx playwright test --config=playwright.service.config.ts --workers=20
234
+
```
235
+
236
+
Depending on the size of your test suite, this command runs your tests on up to 20 parallel workers.
237
+
238
+
After the test completes, you can view the test status in the terminal.
239
+
240
+
```output
241
+
Running 6 tests using 6 workers
242
+
6 passed (18.2s)
243
+
244
+
To open last HTML report run:
245
+
246
+
npx playwright show-report
247
+
```
248
+
249
+
# [Visual Studio Code](#tab/vscode)
250
+
251
+
To run your Playwright test suite in Visual Studio Code with Microsoft Playwright Testing:
252
+
253
+
1. Open the **Test Explorer** view in the activity bar.
254
+
255
+
1. Select the **Run tests** button to run all tests with Microsoft Playwright Testing.
256
+
257
+
When you run all tests, the default profile is used. In the previous step, you configured the default profile to use projects from the service configuration.
258
+
259
+
:::image type="content" source="./media/quickstart-run-end-to-end-tests/visual-studio-code-run-all-tests.png" alt-text="Screenshot that shows how to run all tests in Visual Studio Code." lightbox="./media/quickstart-run-end-to-end-tests/visual-studio-code-run-all-tests.png":::
260
+
261
+
> [!TIP]
262
+
> You can still debug your test code when you run your tests on remote browsers by using the **Debug tests** button.
263
+
264
+
1. Alternately, you can select a specific service configuration from the list to only run the tests for a specific browser configuration.
265
+
266
+
:::image type="content" source="./media/quickstart-run-end-to-end-tests/visual-studio-code-run-all-tests-select-project.png" alt-text="Screenshot that shows how to run all tests for a specific browser configuration, by selecting the project in Visual Studio Code." lightbox="./media/quickstart-run-end-to-end-tests/visual-studio-code-run-all-tests-select-project.png":::
267
+
268
+
1. You can view all test results in the **Test results** tab.
269
+
270
+
---
271
+
272
+
## View test runs in the Playwright portal
273
+
199
274
Go to the [Playwright portal](https://aka.ms/mpt/portal) to view the test run metadata and activity log for your workspace.
200
275
201
276
:::image type="content" source="./media/quickstart-run-end-to-end-tests/playwright-testing-activity-log.png" alt-text="Screenshot that shows the activity log for a workspace in the Playwright Testing portal." lightbox="./media/quickstart-run-end-to-end-tests/playwright-testing-activity-log.png":::
@@ -204,7 +279,11 @@ The activity log lists for each test run the following details: the total test c
204
279
205
280
## Optimize parallel worker configuration
206
281
207
-
Once your tests are running smoothly with the service, experiment with varying the number of parallel workers to determine the optimal configuration that minimizes test completion time. With Microsoft Playwright Testing, you can run with up to 50 parallel workers. Several factors influence the best configuration for your project, such as the CPU, memory, and network resources of your client machine, the target application's load-handling capacity, and the type of actions carried out in your tests.
282
+
Once your tests are running smoothly with the service, experiment with varying the number of parallel workers to determine the optimal configuration that minimizes test completion time.
283
+
284
+
With Microsoft Playwright Testing, you can run with up to 50 parallel workers. Several factors influence the best configuration for your project, such as the CPU, memory, and network resources of your client machine, the target application's load-handling capacity, and the type of actions carried out in your tests.
285
+
286
+
You can specify the number of parallel workers on the Playwright CLI command-line, or configure the `workers` property in the Playwright service configuration file.
208
287
209
288
Learn more about how to [determine the optimal configuration for optimizing test suite completion](./concept-determine-optimal-configuration.md).
0 commit comments