Skip to content

Commit fb87941

Browse files
committed
added NUnit docs
1 parent 7136780 commit fb87941

File tree

3 files changed

+135
-8
lines changed

3 files changed

+135
-8
lines changed

articles/playwright-testing/how-to-use-service-config-file.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ export default defineConfig(
9393
```
9494

9595
* **`runId`**:
96-
- **Description**: This setting allows you to set a unique ID for every test run to distinguish them in the service portal.
96+
- **Description**: This setting allows you to set a unique ID for every test run to distinguish them in the service portal. Using the same runId for multiple test runs results in error.
9797
- **Example**:
9898
```typescript
9999
runId: new Date().toISOString()
@@ -236,7 +236,7 @@ Here's version of the `.runsettings` file with all the available options:
236236
```
237237

238238
* **`runId`**:
239-
- **Description**: This setting allows you to set a unique ID for every test run to distinguish them in the service portal.
239+
- **Description**: This setting allows you to set a unique ID for every test run to distinguish them in the service portal. Using the same runId for multiple test runs results in error. If you don't set it, the service package will generate a unique ID every time you trigger a test run.
240240
- **Example**:
241241
```xml
242242
<Parameter name="RunId" value="sample-run-id1" />
@@ -258,7 +258,7 @@ Here's version of the `.runsettings` file with all the available options:
258258
```
259259

260260
* **`reporter`**
261-
- **Description**: You can publish your test results and artifacts to the service using `microsoft-playwright-testing` logger. You can disable reporting by removing this from your `.runsettings`
261+
- **Description**: You can publish your test results and artifacts to the service using `microsoft-playwright-testing` logger. You can disable reporting by removing this from your `.runsettings` or by setting it to false.
262262
- **Default Value**: true
263263
- **Example**:
264264
```xml

articles/playwright-testing/quickstart-generate-rich-reports-for-tests.md

Lines changed: 118 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ description: 'This quickstart shows how to troubleshoot your test runs using Mic
44
ms.topic: quickstart
55
ms.date: 09/23/2024
66
ms.custom: playwright-testing-preview
7+
zone_pivot_group_filename: playwright-testing/ZonePivotGroups.json
8+
zone_pivot_groups: microsoft-playwright-testing
79
---
810

911
# Quickstart: Troubleshoot tests with Microsoft Playwright Testing Preview
@@ -44,12 +46,15 @@ To get started with publishing test results on Playwright Testing service, first
4446
When the workspace creation finishes, you're redirected to the setup guide.
4547

4648
## Install Microsoft Playwright Testing package
49+
::: zone pivot="playwright-test-runner"
4750

4851
To use the service, install the Microsoft Playwright Testing package.
4952

5053
```npm
5154
npm init @azure/microsoft-playwright-testing
5255
```
56+
> [!NOTE]
57+
> Make sure your project uses `@playwright/test` version 1.47 or above.
5358
5459
This command generates `playwright.service.config.ts` file which serves to:
5560

@@ -75,21 +80,64 @@ export default defineConfig(
7580
```
7681
Setting the value as `false` ensures that cloud-hosted browsers aren't used to run the tests. The tests run on your local machine but the results and artifacts are published on the service.
7782

83+
::: zone-end
84+
85+
::: zone pivot="nunit-test-runner"
86+
87+
To use the service, install Microsoft Playwright Testing package.
88+
89+
```PowerShell
90+
dotnet add package Azure.Developer.MicrosoftPlaywrightTesting.NUnit --prerelease
91+
```
92+
93+
> [!NOTE]
94+
> Make sure your project uses `Microsoft.Playwright.NUnit` version 1.47 or above.
95+
96+
To use only reporting feature, update the following in the `.runsettings` file of your project:
97+
1. Disable cloud-hosted browsers by setting `useCloudHostedBrowsers` as false.
98+
2. Add Microsoft Playwright Testing logger in `Loggers' section.
99+
100+
```xml
101+
<?xml version="1.0" encoding="utf-8"?>
102+
<RunSettings>
103+
<TestRunParameters>
104+
<!--Select if you want to use cloud-hosted browsers to run your Playwright tests.-->
105+
<Parameter name="UseCloudHostedBrowsers" value="true" />
106+
<!--Select the authentication method you want to use with Entra-->
107+
</TestRunParameters>
108+
.
109+
.
110+
.
111+
<LoggerRunSettings>
112+
<Loggers>
113+
<!--Microsoft Playwright Testing service logger for reporting -->
114+
<Logger friendlyName="microsoft-playwright-testing" enabled="True" />
115+
<!--could enable any logger additionally -->
116+
<Logger friendlyName="trx" enabled="false" />
117+
</Loggers>
118+
</LoggerRunSettings>
119+
</RunSettings>
120+
121+
```
122+
::: zone-end
123+
78124
> [!TIP]
79125
> If you wish to accelerate your test run using cloud-hosted browser, you can set `useCloudHostedBrowsers` as true. This will run your tests on the service managed browsers.
80126
81127
## Configure the service region endpoint
82128

83129
In your setup, you have to provide the region-specific service endpoint. The endpoint depends on the Azure region you selected when creating the workspace.
84130

85-
To get the service endpoint URL:
131+
To get the service endpoint URL, perform the following steps:
86132

87133
1. In **Add region endpoint in your setup**, copy the region endpoint for your workspace.
88134

89-
The endpoint URL matches the Azure region that you selected when creating the workspace.
135+
The endpoint URL matches the Azure region that you selected when creating the workspace. Make sure this URL is available in `PLAYWRIGHT_SERVICE_URL` environment variable.
90136

91137
:::image type="content" source="./media/quickstart-run-end-to-end-tests/playwright-testing-region-endpoint.png" alt-text="Screenshot that shows how to copy the workspace region endpoint in the Playwright Testing portal." lightbox="./media/quickstart-run-end-to-end-tests/playwright-testing-region-endpoint.png":::
92138

139+
::: zone pivot="playwright-test-runner"
140+
93141
## Set up your environment
94142

95143
To set up your environment, you have to configure the `PLAYWRIGHT_SERVICE_URL` environment variable with the value you obtained in the previous steps.
@@ -110,6 +158,26 @@ We recommend that you use the `dotenv` module to manage your environment. With `
110158

111159
Make sure to replace the `{MY-REGION-ENDPOINT}` text placeholder with the value you copied earlier.
112160

161+
::: zone-end
162+
163+
::: zone pivot="nunit-test-runner"
164+
## Set up service configuration
165+
166+
Create a file `PlaywrightServiceSetup.cs` in the root directory with the following content.
167+
168+
```csharp
169+
using Azure.Developer.MicrosoftPlaywrightTesting.NUnit;
170+
171+
namespace PlaywrightTests; // Remember to change this as per your project namespace
172+
173+
[SetUpFixture]
174+
public class PlaywrightServiceSetup : PlaywrightServiceNUnit {};
175+
```
176+
177+
> [!NOTE]
178+
> Make sure your project uses `Microsoft.Playwright.NUnit` version 1.47 or above.
179+
180+
::: zone-end
113181

114182
## Set up authentication
115183

@@ -141,17 +209,32 @@ You can generate an access token from your Playwright Testing workspace and use
141209
> We strongly recommend using Microsoft Entra ID for authentication to the service. If you are using access tokens, see [How to Manage Access Tokens](./how-to-manage-access-tokens.md)
142210

143211
## Enable artifacts in Playwright configuration
144-
In the `playwright.config.ts` file of your project, make sure you're collecting all the required artifacts.
212+
::: zone pivot="playwright-test-runner"
213+
214+
In the `playwright.config.ts` file of your project, make sure you are collecting all the required artifacts.
145215
```typescript
146216
use: {
147217
trace: 'on-first-retry',
148218
video:'retain-on-failure',
149219
screenshot:'on'
150-
},
220+
}
151221
```
222+
::: zone-end
223+
224+
::: zone pivot="nunit-test-runner"
225+
226+
Enable artifacts such as screenshot, videos and traces to be captured by Playwright.
227+
- For screenshots, see [capture screenshots](https://playwright.dev/dotnet/docs/screenshots#introduction)
228+
- For videos, see [record videos for your tests](https://playwright.dev/dotnet/docs/videos#introduction)
229+
- For traces, see [recording a trace](https://playwright.dev/dotnet/docs/trace-viewer-intro#recording-a-trace)
230+
231+
Once you collect these artifacts, make sure you attach them to the test path. For more information and example, see [sample configuration for NUnit](https://aka.ms/mpt/nunit-sample)
232+
233+
::: zone-end
152234

153235
## Run your tests and publish results on Microsoft Playwright Testing
154236

237+
::: zone pivot="playwright-test-runner"
155238
You've now prepared the configuration for publishing test results and artifacts with Microsoft Playwright Testing. Run tests using the newly created `playwright.service.config.ts` file and publish test results and artifacts to the service.
156239
157240
```bash
@@ -168,6 +251,37 @@ Running 6 test using 2 worker
168251
169252
Test report: https://playwright.microsoft.com/workspaces/<workspace-id>/runs/<run-id>
170253
```
254+
::: zone-end
255+
256+
::: zone pivot="nunit-test-runner"
257+
258+
You've now prepared the configuration for publishing test results and artifacts with Microsoft Playwright Testing. Run tests using the `.runsettings` file and publish test results and artifacts to the service.
259+
260+
```bash
261+
dotnet test --settings:.runsettings
262+
```
263+
264+
The settings for your test run is defined in `.runsettings` file. See [how to use service package options](./how-to-use-service-config-file.md#options-in-runsettings-file)
265+
266+
> [!NOTE]
267+
> For the Reporting feature of Microsoft Playwright Testing, you get charged based on the number test results published. If you're a first-time user or [getting started with a free trial](./how-to-try-playwright-testing-free.md), you might start with publishing single test result instead of your full test suite to avoid exhausting your free trial limits.
268+
269+
After the test run completes, you can view the test status in the terminal.
270+
271+
```output
272+
Starting test execution, please wait...
273+
274+
Initializing reporting for this test run. You can view the results at: https://playwright.microsoft.com/workspaces/<workspace-id>/runs/<run-id>
275+
276+
A total of 100 test files matched the specified pattern.
277+
278+
Test Report: https://playwright.microsoft.com/workspaces/<workspace-id>/runs/<run-id>
279+
280+
Passed! - Failed: 0, Passed: 100, Skipped: 0, Total: 100, Duration: 10 m - PlaywrightTestsNUnit.dll (net7.0)
281+
282+
Workload updates are available. Run `dotnet workload list` for more information.
283+
```
284+
::: zone-end
171285

172286
> [!CAUTION]
173287
> Depending on the size of your test suite, you might incur additional charges for the test results beyond your allotted free test results.

articles/playwright-testing/quickstart-run-end-to-end-tests.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,9 @@ Enable artifacts such as screenshot, videos and traces to be captured by Playwri
165165
- For screenshots, see [capture screenshots](https://playwright.dev/dotnet/docs/screenshots#introduction)
166166
- For videos, see [record videos for your tests](https://playwright.dev/dotnet/docs/videos#introduction)
167167
- For traces, see [recording a trace](https://playwright.dev/dotnet/docs/trace-viewer-intro#recording-a-trace)
168+
169+
Once you collect these artifacts, make sure you attach them to the test path. For more information, see our [sample](https://aka.ms/mpt/nunit-sample)
170+
168171
::: zone-end
169172

170173
## Run your tests at scale and troubleshoot easily with Microsoft Playwright Testing
@@ -307,10 +310,14 @@ To run your Playwright test suite in Visual Studio Code with Microsoft Playwrigh
307310
Run Playwright tests against browsers managed by the service and see the results in the unified portal using the configuration you created above.
308311
309312
```bash
310-
dotnet test --logger "microsoft-playwright-testing" -- NUnit.NumberOfTestWorkers=20
313+
dotnet test --settings:.runsettings --logger "microsoft-playwright-testing" -- NUnit.NumberOfTestWorkers=20
311314
```
315+
316+
The settings for your test run is defined in `.runsettings` file. See [how to use service package options](./how-to-use-service-config-file.md#options-in-runsettings-file)
317+
312318
After the test run completes, you can view the test status in the terminal.
313319
320+
314321
```output
315322
Starting test execution, please wait...
316323
@@ -345,7 +352,13 @@ Once your tests are running smoothly with the service, experiment with varying t
345352
346353
With Microsoft Playwright Testing, you can run with up to 50 parallel workers. Several factors influence the best configuration for your project, such as the CPU, memory, and network resources of your client machine, the target application's load-handling capacity, and the type of actions carried out in your tests.
347354

355+
::: zone pivot="playwright-test-runner"
348356
You can specify the number of parallel workers on the Playwright CLI command-line, or configure the `workers` property in the Playwright service configuration file.
357+
::: zone-end
358+
359+
::: zone pivot="nunit-test-runner"
360+
You can specify the number of parallel workers on the Playwright CLI command-line, or configure the `NumberOfTestWorkers` property in the `.runsettings` file.
361+
::: zone-end
349362

350363
Learn more about how to [determine the optimal configuration for optimizing test suite completion](./concept-determine-optimal-configuration.md).
351364

0 commit comments

Comments
 (0)