Merge pull request #290539 from MicrosoftDocs/release-ignite-playwright-testing

[Ignite 2024 ship room] Playwright testing content updates for NUnit runner

Author: MikeRayMSFT

articles/playwright-testing/how-to-manage-authentication.md

@@ -4,6 +4,8 @@ description: Learn how to manage authentication and authorization for Microsoft
 ms.topic: how-to
 ms.date: 09/07/2024
 ms.custom: playwright-testing-preview
+zone_pivot_group_filename: playwright-testing/zone-pivots-groups.json
+zone_pivot_groups: microsoft-playwright-testing
 ---
 
 # Manage authentication and authorization for Microsoft Playwright Testing preview
@@ -49,6 +51,8 @@ To enable authentication using access tokens:
 
 ## Set up authentication using access-tokens
 
+::: zone pivot="playwright-test-runner"
+
 1. While running the tests, enable access token auth in the `playwright.service.config.ts` file in your setup. 
 
     ```typescript
@@ -57,14 +61,29 @@ To enable authentication using access tokens:
         serviceAuthType:'ACCESS_TOKEN'
     }));
     ```
+::: zone-end
+
+::: zone pivot="nunit-test-runner"
+
+1. While running the tests, enable access token auth in the `.runsettings` file in your setup. 
+
+    ```xml
+    <TestRunParameters>
+        <!-- Use this option when you want to authenticate using access tokens. This mode of auth should be enabled for the workspace. -->
+         <Parameter name="ServiceAuthType" value="AccessToken" />
+    </TestRunParameters>
+    ```
+::: zone-end
+
+2. Create access token 
 
-1. Create access token 
+    Follow the steps to [create an access token](./how-to-manage-access-tokens.md#generate-a-workspace-access-token). Copy the value of the access token generated.
 
-    Follow the steps to [create an access token](./how-to-manage-access-tokens.md#generate-a-workspace-access-token)
+::: zone pivot="playwright-test-runner"
 
-1. Set up your environment
+3. Set up your environment
 
-    To set up your environment, you have to configure the `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` environment variable with the value you obtained in the previous steps.
+    To set up your environment, configure the `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` environment variable with the value you obtained in the previous steps. Ensure this environment variable is available in your setup where you are running tests.
 
     We recommend that you use the `dotenv` module to manage your environment. With `dotenv`, you define your environment variables in the `.env` file.
 
@@ -74,23 +93,39 @@ To enable authentication using access tokens:
         npm i --save-dev dotenv
         ```
 
-    1. Create a `.env` file alongside the `playwright.config.ts` file in your Playwright project:
+    2. Create a `.env` file alongside the `playwright.config.ts` file in your Playwright project:
 
         ```
         PLAYWRIGHT_SERVICE_ACCESS_TOKEN={MY-ACCESS-TOKEN}
         ```
 
         Make sure to replace the `{MY-ACCESS-TOKEN}` text placeholder with the value you copied earlier.
 
+::: zone-end
+
+::: zone pivot="nunit-test-runner"
+
+3. Set up your environment
+
+    To set up your environment, configure the `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` environment variable with the value you obtained in the previous steps. Ensure this environment variable is available in your setup where you are running tests.
+
+::: zone-end
 
 ## Run tests on the service and publish results
 
 Run Playwright tests against cloud-hosted browsers and publish the results to the service using the configuration you created above.
 
+::: zone pivot="playwright-test-runner"
 ```typescript
 npx playwright test --config=playwright.service.config.ts --workers=20
 ```
+::: zone-end
 
+::: zone pivot="nunit-test-runner"
+```bash
+dotnet test --settings:.runsettings --logger "microsoft-playwright-testing" -- NUnit.NumberOfTestWorkers=20
+```
+::: zone-end
 ## Related content
 
 - Learn more about [managing access tokens](./how-to-manage-access-tokens.md).

articles/playwright-testing/how-to-test-local-applications.md

@@ -1,12 +1,14 @@
 ---
-title: Use remote browsers for local applications
+title: Use remote browsers for local or private applications
 description: Learn how to run end-to-end for locally deployed applications with Microsoft Playwright Testing Preview. Use cloud-hosted browsers to test apps on localhost or private networks.
 ms.topic: how-to
 ms.date: 10/04/2023
 ms.custom: playwright-testing-preview
+zone_pivot_group_filename: playwright-testing/zone-pivots-groups.json
+zone_pivot_groups: microsoft-playwright-testing
 ---
 
-# Use cloud-hosted browsers for locally deployed apps with Microsoft Playwright Testing Preview
+# Use cloud-hosted browsers for locally deployed or privately hosted apps with Microsoft Playwright Testing Preview
 
 Learn how to use Microsoft Playwright Testing Preview to run end-to-end tests for locally deployed applications. Microsoft Playwright Testing uses cloud-hosted, remote browsers for running Playwright tests at scale. You can use the service to run tests for apps on localhost, or that you host on your infrastructure.
 
@@ -21,7 +23,9 @@ To expose local networks and resources to remote browsers, you can use the `expo
 
 You can specify one or multiple networks by using a list of rules. For example, to expose test/staging deployments and [localhost](https://en.wikipedia.org/wiki/Localhost): `*.test.internal-domain,*.staging.internal-domain,<loopback>`.
 
-You can configure the `exposeNetwork` option in `playwright.service.config.ts`. The following example shows how to expose the `localhost` network by using the [`<loopback>`](https://en.wikipedia.org/wiki/Loopback) rule:
+::: zone pivot="playwright-test-runner"
+
+You can configure the `exposeNetwork` option in `playwright.service.config.ts`. The following example shows how to expose the `localhost` network by using the [`<loopback>`](https://en.wikipedia.org/wiki/Loopback) rule. You can also replace `localhost` with a domain that you want to enable for the service.
 
 ```typescript
 import { getServiceConfig, ServiceOS } from "@azure/microsoft-playwright-testing";
@@ -43,6 +47,27 @@ You can now reference `localhost` in the Playwright test code, and run the tests
 ```bash
 npx playwright test --config=playwright.service.config.ts --workers=20
 ```
+::: zone-end
+
+
+::: zone pivot="nunit-test-runner"
+
+You can configure the `ExposeNetwork` option in `.runsettings`. The following example shows how to expose the `localhost` network by using the [`<loopback>`](https://en.wikipedia.org/wiki/Loopback) rule. You can also replace `localhost` with a domain that you want to enable for the service. 
+
+```xml
+    <TestRunParameters>
+        <!--Use this option to connect to local resources from your Playwright test code without having to configure additional firewall-->
+        <Parameter name="ExposeNetwork" value="loopback" />
+    </TestRunParameters>
+```
+
+You can now reference `localhost` in the Playwright test code, and run the tests on cloud-hosted browsers with Microsoft Playwright Testing:
+
+```bash
+dotnet test --settings:.runsettings --logger "microsoft-playwright-testing" -- NUnit.NumberOfTestWorkers=20
+```
+
+::: zone-end
 
 ## Related content
 

articles/playwright-testing/how-to-use-service-config-file.md

@@ -4,11 +4,25 @@ description: Learn how to use options available in configuration file with Micro
 ms.topic: how-to
 ms.date: 09/07/2024
 ms.custom: playwright-testing-preview
+zone_pivot_group_filename: playwright-testing/zone-pivots-groups.json
+zone_pivot_groups: microsoft-playwright-testing
 ---
-# Use options available in configuration file with Microsoft Playwright Testing preview
+
+# Use options available in service package with Microsoft Playwright Testing preview
+
+::: zone pivot="playwright-test-runner"
 
 This article shows you how to use the options available in the `playwright.service.config.ts` file that was generated for you. 
-If you don't have this file in your code, follow the QuickStart guide, see [Quickstart: Run end-to-end tests at scale with Microsoft Playwright Testing Preview](./quickstart-run-end-to-end-tests.md) 
+If you don't have this file in your code, follow [Quickstart: Run end-to-end tests at scale with Microsoft Playwright Testing Preview](./quickstart-run-end-to-end-tests.md) 
+
+::: zone-end
+
+::: zone pivot="nunit-test-runner"
+
+This article shows you how to use the options available in the `.runsettings` file. 
+If you don't have this file in your code, follow [Quickstart: Run end-to-end tests at scale with Microsoft Playwright Testing Preview](./quickstart-run-end-to-end-tests.md) 
+
+::: zone-end
 
 > [!IMPORTANT]
 > Microsoft Playwright Testing is currently in preview. For legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability, see the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
@@ -17,6 +31,8 @@ If you don't have this file in your code, follow the QuickStart guide, see [Quic
 
 * Follow the Quickstart guide and set up a project to run with Microsoft Playwright Testing service. See, [Quickstart: Run end-to-end tests at scale with Microsoft Playwright Testing Preview](./quickstart-run-end-to-end-tests.md) 
 
+::: zone pivot="playwright-test-runner"
+
 Here's version of the `playwright.service.config.ts` file with all the available options:
 
 ```typescript
@@ -77,7 +93,7 @@ export default defineConfig(
       ```
 
 * **`runId`**:
-    - **Description**: This setting allows you to set a unique ID for every test run to distinguish them in the service portal.
+    - **Description**: This setting allows you to set a unique ID for every test run to distinguish them in the service portal. Using the same runId for multiple test runs results in error. For sharding, keep this same across all shards.
     - **Example**:
       ```typescript
       runId: new Date().toISOString()
@@ -136,4 +152,131 @@ export default defineConfig(
         ],
       ]
     ```
+::: zone-end
+
+::: zone pivot="nunit-test-runner"
+
+
+Here's version of the `.runsettings` file with all the available options:
+
+```xml
+<?xml version="1.0" encoding="utf-8"?>
+<RunSettings>
+    <TestRunParameters>
+        <!-- Use this option when you want to authenticate using access tokens. This mode of auth should be enabled for the workspace. -->
+         <Parameter name="ServiceAuthType" value="EntraId" />
+        <!-- Select the operating system where you want to run tests. -->
+        <Parameter name="Os" value="linux" />
+        <!-- Set a unique ID for every test run to distinguish them in the service portal.-->
+        <Parameter name="RunId" value="sample-run-id1" />
+        <!--Select if you want to use cloud-hosted browsers to run your Playwright tests.-->
+        <Parameter name="UseCloudHostedBrowsers" value="true" />
+        <!--Use this option to connect to local resources from your Playwright test code without having to configure additional firewall-->
+        <Parameter name="ExposeNetwork" value="loopback" />
+        <!--Select the authentication method you want to use with Entra-->
+        <Parameter name="AzureTokenCredentialType" value="DefaultAzureCredential" />
+        <!--Enable/disable GitHub summary in GitHub Actions workflow.-->
+        <Parameter name="EnableGitHubSummary" value="false" />
+    </TestRunParameters>
+  <!-- NUnit adapter -->  
+  <NUnit>
+    <!-- Adjust parallel workers, parallel worker would also be bound by number of unit test files -->
+    <NumberOfTestWorkers>10</NumberOfTestWorkers>
+  </NUnit>
+  <!-- General run configuration -->
+  <RunConfiguration>
+    <EnvironmentVariables>
+      <!-- For debugging selectors, it's recommend to set the following environment variable -->
+      <DEBUG>pw:api</DEBUG>
+    </EnvironmentVariables>
+  </RunConfiguration>
+  <!-- Playwright -->  
+  <Playwright>
+    <BrowserName>chromium</BrowserName>
+    <!--Set the timeout for your tests.-->
+    <ExpectTimeout>5000</ExpectTimeout>
+    <LaunchOptions>
+      <Headless>false</Headless>
+      <!--Channel>msedge</Channel-->
+    </LaunchOptions>
+  </Playwright>
+    <LoggerRunSettings>
+        <Loggers>
+            <!--microsoft playwright testing service logger for reporting -->
+            <Logger friendlyName="microsoft-playwright-testing" enabled="true" />
+            <!--could enable any logger additionally -->
+            <Logger friendlyName="trx" enabled="false" />
+        </Loggers>
+    </LoggerRunSettings>
+</RunSettings>
+
+```
 
+## Options in `.runsettings` file
+
+* **`serviceAuthType`**:
+    - **Description**: This setting allows you to choose the authentication method you want to use for your test run. 
+    - **Available Options**:
+        - `AccessToken` to use access tokens. You need to enable authentication using access tokens if you want to use this option, see [manage authentication](./how-to-manage-authentication.md).
+        - `EntraId` to use Microsoft Entra ID for authentication. It's the default mode. 
+    - **Default Value**: `EntraId`
+    - **Example**:
+      ```xml
+      <Parameter name="ServiceAuthType" value="EntraId" />
+      ```
+
+* **`os`**:
+    - **Description**: This setting allows you to choose the operating system where the browsers running Playwright tests are hosted.
+    - **Available Options**:
+        - "windows" for Windows OS.
+        - "linux" for Linux OS.
+    - **Default Value**: "linux"
+    - **Example**:
+      ```xml
+      <Parameter name="Os" value="linux" />
+      ```
+
+* **`RunId`**:
+    - **Description**: This setting allows you to set a unique ID for every test run to distinguish them in the service portal. Using the same runId for multiple test runs results in error. If you don't set it, the service package will generate a unique ID every time you trigger a test run. For sharding, keep this same across all shards.
+    - **Example**:
+      ```xml
+      <Parameter name="RunId" value="sample-run-id1" />
+      ```
+
+* **`AzureTokenCredentialType`**:
+    - **Description**: This setting allows you to select the authentication method you want to use with Microsoft Entra ID.
+    - **Example**:
+      ```xml
+      <Parameter name="AzureTokenCredentialType" value="DefaultAzureCredential" />
+      ```
+
+* **`UseCloudHostedBrowsers`**
+    - **Description**: This setting allows you to choose whether to use cloud-hosted browsers or the browsers on your client machine to run your Playwright tests. If you disable this option, your tests run on the browsers of your client machine instead of cloud-hosted browsers, and you don't incur any charges.
+    - **Default Value**: true
+    - **Example**:
+      ```xml
+      <Parameter name="UseCloudHostedBrowsers" value="true" />
+      ```   
+* **`ExposeNetwork`**
+    - **Description**: This setting allows you to connect to local resources from your Playwright test code without having to configure another firewall settings. To learn more, see [how to test local applications](./how-to-test-local-applications.md)
+    - **Example**:
+      ```xml
+      <Parameter name="ExposeNetwork" value="loopback" />
+      ```
+
+* **`reporter`**
+    - **Description**: You can publish your test results and artifacts to the service using `microsoft-playwright-testing` logger. You can disable reporting by removing this from your `.runsettings` or by setting it to false. 
+    - **Default Value**: true
+    - **Example**:
+      ```xml
+      <Logger friendlyName="microsoft-playwright-testing" enabled="true" />
+      ```
+
+* **`EnableGitHubSummary`**:
+    - **Description**: This setting allows you to configure the Microsoft Playwright Testing service reporter. You can choose whether to include the test run summary in the GitHub summary when running in GitHub Actions.
+    - **Default Value**: true
+    - **Example**:
+    ```xml
+      <Parameter name="EnableGitHubSummary" value="false" />
+    ```
+::: zone-end

articles/playwright-testing/how-to-use-service-features.md

@@ -4,6 +4,8 @@ description: Learn how to use different features offered by Microsoft Playwright
 ms.topic: how-to
 ms.date: 09/07/2024
 ms.custom: playwright-testing-preview
+zone_pivot_group_filename: playwright-testing/zone-pivots-groups.json
+zone_pivot_groups: microsoft-playwright-testing
 ---
 
 # Use features of Microsoft Playwright Testing preview
@@ -48,7 +50,7 @@ You can also choose to use either feature or both for a test run.
 
 > [!IMPORTANT]
 > You can only use a feature in a test run if it is enabled for the workspace.
-
+::: zone pivot="playwright-test-runner"
 1. In your Playwright setup, go to `playwright.service.config.ts` file and use these settings for feature management. 
 
 ```typescript
@@ -87,7 +89,51 @@ export default defineConfig(
       ["@azure/microsoft-playwright-testing/reporter"]],
       ```
 
+::: zone-end
+
+::: zone pivot="nunit-test-runner"
+
+1. In your Playwright setup, go to `.runsettings` file and use these settings for feature management. 
+
+```xml
+<?xml version="1.0" encoding="utf-8"?>
+<RunSettings>
+    <TestRunParameters>
+        <!--Select if you want to use cloud-hosted browsers to run your Playwright tests.-->
+        <Parameter name="UseCloudHostedBrowsers" value="true" />
+    </TestRunParameters>
+  <!-- NUnit adapter -->  
+  .
+  .
+  .
+    <LoggerRunSettings>
+        <Loggers>
+            <!--microsoft playwright testing service logger for reporting -->
+            <Logger friendlyName="microsoft-playwright-testing" enabled="true" />
+            <!--could enable any logger additionally -->
+            <Logger friendlyName="trx" enabled="false" />
+        </Loggers>
+    </LoggerRunSettings>
+</RunSettings>
+
+```
 
+* **`UseCloudHostedBrowsers`**
+    - **Description**: This setting allows you to choose whether to use cloud-hosted browsers or the browsers on your client machine to run your Playwright tests. If you disable this option, your tests run on the browsers of your client machine instead of cloud-hosted browsers, and you don't incur any charges.
+    - **Default Value**: true
+    - **Example**:
+      ```xml
+      <Parameter name="UseCloudHostedBrowsers" value="true" />
+      ```   
+
+* **`reporter`**
+    - **Description**: You can publish your test results and artifacts to the service using `microsoft-playwright-testing` logger. You can disable reporting by removing this from your `.runsettings` or by setting it to false. 
+    - **Default Value**: true
+    - **Example**:
+      ```xml
+      <Logger friendlyName="microsoft-playwright-testing" enabled="true" />
+      ```
+::: zone-end
 ## Related content
 
 - Learn more about [Microsoft Playwright Testing preview pricing](https://aka.ms/mpt/pricing).