Skip to content

Commit 2fa427e

Browse files
committed
updated article for pipelines
1 parent fb87941 commit 2fa427e

File tree

3 files changed

+170
-5
lines changed

3 files changed

+170
-5
lines changed

articles/playwright-testing/quickstart-automate-end-to-end-testing.md

Lines changed: 168 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ description: In this quickstart, you learn how to run your Playwright tests at s
44
ms.topic: quickstart
55
ms.date: 10/04/2023
66
ms.custom: playwright-testing-preview, build-2024
7+
zone_pivot_group_filename: playwright-testing/ZonePivotGroups.json
8+
zone_pivot_groups: microsoft-playwright-testing
79
---
810

911
# Quickstart: Set up continuous end-to-end testing with Microsoft Playwright Testing Preview
@@ -57,6 +59,7 @@ To get the service endpoint URL and store it as a CI workflow secret, perform th
5759
| ----------- | ------------ |
5860
| *PLAYWRIGHT_SERVICE_URL* | Paste the endpoint URL you copied previously. |
5961

62+
::: zone pivot="playwright-test-runner"
6063
## Add service configuration file
6164

6265
If you haven't configured your Playwright tests yet for running them on cloud-hosted browsers, add a service configuration file to your repository. In the next step, you then specify this service configuration file on the Playwright CLI.
@@ -73,7 +76,7 @@ If you haven't configured your Playwright tests yet for running them on cloud-ho
7376
- Accelerate build pipelines by running tests in parallel using cloud-hosted browsers.
7477
- Simplify troubleshooting with easy access to test results and artifacts published to the service.
7578

76-
However, you can choose to use either of these features or both. See [How to use service features](./how-to-use-service-features.md#manage-features-while-running-tests) and update the service configuration file as per your requirement.
79+
However, you can choose to use either of these features or both. See [How to use service features](./how-to-use-service-features.md#manage-features-while-running-tests) and update the service configuration file as per your requirements.
7780

7881
3. Save and commit the file to your source code repository.
7982

@@ -97,9 +100,40 @@ In the `playwright.config.ts` file of your project, make sure you are collecting
97100
screenshot:'on'
98101
},
99102
```
103+
::: zone-end
104+
105+
::: zone pivot="nunit-test-runner"
106+
## Set up service configuration
107+
108+
1. Create a new file `PlaywrightServiceSetup.cs` in the root directory of your project. This file facilitates authentication of your client with the service.
109+
2. Add the following content to it:
110+
111+
:::code language="typescript" source="~/playwright-testing-service/samples/.NET/NUnit/PlaywrightServiceSetup.cs":::
112+
113+
3. Save and commit the file to your source code repository.
114+
115+
## Update .scproj file for your project
116+
117+
Update the `csproj` file in your repository to add details about Microsoft Playwright Testing service package in `ItemGroup` section.
118+
119+
```xml
120+
<ItemGroup>
121+
<PackageReference Include="Azure.Developer.MicrosoftPlaywrightTesting.NUnit" Version="1.0.0-beta.2" />
122+
</ItemGroup>
123+
```
124+
## Enable artifacts in your Playwright setup
125+
126+
Enable artifacts such as screenshot, videos and traces to be captured by Playwright.
127+
- For screenshots, see [capture screenshots](https://playwright.dev/dotnet/docs/screenshots#introduction)
128+
- For videos, see [record videos for your tests](https://playwright.dev/dotnet/docs/videos#introduction)
129+
- For traces, see [recording a trace](https://playwright.dev/dotnet/docs/trace-viewer-intro#recording-a-trace)
130+
131+
Once you collect these artifacts, make sure you attach them to the test path. For more information, see our [sample](https://aka.ms/mpt/nunit-sample)
132+
::: zone-end
100133

101134
## Update the workflow definition
102135

136+
::: zone pivot="playwright-test-runner"
103137
Update the CI workflow definition to run your Playwright tests with the Playwright CLI. Pass the [service configuration file](#add-service-configuration-file) as an input parameter for the Playwright CLI. You configure your environment by specifying environment variables.
104138

105139
1. Open the CI workflow definition
@@ -114,6 +148,20 @@ Update the CI workflow definition to run your Playwright tests with the Playwrig
114148

115149
# This step is to sign-in to Azure to run tests from GitHub Action workflow.
116150
# Choose how to set up authentication to Azure from GitHub Actions. This is one example.
151+
on:
152+
push:
153+
branches: [ main, master ]
154+
pull_request:
155+
branches: [ main, master ]
156+
permissions: # Required when using Micosoft Entra ID to authenticate
157+
id-token: write
158+
contents: read
159+
jobs:
160+
test:
161+
timeout-minutes: 60
162+
runs-on: ubuntu-latest
163+
steps:
164+
- uses: actions/checkout@v4
117165
- name: Login to Azure with AzPowershell (enableAzPSSession true)
118166
uses: azure/login@v2
119167
with:
@@ -177,23 +225,140 @@ Update the CI workflow definition to run your Playwright tests with the Playwrig
177225
```
178226
179227
---
228+
::: zone-end
229+
230+
::: zone pivot="nunit-test-runner"
231+
232+
Update the CI workflow definition to run your Playwright tests with the Playwright NUnit CLI. Pass the `.runsettings` file as an input parameter for the Playwright CLI. You configure your environment by specifying environment variables.
233+
234+
1. Open the CI workflow definition
235+
236+
1. Add the following steps to run your Playwright tests in Microsoft Playwright Testing.
237+
238+
The following steps describe the workflow changes for GitHub Actions or Azure Pipelines. Similarly, you can run your Playwright tests by using the Playwright CLI in other CI platforms.
239+
240+
# [GitHub Actions](#tab/github)
241+
242+
```yml
243+
on:
244+
push:
245+
branches: [ main, master ]
246+
pull_request:
247+
branches: [ main, master ]
248+
permissions: # Required when using AuthType as EntraId
249+
id-token: write
250+
contents: read
251+
jobs:
252+
test:
253+
timeout-minutes: 60
254+
runs-on: ubuntu-latest
255+
steps:
256+
- uses: actions/checkout@v4
257+
# This step is to sign-in to Azure to run tests from GitHub Action workflow.
258+
# Choose how to set up authentication to Azure from GitHub Actions. This is one example.
259+
260+
- name: Login to Azure with AzPowershell (enableAzPSSession true)
261+
uses: azure/login@v2
262+
with:
263+
client-id: ${{ secrets.AZURE_CLIENT_ID }}
264+
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
265+
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
266+
enable-AzPSSession: true
267+
268+
- name: Setup .NET
269+
uses: actions/setup-dotnet@v4
270+
with:
271+
dotnet-version: 8.0.x
272+
273+
- name: Restore dependencies
274+
run: dotnet restore
275+
working-directory: path/to/playwright/folder # update accordingly
276+
277+
- name: Build
278+
run: dotnet build --no-restore
279+
working-directory: path/to/playwright/folder # update accordingly
280+
281+
- name: Run Playwright tests
282+
working-directory: path/to/playwright/folder # update accordingly
283+
env:
284+
# Regional endpoint for Microsoft Playwright Testing
285+
PLAYWRIGHT_SERVICE_URL: ${{ secrets.PLAYWRIGHT_SERVICE_URL }}
286+
PLAYWRIGHT_SERVICE_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}-${{ github.sha }}
287+
run: dotnet test --settings:.runsettings --logger "microsoft-playwright-testing" -- NUnit.NumberOfTestWorkers=20
288+
289+
- name: Upload Playwright report
290+
uses: actions/upload-artifact@v3
291+
if: always()
292+
with:
293+
name: playwright-report
294+
path: path/to/playwright/folder/playwright-report/ # update accordingly
295+
retention-days: 10
296+
```
297+
298+
# [Azure Pipelines](#tab/pipelines)
299+
300+
```yml
301+
- task: UseDotNet@2
302+
inputs:
303+
packageType: 'sdk'
304+
version: '8.x'
305+
installationPath: $(Agent.ToolsDirectory)/dotnet
306+
307+
- checkout: self
308+
309+
- script: dotnet restore
310+
displayName: 'Restore dependencies'
311+
workingDirectory: path/to/playwright/folder # update accordingly
312+
313+
- script: dotnet build --no-restore
314+
displayName: 'Build'
315+
workingDirectory: path/to/playwright/folder # update accordingly
316+
317+
- task: AzureCLI@2
318+
displayName: Run Playwright Test
319+
env:
320+
PLAYWRIGHT_SERVICE_URL: $(PLAYWRIGHT_SERVICE_URL)
321+
PLAYWRIGHT_SERVICE_RUN_ID: ${{ parameters.runIdPrefix }}$(Build.DefinitionName) - $(Build.BuildNumber) - $(System.JobAttempt)
322+
inputs:
323+
azureSubscription: My_Service_Connection # Service connection used to authenticate this pipeline with Azure to use the service
324+
scriptType: 'pscore'
325+
scriptLocation: 'inlineScript'
326+
inlineScript: |
327+
dotnet test --settings:.runsettings --logger "microsoft-playwright-testing" -- NUnit.NumberOfTestWorkers=20
328+
addSpnToEnvironment: true
329+
workingDirectory: path/to/playwright/folder # update accordingly
330+
331+
- task: PublishPipelineArtifact@1
332+
displayName: Upload Playwright report
333+
inputs:
334+
targetPath: path/to/playwright/folder/playwright-report/ # update accordingly
335+
artifact: 'Playwright tests'
336+
publishLocation: 'pipeline'
337+
```
338+
339+
---
340+
341+
::: zone-end
180342

181343
1. Save and commit your changes.
182344

183-
When the CI workflow is triggered, your Playwright tests run in your Microsoft Playwright Testing workspace on cloud-hosted browsers, across 20 parallel workers.
345+
When the CI workflow is triggered, your Playwright tests run in your Microsoft Playwright Testing workspace on cloud-hosted browsers, across 20 parallel workers. The results and artifacts collected are published to the service and can be viewed on service portal.
184346

185347
> [!NOTE]
186348
> Reporting feature is enabled by default for existing workspaces. This is being rolled out in stages and will take a few days. To avoid failures, confirm that `Rich diagnostics using reporting` setting is ON for your workspace before proceeding. See, [Enable reporting for workspace](./how-to-use-service-features.md#manage-feature-for-the-workspace).
187349

188350
> [!CAUTION]
189-
> With Microsoft Playwright Testing, you get charged based on the number of total test minutes and test result published. If you're a first-time user or [getting started with a free trial](./how-to-try-playwright-testing-free.md), you might start with running a single test at scale instead of your full test suite to avoid exhausting your free test minutes and test results.
351+
> With Microsoft Playwright Testing, you get charged based on the number of total test minutes consumed and test results published. If you're a first-time user or [getting started with a free trial](./how-to-try-playwright-testing-free.md), you might start with running a single test at scale instead of your full test suite to avoid exhausting your free test minutes and test results.
190352
>
191353
> After you validate that the test runs successfully, you can gradually increase the test load by running more tests with the service.
192354
>
355+
::: zone pivot="playwright-test-runner"
356+
193357
> You can run a single test with the service by using the following command-line:
194358
>
195359
> ```npx playwright test {name-of-file.spec.ts} --config=playwright.service.config.ts```
196360

361+
::: zone-end
197362
## View test runs and results in the Playwright portal
198363

199364
You can now troubleshoot the CI pipeline in the Playwright portal,

articles/playwright-testing/quickstart-generate-rich-reports-for-tests.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -208,7 +208,7 @@ You can generate an access token from your Playwright Testing workspace and use
208208
> [!CAUTION]
209209
> We strongly recommend using Microsoft Entra ID for authentication to the service. If you are using access tokens, see [How to Manage Access Tokens](./how-to-manage-access-tokens.md)
210210

211-
## Enable artifacts in Playwright configuration
211+
## Enable artifacts in your Playwright setup
212212
::: zone pivot="playwright-test-runner"
213213

214214
In the `playwright.config.ts` file of your project, make sure you are collecting all the required artifacts.

articles/playwright-testing/quickstart-run-end-to-end-tests.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ You can generate an access token from your Playwright Testing workspace and use
146146
> We strongly recommend using Microsoft Entra ID for authentication to the service. If you are using access tokens, see [How to Manage Access Tokens](./how-to-manage-access-tokens.md)
147147

148148

149-
## Enable artifacts in Playwright configuration
149+
## Enable artifacts in your Playwright setup
150150
::: zone pivot="playwright-test-runner"
151151

152152
In the `playwright.config.ts` file of your project, make sure you are collecting all the required artifacts.

0 commit comments

Comments
 (0)