Skip to content

Commit dbfd3f6

Browse files
vidai-msftisra-fel
andauthored
Updated script to build TestFx environment as well as the document (#20592)
* Updated script to build TestFx environment as well as the document * Update documentation/testing-docs/using-azure-test-framework.md Co-authored-by: Yeming Liu <[email protected]> Co-authored-by: Yeming Liu <[email protected]>
1 parent c15aaf3 commit dbfd3f6

File tree

5 files changed

+491
-543
lines changed

5 files changed

+491
-543
lines changed

documentation/testing-docs/using-azure-test-framework.md

Lines changed: 111 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -1,134 +1,153 @@
1-
# Using Microsoft.Rest.ClientRuntime.Azure.TestFramework #
2-
3-
- [Getting Started](#getting-started)
4-
- [Acquiring TestFramework](#acquiring-testframework)
5-
- [Setup prior to Record/Playback tests](#setup-prior-to-record-or-playback-of-tests)
6-
- [New-TestCredential](#new-testcredential)
7-
- [Create New Service Principal](#create-new-service-principal)
8-
- [Use Existing Service Principal](#use-existing-service-principal)
9-
- [Set-TestEnvironment](#set-testenvironment)
10-
- [Existing Service Principal](#existing-service-principal)
11-
- [Manually Set Environment Variables](#manually-set-environment-variables)
12-
- [Environment Variables](#environment-variables)
13-
- [Playback Test](#playback-test)
14-
- [Record Test with Interactive login using OrgId](#record-test-with-interactive-login-using-orgid)
15-
- [Record Test with ServicePrincipal](#record-test-with-serviceprincipal)
16-
- [Record/Playback tests](#record-or-playback-tests)
17-
- [Change Test Environment settings at run-time](#change-test-environment-settings-at-run-time)
18-
- [Troubleshooting](#troubleshooting)
19-
- [Supported Key Value pairs in ConnectionString](#supported-key-value-pairs-in-connectionstring)
20-
- [Environment Variable Reference](#supported-environment-in-test-framework)
1+
# Using Azure PowerShell Test Framework
2+
3+
- [Using Microsoft.Rest.ClientRuntime.Azure.TestFramework](#using-microsoftrestclientruntimeazuretestframework)
4+
- [Getting Started](#getting-started)
5+
- [Azure PowerShell Test Framework](#azure-powershell-test-framework)
6+
- [Setup prior to Record or Playback of tests](#setup-prior-to-record-or-playback-of-tests)
7+
- [Run Command Set-TestFxEnvironment to Build Connection String (Recommended)](#run-command-set-testfxenvironment-to-build-connection-string-recommended)
8+
- [Create New Service Principal](#create-new-service-principal)
9+
- [Use Existing Service Principal](#use-existing-service-principal)
10+
- [Manually Set Environment Variables to Build Connection String](#manually-set-environment-variables-to-build-connection-string)
11+
- [Environment Variables](#environment-variables)
12+
- [Record Test with service principal](#record-test-with-service-principal)
13+
- [Playback Test](#playback-test)
14+
- [JSON Config File V.S. Environment Variables](#json-config-file-vs-environment-variables)
15+
- [Record or Playback Tests](#record-or-playback-tests)
16+
- [Change Test Environment settings at run-time](#change-test-environment-settings-at-run-time)
17+
- [Once you set your connection string, you can add or update key/value settings](#once-you-set-your-connection-string-you-can-add-or-update-keyvalue-settings)
18+
- [Note:](#note)
19+
- [Troubleshooting](#troubleshooting)
20+
- [Issue: exceptions in Microsoft.Azure.Test.HttpRecorder](#issue-exceptions-in-microsoftazuretesthttprecorder)
21+
- [Supported Environments in Test Framework](#supported-environments-in-test-framework)
22+
- [Default Environments and associated Uri](#default-environments-and-associated-uri)
23+
- [Environment = Prod](#environment--prod)
24+
- [Environment = Dogfood](#environment--dogfood)
25+
- [Environment = Next](#environment--next)
26+
- [Environment = Current](#environment--current)
27+
- [Environment = Custom](#environment--custom)
2128

2229
## Getting Started
23-
- Install the latest Az.Resources from the [PSGallery](https://www.powershellgallery.com/) into Windows PowerShell
24-
- Run Windows PowerShell as administrator and execute following command
25-
- `Install-Module -Name Az.Resources -Scope AllUsers -AllowClobber`
26-
- Double click `.\tools\PS-VSPrompt` shortcut
27-
- This starts the VS Developer command prompt in PowerShell inside the `azure-powershell/tools` directory
28-
- Import the `Repo-Tasks` module that helps to perform basic repository tasks
29-
- Run the command `Import-Module .\Repo-Tasks.psd1`
3030

31-
## Azure PowerShell TestFramework
31+
- Install the latest `Az.Resources` from the [PSGallery](https://www.powershellgallery.com/) into PowerShell
32+
- Run PowerShell as administrator and execute the following command
33+
- `Install-Module -Name Az.Resources -Scope AllUsers -AllowClobber -Force`
34+
- Import the `TestFx-Tasks` module that helps to configure the settings
35+
- Run the command `Import-Module ./tools/Modules/TestFx-Tasks.psd1`
36+
37+
## Azure PowerShell Test Framework
3238

3339
Azure PowerShell repo now has its own test framework located under `tools\TestFx`, which supports recording all the HTTP requests from behind Azure PowerShell cmdlets and then playing them back.
3440

3541
The target framework of test is .Net Core 3.1, please ensure .Net runtime Microsoft.NETCore.App 3.1 is installed. You can list all installed version via `dotnet --info`.
3642

3743
## Setup prior to Record or Playback of tests
3844

39-
In order to Record/Playback a test, you need to setup a connection string that consists of various key/value pairs that provide information to the test environment.
45+
In order to Record/Playback a test, test framework needs to setup a connection string that consists of various key/value pairs that provide necessary information.
46+
47+
You can choose either option to configure the settings:
4048

41-
You have three options to set up the connection string:
42-
- Run the [`New-TestCredential` cmdlet](#new-testcredential) (recommended for PowerShell development)
43-
- Run the [`Set-TestEnvironment` cmdlet](#set-testenvironment)
44-
- [Manually set the environment variables](#manually-set-environment-variables).
49+
- Run the [`Set-TestFxEnvironment` cmdlet](#run-command-set-testfxenvironment-to-build-connection-string) (Recommended)
50+
- [Manually set the environment variables](#manually-set-environment-variables-to-build-connection-string)
4551

46-
### New-TestCredential
52+
### Run Command Set-TestFxEnvironment to Build Connection String (Recommended)
4753

48-
This cmdlet, located in the [`Repo-Tasks` module](/tools/Repo-Tasks.psd1), which pulls in the [`TestFx-Tasks` module](/tools/Modules/TestFx-Tasks.psd1) and [`Build-Tasks` module](/tools/Modules/Build-Tasks.psd1), will allow you to create a credentials file (located in `C:/Users/\<currentuser\>/.azure/testcredentials.json`) that will be used to set the environment variable when scenario tests are run. This credentials file will be used in all future sessions unless it is deleted or the environment variables are manually set. This cmdlet is not currently available for .NET SDK development.
54+
This cmdlet will allow you to create a credentials file (located in `C:/Users/<currentuser>/.Azure/testcredentials.json`) that will be used to set the connection string when scenario tests are run. This credentials file will be used in all future sessions unless it is deleted.
4955

5056
#### Create New Service Principal
5157

52-
Using a service principal is the preferred option for recording tests because it works with both .NET Framework and .NET Core. In order to create a new service principal, run this command with an unused service principal display name:
58+
Using a service principal is the preferred option for recording tests because it works with both .NET Framework and .NET Core.
59+
In order to create a new service principal, run the following command with an unused service principal display name:
5360

5461
```powershell
55-
New-TestCredential -ServicePrincipalDisplayName "ScenarioTestCredentials" -SubscriptionId `
56-
<subscriptionId> -TenantId <tenantId> -RecordMode "Record"
62+
Set-TestFxEnvironment -ServicePrincipalDisplayName <DisplayName> -SubscriptionId <SubscriptionId> -TenantId <TenantId> -RecordMode "Record"
5763
```
5864

59-
This command will create a new service principal, set the correct role assignment for this service principal based upon the subscription provided, and place the service principal id and automatically generated secret into the credentials file.
65+
This command will first create a new service principal. And then set the `Contributor` role assignment for this service principal based upon the subscription provided. After that, it will place the service principal application id and automatically generated secret into the credentials file.
6066

61-
Alternatively, to create a service principal, follow the [Azure AD guide to create a Application Service Principal](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#create-an-active-directory-application). The application type should be `Web app / API` and the sign-on URL value is irrelevant (you can set any value).
67+
If the display name of the service principal already exists, it will prompt if you would like to create a new one with the same name.
68+
If the answer is "Y", the new generated application id and the secret will be saved.
69+
70+
Alternatively, if you prefer creating a service principal by yourself from Azure portal, follow the [Azure AD guide to create a Application Service Principal](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#create-an-active-directory-application).
6271

6372
#### Use Existing Service Principal
6473

65-
If you would like to use an existing service principal, run this command with an existing service principal display name and secret:
74+
If you would like to use an existing service principal, run the following command with an existing service principal application id and secret:
6675

6776
```powershell
68-
New-TestCredential -ServicePrincipalDisplayName "Existing Service Principal" -ServicePrincipalSecret `
69-
"testpassword" -SubscriptionId <subscriptionId> -TenantId <tenantId> -RecordMode "Record"
77+
Set-TestFxEnvironment -ServicePrincipalId <ServicePrincipalApplicationId> -ServicePrincipalSecret <ServicePrincipalSecret> -SubscriptionId <SubscriptionId> -TenantId <TenantId> -RecordMode "Record"
7078
```
7179

72-
### Set-TestEnvironment
73-
74-
This cmdlet, located in the [`Repo-Tasks` module](/tools/Repo-Tasks.psd1), will directly set the environment variable for the session.
80+
For existing service principal, this command will respect your own settings and won't assign the `Contributor` role automatically.
7581

76-
#### Existing Service Principal
77-
78-
This is the preferred option for recording tests because it works with both .NET Framework and .NET Core.
79-
80-
```powershell
81-
Set-TestEnvironment -ServicePrincipalId <servicePrincipalId> -ServicePrincipalSecret `
82-
"testpassword" -SubscriptionId <subscriptionId> -TenantId <tenantId> -RecordMode "Record"
83-
```
82+
8483

85-
### Manually Set Environment Variables
84+
### Manually Set Environment Variables to Build Connection String
8685

8786
#### Environment Variables
8887

8988
`TEST_CSM_ORGID_AUTHENTICATION`
90-
* This is the connection string that determines how to connect to Azure. This includes both your authentication and the Azure environment to connect to.
9189

92-
`AZURE_TEST_MODE`
93-
* This specifies whether the test framework will `Record` test sessions or `Playback` previously recorded test sessions.
90+
* This determines how to connect to Azure. It includes both your authentication and the Azure environment information.
9491

95-
#### Playback Test
96-
97-
The default test mode is `Playback` mode, so setting up the connection string is not required. You can optionally set environment variables:
92+
`AZURE_TEST_MODE`
9893

99-
```
100-
TEST_CSM_ORGID_AUTHENTICATION=
101-
AZURE_TEST_MODE=Playback
102-
```
94+
* This specifies whether the test framework will `Record` test sessions or `Playback` previously recorded test sessions.
10395

104-
#### Record Test with service principal
96+
#### Record Tests
10597

106-
After the service principal is created, you will need to give it access to Azure resources. This can be done with the following PowerShell command, with the [Service Principal Application ID](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key) (this is a guid, not the display name of the service principal) substituted in for `{clientId}`.
98+
After the service principal is created, you will need to give it access to Azure resources. This can be done with the following PowerShell command. The argument for this command is the application id (See [Service Principal Application ID](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key))
10799

108100
```powershell
109-
New-AzRoleAssignment -ServicePrincipalName {clientId} -RoleDefinitionName Contributor
101+
New-AzRoleAssignment -ApplicationId <ApplicationId> -Scope "/subscriptions/<SubscriptionId>" -RoleDefinitionName Contributor
110102
```
111103

112-
To use this option, set the following environment variable before starting Visual Studio. The following values are substituted into the below connection string:
104+
To use this option, set the following environment variable before starting Visual Studio. The following values are substituted into the below environment variable:
105+
106+
`ClientId`
113107

114-
`clientId`
115108
* The [Service Principal Application ID](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key)
116109

117-
`clientSecret`
118-
* A [Service Principal Authentication Key](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key)
110+
`ClientSecret`
119111

120-
`tenantId`
121-
* The [AAD Tenant ID](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-tenant-id)
112+
* The [Service Principal Authentication Key](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key)
122113

123114
```
124-
TEST_CSM_ORGID_AUTHENTICATION=SubscriptionId={SubId};ServicePrincipal={clientId};ServicePrincipalSecret={clientSecret};TenantId={tenantId};Environment={env};HttpRecorderMode=Record;
115+
TEST_CSM_ORGID_AUTHENTICATION=Environment=Prod;SubscriptionId=<SubscriptionId>;TenantId=<TenantId>;ServicePrincipal=<ClientId>;ServicePrincipalSecret=<ClientSecret>;HttpRecorderMode=Record;
125116
AZURE_TEST_MODE=Record
126117
```
127118

119+
#### Playback Tests
120+
121+
The default test mode is `Playback`, so setting up the `AZURE_TEST_MODE` is not required. You can optionally set environment variables:
122+
123+
```
124+
TEST_CSM_ORGID_AUTHENTICATION=Environment=Prod;SubscriptionId=<SubscriptionId>;TenantId=<TenantId>;ServicePrincipal=<ClientId>;ServicePrincipalSecret=<ClientSecret>;HttpRecorderMode=Playback;
125+
AZURE_TEST_MODE=Playback
126+
```
127+
128+
## JSON Config File V.S. Environment Variables
129+
130+
Opting for config file is the recommended way to build connection string because any changes you make will take effect immediately without having to restart Visual Studio. However, updating the environment variables is different. It requires rebooting Visual Studio before it can read the updated values. So following is the steps how Test Framework detects the settings.
131+
132+
- If JSON config file exists
133+
- It will be used to build the connection string. Anything set in the environment variables will be ignored
134+
- If JSON config file does not exist
135+
- Test framework will first retrieve the environment variable `TEST_CSM_ORGID_AUTHENTICATION` and use its value to build the connection string except for the test mode (Record/Playback).
136+
- Then test framework will try to get the value of the environment variable `AZURE_TEST_MODE`
137+
- If `AZURE_TEST_MODE` is set, its value will be used as the test mode
138+
- Otherwise, the property named `HttpRecorderMode` configured in `TEST_CSM_ORGID_AUTHENTICATION` will be used
139+
- If the property `HttpRecorderMode` is also not set, `Playback` will be applied as the default value
140+
141+
If you are not sure the settings on your machine, please run command `Get-TestFxEnvironment`. It will consolidate above steps and display the ultimate result.
142+
128143
## Record or Playback Tests
129144

130145
- [Run the tests](https://github.com/Azure/azure-powershell/blob/main/documentation/development-docs/azure-powershell-developer-guide.md#recordingrunning-tests) and make sure that you got a generated `.json` file that matches the test name under the `SessionRecords` folder in the test project.
131-
- To assure that the records work fine, delete the connection string (default mode is Playback mode) OR change HttpRecorderMode within the connection string to "Playback" and run the tests
146+
- If you want to switch from Record to Playback or from Playback to Record, consider below steps.
147+
- If you opt for JSON config file, update the value of the property `HttpRecorderMode` in the JSON.
148+
- If you opt for environment variables
149+
- If you have `AZURE_TEST_MODE` set, update the value of this variable
150+
- Otherwise, update the value of the property `HttpRecorderMode` defined in the variable `TEST_CSM_ORGID_AUTHENTICATION`
132151

133152
## Change Test Environment settings at run-time
134153

@@ -146,6 +165,7 @@ TestEnvironment.Endpoints.GraphUri = new Uri("https://newGraphUri.windows.net");
146165
```
147166

148167
### Note:
168+
149169
Changing the above properties at run-time has the potential to hard code few things in your tests. Best practice would be to use these properties to change values at run-time from immediate window at run-time and avoid hard-coding certain values.
150170

151171
## Troubleshooting
@@ -160,21 +180,21 @@ Ensure that the `HttpRecorderMode` in the `TEST_CSM_ORGID_AUTHENTICATION` enviro
160180

161181
##### Environment = Prod
162182

163-
AADAuthUri = "https://login.microsoftonline.com"
164-
GalleryUri = "https://gallery.azure.com/"
165-
GraphUri = "https://graph.windows.net/"
166-
IbizaPortalUri = "https://portal.azure.com/"
167-
RdfePortalUri = "http://go.microsoft.com/fwlink/?LinkId=254433"
168-
ResourceManagementUri = "https://management.azure.com/"
169-
ServiceManagementUri = "https://management.core.windows.net"
170-
AADTokenAudienceUri = "https://management.core.windows.net"
171-
GraphTokenAudienceUri = "https://graph.windows.net/"
172-
DataLakeStoreServiceUri = "https://azuredatalakestore.net"
173-
DataLakeAnalyticsJobAndCatalogServiceUri = "https://azuredatalakeanalytics.net"
183+
AADAuthUri = "https://login.microsoftonline.com"
184+
GalleryUri = "https://gallery.azure.com/"
185+
GraphUri = "https://graph.windows.net/"
186+
IbizaPortalUri = "https://portal.azure.com/"
187+
RdfePortalUri = "http://go.microsoft.com/fwlink/?LinkId=254433"
188+
ResourceManagementUri = "https://management.azure.com/"
189+
ServiceManagementUri = "https://management.core.windows.net"
190+
AADTokenAudienceUri = "https://management.core.windows.net"
191+
GraphTokenAudienceUri = "https://graph.windows.net/"
192+
DataLakeStoreServiceUri = "https://azuredatalakestore.net"
193+
DataLakeAnalyticsJobAndCatalogServiceUri = "https://azuredatalakeanalytics.net"
174194

175195
##### Environment = Dogfood
176196

177-
AADAuthUri = "https://login.windows-ppe.net";
197+
AADAuthUri = "https://login.windows-ppe.net";
178198
GalleryUri = "https://df.gallery.azure-test.net/";
179199
GraphUri = "https://graph.ppe.windows.net/";
180200
IbizaPortalUri = "http://df.onecloud.azure-test.net";
@@ -188,7 +208,7 @@ Ensure that the `HttpRecorderMode` in the `TEST_CSM_ORGID_AUTHENTICATION` enviro
188208

189209
##### Environment = Next
190210

191-
AADAuthUri = "https://login.windows-ppe.net"
211+
AADAuthUri = "https://login.windows-ppe.net"
192212
GalleryUri = "https://next.gallery.azure-test.net/"
193213
GraphUri = "https://graph.ppe.windows.net/"
194214
IbizaPortalUri = "http://next.onecloud.azure-test.net"
@@ -202,7 +222,7 @@ Ensure that the `HttpRecorderMode` in the `TEST_CSM_ORGID_AUTHENTICATION` enviro
202222

203223
##### Environment = Current
204224

205-
AADAuthUri = "https://login.windows-ppe.net"
225+
AADAuthUri = "https://login.windows-ppe.net"
206226
GalleryUri = "https://df.gallery.azure-test.net/"
207227
GraphUri = "https://graph.ppe.windows.net/"
208228
IbizaPortalUri = "http://df.onecloud.azure-test.net"
@@ -215,6 +235,7 @@ Ensure that the `HttpRecorderMode` in the `TEST_CSM_ORGID_AUTHENTICATION` enviro
215235
DataLakeAnalyticsJoAbndCatalogServiceUri = "https://konaaccountdogfood.net"
216236

217237
##### Environment = Custom
238+
218239
When specified, test framework expect all Uri's to be provided by the user as part of the connection string.
219240

220241
What is also supported is as below (connections string example)

0 commit comments

Comments
 (0)