|
1 | | -# AzureStack Canary validator |
2 | | -Canary validator provides a breadth customer experience with the Azure Stack deployment. It tries to exercise the various customer scenarios/usecases on the deployment. |
3 | | - |
4 | | -Instructions are relative to the .\CanaryValidator directory. |
5 | | -Canary can be invoked either as Service Administrator or Tenant Administrator. |
6 | | - |
7 | | -## Download Canary |
8 | | - |
9 | | -```powershell |
10 | | -Invoke-WebRequest https://github.com/Azure/AzureStack-Tools/archive/master.zip -OutFile master.zip |
11 | | -Expand-Archive master.zip -DestinationPath . -Force |
12 | | -Set-Location -Path ".\AzureStack-Tools-master\CanaryValidator" -PassThru |
13 | | -``` |
14 | | - |
15 | | -## To execute Canary as Tenant Administrator (if Windows Server 2016 or Windows Server 2012-R2 images are already present in the PIR) |
16 | | - |
17 | | -```powershell |
18 | | -# Install-Module -Name 'AzureRm.Bootstrapper' |
19 | | -# Install-AzureRmProfile -profile '2017-03-09-profile' -Force |
20 | | -# Install-Module -Name AzureStack -RequiredVersion 1.2.10 |
21 | | -$TenantAdminCreds = New-Object System.Management.Automation.PSCredential "<Tenant Admin username>", (ConvertTo-SecureString "<Tenant Admin password>" -AsPlainText -Force) |
22 | | -$ServiceAdminCreds = New-Object System.Management.Automation.PSCredential "<Service Admin username>", (ConvertTo-SecureString "<Service Admin password>" -AsPlainText -Force) |
23 | | -.\Canary.Tests.ps1 -TenantID "<TenantID from Azure Active Directory>" -AdminArmEndpoint "<Administrative ARM endpoint>" -ServiceAdminCredentials $ServiceAdminCreds -TenantArmEndpoint "<Tenant ARM endpoint>" -TenantAdminCredentials $TenantAdminCreds |
24 | | -``` |
25 | | - |
26 | | -## To execute Canary as Tenant Administrator (if Windows Server 2016 or Windows Server 2012-R2 images are not present in PIR) |
27 | | - |
28 | | -```powershell |
29 | | -# Download the WS2016 ISO image from: https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2016, and place it on your local machine |
30 | | -# Install-Module -Name 'AzureRm.Bootstrapper' |
31 | | -# Install-AzureRmProfile -profile '2017-03-09-profile' -Force |
32 | | -# Install-Module -Name AzureStack -RequiredVersion 1.2.10 |
33 | | -$TenantAdminCreds = New-Object System.Management.Automation.PSCredential "<Tenant Admin username>", (ConvertTo-SecureString "<Tenant Admin password>" -AsPlainText -Force) |
34 | | -$ServiceAdminCreds = New-Object System.Management.Automation.PSCredential "<Service Admin username>", (ConvertTo-SecureString "<Service Admin password>" -AsPlainText -Force) |
35 | | -.\Canary.Tests.ps1 -TenantID "<TenantID from Azure Active Directory>" -AdminArmEndpoint "<Administrative ARM endpoint>" -ServiceAdminCredentials $ServiceAdminCreds -TenantArmEndpoint "<Tenant ARM endpoint>" -TenantAdminCredentials $TenantAdminCreds -WindowsISOPath "<path where the WS2016 ISO is present>" |
36 | | -``` |
37 | | -## NOTE: When running Canary against ADFS environment, please make sure to pass in the TenantAdminObjectId parameter |
38 | | - |
39 | | -## To list the usecases in Canary |
40 | | - |
41 | | -```powershell |
42 | | -# Install-Module -Name 'AzureRm.Bootstrapper' |
43 | | -# Install-AzureRmProfile -profile '2017-03-09-profile' -Force |
44 | | -# Install-Module -Name AzureStack -RequiredVersion 1.2.10 |
45 | | -.\Canary.Tests.ps1 -ListAvailable |
46 | | -
|
47 | | -Sample output: |
48 | | -PS C:\AzureStack-Tools-vnext\CanaryValidator> .\Canary.Tests.ps1 -ListAvailable |
49 | | -List of scenarios in Canary: |
50 | | - CreateAdminAzureStackEnv |
51 | | - LoginToAzureStackEnvAsSvcAdmin |
52 | | - SelectDefaultProviderSubscription |
53 | | - ListFabricResourceProviderInfo |
54 | | - |-- GetAzureStackInfraRole |
55 | | - |-- GetAzureStackInfraRoleInstance |
56 | | - |-- GetAzureStackLogicalNetwork |
57 | | - |-- GetAzureStackStorageCapacity |
58 | | - |-- GetAzureStackInfrastructureShare |
59 | | - |-- GetAzureStackScaleUnit |
60 | | - |-- GetAzureStackScaleUnitNode |
61 | | - |-- GetAzureStackIPPool |
62 | | - |-- GetAzureStackMacPool |
63 | | - |-- GetAzureStackGatewayPool |
64 | | - |-- GetAzureStackSLBMux |
65 | | - |-- GetAzureStackGateway |
66 | | - ListHealthResourceProviderAlerts |
67 | | - |-- GetAzureStackAlert |
68 | | - ListUpdatesResourceProviderInfo |
69 | | - |-- GetAzureStackUpdateSummary |
70 | | - |-- GetAzureStackUpdateToApply |
71 | | - UploadLinuxImageToPIR |
72 | | - CreateTenantAzureStackEnv |
73 | | - CreateResourceGroupForTenantSubscription |
74 | | - CreateTenantPlan |
75 | | - CreateTenantOffer |
76 | | - CreateTenantDefaultManagedSubscription |
77 | | - LoginToAzureStackEnvAsTenantAdmin |
78 | | - CreateTenantSubscription |
79 | | - RoleAssignmentAndCustomRoleDefinition |
80 | | - |-- ListAssignedRoles |
81 | | - |-- ListExistingRoleDefinitions |
82 | | - |-- GetProviderOperations |
83 | | - |-- AssignReaderRole |
84 | | - |-- VerifyReaderRoleAssignment |
85 | | - |-- RemoveReaderRoleAssignment |
86 | | - |-- CustomRoleDefinition |
87 | | - |-- ListRoleDefinitionsAfterCustomRoleCreation |
88 | | - |-- RemoveCustomRoleDefinition |
89 | | - RegisterResourceProviders |
90 | | - CreateResourceGroupForUtilities |
91 | | - CreateStorageAccountForUtilities |
92 | | - CreateStorageContainerForUtilities |
93 | | - CreateDSCScriptResourceUtility |
94 | | - CreateCustomScriptResourceUtility |
95 | | - CreateDataDiskForVM |
96 | | - UploadUtilitiesToBlobStorage |
97 | | - CreateKeyVaultStoreForCertSecret |
98 | | - CreateResourceGroupForVMs |
99 | | - DeployARMTemplate |
100 | | - RetrieveResourceDeploymentTimes |
101 | | - QueryTheVMsDeployed |
102 | | - CheckVMCommunicationPreVMReboot |
103 | | - TransmitMTUSizedPacketsBetweenTenantVMs |
104 | | - AddDatadiskToVMWithPrivateIP |
105 | | - |-- StopDeallocateVMWithPrivateIPBeforeAddingDatadisk |
106 | | - |-- AddTheDataDiskToVMWithPrivateIP |
107 | | - |-- StartVMWithPrivateIPAfterAddingDatadisk |
108 | | - ApplyDataDiskCheckCustomScriptExtensionToVMWithPrivateIP |
109 | | - |-- CheckForExistingCustomScriptExtensionOnVMWithPrivateIP |
110 | | - |-- ApplyCustomScriptExtensionToVMWithPrivateIP |
111 | | - RestartVMWithPublicIP |
112 | | - StopDeallocateVMWithPrivateIP |
113 | | - StartVMWithPrivateIP |
114 | | - CheckVMCommunicationPostVMReboot |
115 | | - CheckExistenceOfScreenShotForVMWithPrivateIP |
116 | | - EnumerateAllResources |
117 | | - DeleteVMWithPrivateIP |
118 | | - DeleteVMResourceGroup |
119 | | - DeleteUtilitiesResourceGroup |
120 | | - TenantRelatedcleanup |
121 | | - |-- DeleteTenantSubscriptions |
122 | | - |-- LoginToAzureStackEnvAsSvcAdminForCleanup |
123 | | - |-- RemoveLinuxImageFromPIR |
124 | | - |-- DeleteSubscriptionResourceGroup |
125 | | -``` |
126 | | - |
127 | | -## To exclude certain usecases from getting executed |
128 | | - |
129 | | -```powershell |
130 | | -# Install-Module -Name 'AzureRm.Bootstrapper' |
131 | | -# Install-AzureRmProfile -profile '2017-03-09-profile' -Force |
132 | | -# Install-Module -Name AzureStack -RequiredVersion 1.2.10 |
133 | | -# A new paramter called ExclusionList has been added which is a string array. Pass in the list of usecases you don't want to execute to this parameter. |
134 | | -$ServiceAdminCreds = New-Object System.Management.Automation.PSCredential "<Service Admin username>", (ConvertTo-SecureString "<Service Admin password>" -AsPlainText -Force) |
135 | | -.\Canary.Tests.ps1 -TenantID "<TenantID from Azure Active Directory>" -AdminArmEndpoint "<Administrative ARM endpoint>" -ServiceAdminCredentials $ServiceAdminCreds -ExclusionList "ListFabricResourceProviderInfo","ListUpdateResourceProviderInfo" |
136 | | -``` |
137 | | - |
138 | | -## Reading the results & logs |
139 | | - |
140 | | -Canary generates log files in the TMP directory ($env:TMP). The logs can be found under the directory "CanaryLogs[DATETIME]". There are two types of logs generated, a text log and a JSON log. JSON log provides a quick and easy view of all the usecases and their corresponding results. Text log provides a more detailed output of each usecase execution, its output and results. |
141 | | - |
142 | | -Each usecase entry in the JSON log consists of the following fields. |
143 | | - |
144 | | -- Name |
145 | | -- Description |
146 | | -- StartTime |
147 | | -- EndTime |
148 | | -- Result |
149 | | -- Exception (in case a scenario fails) |
150 | | - |
151 | | -The exception field is helpful to debug failed use cases. |
| 1 | +# AzureStack Canary validator |
| 2 | +Canary validator provides a breadth customer experience with the Azure Stack deployment. It tries to exercise the various customer scenarios/usecases on the deployment. |
| 3 | + |
| 4 | +Instructions are relative to the .\CanaryValidator directory. |
| 5 | +Canary can be invoked either as Service Administrator or Tenant Administrator. |
| 6 | + |
| 7 | +## Download Canary |
| 8 | + |
| 9 | +```powershell |
| 10 | +Invoke-WebRequest https://github.com/Azure/AzureStack-Tools/archive/master.zip -OutFile master.zip |
| 11 | +Expand-Archive master.zip -DestinationPath . -Force |
| 12 | +Set-Location -Path ".\AzureStack-Tools-master\CanaryValidator" -PassThru |
| 13 | +``` |
| 14 | + |
| 15 | +## To execute Canary as Tenant Administrator (if Windows Server 2016 or Windows Server 2012-R2 images are already present in the PIR) |
| 16 | + |
| 17 | +```powershell |
| 18 | +# Install-Module -Name 'AzureRm.Bootstrapper' |
| 19 | +# Install-AzureRmProfile -profile '2017-03-09-profile' -Force |
| 20 | +# Install-Module -Name AzureStack -RequiredVersion 1.2.10 |
| 21 | +$TenantAdminCreds = New-Object System.Management.Automation.PSCredential "<Tenant Admin username>", (ConvertTo-SecureString "<Tenant Admin password>" -AsPlainText -Force) |
| 22 | +$ServiceAdminCreds = New-Object System.Management.Automation.PSCredential "<Service Admin username>", (ConvertTo-SecureString "<Service Admin password>" -AsPlainText -Force) |
| 23 | +.\Canary.Tests.ps1 -TenantID "<TenantID from Azure Active Directory>" -AdminArmEndpoint "<Administrative ARM endpoint>" -ServiceAdminCredentials $ServiceAdminCreds -TenantArmEndpoint "<Tenant ARM endpoint>" -TenantAdminCredentials $TenantAdminCreds |
| 24 | +``` |
| 25 | + |
| 26 | +## To execute Canary as Tenant Administrator (if Windows Server 2016 or Windows Server 2012-R2 images are not present in PIR) |
| 27 | + |
| 28 | +```powershell |
| 29 | +# Download the WS2016 ISO image from: https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2016, and place it on your local machine |
| 30 | +# Install-Module -Name 'AzureRm.Bootstrapper' |
| 31 | +# Install-AzureRmProfile -profile '2017-03-09-profile' -Force |
| 32 | +# Install-Module -Name AzureStack -RequiredVersion 1.2.10 |
| 33 | +$TenantAdminCreds = New-Object System.Management.Automation.PSCredential "<Tenant Admin username>", (ConvertTo-SecureString "<Tenant Admin password>" -AsPlainText -Force) |
| 34 | +$ServiceAdminCreds = New-Object System.Management.Automation.PSCredential "<Service Admin username>", (ConvertTo-SecureString "<Service Admin password>" -AsPlainText -Force) |
| 35 | +.\Canary.Tests.ps1 -TenantID "<TenantID from Azure Active Directory>" -AdminArmEndpoint "<Administrative ARM endpoint>" -ServiceAdminCredentials $ServiceAdminCreds -TenantArmEndpoint "<Tenant ARM endpoint>" -TenantAdminCredentials $TenantAdminCreds -WindowsISOPath "<path where the WS2016 ISO is present>" |
| 36 | +``` |
| 37 | +## NOTE: When running Canary against ADFS environment, please make sure to pass in the TenantAdminObjectId parameter |
| 38 | + |
| 39 | +## To list the usecases in Canary |
| 40 | + |
| 41 | +```powershell |
| 42 | +# Install-Module -Name 'AzureRm.Bootstrapper' |
| 43 | +# Install-AzureRmProfile -profile '2017-03-09-profile' -Force |
| 44 | +# Install-Module -Name AzureStack -RequiredVersion 1.2.10 |
| 45 | +.\Canary.Tests.ps1 -ListAvailable |
| 46 | +
|
| 47 | +Sample output: |
| 48 | +PS C:\AzureStack-Tools-vnext\CanaryValidator> .\Canary.Tests.ps1 -ListAvailable |
| 49 | +List of scenarios in Canary: |
| 50 | + CreateAdminAzureStackEnv |
| 51 | + LoginToAzureStackEnvAsSvcAdmin |
| 52 | + SelectDefaultProviderSubscription |
| 53 | + ListFabricResourceProviderInfo |
| 54 | + |-- GetAzureStackInfraRole |
| 55 | + |-- GetAzureStackInfraRoleInstance |
| 56 | + |-- GetAzureStackLogicalNetwork |
| 57 | + |-- GetAzureStackStorageCapacity |
| 58 | + |-- GetAzureStackInfrastructureShare |
| 59 | + |-- GetAzureStackScaleUnit |
| 60 | + |-- GetAzureStackScaleUnitNode |
| 61 | + |-- GetAzureStackIPPool |
| 62 | + |-- GetAzureStackMacPool |
| 63 | + |-- GetAzureStackGatewayPool |
| 64 | + |-- GetAzureStackSLBMux |
| 65 | + |-- GetAzureStackGateway |
| 66 | + ListHealthResourceProviderAlerts |
| 67 | + |-- GetAzureStackAlert |
| 68 | + ListUpdatesResourceProviderInfo |
| 69 | + |-- GetAzureStackUpdateSummary |
| 70 | + |-- GetAzureStackUpdateToApply |
| 71 | + UploadLinuxImageToPIR |
| 72 | + CreateTenantAzureStackEnv |
| 73 | + CreateResourceGroupForTenantSubscription |
| 74 | + CreateTenantPlan |
| 75 | + CreateTenantOffer |
| 76 | + CreateTenantDefaultManagedSubscription |
| 77 | + LoginToAzureStackEnvAsTenantAdmin |
| 78 | + CreateTenantSubscription |
| 79 | + RoleAssignmentAndCustomRoleDefinition |
| 80 | + |-- ListAssignedRoles |
| 81 | + |-- ListExistingRoleDefinitions |
| 82 | + |-- GetProviderOperations |
| 83 | + |-- AssignReaderRole |
| 84 | + |-- VerifyReaderRoleAssignment |
| 85 | + |-- RemoveReaderRoleAssignment |
| 86 | + |-- CustomRoleDefinition |
| 87 | + |-- ListRoleDefinitionsAfterCustomRoleCreation |
| 88 | + |-- RemoveCustomRoleDefinition |
| 89 | + RegisterResourceProviders |
| 90 | + CreateResourceGroupForUtilities |
| 91 | + CreateStorageAccountForUtilities |
| 92 | + CreateStorageContainerForUtilities |
| 93 | + CreateDSCScriptResourceUtility |
| 94 | + CreateCustomScriptResourceUtility |
| 95 | + CreateDataDiskForVM |
| 96 | + UploadUtilitiesToBlobStorage |
| 97 | + CreateKeyVaultStoreForCertSecret |
| 98 | + CreateResourceGroupForVMs |
| 99 | + DeployARMTemplate |
| 100 | + RetrieveResourceDeploymentTimes |
| 101 | + QueryTheVMsDeployed |
| 102 | + CheckVMCommunicationPreVMReboot |
| 103 | + TransmitMTUSizedPacketsBetweenTenantVMs |
| 104 | + AddDatadiskToVMWithPrivateIP |
| 105 | + |-- StopDeallocateVMWithPrivateIPBeforeAddingDatadisk |
| 106 | + |-- AddTheDataDiskToVMWithPrivateIP |
| 107 | + |-- StartVMWithPrivateIPAfterAddingDatadisk |
| 108 | + ApplyDataDiskCheckCustomScriptExtensionToVMWithPrivateIP |
| 109 | + |-- CheckForExistingCustomScriptExtensionOnVMWithPrivateIP |
| 110 | + |-- ApplyCustomScriptExtensionToVMWithPrivateIP |
| 111 | + RestartVMWithPublicIP |
| 112 | + StopDeallocateVMWithPrivateIP |
| 113 | + StartVMWithPrivateIP |
| 114 | + CheckVMCommunicationPostVMReboot |
| 115 | + CheckExistenceOfScreenShotForVMWithPrivateIP |
| 116 | + EnumerateAllResources |
| 117 | + DeleteVMWithPrivateIP |
| 118 | + DeleteVMResourceGroup |
| 119 | + DeleteUtilitiesResourceGroup |
| 120 | + TenantRelatedcleanup |
| 121 | + |-- DeleteTenantSubscriptions |
| 122 | + |-- LoginToAzureStackEnvAsSvcAdminForCleanup |
| 123 | + |-- RemoveLinuxImageFromPIR |
| 124 | + |-- DeleteSubscriptionResourceGroup |
| 125 | +``` |
| 126 | + |
| 127 | +## To exclude certain usecases from getting executed |
| 128 | + |
| 129 | +```powershell |
| 130 | +# Install-Module -Name 'AzureRm.Bootstrapper' |
| 131 | +# Install-AzureRmProfile -profile '2017-03-09-profile' -Force |
| 132 | +# Install-Module -Name AzureStack -RequiredVersion 1.2.10 |
| 133 | +# A new paramter called ExclusionList has been added which is a string array. Pass in the list of usecases you don't want to execute to this parameter. |
| 134 | +$ServiceAdminCreds = New-Object System.Management.Automation.PSCredential "<Service Admin username>", (ConvertTo-SecureString "<Service Admin password>" -AsPlainText -Force) |
| 135 | +.\Canary.Tests.ps1 -TenantID "<TenantID from Azure Active Directory>" -AdminArmEndpoint "<Administrative ARM endpoint>" -ServiceAdminCredentials $ServiceAdminCreds -ExclusionList "ListFabricResourceProviderInfo","ListUpdateResourceProviderInfo" |
| 136 | +``` |
| 137 | + |
| 138 | +## Reading the results & logs |
| 139 | + |
| 140 | +Canary generates log files in the TMP directory ($env:TMP). The logs can be found under the directory "CanaryLogs[DATETIME]". There are two types of logs generated, a text log and a JSON log. JSON log provides a quick and easy view of all the usecases and their corresponding results. Text log provides a more detailed output of each usecase execution, its output and results. |
| 141 | + |
| 142 | +Each usecase entry in the JSON log consists of the following fields. |
| 143 | + |
| 144 | +- Name |
| 145 | +- Description |
| 146 | +- StartTime |
| 147 | +- EndTime |
| 148 | +- Result |
| 149 | +- Exception (in case a scenario fails) |
| 150 | + |
| 151 | +The exception field is helpful to debug failed use cases. |
0 commit comments