Managed HSM (#13287)

Force merge after CI passed

* [KeyVault] Managed HSM (#13259)

* Enable MHSM Management via *-AzKeyVault (#12575)

* Support creating a MHSM pool.

* Supporting querying MHSM objects

* Support deleting MHSM

* Support updating mhsm

* Add test cases

* Hide unavailable services

* Add test cases

* expose EnablePurgeProtection for MHSM

* correct indent of ps1xml

* upload localfeed

* Hide enablePurgeProtection

* Update ChangeLog.md and help.md

* Modify codes according to comments

* Update help.md

* Update VaultCreationParameters.cs

* Update get-azkeyvault.md

* Update KeyVaultManagementTests.ps1

* Record scenario test for Managed Hsm (#12631)

* Record scenario tests for managed hsm

* add default parameter set for

* Re-record scenario tests.

* [KeyVault] Update to official SDK (#12767)

* local 3.1.0 sdk

* Use production key vault mgmt sdk

* remove security domain from format xml

* combine track 1&2 sdk (#13018)

* combine track 1&2 sdk

* update azure.core to 1.5.0

* add exception; revert addkeyvaultkey

* add dll to psd1

* fix exception

* Detach managed hsm from key vault command (#13187)

* get managed hsm

* new managed hsm

* remove managed hsm

* update help.md

* update managed hsm

* add online version for new help.md

* Convert mhsm test to liveonly

* add test record

* Support data plane of managed HSM (#13216)

* create managed hsm key

* get managed hsm key

* remove managed hsm key

* update managed hsm key

* undo managed hsm key removal

* back up and restore managed hsm key

* add help.md

* import/download managed hsm RSA key

* Update help.md

* Update changelog.md

* suppress signature issues

* Update all help markdowns

* add logger for track2sdk

* add metadata for oct-HSM

Co-authored-by: Yeming Liu <[email protected]>

* Limit KeyType to be required only when create managed HSM key (#13242)

* limit KeyType to be required only when create managed HSM key

* add pester test

* Security domain (#13226)

* wip

* wip

* wip

* wip

* wip

* wip

* support securestring

* wip

* wip

* wip

* generate docs

* docs & error handling

* move crypto alg inside security domain

* resource strings

* remove extra code

* write help markdown

* resolve relative path to absolute path

* suppress signature issues

Co-authored-by: Yeming Liu <[email protected]>
Co-authored-by: Beisi Zhou <[email protected]>
Co-authored-by: Beisi Zhou <[email protected]>

* remove pfx file

* full backup restore + rbac (#13261)

Co-authored-by: Yeming Liu <[email protected]>

* small fixes

changelog; azure.core; sdk version; online version; breaking change csv;
shared assembly conflict; failed tests;

* license of BouncyCastle.NetCore

* remove local feed

Co-authored-by: Yeming Liu <[email protected]>
Co-authored-by: Yeming Liu <[email protected]>

* use portable.bouncycastle

* bug fix...

...replace bouncycastle lib;
fix security domain issue on windows powershell

* Add pester test for RBAC and full-backup managed HSM

* fix dll list

* support relative path when restoring SD

* update license (bouncycastle.netcore -> portable)

* remove dependency Microsoft.IdentityModel.Tokens

Co-authored-by: Beisi Zhou <[email protected]>
Co-authored-by: Yeming Liu <[email protected]>
Co-authored-by: Beisi Zhou <[email protected]>
Co-authored-by: Dingmeng Xue <[email protected]>

Author: 4 people

LICENSE.txt

@@ -223,5 +223,19 @@ Permission is hereby granted, free of charge, to any person obtaining a copy of
 The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
+***************
+
+The software includes Portable.BouncyCastle. The MIT License set out below is provided for informational purposes only. It is not the license that governs any part of the software.
+
+Portable.BouncyCastle
+
+LICENSE
+Copyright (c) 2000 - 2017 The Legion of the Bouncy Castle Inc. (http://www.bouncycastle.org)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 -------------END OF THIRD PARTY NOTICE----------------------------------------
 

src/Accounts/Authentication/Utilities/CustomAssemblyResolver.cs

@@ -26,6 +26,7 @@ public static class CustomAssemblyResolver
                 {"System.Reflection.DispatchProxy", new Version("4.0.3.0")},
                 {"System.Runtime.CompilerServices.Unsafe", new Version("4.0.5.0")},
                 {"System.Security.AccessControl", new Version("4.1.1.0")},
+                {"System.Security.Cryptography.Cng", new Version("4.3.0.0")},
                 {"System.Security.Permissions", new Version("4.0.1.0")},
                 {"System.Security.Principal.Windows", new Version("4.1.1.0")},
                 {"System.ServiceModel.Primitives", new Version("4.2.0.0")},

src/Compute/Compute.Test/Compute.Test.csproj

@@ -13,7 +13,7 @@
   <ItemGroup>
     <PackageReference Include="Microsoft.Azure.Graph.RBAC" Version="3.4.0-preview" />
     <PackageReference Include="Microsoft.Azure.Management.Compute" Version="39.0.0" />
-    <PackageReference Include="Microsoft.Azure.Management.KeyVault" Version="3.0.0" />
+    <PackageReference Include="Microsoft.Azure.Management.KeyVault" Version="3.1.0-preview.2" />
     <PackageReference Include="Microsoft.Azure.Management.Network" Version="20.1.1" />
   </ItemGroup>
 

src/EventHub/EventHub.Test/EventHub.Test.csproj

@@ -13,7 +13,7 @@
   <ItemGroup>
     <PackageReference Include="Microsoft.Azure.KeyVault" Version="3.0.5" />
     <PackageReference Include="Microsoft.Azure.Management.EventHub" Version="2.7.2" />
-    <PackageReference Include="Microsoft.Azure.Management.KeyVault" Version="3.0.0" />
+    <PackageReference Include="Microsoft.Azure.Management.KeyVault" Version="3.1.0-preview.2" />
   </ItemGroup>
 
   <ItemGroup>

src/HDInsight/HDInsight.Test/HDInsight.Test.csproj

@@ -14,7 +14,7 @@
     <PackageReference Include="Microsoft.Azure.KeyVault" Version="3.0.5" />
     <PackageReference Include="Microsoft.Azure.Management.HDInsight" Version="5.6.0" />
     <PackageReference Include="Microsoft.Azure.Management.HDInsight.Job" Version="2.0.7" />
-    <PackageReference Include="Microsoft.Azure.Management.KeyVault" Version="3.0.0" />
+    <PackageReference Include="Microsoft.Azure.Management.KeyVault" Version="3.1.0-preview.2" />
     <PackageReference Include="Microsoft.Azure.Management.ManagedServiceIdentity" Version="0.11.0" />
     <PackageReference Include="Microsoft.Azure.Management.Network" Version="20.1.1" />
     <PackageReference Include="Microsoft.Azure.Management.OperationalInsights" Version="0.21.0-preview" />

src/KeyVault/KeyVault.Test/KeyVault.Test.csproj

@@ -13,7 +13,7 @@
   <ItemGroup>
     <PackageReference Include="Microsoft.Azure.KeyVault" Version="3.0.1" />
     <PackageReference Include="Microsoft.Azure.KeyVault.WebKey" Version="3.0.1" />
-    <PackageReference Include="Microsoft.Azure.Management.KeyVault" Version="3.0.0" />
+    <PackageReference Include="Microsoft.Azure.Management.KeyVault" Version="3.1.0-preview.2" />
     <PackageReference Include="Microsoft.Azure.Management.Network" Version="20.1.1" />
   </ItemGroup>
 
@@ -23,7 +23,10 @@
 
   <ItemGroup>
     <None Update="Scripts\ControlPlane\KeyVaultManagementTests.ps1" CopyToOutputDirectory="PreserveNewest" />
-    <None Update="Resources\pshtest.pfx" CopyToOutputDirectory="PreserveNewest" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <Folder Include="ScenarioTests\PesterTests\" />
   </ItemGroup>
 
 </Project>

src/KeyVault/KeyVault.Test/Models/UtilitiesTests.cs

@@ -58,22 +58,5 @@ public void GetWebKeyFromByok()
             Assert.True(webKey.T.SequenceEqual(byokBlob));
             Assert.Equal(webKey.Kty, JsonWebKeyType.RsaHsm);
         }
-
-        [Fact]
-        [Trait(Category.AcceptanceType, Category.CheckIn)]
-        public void GetWebKeyFromCertificate()
-        {
-            string password = "123";
-            // This allows the test to run in Visual Studio and in the console runner. The file will exist in one of the two locations depending on the environment.
-            var consolePath = Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) ?? String.Empty, "Resources", "pshtest.pfx");
-            var vsPath = Path.Combine(Directory.GetCurrentDirectory(), "Resources", "pshtest.pfx");
-
-            IWebKeyConverter converters = WebKeyConverterFactory.CreateConverterChain();
-            var webKey = converters.ConvertKeyFromFile(new FileInfo(File.Exists(consolePath) ? consolePath : vsPath), password.ConvertToSecureString());
-
-            Assert.True(webKey.HasPrivateKey());
-            Assert.True(webKey.IsValid());
-            Assert.Equal(webKey.Kty, JsonWebKeyType.Rsa);
-        }
     }
 }

src/KeyVault/KeyVault.Test/PesterTests/ManagedHsmDatePlaneTests.Tests.ps1

@@ -0,0 +1,262 @@
+$here = Split-Path -Parent $MyInvocation.MyCommand.Path
+$sut = (Split-Path -Leaf $MyInvocation.MyCommand.Path) -replace '\.Tests\.', '.'
+. "$here\$sut"
+
+BeforeAll {
+    . $PSScriptRoot/ManagedHsmDatePlaneTests.ps1
+    ImportModules
+    $hsmName = GetAzManagedHsm
+}
+
+Describe "AddAzManagedHsmKey" {
+    It "Create a RSA key inside a managed HSM" {
+        $keyName = GetRandomName -Prefix "key"
+        $keyType = "RSA"
+        $rsaKey = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType $keyType
+        $rsaKey.VaultName | Should -BeExactly $hsmName
+        $rsaKey.Name | Should -BeExactly $keyName
+        $rsaKey.Attributes.KeyType | Should -Be "RSA-HSM"
+    }
+
+    It "Create an EC key with curve P-256 inside a managed HSM" {
+        $keyName = GetRandomName -Prefix "key"
+        $keyType = "EC"
+        $curveName = "P-256"
+        $rsaKey = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType $keyType -CurveName $curveName
+        $rsaKey.VaultName | Should -BeExactly $hsmName
+        $rsaKey.Name | Should -BeExactly $keyName
+        $rsaKey.Attributes.KeyType | Should -Be "EC-HSM"
+        $rsaKey.Key.CurveName | Should -Be $curveName
+    }
+
+    It "Create an oct key inside a managed HSM" {
+        $keyName = GetRandomName -Prefix "key"
+        $keyType = "oct"
+        $rsaKey = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType $keyType
+        $rsaKey.VaultName | Should -BeExactly $hsmName
+        $rsaKey.Name | Should -BeExactly $keyName
+        $rsaKey.Attributes.KeyType | Should -Be "oct-HSM"
+    }
+
+     It "Create an oct key inside a managed HSM" {
+        $keyName = GetRandomName -Prefix "key"
+        $keyType = "oct"
+        $rsaKey = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType $keyType
+        $rsaKey.VaultName | Should -BeExactly $hsmName
+        $rsaKey.Name | Should -BeExactly $keyName
+        $rsaKey.Attributes.KeyType | Should -Be "oct-HSM"
+    }
+
+     It "Create a key with non-default values inside a managed HSM" {
+        $keyName = GetRandomName -Prefix "key"
+        $keyType = "RSA"
+        $KeyOps = 'decrypt', 'verify'
+        # Expires & NotBefore is hard to cmpare, may add in the furture
+        $Tags = @{'Severity' = 'high'; 'Accounting' = "true"}
+
+        $key = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType $keyType -KeyOps $KeyOps -Disable -Tag $Tags
+        
+        $key.Attributes.KeyOps | Should -Be $KeyOps
+        $key.Tags.Count | Should -Be 2
+        $key.Enabled | Should -Be $false
+    }
+
+    # It "Import a RSA key from pfx file into a managed HSM" {
+    #     $keyName = GetRandomName -Prefix "key"
+    #     $keyFilePath = Join-Path $PSScriptRoot ../Resources/testImportKey.pfx -Resolve
+    #     $keyFilePwd = $null
+    #     $key = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyFilePath $keyFilePath -KeyFilePassword $keyFilePwd
+    #     $key.Name | Should -BeExactly $keyName
+    # }
+}
+
+Describe "GetAzManagedHsmKey"{
+    BeforeEach{
+        # Add a key
+        $keyName = GetRandomName -Prefix "key"
+        $key = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType "RSA"
+    }
+
+    It "List all the keys in a managed HSM" {
+        $keys = Get-AzManagedHsmKey -HsmName $hsmName
+        $keys.Count | Should -BeGreaterThan 0
+    }
+
+    It "Get a specific key in a managed HSM" {
+        $got = Get-AzManagedHsmKey -HsmName $hsmName -KeyName $keyName
+        $got.Id | Should -Be $key.Id
+    }
+
+    It "List all the keys that have been deleted in a managed HSM" {
+        Remove-AzManagedHsmKey -HsmName $hsmName -Name $keyName -Force
+        $deletedKey = Get-AzManagedHsmKey -HsmName $hsmName -KeyName $keyName -InRemovedState
+        $deletedKey.Id | Should -Be $key.Id
+    }
+    
+    It "Download a key from a managed HSM" {
+        $filePath = "$PSScriptRoot\public.pem"
+        Get-AzManagedHsmKey -HsmName $hsmName -KeyName $keyName -OutFile $filePath
+        $filePath | Should -Exist
+    }
+}
+
+Describe "RemoveAzManagedHsmKey"{
+    It "Remove a key from a managed HSM" {
+        $keyName = GetRandomName -Prefix "key"
+        $key = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType "RSA"
+        $deletedKey = Remove-AzManagedHsmKey -HsmName $hsmName -Name $keyName -Force -PassThru
+        $deletedKey.Id | Should -Be $key.Id
+    }
+
+    It "Purge a deleted key from a managed HSM" {
+        $keyName = GetRandomName -Prefix "key"
+        Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType "RSA"
+        Remove-AzManagedHsmKey -HsmName $hsmName -Name $keyName -Force -PassThru
+        Remove-AzManagedHsmKey -HsmName $hsmName -Name $keyName -InRemovedState -Force -PassThru
+        $deletedKey = Get-AzManagedHsmKey -HsmName $hsmName -Name $keyName -InRemovedState
+        $deletedKey | Should -Be $null
+    }
+
+    It "Remove keys by using piping" {
+        Get-AzManagedHsmKey -HsmName $hsmName | Remove-AzManagedHsmKey -Force
+        $keys = Get-AzManagedHsmKey -HsmName $hsmName
+        $keys.Count | Should -Be 0
+    }
+}
+
+Describe "UpdateAzManagedHsmKey"{
+    It "Enable a key and set tags" {
+        $keyName = GetRandomName -Prefix "key"
+        $key = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType "RSA" -Disable
+        $Tags = @{'Severity' = 'high'; 'Accounting' = 'true'}
+
+        $updatedKey = Update-AzManagedHsmKey -HsmName $hsmName -Name $keyName -Enable $True -Tag $Tags -PassThru
+       
+        $updatedKey.Id | Should -Be $key.Id
+        $updatedKey.Enabled | Should -Be $True
+        $updatedKey.Tags.Count | Should -Be 2
+    }
+}
+
+Describe "UndoAzManagedHsmKeyRemoval"{
+    It "Undo a key removal" {      
+        $keyName = GetRandomName -Prefix "key"
+        $key = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType "RSA"
+        $deletedKey = Remove-AzManagedHsmKey -HsmName $hsmName -Name $keyName -Force -PassThru
+        $recoveredKey = $deletedKey | Undo-AzManagedHsmKeyRemoval
+        $recoveredKey.Id | Should -Be $key.Id
+    }
+}
+
+Describe "BackupAndRestoreAzManagedHsmKey"{
+    It "Backup and restore a key" {      
+        $keyName = GetRandomName -Prefix "key"
+        $key = Add-AzManagedHsmKey -HsmName $hsmName -Name $keyName -KeyType "RSA"
+        $filePath = "$PSScriptRoot/backupkey.blob"  
+        $key | Backup-AzManagedHsmKey -OutputFile $filePath -Force
+        $filePath | Should -Exist
+ 
+        Remove-AzManagedHsmKey -HsmName $hsmName -Name $keyName -Force
+        Remove-AzManagedHsmKey -HsmName $hsmName -Name $keyName -InRemovedState -Force
+        $restoredKey = Restore-AzManagedHsmKey -HsmName $hsmName -InputFile $filePath
+        $restoredKey.Id | Should -Be $key.Id
+    }
+}
+
+Describe "BackupAndRestoreAzManagedHsm"{
+    BeforeEach{
+        $sasToken = ConvertTo-SecureString -AsPlainText -Force "?sv=2019-12-12&ss=bfqt&srt=sco&sp=rwdlacupx&se=2020-10-21T13:11:01Z&st=2020-10-21T05:11:01Z&spr=https&sig=******"
+        $containerUri = "https://{accountName}.blob.core.windows.net/{containerName}"
+    }
+
+    It "Backup a managed HSM" {
+        $uri = Backup-AzManagedHsm -Name $hsmName -StorageContainerUri $containerUri -SasToken $sasToken
+        $uri | Should -Not -Be $null
+    }
+
+    It "Restore a managed HSM" {
+        $restoreResult = Restore-AzManagedHsm -Name $hsmName -StorageContainerUri $containerUri -BackupFolder "mhsm-$hsmName-2020102105402658" -SasToken $sasToken -PassThru
+        $restoreResult | Should -Be $True
+    }
+}
+
+Describe "GetAzManagedHsmRoleDefinition"{
+    It "List all the roles at '/keys' scope" {
+        $roles = Get-AzManagedHsmRoleDefinition -HsmName $hsmName -Scope "/keys"
+        $roles.Count | Should -BeGreaterThan 0
+    }
+
+    It "Get a specific role" {
+        $backupRole = Get-AzManagedHsmRoleDefinition -HsmName $hsmName -RoleDefinitionName "managed hsm backup"
+        $backupRole | Should -Not -Be $null
+        $backupRole.Permissions | Should -Not -Be $null
+        $backupRole.Permissions.AllowedDataActions | Should -Not -Be $null
+    }
+}
+
+Describe "NewAzManagedHsmRoleAssignment"{
+    BeforeEach{
+        $signInName = "[email protected]"
+        $roleDefinitionName = "Managed HSM Backup"
+        # Clean role
+        $roleAssignment = Get-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName
+        if($roleAssignment){
+            Remove-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName
+        } 
+    }
+
+    It "Assign a role to user" {        
+        # Assign role
+        $roleAssignment = New-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName
+        $roleAssignment | Should -Not -Be $null
+        $roleAssignment.RoleDefinitionName | Should -Be $roleDefinitionName
+    }
+}
+
+Describe "RemoveAzManagedHsmRoleAssignment"{
+    BeforeEach{
+        # Assign role
+        $signInName = "[email protected]"
+        $roleDefinitionName = "Managed HSM Backup"
+        $roleAssignment = Get-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName
+        if(!$roleAssignment){
+            $roleAssignment = New-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName
+        } 
+    }
+
+    It "Revoke a role from user at '/keys' scope" {        
+        Remove-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName -Scope "/keys"
+        $roleAssignment = Get-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName
+        $roleAssignment | Should -Be $null       
+    }
+}
+
+Describe "GetAzManagedHsmRoleAssignment"{
+    BeforeEach{
+        # Assign role
+        $signInName = "[email protected]"
+        $roleDefinitionName = "Managed HSM Backup"
+        $roleAssignment = Get-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName
+        if(!$roleAssignment){
+            $roleAssignment = New-AzManagedHsmRoleAssignment -HsmName $hsmName -RoleDefinitionName $roleDefinitionName -SignInName $signInName
+        } 
+    }
+
+    It "List all role assignmentss in a managed HSM" {
+        $roleAssignments = Get-AzManagedHsmRoleAssignment -HsmName $hsmName
+        $roleAssignments | Should -Not -Be $null       
+        $roleAssignments.Count | Should -BeGreaterThan 0
+    }
+
+    It "List a user's role assignments in a managed HSM on '/keys' scope" {        
+        $roleAssignments = Get-AzManagedHsmRoleAssignment -HsmName $hsmName -SignInName $signInName -Scope "/keys"
+        $roleAssignments | Should -Not -Be $null       
+        $roleAssignments.Count | Should -BeGreaterThan 0
+    }
+}
+
+# to do: manually remove all stuffs in resource group
+# AfterAll {
+    # $hsm = Get-AzManagedHsm -Name $hsmName
+    # Remove-AzResourceGroup -Name $hsm.ResourceGroupName -Force
+# }