(GH-121) Add Azure Arc support

README.md

@@ -11,7 +11,7 @@
 
 1. [Description](#description)
 1. [Setup](#setup)
-1. [Managed Service Identity (MSI) vs Service Principal Credentials](#managed-service-identity-msi-vs-service-principal-credentials)
+1. [Managed Service Identity (MSI) vs Managed Identity for Azure Arc-enabled servers vs Service Principal Credentials](#managed-service-identity-msi-vs-managed-identity-for-azure-arc-enabled-servers-vs-service-principal-credentials)
 1. [How it works](#how-it-works)
     * [Puppet Function](#puppet-function)
     * [Hiera Backend](#hiera-backend)
@@ -38,12 +38,14 @@ The module requires the following:
   * Managed Service Identity ( MSI )
     * Puppet Server running on a machine with Managed Service Identity ( MSI ) and assigned the appropriate permissions
   to pull secrets from the vault. To learn more or get help with this please visit https://docs.microsoft.com/en-us/azure/active-directory/managed-service-identity/tutorial-windows-vm-access-nonaad
+  * Managed Identity for Azure Arc-enabled servers
+    * Follow Microsofts documentation on setting up an Azure Arc-enabled server.  To learn more or get help with this please visit https://learn.microsoft.com/en-us/azure/azure-arc/servers/learn/quick-enable-hybrid-vm
   * Service Principal
     * Following the required steps to setup a Service Principal.  To learn more or get help with this please visit https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app
 
-# Managed Service Identity (MSI) vs Service Principal Credentials
+# Managed Service Identity (MSI) vs Managed Identity for Azure Arc-enabled servers vs Service Principal Credentials
 
-This module provides 2 ways for users to authenticate with azure key vault and pull secrets. These 2 options are Managed Service Identity ( MSI ) and Service Principal Credentials.  We highly recommend you utilize Managed Service Identity over service principal credentials whenever possible.  This is because you do not have to manage and secure a file on our machines that contain credentials!  In some cases, Managed Service Identity ( MSI ) might not be an option for you.  One example of this is  if your Puppet server and some of your puppet agents are not hosted in Azure.  In that case, you can create a Service Principal in Azure Active Directory, assign the appropriate permissions to this Service Principal, and both the function and Hiera Backend provided in this module can authenticate to Azure Keyvault using the credentials of this Service Principal.
+This module provides 3 ways for users to authenticate with azure key vault and pull secrets. These 3 options are Managed Service Identity ( MSI ), Managed Identity for Azure Arc-enabled servers, Service Principal Credentials.  We highly recommend you utilize Managed Service Identity over service principal credentials whenever possible.  This is because you do not have to manage and secure a file on our machines that contain credentials!  In some cases, Managed Service Identity ( MSI ) might not be an option for you.  One example of this is  if your Puppet server and some of your puppet agents are not hosted in Azure.  In this case, we highly recommend you look at and use Azure Arc-enabled servers.  If for some reason this cannot be done, you should fallback to Service Principal Credentials.  This would require you to create a Service Principal in Azure Active Directory, assign the appropriate permissions to this Service Principal, and both the function and Hiera Backend provided in this module can authenticate to Azure Keyvault using the credentials of this Service Principal.
 
 ## How it works
 
@@ -69,9 +71,19 @@ In the above example the api_versions hash is important.  It is pinning both of
 * Instance Metadata Service Versions ( https://docs.microsoft.com/en-us/azure/virtual-machines/windows/instance-metadata-service )
 * Vault Versions ( TBD )
 
+#### Using Managed Identity for Azure Arc-enabled servers
 
-#### Using Service Principal Credentials
+```puppet
+$important_secret = azure_key_vault::secret('production-vault', 'important-secret', {
+  vault_api_version            => '2016-10-01',
+  metadata_api_version         => '2018-04-02',
+  use_azure_arc_authentication => true,
+})
+```
+
+This example shows how to utilize Managed Identity for Azure Arc-enabled servers. Similar to the above, the metadata endpoint on the Azure Arc-enabled service will be accessed to generate a secret file on the local machine. The secret within the file will be read and used to authenticate the machine to the secret in the corresponding vault you requested.
 
+#### Using Service Principal Credentials
 
 ```puppet
 $important_secret = azure_key_vault::secret('production-vault', 'important-secret', {
@@ -90,7 +102,6 @@ This example show how to utilize service principal credentials if you for some r
 
 This module contains a Hiera 5 backend that allows you to securely retrieve secrets from Azure key vault and use them in hiera.
 
-
 #### Using Managed Service Identity ( MSI )
 
 Add a new entry to the `hierarchy` hash in `hiera.yaml` providing the following required lookup options:
@@ -109,6 +120,25 @@ Add a new entry to the `hierarchy` hash in `hiera.yaml` providing the following
         - '^password.*'
 ```
 
+#### Using Managed Identity for Azure Arc-enabled servers
+
+To utilize Managed Identity for Azure Arc-enabled servers in hiera simply add `use_azure_arc_authentication` with the value of `true`.
+
+```yaml
+- name: 'Azure Key Vault Secrets'
+    lookup_key: azure_key_vault::lookup
+    options:
+      vault_name: production-vault
+      vault_api_version: '2016-10-01'
+      use_azure_arc_authentication: true
+      metadata_api_version: '2018-04-02'
+      key_replacement_token: '-'
+      confine_to_keys:
+        - '^azure_.*'
+        - '^.*_password$'
+        - '^password.*'
+```
+
 #### Using Service Principal Credentials
 
 To utilize service principal credentials in hiera simply replace `metadata_api_version` with `service_principal_credentials` and ensure it points to a valid yaml file that contains the service principal credentials you would like to use.

lib/puppet/functions/azure_key_vault/lookup.rb

@@ -9,7 +9,8 @@
       Optional[metadata_api_version] => String,
       confine_to_keys => Array[String],
       Optional[key_replacement_token] => String,
-      Optional[service_principal_credentials] => String
+      Optional[service_principal_credentials] => String,
+      Optional[use_azure_arc_authentication] => Boolean
     }]', :options
     param 'Puppet::LookupContext', :context
     return_type 'Variant[Sensitive, Undef]'
@@ -44,6 +45,8 @@ def lookup_key(secret_name, options, context)
     if access_token.nil?
       metadata_api_version = options['metadata_api_version']
       service_principal_credentials = options['service_principal_credentials']
+      use_azure_arc_authentication = options['use_azure_arc_authentication']
+
       if metadata_api_version && service_principal_credentials
         raise ArgumentError, 'metadata_api_version and service_principal_credentials cannot be used together'
       end
@@ -54,6 +57,8 @@ def lookup_key(secret_name, options, context)
       if service_principal_credentials
         credentials = YAML.load_file(service_principal_credentials)
         access_token = TragicCode::Azure.get_access_token_service_principal(credentials)
+      elsif use_azure_arc_authentication
+        access_token = TragicCode::Azure.get_access_token_azure_arc(metadata_api_version)
       else
         access_token = TragicCode::Azure.get_access_token(metadata_api_version)
       end

lib/puppet/functions/azure_key_vault/secret.rb

@@ -18,7 +18,8 @@
         tenant_id => String,
         client_id => String,
         client_secret => String
-      }]
+      }],
+      Optional[use_azure_arc_authentication] => Boolean
     }]', :api_endpoint_hash
     optional_param 'String', :secret_version
     return_type 'Sensitive[String]'
@@ -34,23 +35,40 @@ def secret(cache, vault_name, secret_name, api_endpoint_hash, secret_version = '
       partial_credentials = api_endpoint_hash['service_principal_credentials'].slice('tenant_id', 'client_id')
       Puppet.debug("service_principal_credentials: #{partial_credentials}")
     end
+    Puppet.debug("use_azure_arc_authentication: #{api_endpoint_hash['use_azure_arc_authentication']}")
     cache_hash = cache.retrieve(self)
     access_token_id = :"access_token_#{vault_name}"
     unless cache_hash.key?(access_token_id)
       Puppet.debug("retrieving access token since it's not in the cache")
       metadata_api_version = api_endpoint_hash['metadata_api_version']
       service_principal_credentials = api_endpoint_hash['service_principal_credentials']
+      use_azure_arc_authentication = api_endpoint_hash['use_azure_arc_authentication']
+
+      if !metadata_api_version && !service_principal_credentials
+        raise ArgumentError, 'hash must contain at least one of metadata_api_version or service_principal_credentials.'
+      end
+
+      # Service principal validation
       if metadata_api_version && service_principal_credentials
-        raise ArgumentError, 'metadata_api_version and service_principal_credentials cannot be used together'
+        raise ArgumentError, 'metadata_api_version and service_principal_credentials cannot be used together.'
       end
 
-      if service_principal_credentials
-        access_token = TragicCode::Azure.get_access_token_service_principal(service_principal_credentials)
-      elsif metadata_api_version
-        access_token = TragicCode::Azure.get_access_token(metadata_api_version)
-      else
-        raise ArgumentError, 'hash must contain at least one of metadata_api_version or service_principal_credentials'
+      # Azure arc validation
+      if service_principal_credentials && use_azure_arc_authentication
+        raise ArgumentError, 'service_principal_credentials and use_azure_arc_authentication cannot be used together.'
       end
+
+      if !metadata_api_version && use_azure_arc_authentication
+        raise ArgumentError, 'use_azure_arc_authentication must be used together with metadata_api_version.'
+      end
+
+      access_token = if service_principal_credentials
+                       TragicCode::Azure.get_access_token_service_principal(service_principal_credentials)
+                     elsif use_azure_arc_authentication
+                       TragicCode::Azure.get_access_token_azure_arc(metadata_api_version)
+                     else
+                       TragicCode::Azure.get_access_token(metadata_api_version)
+                     end
       cache_hash[access_token_id] = access_token
     end
 

lib/puppet_x/tragiccode/azure.rb

@@ -4,14 +4,43 @@
 module TragicCode
   # Azure API functions
   class Azure
+    @azure_arc_instance_metadata_endpoint_ip = '127.0.0.1'.freeze
+
     def self.normalize_object_name(object_name, replacement)
       object_name.gsub(%r{[^0-9a-zA-Z-]}, replacement)
     end
 
+    def self.get_access_token_azure_arc(api_version)
+      # Generate File and Read Challenge Token
+      uri = URI("http://#{@azure_arc_instance_metadata_endpoint_ip}/metadata/identity/oauth2/token?api-version=#{api_version}&resource=https%3A%2F%2Fvault.azure.net")
+      req = Net::HTTP::Get.new(uri.request_uri)
+      req['Metadata'] = 'true'
+      res = Net::HTTP.start(uri.hostname, uri.port) do |http|
+        http.request(req)
+      end
+
+      # 403 is expected here. we do not provide ANY key
+      raise res.body unless res.is_a?(Net::HTTPUnauthorized)
+      raise 'Response header Www-Authenticate is missing' unless res['Www-Authenticate']
+
+      challenge_token_file_path = res['Www-Authenticate'].sub(%r{\s*Basic\s+realm=}, '')
+      challenge_token = File.read(challenge_token_file_path)
+
+      # Get Access Token using challenge token
+      internal_get_access_token(api_version, @azure_arc_instance_metadata_endpoint_ip, { 'Authorization' => "Basic #{challenge_token}" })
+    end
+
     def self.get_access_token(api_version)
-      uri = URI("http://169.254.169.254/metadata/identity/oauth2/token?api-version=#{api_version}&resource=https%3A%2F%2Fvault.azure.net")
+      internal_get_access_token(api_version, '169.254.169.254')
+    end
+
+    def self.internal_get_access_token(api_version, instance_metadata_service_endpoint = '169.254.169.254', extra_http_headers_hash = {})
+      uri = URI("http://#{instance_metadata_service_endpoint}/metadata/identity/oauth2/token?api-version=#{api_version}&resource=https%3A%2F%2Fvault.azure.net")
       req = Net::HTTP::Get.new(uri.request_uri)
       req['Metadata'] = 'true'
+      extra_http_headers_hash.each do |key, value|
+        req[key] = value
+      end
       res = Net::HTTP.start(uri.hostname, uri.port) do |http|
         http.request(req)
       end

spec/functions/azure_key_vault_secret_spec.rb

@@ -23,6 +23,14 @@
       is_expected.to run.with_params.and_raise_error(ArgumentError, %r{expects between 3 and 4 arguments}i)
     end
 
+    it "errors when missing both 'metadata_api_version' and 'service_principal_credentials'" do
+      bad_hash = api_versions_hash
+      bad_hash.delete('metadata_api_version')
+      is_expected.to run.with_params(
+        vault_name, secret_name, bad_hash
+      ).and_raise_error(%r{hash must contain at least one of metadata_api_version or service_principal_credentials})
+    end
+
     it "errors when using both 'metadata_api_version' and 'service_principal_credentials'" do
       bad_hash = {
         'metadata_api_version' => 'test',
@@ -37,14 +45,6 @@
         vault_name, secret_name, bad_hash
       ).and_raise_error(%r{metadata_api_version and service_principal_credentials cannot be used together})
     end
-
-    it "errors when missing both 'metadata_api_version' and 'service_principal_credentials'" do
-      bad_hash = api_versions_hash
-      bad_hash.delete('metadata_api_version')
-      is_expected.to run.with_params(
-        vault_name, secret_name, bad_hash
-      ).and_raise_error(%r{hash must contain at least one of metadata_api_version or service_principal_credentials})
-    end
   end
 
   context 'when getting the latest version of a secret' do

spec/functions/tragic_code/azure_spec.rb

@@ -25,6 +25,50 @@
     end
   end
 
+  context '.get_access_token_azure_arc' do
+    it 'returns a bearer token' do
+      File.stub(:read).and_return('magical-token-from-file')
+
+      stub_request(:get, %r{127.0.0.1})
+        .to_return(
+          body: '{"access_token": "token"}',
+          status: 401,
+          headers: { 'Www-Authenticate' => 'Basic realm=C:\\ProgramData\\AzureConnectedMachineAgent\\Tokens\\f1da0584-97f4-42fd-a671-879ad3de86fa.key' },
+        )
+
+      stub_request(:get, %r{127.0.0.1})
+        .with(headers: { 'Authorization' => 'Basic magical-token-from-file' })
+        .to_return(body: '{"access_token": "token"}', status: 200)
+
+      expect(described_class.get_access_token_azure_arc('api')).to eq('token')
+    end
+
+    it 'throws error with response body when response is not 401 (unauthorized) when attempting to generate secret file' do
+      stub_request(:get, %r{127.0.0.1})
+        .to_return(body: 'some_error', status: 200)
+      expect { described_class.get_access_token_azure_arc('api') }.to raise_error('some_error')
+    end
+
+    it 'throws error when the 401 (unauthorized) response is missing the Www-Authenticate header' do
+      stub_request(:get, %r{127.0.0.1})
+        .to_return(body: 'some_error', status: 401)
+      expect { described_class.get_access_token_azure_arc('api') }.to raise_error('Response header Www-Authenticate is missing')
+    end
+
+    it 'throws error with response body when response is not 2xx when getting the auth token' do
+      File.stub(:read).and_return('magical-token-from-file')
+      # rubocop:disable Layout/LineLength
+      stub_request(:get, %r{127.0.0.1})
+        .to_return(body: '{"access_token": "token"}', status: 401, headers: { 'Www-Authenticate' => 'Basic realm=C:\\ProgramData\\AzureConnectedMachineAgent\\Tokens\\f1da0584-97f4-42fd-a671-879ad3de86fa.key' })
+      # rubocop:enable Layout/LineLength
+      stub_request(:get, %r{127.0.0.1})
+        .with(headers: { 'Authorization' => 'Basic magical-token-from-file' })
+        .to_return(body: 'some_error', status: 400)
+
+      expect { described_class.get_access_token_azure_arc('api') }.to raise_error('some_error')
+    end
+  end
+
   context '.get_access_token_service_principal' do
     let(:credentials) { { 'client_id' => '', 'tenant_id' => '', 'client_secret' => '' } }