diff --git a/README.md b/README.md index c6ca703..5295848 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ 1. [Description](#description) 1. [Setup](#setup) -1. [Managed Service Identity (MSI) vs Service Principal Credentials](#managed-service-identity-msi-vs-service-principal-credentials) +1. [Managed Service Identity (MSI) vs Managed Identity for Azure Arc-enabled servers vs Service Principal Credentials](#managed-service-identity-msi-vs-managed-identity-for-azure-arc-enabled-servers-vs-service-principal-credentials) 1. [How it works](#how-it-works) * [Puppet Function](#puppet-function) * [Hiera Backend](#hiera-backend) @@ -38,12 +38,14 @@ The module requires the following: * Managed Service Identity ( MSI ) * Puppet Server running on a machine with Managed Service Identity ( MSI ) and assigned the appropriate permissions to pull secrets from the vault. To learn more or get help with this please visit https://docs.microsoft.com/en-us/azure/active-directory/managed-service-identity/tutorial-windows-vm-access-nonaad + * Managed Identity for Azure Arc-enabled servers + * Follow Microsofts documentation on setting up an Azure Arc-enabled server. To learn more or get help with this please visit https://learn.microsoft.com/en-us/azure/azure-arc/servers/learn/quick-enable-hybrid-vm * Service Principal * Following the required steps to setup a Service Principal. To learn more or get help with this please visit https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app -# Managed Service Identity (MSI) vs Service Principal Credentials +# Managed Service Identity (MSI) vs Managed Identity for Azure Arc-enabled servers vs Service Principal Credentials -This module provides 2 ways for users to authenticate with azure key vault and pull secrets. These 2 options are Managed Service Identity ( MSI ) and Service Principal Credentials. We highly recommend you utilize Managed Service Identity over service principal credentials whenever possible. This is because you do not have to manage and secure a file on our machines that contain credentials! In some cases, Managed Service Identity ( MSI ) might not be an option for you. One example of this is if your Puppet server and some of your puppet agents are not hosted in Azure. In that case, you can create a Service Principal in Azure Active Directory, assign the appropriate permissions to this Service Principal, and both the function and Hiera Backend provided in this module can authenticate to Azure Keyvault using the credentials of this Service Principal. +This module provides 3 ways for users to authenticate with azure key vault and pull secrets. These 3 options are Managed Service Identity ( MSI ), Managed Identity for Azure Arc-enabled servers, Service Principal Credentials. We highly recommend you utilize Managed Service Identity over service principal credentials whenever possible. This is because you do not have to manage and secure a file on our machines that contain credentials! In some cases, Managed Service Identity ( MSI ) might not be an option for you. One example of this is if your Puppet server and some of your puppet agents are not hosted in Azure. In this case, we highly recommend you look at and use Azure Arc-enabled servers. If for some reason this cannot be done, you should fallback to Service Principal Credentials. This would require you to create a Service Principal in Azure Active Directory, assign the appropriate permissions to this Service Principal, and both the function and Hiera Backend provided in this module can authenticate to Azure Keyvault using the credentials of this Service Principal. ## How it works @@ -69,9 +71,19 @@ In the above example the api_versions hash is important. It is pinning both of * Instance Metadata Service Versions ( https://docs.microsoft.com/en-us/azure/virtual-machines/windows/instance-metadata-service ) * Vault Versions ( TBD ) +#### Using Managed Identity for Azure Arc-enabled servers -#### Using Service Principal Credentials +```puppet +$important_secret = azure_key_vault::secret('production-vault', 'important-secret', { + vault_api_version => '2016-10-01', + metadata_api_version => '2018-04-02', + use_azure_arc_authentication => true, +}) +``` + +This example shows how to utilize Managed Identity for Azure Arc-enabled servers. Similar to the above, the metadata endpoint on the Azure Arc-enabled service will be accessed to generate a secret file on the local machine. The secret within the file will be read and used to authenticate the machine to the secret in the corresponding vault you requested. +#### Using Service Principal Credentials ```puppet $important_secret = azure_key_vault::secret('production-vault', 'important-secret', { @@ -90,7 +102,6 @@ This example show how to utilize service principal credentials if you for some r This module contains a Hiera 5 backend that allows you to securely retrieve secrets from Azure key vault and use them in hiera. - #### Using Managed Service Identity ( MSI ) Add a new entry to the `hierarchy` hash in `hiera.yaml` providing the following required lookup options: @@ -109,6 +120,25 @@ Add a new entry to the `hierarchy` hash in `hiera.yaml` providing the following - '^password.*' ``` +#### Using Managed Identity for Azure Arc-enabled servers + +To utilize Managed Identity for Azure Arc-enabled servers in hiera simply add `use_azure_arc_authentication` with the value of `true`. + +```yaml +- name: 'Azure Key Vault Secrets' + lookup_key: azure_key_vault::lookup + options: + vault_name: production-vault + vault_api_version: '2016-10-01' + use_azure_arc_authentication: true + metadata_api_version: '2018-04-02' + key_replacement_token: '-' + confine_to_keys: + - '^azure_.*' + - '^.*_password$' + - '^password.*' +``` + #### Using Service Principal Credentials To utilize service principal credentials in hiera simply replace `metadata_api_version` with `service_principal_credentials` and ensure it points to a valid yaml file that contains the service principal credentials you would like to use. diff --git a/lib/puppet/functions/azure_key_vault/lookup.rb b/lib/puppet/functions/azure_key_vault/lookup.rb index 8ba54e2..fa02cb2 100644 --- a/lib/puppet/functions/azure_key_vault/lookup.rb +++ b/lib/puppet/functions/azure_key_vault/lookup.rb @@ -9,7 +9,8 @@ Optional[metadata_api_version] => String, confine_to_keys => Array[String], Optional[key_replacement_token] => String, - Optional[service_principal_credentials] => String + Optional[service_principal_credentials] => String, + Optional[use_azure_arc_authentication] => Boolean }]', :options param 'Puppet::LookupContext', :context return_type 'Variant[Sensitive, Undef]' @@ -44,6 +45,8 @@ def lookup_key(secret_name, options, context) if access_token.nil? metadata_api_version = options['metadata_api_version'] service_principal_credentials = options['service_principal_credentials'] + use_azure_arc_authentication = options['use_azure_arc_authentication'] + if metadata_api_version && service_principal_credentials raise ArgumentError, 'metadata_api_version and service_principal_credentials cannot be used together' end @@ -54,6 +57,8 @@ def lookup_key(secret_name, options, context) if service_principal_credentials credentials = YAML.load_file(service_principal_credentials) access_token = TragicCode::Azure.get_access_token_service_principal(credentials) + elsif use_azure_arc_authentication + access_token = TragicCode::Azure.get_access_token_azure_arc(metadata_api_version) else access_token = TragicCode::Azure.get_access_token(metadata_api_version) end diff --git a/lib/puppet/functions/azure_key_vault/secret.rb b/lib/puppet/functions/azure_key_vault/secret.rb index 1e4e463..8ac421c 100644 --- a/lib/puppet/functions/azure_key_vault/secret.rb +++ b/lib/puppet/functions/azure_key_vault/secret.rb @@ -18,7 +18,8 @@ tenant_id => String, client_id => String, client_secret => String - }] + }], + Optional[use_azure_arc_authentication] => Boolean }]', :api_endpoint_hash optional_param 'String', :secret_version return_type 'Sensitive[String]' @@ -34,23 +35,40 @@ def secret(cache, vault_name, secret_name, api_endpoint_hash, secret_version = ' partial_credentials = api_endpoint_hash['service_principal_credentials'].slice('tenant_id', 'client_id') Puppet.debug("service_principal_credentials: #{partial_credentials}") end + Puppet.debug("use_azure_arc_authentication: #{api_endpoint_hash['use_azure_arc_authentication']}") cache_hash = cache.retrieve(self) access_token_id = :"access_token_#{vault_name}" unless cache_hash.key?(access_token_id) Puppet.debug("retrieving access token since it's not in the cache") metadata_api_version = api_endpoint_hash['metadata_api_version'] service_principal_credentials = api_endpoint_hash['service_principal_credentials'] + use_azure_arc_authentication = api_endpoint_hash['use_azure_arc_authentication'] + + if !metadata_api_version && !service_principal_credentials + raise ArgumentError, 'hash must contain at least one of metadata_api_version or service_principal_credentials.' + end + + # Service principal validation if metadata_api_version && service_principal_credentials - raise ArgumentError, 'metadata_api_version and service_principal_credentials cannot be used together' + raise ArgumentError, 'metadata_api_version and service_principal_credentials cannot be used together.' end - if service_principal_credentials - access_token = TragicCode::Azure.get_access_token_service_principal(service_principal_credentials) - elsif metadata_api_version - access_token = TragicCode::Azure.get_access_token(metadata_api_version) - else - raise ArgumentError, 'hash must contain at least one of metadata_api_version or service_principal_credentials' + # Azure arc validation + if service_principal_credentials && use_azure_arc_authentication + raise ArgumentError, 'service_principal_credentials and use_azure_arc_authentication cannot be used together.' end + + if !metadata_api_version && use_azure_arc_authentication + raise ArgumentError, 'use_azure_arc_authentication must be used together with metadata_api_version.' + end + + access_token = if service_principal_credentials + TragicCode::Azure.get_access_token_service_principal(service_principal_credentials) + elsif use_azure_arc_authentication + TragicCode::Azure.get_access_token_azure_arc(metadata_api_version) + else + TragicCode::Azure.get_access_token(metadata_api_version) + end cache_hash[access_token_id] = access_token end diff --git a/lib/puppet_x/tragiccode/azure.rb b/lib/puppet_x/tragiccode/azure.rb index 1444790..4dd6251 100644 --- a/lib/puppet_x/tragiccode/azure.rb +++ b/lib/puppet_x/tragiccode/azure.rb @@ -4,14 +4,43 @@ module TragicCode # Azure API functions class Azure + @azure_arc_instance_metadata_endpoint_ip = '127.0.0.1'.freeze + def self.normalize_object_name(object_name, replacement) object_name.gsub(%r{[^0-9a-zA-Z-]}, replacement) end + def self.get_access_token_azure_arc(api_version) + # Generate File and Read Challenge Token + uri = URI("http://#{@azure_arc_instance_metadata_endpoint_ip}/metadata/identity/oauth2/token?api-version=#{api_version}&resource=https%3A%2F%2Fvault.azure.net") + req = Net::HTTP::Get.new(uri.request_uri) + req['Metadata'] = 'true' + res = Net::HTTP.start(uri.hostname, uri.port) do |http| + http.request(req) + end + + # 403 is expected here. we do not provide ANY key + raise res.body unless res.is_a?(Net::HTTPUnauthorized) + raise 'Response header Www-Authenticate is missing' unless res['Www-Authenticate'] + + challenge_token_file_path = res['Www-Authenticate'].sub(%r{\s*Basic\s+realm=}, '') + challenge_token = File.read(challenge_token_file_path) + + # Get Access Token using challenge token + internal_get_access_token(api_version, @azure_arc_instance_metadata_endpoint_ip, { 'Authorization' => "Basic #{challenge_token}" }) + end + def self.get_access_token(api_version) - uri = URI("http://169.254.169.254/metadata/identity/oauth2/token?api-version=#{api_version}&resource=https%3A%2F%2Fvault.azure.net") + internal_get_access_token(api_version, '169.254.169.254') + end + + def self.internal_get_access_token(api_version, instance_metadata_service_endpoint = '169.254.169.254', extra_http_headers_hash = {}) + uri = URI("http://#{instance_metadata_service_endpoint}/metadata/identity/oauth2/token?api-version=#{api_version}&resource=https%3A%2F%2Fvault.azure.net") req = Net::HTTP::Get.new(uri.request_uri) req['Metadata'] = 'true' + extra_http_headers_hash.each do |key, value| + req[key] = value + end res = Net::HTTP.start(uri.hostname, uri.port) do |http| http.request(req) end diff --git a/spec/functions/azure_key_vault_secret_spec.rb b/spec/functions/azure_key_vault_secret_spec.rb index e50e9e4..72c045b 100644 --- a/spec/functions/azure_key_vault_secret_spec.rb +++ b/spec/functions/azure_key_vault_secret_spec.rb @@ -23,6 +23,14 @@ is_expected.to run.with_params.and_raise_error(ArgumentError, %r{expects between 3 and 4 arguments}i) end + it "errors when missing both 'metadata_api_version' and 'service_principal_credentials'" do + bad_hash = api_versions_hash + bad_hash.delete('metadata_api_version') + is_expected.to run.with_params( + vault_name, secret_name, bad_hash + ).and_raise_error(%r{hash must contain at least one of metadata_api_version or service_principal_credentials}) + end + it "errors when using both 'metadata_api_version' and 'service_principal_credentials'" do bad_hash = { 'metadata_api_version' => 'test', @@ -37,14 +45,6 @@ vault_name, secret_name, bad_hash ).and_raise_error(%r{metadata_api_version and service_principal_credentials cannot be used together}) end - - it "errors when missing both 'metadata_api_version' and 'service_principal_credentials'" do - bad_hash = api_versions_hash - bad_hash.delete('metadata_api_version') - is_expected.to run.with_params( - vault_name, secret_name, bad_hash - ).and_raise_error(%r{hash must contain at least one of metadata_api_version or service_principal_credentials}) - end end context 'when getting the latest version of a secret' do diff --git a/spec/functions/tragic_code/azure_spec.rb b/spec/functions/tragic_code/azure_spec.rb index 453ae21..e269c5c 100644 --- a/spec/functions/tragic_code/azure_spec.rb +++ b/spec/functions/tragic_code/azure_spec.rb @@ -25,6 +25,50 @@ end end + context '.get_access_token_azure_arc' do + it 'returns a bearer token' do + File.stub(:read).and_return('magical-token-from-file') + + stub_request(:get, %r{127.0.0.1}) + .to_return( + body: '{"access_token": "token"}', + status: 401, + headers: { 'Www-Authenticate' => 'Basic realm=C:\\ProgramData\\AzureConnectedMachineAgent\\Tokens\\f1da0584-97f4-42fd-a671-879ad3de86fa.key' }, + ) + + stub_request(:get, %r{127.0.0.1}) + .with(headers: { 'Authorization' => 'Basic magical-token-from-file' }) + .to_return(body: '{"access_token": "token"}', status: 200) + + expect(described_class.get_access_token_azure_arc('api')).to eq('token') + end + + it 'throws error with response body when response is not 401 (unauthorized) when attempting to generate secret file' do + stub_request(:get, %r{127.0.0.1}) + .to_return(body: 'some_error', status: 200) + expect { described_class.get_access_token_azure_arc('api') }.to raise_error('some_error') + end + + it 'throws error when the 401 (unauthorized) response is missing the Www-Authenticate header' do + stub_request(:get, %r{127.0.0.1}) + .to_return(body: 'some_error', status: 401) + expect { described_class.get_access_token_azure_arc('api') }.to raise_error('Response header Www-Authenticate is missing') + end + + it 'throws error with response body when response is not 2xx when getting the auth token' do + File.stub(:read).and_return('magical-token-from-file') + # rubocop:disable Layout/LineLength + stub_request(:get, %r{127.0.0.1}) + .to_return(body: '{"access_token": "token"}', status: 401, headers: { 'Www-Authenticate' => 'Basic realm=C:\\ProgramData\\AzureConnectedMachineAgent\\Tokens\\f1da0584-97f4-42fd-a671-879ad3de86fa.key' }) + # rubocop:enable Layout/LineLength + stub_request(:get, %r{127.0.0.1}) + .with(headers: { 'Authorization' => 'Basic magical-token-from-file' }) + .to_return(body: 'some_error', status: 400) + + expect { described_class.get_access_token_azure_arc('api') }.to raise_error('some_error') + end + end + context '.get_access_token_service_principal' do let(:credentials) { { 'client_id' => '', 'tenant_id' => '', 'client_secret' => '' } }