Release 3.0 (#23)

Reconcile 3.0.0 to main

* Added additional store types, pfx discovery and inventory complete.
* jks stores discovery and inventory complete
* PKCS12 store discovery and inventory completed.
* added add, remove, and create functionality to JKS store.  Added store type classes.
* Completed add/remove/create for PFX and PKCS12
* Added capability to PEM cert stores for creating an empty store location.
* Update integration-manifest.json removing storepathtype and storepathvalue to prevent errors in KFUtil when creating store types.

Author: fiddlermikey

CHANGELOG.md

@@ -1,3 +1,13 @@
+## 3.0.0
+
+* Added support for JKS, PKCS12 and PFX file stores in the Hashicorp Vault Key-Value secrets engine.
+* Added PAM support for server credentials.
+
+* **Breaking Changes**
+    * The server url and Vault Token have been moved to the server username and server password fields of server credentials, respectively.
+    * The HCVKV store type for PEM files has been renamed to HCVKVPEM
+    
+
 ## 2.0.0
 
 * Added support for storing certs in sub-paths

README.md

@@ -14,16 +14,13 @@ The Universal Orchestrator is part of the Keyfactor software distribution and is
 The Universal Orchestrator is the successor to the Windows Orchestrator. This Orchestrator Extension plugin only works with the Universal Orchestrator and does not work with the Windows Orchestrator.
 
 
-
-
 ## Support for Orchestrator Extension for Hashicorp Vault
 
 Orchestrator Extension for Hashicorp Vault is supported by Keyfactor for Keyfactor customers. If you have a support issue, please open a support ticket with your Keyfactor representative.
 
 ###### To report a problem or suggest a new feature, use the **[Issues](../../issues)** tab. If you want to contribute actual bug fixes or proposed enhancements, use the **[Pull requests](../../pulls)** tab.
 
 
-
 ---
 
 
@@ -62,17 +59,39 @@ This integration supports 3 Hashicorp Secrets Engines; PKI, Key-Value store, and
 
 ### The Key-Value secrets engine
 
-The Following operations are supported by this integration **only** for the Key-Value secrets engine.
+For the Key-Value secrets engine, we have 4 store types that can be used.  
 
-1. Discovery - Discover all sub-paths containing certificate.
-1. Inventory - Return all certificates stored in a path.
+- *HCVKVJKS* - For JKS certificate files, treats each file as it's own store.
+- *HCVKVPFX* - For PFX certificate files, treats each file as it's own store.
+- *HCVKVP12* - For PKCS12 certificate files, treats each file as it's own store.
+- *HCVKVPEM* - For PEM encoded certificates, treats each _path_ as it's own store.  Each certificate exists in a sub-path from the store path.
+
+The following operations are supported by this integration for all of the Key-Value secrets engine types:
+
+1. Discovery - Discovery all file repositories for the type
+1. Inventory - Inventory all certificates in the path
 1. Management (Add) - Add a certificate to a defined certificate store.
 1. Management (Remove) - Remove a certificate from a defined certificate store.
+1. Create - Create a new, empty certificate store at the path defined in Store Path.
+
+
+Excluding *HCVKVPEM*, the discovery process requires that:
+1. The entry for the certificate contain the base64 encoded certificate file.
+1. The name (key) for the entry ends with the suffix corresponding to the certificate store type:
+ 1. *HCVKVJKS* - `*_jks` 
+ 1. *HCVKVPFX* - `*_pfx`
+ 1. *HCVKVP12* - `*_p12`
+ 1. *HCVKVPEM* - `certificate`
+1. For all except *HCVKVPEM*, there be an entry named `passphrase` that contains the password for the store.
+1. For *HCVKVPEM*, there be an entry named `private_key` containing the private key portion of the key-pair.
+
+**Note**: Key/Value secrets that do not include the expected keys will be ignored during inventory scans.
 
 ### The Hashicorp PKI and Keyfactor Plugin secrets engines
 
-Both the Hashicorp PKI and Keyfactor plugin are designed to allow managing certifications directly on the Hashicorp Vault instance.
-This integration does support the following in order to view your certificates from the platform:
+Both the Hashicorp PKI and Keyfactor Secrets Engine plugins are designed to allow managing certifications directly on the Hashicorp Vault instance.
+The store type for the PKI and/or the Keyfactor secrets engine is the same; `HCVPKI`.
+This integration supports the following in order to view your certificates from the platform:
 
 1. Inventory - Return all certificates stored in a path.
 
@@ -90,13 +109,6 @@ This integration was built on the .NET Core 3.1 target framework and are compati
 
 1. It is not necessary to use the Vault root token when creating a Certificate Store for HashicorpVault.  We recommend creating a token with policies that reflect the minimum permissions necessary to perform the intended operations.
 
-1. For the Key-Value secrets engine, the certificates are stored as an entry with these fields.  
-
-- `certificate` - The PEM formatted certificate and intermediate CA chain (if selected)
-- `private_key` - The certificate private key
-
-**Note**: Key/Value secrets that do not include the keys `certificate` and `private_key` will be ignored during inventory scans. 
-
 ## Extension Configuration
 
 ### On the Orchestrator Agent Machine
@@ -111,7 +123,7 @@ This integration was built on the .NET Core 3.1 target framework and are compati
 
 ### In the Keyfactor Platform
 
-#### Add a new Certificate Store Type - **Key-Value Secrets Engine**
+#### Add a new Certificate Store Type - **Hashicorp Vault Key-Value PEM**
 
 - Log into Keyfactor as Administrator or a user with permissions to add certificate store types.
 - Click on the gear icon in the top right and then navigate to the "Certificate Store Types"
@@ -120,9 +132,11 @@ This integration was built on the .NET Core 3.1 target framework and are compati
 ![](images/store_type_add.png)
 
 - Set the following values in the "Basic" tab:
-  - **Name:** "Hashicorp Vault Key-Value" (or another preferred name)
-  - **Short Name:** "HCVKV"
+  - **Name:** "Hashicorp Vault Key-Value PEM" (or another preferred name)
+  - **Short Name:** "HCVKVPEM"
   - **Supported Job Types** - "Inventory", "Add", "Remove", "Discovery"
+    - **NOTE** If you are setting up "`HCVKVJKS`, `HCVKVPFX`, or `HCVKVP12` the supported job types will be "Inventory, Discovery".
+  - **Needs Server** - should be checked (true).
 
 ![](images/store-type-kv.PNG)
 
@@ -134,13 +148,14 @@ This integration was built on the .NET Core 3.1 target framework and are compati
 
 - Click the "Custom Fields" tab to add the following custom fields:
   - **MountPoint** - type: *string*
-  - **VaultServerUrl** - type: *string*, *required*
-  - **VaultToken** - type: *secret*, *required*
   - **SubfolderInventory** - type: *bool* (By default, this is set to false. Not a required field)
   - **IncludeCertChain** - type: *bool* (If true, the available intermediate certificates will also be written to Vault during enrollment)
 
 ![](images/store_type_fields.png)
 
+**Note**
+The 3 highlighted fields above will be added automatically by the platform, you will not need to include them when creating the certificate store type.
+
 - Click **Save** to save the new Store Type.
 
 #### Add the Hashicorp Vault Certificate Store - **Key-Value Secrets Engine**
@@ -154,16 +169,19 @@ In Keyfactor Command create a new Certificate Store that resembles the one below
 
 ![](images/cert_store_fields.png)
 
-- **Client Machine** - Enter the URL for the Vault host machine
+- **Client Machine** - Enter an identifier for the client machine.  This could be the Orchestrator host name, or anything else useful.  This value is not used by the extension.
 - **Store Path** - This is the path after mount point where the certs will be stored.
   - example: `kv-v2\kf-secrets\certname` would use the path "\kf-secrets"
 - **Mount Point** - This is the mount point name for the instance of the Key Value secrets engine.  
   - If left blank, will default to "kv-v2".
   - If your organization utilizes Vault enterprise namespaces, you should include the namespace here.
-- **Vault Token** - This is the access token that will be used by the orchestrator for requests to Vault.
-- **Vault Server Url** - the full url and port of the Vault server instance
 - **Subfolder Inventory** - Set to 'True' if it is a requirement to inventory secrets at the subfolder/component level. The default, 'False' will inventory secrets stored at the root of the "Store Path", but will not look at secrets in subfolders. **Note** that there is a limit on the number of certificates that can be in a certificate store. In certain environments enabling Subfolder Inventory may exceed this limit and cause inventory job failure. Inventory job results are currently submitted to the Command platform as a single HTTP POST. There is not a specific limit on the number of certificates in a store, rather the limit is based on the size of the actual certificates and the HTTP POST size limit configured on the Command web server.
 
+#### Set the server name and password
+
+- The server name should be the full URL to the instance of Vault that will be accessible by the orchestrator. (example: `http://127.0.0.1:8200`)
+- The server password should be the Vault token that will be used for authenticating.
+
 ### For the Keyfactor and PKI plugins
 
 - Add a new Certificate Store Type

hashicorp-vault-orchestrator.sln

@@ -14,6 +14,11 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution
 		readme_source.md = readme_source.md
 	EndProjectSection
 EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "readme-src", "readme-src", "{3266961F-0B1D-4DB6-9A58-C0DA958EB832}"
+	ProjectSection(SolutionItems) = preProject
+		readme-pam-support.md = readme-pam-support.md
+	EndProjectSection
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
@@ -28,6 +33,9 @@ Global
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE
 	EndGlobalSection
+	GlobalSection(NestedProjects) = preSolution
+		{3266961F-0B1D-4DB6-9A58-C0DA958EB832} = {83623EBF-AC4C-4158-922D-959AEFC75453}
+	EndGlobalSection
 	GlobalSection(ExtensibilityGlobals) = postSolution
 		SolutionGuid = {F1706189-FE1B-4EE6-9AE1-EF4A777B2F06}
 	EndGlobalSection

hashicorp-vault-orchestrator/CertUtility.cs

@@ -0,0 +1,112 @@
+// Copyright 2023 Keyfactor
+// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+// and limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Security.Cryptography;
+using System.Text;
+using Keyfactor.Logging;
+using Keyfactor.Orchestrators.Extensions;
+using Microsoft.Extensions.Logging;
+using Org.BouncyCastle.OpenSsl;
+using Org.BouncyCastle.Pkcs;
+
+namespace Keyfactor.Extensions.Orchestrator.HashicorpVault
+{
+    public static class CertUtility
+    {
+        public static List<CurrentInventoryItem> CurrentInventoryFromPkcs12(Pkcs12Store store)
+        {
+            var logger = LogHandler.GetClassLogger<HcvKeyValueClient>();
+
+            logger.MethodEntry();
+            var certs = new List<CurrentInventoryItem>();
+
+            try
+            {
+                using (var memoryStream = new MemoryStream())
+                {
+                    using (TextWriter streamWriter = new StreamWriter(memoryStream))
+                    {
+                        logger.LogTrace("Extracting Private Key...");
+                        var pemWriter = new PemWriter(streamWriter);
+                        logger.LogTrace("Created pemWriter...");
+                        var aliases = store.Aliases.Cast<string>().Where(a => store.IsKeyEntry(a));
+                        //logger.LogTrace($"Alias = {alias}");
+                        foreach (var alias in aliases)
+                        {
+                            var certInventoryItem = new CurrentInventoryItem { Alias = alias };
+
+                            var entryCerts = new List<string>();
+                            logger.LogTrace("extracting public key");
+                            var publicKey = store.GetCertificate(alias).Certificate.GetPublicKey();
+                            var privateKeyEntry = store.GetKey(alias);
+                            if (privateKeyEntry != null) certInventoryItem.PrivateKeyEntry = true;
+                            pemWriter.WriteObject(publicKey);
+                            streamWriter.Flush();
+                            var publicKeyString = Encoding.ASCII.GetString(memoryStream.GetBuffer()).Trim()
+                                .Replace("\r", "").Replace("\0", "");
+                            entryCerts.Add(publicKeyString);
+
+                            var pemChain = new List<string>();
+
+                            logger.LogTrace("getting chain certs");
+
+                            var chain = store.GetCertificateChain(alias).ToList();
+
+                            chain.ForEach(c =>
+                            {
+                                var cert = c.Certificate.GetEncoded();
+                                var encoded = Pemify(Convert.ToBase64String(cert));
+                                pemChain.Add(encoded);
+                            });
+
+                            if (chain.Count() > 0)
+                            {
+                                certInventoryItem.UseChainLevel = true;
+                                entryCerts.AddRange(pemChain);
+                            }
+                            certInventoryItem.Certificates = pemChain;
+                            certs.Add(certInventoryItem);
+                        }
+                        memoryStream.Close();
+                        streamWriter.Close();
+                    }
+                    logger.MethodExit();
+                    return certs;
+                }
+            }
+            catch (Exception ex)
+            {
+                logger.LogError("error extracting certs from pkcs12", ex);
+                throw;
+            }
+        }
+
+        public static Func<string, string> Pemify = base64Cert =>
+        {
+            string FormatBase64(string ss) =>
+                ss.Length <= 64 ? ss : ss.Substring(0, 64) + "\n" + FormatBase64(ss.Substring(64));
+
+            string header = "-----BEGIN CERTIFICATE-----\n";
+            string footer = "\n-----END CERTIFICATE-----";
+
+            return header + FormatBase64(base64Cert) + footer;
+        };
+
+        public static string GenerateRandomString(int length)
+        {
+            using (Aes crypto = Aes.Create())
+            {
+                crypto.GenerateKey();
+                return Convert.ToBase64String(crypto.Key).Substring(0, length);
+            }
+        }
+    }
+}

hashicorp-vault-orchestrator/Constants.cs

@@ -1,4 +1,4 @@
-// Copyright 2022 Keyfactor
+// Copyright 2023 Keyfactor
 // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS,
@@ -13,12 +13,28 @@ static class AzureKeyVaultConstants
         public const string PKI_STORE_TYPE = "HCV"; //same for Keyfactor plugin store type
     }
 
-    static class JobTypes
+    static class JobType
     {
         public const string CREATE = "Create";
         public const string DISCOVERY = "Discovery";
         public const string INVENTORY = "Inventory";
         public const string MANAGEMENT = "Management";
         public const string REENROLLMENT = "Enrollment";
     }
+
+    static class StoreType
+    {
+        public const string HCVKVPEM = "HCVKVPEM";
+        public const string KCVKVJKS = "HCVKVJKS";
+        public const string HCVKVPKCS12 = "HCVKVP12";
+        public const string HCVKVPFX = "HCVKVPFX";
+        public const string HCVPKI = "HCVPKI";        
+    }
+
+    static class StoreFileExtensions {
+        public const string HCVKVJKS = "_jks";
+        public const string HCVKVPKCS12 = "_p12";
+        public const string HCVKVPFX = "_pfx";
+        public const string HCVKVPEM = "certificate";
+    }
 }

hashicorp-vault-orchestrator/FileStores/IFileStore.cs

@@ -0,0 +1,20 @@
+// Copyright 2023 Keyfactor
+// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+// and limitations under the License.
+
+using System.Collections.Generic;
+using Keyfactor.Orchestrators.Extensions;
+
+namespace Keyfactor.Extensions.Orchestrator.HashicorpVault
+{
+    public interface IFileStore
+    {
+        string AddCertificate(string alias, string pfxPassword, string entryContents, bool includeChain, string certContent, string passphrase);
+        string RemoveCertificate(string alias, string passphrase, string storeFileContent);
+        byte[] CreateFileStore(string passphrase);
+        IEnumerable<CurrentInventoryItem> GetInventory(Dictionary<string, object> certFields);
+    }
+}