Fixes ab#47259

* Fixed issue with additional_emails field not syncing.
* Added independent logging via NLog.

Author: mkachk

CHANGELOG.md

@@ -1,14 +1,20 @@
+Version 2.0.2
+
+    Fixed issue with additional_emails field not syncing.
+    Added independent logging via NLog.
+
 Version 2.0.1
-- Fixed issue with no input for either custom or manual fields leading to a crash.
-- Fixed issue with data for imported DigiCert fields renamed with a replacement character not syncing back to DigiCert.
-- Fixed possible crash caused by importing DigiCert custom fields with "Anything" data type.
+
+    Fixed issue with no input for either custom or manual fields leading to a crash.
+    Fixed issue with data for imported DigiCert fields renamed with a replacement character not syncing back to DigiCert.
+    Fixed possible crash caused by importing DigiCert custom fields with "Anything" data type.
 
 Version 2.0.0
 
-- Added ability to sync custom fields from Keyfactor to DigiCert.
-- Tool now requires command line argument to specify sync direction: "dctokf" for DigiCert to Keyfactor and "kftodc" for Keyfactor to DigiCert.
-- New DigiCert API Key with restrictions set to "None" in DigiCert config required to perform sync from Keyfactor to Digicert.
+    Added ability to sync custom fields from Keyfactor to DigiCert.
+    Tool now requires command line argument to specify sync direction: "dctokf" for DigiCert to Keyfactor and "kftodc" for Keyfactor to DigiCert.
+    New DigiCert API Key with restrictions set to "None" in DigiCert config required to perform sync from Keyfactor to Digicert.
 
 Version 1.0
 
-- Initial Release
+    Initial Release

README.md

@@ -4,9 +4,7 @@ A tool to automatically synchronize metadata fields and their content from DigiC
 
 #### Integration status: Production - Ready for use in production environments.
 
-## About the Keyfactor API Client
 
-This API client allows for programmatic management of Keyfactor resources.
 
 
 
@@ -19,30 +17,19 @@ Digicert Metadata Sync is open source and there is **no SLA** for this tool/libr
 
 
 
-
-
 ## Overview
-This tool primarily sets up metadata fields in Keyfactor for the custom metadata fields in DigiCert, which are named as such, but can also setup metadata fields in Keyfactor for non-custom fields available in DigiCert and unavailable in Keyfactor by default,   such as the Digicert Cert ID and the Organization contact.  These fields are referred to as manual fields in the context of this tool. After setting up these fields, the tool proceeds to update the contents of these fields. This tool only adds metadata to certificates that have already been imported into Keyfactor. Additionally, this tool requires a properly installed and functioning AnyGateway configured to work with Keyfactor and Digicert. The latest update allows for syncronization of custom field contents from Keyfactor to DigiCert. New fields are created in Keyfactor and DigiCert to accomodate for this.
+This tool primarily sets up metadata fields in Keyfactor for the custom metadata fields in DigiCert, which are named as such, but can also setup metadata fields in Keyfactor for non-custom fields available in DigiCert and unavailable in Keyfactor by default,   such as the Digicert Cert ID and the Organization contact.  These fields are referred to as manual fields in the context of this tool. After setting up these fields, the tool proceeds to update the contents of these fields. This tool only adds metadata to certificates that have already been imported into Keyfactor. Additionally, this tool requires a properly installed and functioning AnyGateway configured to work with Keyfactor and Digicert.
 
 ## Installation and Usage
-The tool comes as a Windows executable. The tool performs synchronization each time its run. For the tool to run automatically, it needs to be added as a scheduled process using Windows. The advised interval for running it is once per week. The files App.config and manualfields.json need to be present in the same directory as the tool for it to run correctly. The specific location from which the tool is ran does not matter, but it needs to have access to both the Keyfactor API endpoint as well as  Digicert, and appropriate permissions for access to the configuration files. 
+The tool comes as a Windows executable. The tool performs synchronization each time its run. For the tool to run automatically, it needs to be added as a scheduled process using Windows. The advised interval for running it is once per week. The files DigicertMetadataSync.dll.config and manualfields.json need to be present in the same directory as the tool for it to run correctly. The specific location from which the tool is ran does not matter, but it needs to have access to both the Keyfactor API endpoint as well as  Digicert, and appropriate permissions for access to the configuration files. 
 An explanation for the settings found in these files is given below. 
 
-## Command Line Arguments
-One of these two arguments needs to be used for the tool to run.
-- <b>"kftodc"</b>  
-Syncronizes the contents of custom fields listed in manualfields.json from Keyfactor to DigiCert. If the fields in manualfields.json do not exist in Keyfactor or DigiCert, they are created first. Example: ```.\DigicertMetadataSync.exe kftodc```
-- <b>"dctokf"</b>  
-Syncronizes the contents of both custom and non-custom fields from DigiCert to Keyfactor. The fields are listed in manualfields.json, and are created if necessary.
-Example: ```.\DigicertMetadataSync.exe dctokf```
 
 ## Settings
 The settings currently present in these files are shown as an example and need to be configured for your specific situation.
-### app.config settings
+### DigicertMetadataSync.dll.config settings
 - <b>DigicertAPIKey</b>  
-Standard DigiCert API access key.
-- <b>DigicertAPIKeyTopPerm</b>  
-DigiCert API access key with restrictions set to "None" - <b>required for sync from Keyfactor to DigiCert</b>. 
+Standard DigiCert API access key 
 - <b>KeyfactorDomainAndUser</b>  
 Same credential as used when logging into Keyfactor Command. A different set of credentials can be used provided they have adequate access permissions.
 - <b>KeyfactorPassword</b>  
@@ -86,4 +73,6 @@ String to be input into Keyfactor as the metadata field hint.
 - <b>KeyfactorAllowAPI</b>  
 Allows API management of this metadata field in Keyfactor. Should be set to true for continuous synchronization with this tool.
 
+### Logging
+Logging functionality can be configured via entering either "Debug" or "Trace" into the value of `<variable name="minLogLevel" value="Debug" />` in NLog.config.
 

digicert-metadata-sync/AddFieldsToKeyfactor.cs

@@ -20,21 +20,19 @@ namespace DigicertMetadataSync;
 
 // This fuction adds the fields to keyfactor.
 // It will only add new fields.
-partial class DigicertSync
+internal partial class DigicertSync
 {
-    public static Tuple<int,List<string>> AddFieldsToKeyfactor(List<KeyfactorMetadataInstanceSendoff> inputlist,
+    public static Tuple<int, List<string>> AddFieldsToKeyfactor(List<KeyfactorMetadataInstanceSendoff> inputlist,
         List<KeyfactorMetadataInstance> existingmetadatalist, bool noexistingfields, string keyfactorusername,
         string keyfactorpassword, string keyfactorapilocation)
     {
         var addfieldstokeyfactorurl = keyfactorapilocation + "MetadataFields";
         var addfieldsclient = new RestClient();
         addfieldsclient.Authenticator = new HttpBasicAuthenticator(keyfactorusername, keyfactorpassword);
-        int totalnumberadded = 0;
-        List<string> newfields = new List<string>();
+        var totalnumberadded = 0;
+        var newfields = new List<string>();
         if (inputlist.Count != 0)
-        {
             foreach (var metadatainstance in inputlist)
-            {
                 if (noexistingfields == false)
                 {
                     var fieldquery = from existingmetadatainstance in existingmetadatalist
@@ -50,7 +48,7 @@ public static Tuple<int,List<string>> AddFieldsToKeyfactor(List<KeyfactorMetadat
                         addfieldrequest.AddHeader("x-keyfactor-requested-with", "APIClient");
                         var serializedfield = JsonConvert.SerializeObject(metadatainstance);
                         addfieldrequest.AddParameter("application/json", serializedfield, ParameterType.RequestBody);
-                        RestResponse metadataresponse = new RestResponse();
+                        var metadataresponse = new RestResponse();
                         try
                         {
                             metadataresponse = addfieldsclient.Post(addfieldrequest);
@@ -68,7 +66,7 @@ public static Tuple<int,List<string>> AddFieldsToKeyfactor(List<KeyfactorMetadat
                         if (fieldquery.FirstOrDefault().DataType != metadatainstance.DataType)
                         {
                             //Throw error if datatype included in keyfactor does not match the digicert one.
-                            NotSupportedException mismatchedtypes = new NotSupportedException();
+                            var mismatchedtypes = new NotSupportedException();
                             throw mismatchedtypes;
                         }
                     }
@@ -82,7 +80,7 @@ public static Tuple<int,List<string>> AddFieldsToKeyfactor(List<KeyfactorMetadat
                     addfieldrequest.AddHeader("x-keyfactor-requested-with", "APIClient");
                     var serializedfield = JsonConvert.SerializeObject(metadatainstance);
                     addfieldrequest.AddParameter("application/json", serializedfield, ParameterType.RequestBody);
-                    RestResponse metadataresponse = new RestResponse();
+                    var metadataresponse = new RestResponse();
                     try
                     {
                         metadataresponse = addfieldsclient.Post(addfieldrequest);
@@ -94,9 +92,8 @@ public static Tuple<int,List<string>> AddFieldsToKeyfactor(List<KeyfactorMetadat
                         throw e;
                     }
                 }
-            }
-        }
-        Tuple<int, List<string>> returnvals = new Tuple<int, List<string>>(totalnumberadded, newfields);
+
+        var returnvals = new Tuple<int, List<string>>(totalnumberadded, newfields);
 
         return returnvals;
     }

digicert-metadata-sync/App.config

@@ -5,10 +5,10 @@
 		<add key="DigicertAPIKeyTopPerm" value="" />
 		<add key="KeyfactorDomainAndUser" value="" />
 		<add key="KeyfactorPassword" value="" />
-		<add key="KeyfactorCertSearchReturnLimit" value="500" /> 
+		<add key="KeyfactorCertSearchReturnLimit" value="" />
 		<add key="KeyfactorAPIEndpoint" value="" />
 		<add key="KeyfactorDigicertIssuedCertQueryTerm" value="DigiCert" />
 		<add key="ImportAllCustomDigicertFields" value="False" />
 		<add key="ReplaceDigicertWhiteSpaceCharacterInName" value="_-_" />
 	</appSettings>
-</configuration>
+</configuration>

digicert-metadata-sync/DigicertMetadataSync.csproj

@@ -10,11 +10,12 @@
 
   <ItemGroup>
     <PackageReference Include="Keyfactor.Logging" Version="1.1.0" />
-    <PackageReference Include="Microsoft.Extensions.Configuration.Binder" Version="6.0.0" />
-    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
-    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
-    <PackageReference Include="RestSharp" Version="108.0.1" />
-    <PackageReference Include="System.Configuration.ConfigurationManager" Version="6.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Configuration.Binder" Version="7.0.4" />
+    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="7.0.0" />
+    <PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
+    <PackageReference Include="NLog" Version="5.2.0" />
+    <PackageReference Include="RestSharp" Version="108.0.3" />
+    <PackageReference Include="System.Configuration.ConfigurationManager" Version="7.0.0" />
 	  <None Update="App.config">
 		  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
 	  </None>
@@ -24,6 +25,9 @@
 	  <None Update="customfields.json">
 		  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
 	  </None>
+	  <None Update="NLog.config">
+	    <CopyToOutputDirectory>Always</CopyToOutputDirectory>
+	  </None>
   </ItemGroup>
 
 </Project>

digicert-metadata-sync/GrabCustomFieldsFromDigiCert.cs

@@ -26,7 +26,6 @@ partial class DigicertSync
 {
     public static List<CustomDigicertMetadataInstance> GrabCustomFieldsFromDigiCert(string apikey)
     {
-        ILogger logger = LogHandler.GetClassLogger<DigicertSync>();
         var digicertclient = new RestClient();
         var customfieldsretrieval = "https://www.digicert.com/services/v2/account/metadata";
         var digicertrequest = new RestRequest(customfieldsretrieval);
@@ -38,7 +37,7 @@ public static List<CustomDigicertMetadataInstance> GrabCustomFieldsFromDigiCert(
         trimmeddigicertresponse = trimmeddigicertresponse.Remove(lengthofresponse - 1, 1);
         var fieldlist = JsonConvert.DeserializeObject<List<CustomDigicertMetadataInstance>>(trimmeddigicertresponse);
         Console.WriteLine("Obtained custom fields from DigiCert.");
-        logger.LogDebug("Obtained custom fields from DigiCert.");
+        _logger.Debug("Obtained custom fields from DigiCert.");
         return fieldlist;
     }
 }

digicert-metadata-sync/Helpers.cs

@@ -12,57 +12,46 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-using System.Reflection;
 using System.Text.RegularExpressions;
+using Newtonsoft.Json.Linq;
 
 namespace DigicertMetadataSync;
 
-partial class DigicertSync
+internal partial class DigicertSync
 {
     public static int TypeMatcher(string digicerttype)
     {
         if (digicerttype.Contains("int") || digicerttype.Contains("Int"))
-        {
             // 2 matches the keyfactor int type metadata field
             return 2;
-        }
-        else
-        {
-            //1 matches the keyfactor string type
-            return 1;
-        }
+        //1 matches the keyfactor string type
+        return 1;
     }
 }
 
-partial class DigicertSync
+internal partial class DigicertSync
 {
     public static Dictionary<string, object> ClassConverter(object obj)
     {
         if (obj != null && obj != "")
         {
-            Dictionary<string, object> resultdict = new Dictionary<string, object>();
+            var resultdict = new Dictionary<string, object>();
             var propertylist = obj.GetType().GetProperties();
 
-            foreach (PropertyInfo prop in propertylist)
+            foreach (var prop in propertylist)
             {
-                string propName = prop.Name;
+                var propName = prop.Name;
                 var val = obj.GetType().GetProperty(propName).GetValue(obj, null);
                 if (val != null)
-                {
                     resultdict.Add(propName, val);
-                }
                 else
-                {
                     resultdict.Add(propName, "");
-                }
             }
 
             return resultdict;
         }
-        else
-        {
-            return null;
-        }
+
+        return null;
     }
 
     public static string ReplaceAllWhiteSpaces(string str, string replacement)
@@ -72,78 +61,47 @@ public static string ReplaceAllWhiteSpaces(string str, string replacement)
 
     public static bool CheckMode(string mode)
     {
-        if ((mode == "kftodc") || (mode == "dctokf")){
-            return true;
-        }
+        if (mode == "kftodc" || mode == "dctokf") return true;
         return false;
     }
 
     private static List<KeyfactorMetadataInstanceSendoff> convertlisttokf(List<ReadInMetadataField> inputlist,
         string replacementcharacter)
     {
-        List<KeyfactorMetadataInstanceSendoff> formattedlist = new List<KeyfactorMetadataInstanceSendoff>();
+        var formattedlist = new List<KeyfactorMetadataInstanceSendoff>();
         if (inputlist.Count != 0)
-        {
-            foreach (ReadInMetadataField input in inputlist)
+            foreach (var input in inputlist)
             {
-                KeyfactorMetadataInstanceSendoff formatinstance = new KeyfactorMetadataInstanceSendoff();
+                var formatinstance = new KeyfactorMetadataInstanceSendoff();
                 if (input.KeyfactorMetadataFieldName == null || input.KeyfactorMetadataFieldName == "")
-                {
                     //If name is emtpy, use autocomplete.
                     formatinstance.Name = ReplaceAllWhiteSpaces(input.DigicertFieldName, replacementcharacter);
-                }
                 else
-                {
                     //Use user input preferred name.
                     formatinstance.Name = input.KeyfactorMetadataFieldName;
-                }
 
                 formatinstance.AllowAPI = Convert.ToBoolean(input.KeyfactorAllowAPI);
                 formatinstance.Hint = input.KeyfactorHint;
                 formatinstance.DataType = TypeMatcher(input.KeyfactorDataType);
                 formatinstance.Description = input.KeyfactorDescription;
                 formattedlist.Add(formatinstance);
             }
-        }
 
         return formattedlist;
     }
 
-    private static Dictionary<string, object> recursiveopener(Dictionary<string, object> dictin, List<string> keys,
-        int limit)
+    public static JObject Flatten(JObject jObject, string parentName = "")
     {
-        // Recursion steps
-        // Each iteration: access dict at keys[0], pop back off keys. K
-        object result = new object();
-        if (dictin == null)
+        var result = new JObject();
+        foreach (var property in jObject.Properties())
         {
-            return null;
-        }
-
-        if (limit != 0)
-        {
-            var location = keys[0];
-            keys.RemoveAt(0);
-            --limit;
-            if (limit == 0)
-            {
-                //At the bottom of keys
-                Dictionary<string, object> newdict = new Dictionary<string, object>();
-                newdict[location] = dictin[location];
-                return newdict;
-            }
+            var propName = string.IsNullOrEmpty(parentName) ? property.Name : $"{parentName}.{property.Name}";
+            if (property.Value is JObject nestedObject)
+                result.Merge(Flatten(nestedObject, propName));
             else
-            {
-                //Keep recursion
-
-                Dictionary<string, object> newdict = ClassConverter(dictin[location]);
-                return recursiveopener(newdict, keys, limit);
-            }
-        }
-        else
-        {
-            //Found nothing.
-            return null;
+                result[propName] = property.Value;
         }
+
+        return result;
     }
 }