Add new parameters for interactive login, new params to export empty fields with reandmes adjusted.

Author: dotnetprog

Guide-XrmToolBox.md

@@ -9,10 +9,15 @@ This tool was created to support more data types that the official tool does.
 ✔️ Schema definition file for export/import 
 
 ### Upcoming features 🔜
-🔜 Configuration Data Importation \
-🔜 Configuration Data Exportation  
+🔜 Supports for multiselect optionset \
+🔜 ~~Configuration Data Importation~~ <span style="color:red">This is no longer planned.</span>\
+🔜 ~~Configuration Data Exportation~~  <span style="color:red">This is no longer planned.</span>
+
+> [!IMPORTANT]  
+> Data import/export features will only be available through the cli tool. 
+> Since the cli tool is made with .Net Core and XTB Plugins is in .Net Framework, It's really hard to keep a sharable codebase and retricts the usage of some modern libraries.
+> CLI tool documentation [here](https://github.com/dotnetprog/dataverse-configuration-migration-tool)
 
->**Note**:  Those features are available through the cli tool. More Info [here](https://github.com/dotnetprog/dataverse-configuration-migration-too)
 
 ##  What's a schema definition file exactly 🤔❓
 

README.md

@@ -5,7 +5,8 @@ This repository contains a custom .NET CLI tool designed to export and import co
 ### Download latest release
 Get latest version of the tool built on this [release](https://github.com/dotnetprog/dataverse-configuration-migration-tool/releases/latest)
 > [!NOTE]  
-> If you want to use the built version of the tool , `appsettings.Production.json` will need to be setup manually with your azure service principal credentials.
+> Interactive Login is now available with the CLI tool. The use of service principal is no more mandatory but still recommanded for automation scenarios. \
+> To automate the use of the tool, let's say within a pipeline (e.g Github Action,Azure Devops Pipeline),`appsettings.Production.json` will need to be setup manually with your azure service principal credentials.
 > [Quick Guide](https://recursion.no/blogs/dataverse-setup-service-principal-access-for-environment/) to create an azure service principal
 ## Why ❓
 
@@ -17,7 +18,8 @@ This new tool enables you to:
 - runs on windows and **linux**
 
 ## ⭐Features⭐
-
+🆕 :heavy_check_mark: **Interactive Login** is now available by using `--il` parameter. Service Principal is no more required.🆕 \
+🆕 :heavy_check_mark: Data export now supports a new parameter `--AllowEmptyFields` or `--enable-empty-fields` to export empty fields. *Useful to clear values on target environments* 🆕 \
 :heavy_check_mark: Import configuration data into Dataverse \
 :heavy_check_mark: Export configuration data from Dataverse \
 :heavy_check_mark: Schema validation and rule-based checks \
@@ -36,6 +38,13 @@ This new tool enables you to:
 - Image
 - File
 
+### Upcoming features 🔜
+🔜 Supports for multiselect optionset 
+
+
+> [!IMPORTANT]  
+> Data import/export features for XrmToolBox is no longer planned.
+> Since the cli tool is made with .Net Core and XTB Plugins is in .Net Framework, It's really hard to keep a sharable codebase and retricts the usage of some modern libraries.
 
 
 
@@ -59,15 +68,20 @@ This new tool enables you to:
 
 ### Usage
 
+Before running the tool, set your dataverse variables securely using [dotnet user-secrets](https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets):
 
-Before running the tool, set your `clientId`, `clientSecret` and `url` securely using [dotnet user-secrets](https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets):
-
+#### For Service principal
 ```powershell
 cd src/Dataverse.ConfigurationMigrationTool/Dataverse.ConfigurationMigrationTool.Console
 dotnet user-secrets set "Dataverse:ClientId" "<your-client-id>"
 dotnet user-secrets set "Dataverse:ClientSecret" "<your-client-secret>"
 dotnet user-secrets set "Dataverse:Url" "<your-env-url>"
 ```
+#### For Interactive Login
+```powershell
+cd src/Dataverse.ConfigurationMigrationTool/Dataverse.ConfigurationMigrationTool.Console
+dotnet user-secrets set "Dataverse:Url" "<your-env-url>"
+```
 
 Run the CLI tool with the required arguments (no need to pass clientId or clientSecret on the command line):
 #### example
@@ -77,13 +91,51 @@ dotnet run --environment DOTNET_ENVIRONMENT=Development --project Dataverse.Conf
 
 #### 💻 Command Line Arguments 💻
 
+
 Verb: `import`
-- `--data` : Path to the data xml file, you can use `export-data` command or the microsoft tool (see last section).
-- `--schema` : Path to the schema XML file
+- `--data` : Path to the data xml file, you can use `export-data` command or the microsoft tool (see last section).(Absolute or relative path)
+- `--schema` : Path to the schema XML file (Absolute or relative path)
+- 🆕`--il` or `-il` : **Optional** flag to use interactive login (user will be prompted to log in) instead of service principal. Useful for local testing/execution.
+
+> [!NOTE]  
+> using interactive login to import data will request user to login twice as it instantiates two different connections to dataverse for performance purposes.
+
+**Example**
+
+using dotnet run from the solution folder (.sln)
+```powershell
+dotnet run --environment DOTNET_ENVIRONMENT=Development --project Dataverse.ConfigurationMigrationTool.Console -- import --data "C:\temp\data.xml" --schema "C:\temp\data_schema.xml" --il
+```
+using directly the tool executable
+```powershell
+cd path/to/tool__executable_folder
+Dataverse.ConfigurationMigrationTool.Console.exe import --data "C:\temp\data.xml" --schema "C:\temp\data_schema.xml" --il
+```
 
 Verb: `export-data`
-- `--schema` : Path to the schema XML file
-- `--output` : output file path to save the exported data. This file can be used for the `import` command.
+- `--schema` : Path to the schema XML file (Absolute or relative path)
+- `--output` : output file path to save the exported data. This file can be used for the `import` command. (Absolute or relative path)
+- 🆕`--AllowEmptyFields` or `--enable-empty-fields` : **Optional** flag to include fields with empty values in the export. Useful for clearing values in the target environment.
+- 🆕`--il` or `-il` : **Optional** flag to use interactive login (user will be prompted to log in) instead of service principal. Useful for local testing/execution.
+
+**Example**
+
+using dotnet run from the solution folder (.sln)
+```powershell
+dotnet run --environment DOTNET_ENVIRONMENT=Development --project Dataverse.ConfigurationMigrationTool.Console -- export-data --schema "C:\temp\data_schema.xml" --output "C:\temp\exported_data.xml" --enable-empty-fields --il
+```
+using directly the tool executable
+```powershell
+cd path/to/tool__executable_folder
+Dataverse.ConfigurationMigrationTool.Console.exe export-data --schema "C:\temp\data_schema.xml" --output "C:\temp\exported_data.xml" --enable-empty-fields --il
+```
+
+> [!TIP]
+> To use service principal credentials, the interactive login flag `--il` or `-il` must be omitted.
+> If you use the executable directly, you must edit the `appsettings.Production.json` file with your service principal credentials
+> If you use `dotnet run`, you can set the secrets using `dotnet user-secrets` as shown above or edit the `appsettings.Development.json` file.
+> you can also set environment variables `Dataverse__ClientId`, `Dataverse__ClientSecret` and `Dataverse__Url` to override the settings in the json files.
+> you can also use these commandline arguments `--Dataverse:ClientId`, `--Dataverse:ClientSecret` and `--Dataverse:Url` to override the settings in the json files.
 
 ## 🤝 Contributing 🤝
 

src/Dataverse.ConfigurationMigrationTool/Console.Tests/ConfigurationProviders/CustomCommandLineConfigurationExtensionsTests.cs

@@ -0,0 +1,26 @@
+using Dataverse.ConfigurationMigrationTool.Console.ConfigurationProviders;
+using Microsoft.Extensions.Configuration;
+using NSubstitute;
+
+namespace Dataverse.ConfigurationMigrationTool.Console.Tests.ConfigurationProviders;
+public class CustomCommandLineConfigurationExtensionsTests
+{
+    private readonly IConfigurationBuilder _builder = Substitute.For<IConfigurationBuilder>();
+    public CustomCommandLineConfigurationExtensionsTests()
+    {
+        _builder.Add(Arg.Any<IConfigurationSource>()).Returns(_builder);
+    }
+    [Fact]
+    public void GivenAConfigurationBuilder_WhenItAddsCustomCommandline_ThenItAddsTheCustomCommandLineConfigurationProvider()
+    {
+        // Arrange
+        var args = new[] { "--key=value" };
+        // Act
+        _builder.AddCustomCommandline(args);
+        // Assert
+        _builder.Received(1).Add(Arg.Is<CustomCommandLineConfigurationSource>(source =>
+            source.Args.SequenceEqual(args) &&
+            source.SwitchMappings == null &&
+            source.FlagMappings == null));
+    }
+}

src/Dataverse.ConfigurationMigrationTool/Console.Tests/ConfigurationProviders/CustomCommandLineConfigurationProviderTests.cs

@@ -0,0 +1,205 @@
+using Dataverse.ConfigurationMigrationTool.Console.ConfigurationProviders;
+using Shouldly;
+
+namespace Dataverse.ConfigurationMigrationTool.Console.Tests.ConfigurationProviders;
+
+public class CustomCommandLineConfigurationProviderTests
+{
+    [Fact]
+    public void Given_ArgsWithDoubleDash_When_Load_Then_ParsesKeyValueCorrectly()
+    {
+        // Arrange
+        var args = new[] { "--key=value" };
+        var provider = BuildProvider(args);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "key").ShouldBe("value");
+    }
+    [Fact]
+    public void Given_ArgsWithDoubleDashWithNoSeparator_When_Load_Then_ParsesKeyValueCorrectly()
+    {
+        // Arrange
+        var args = new[] { "--key", "value" };
+        var provider = BuildProvider(args);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "key").ShouldBe("value");
+    }
+    [Fact]
+    public void Given_ArgsWithSingleDashWithNoSeparator_When_Load_Then_ParsesKeyValueCorrectly()
+    {
+        // Arrange
+        var args = new[] { "-key", "value" };
+        var switchMappings = new Dictionary<string, string> { { "-key", "key" } };
+        var provider = BuildProvider(args, switchMappings);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "key").ShouldBe("value");
+    }
+    [Fact]
+    public void Given_ArgsWithSingleDash_When_Load_Then_ParsesKeyValueCorrectly()
+    {
+        // Arrange
+        var args = new[] { "-key=value" };
+        var switchMappings = new Dictionary<string, string> { { "-key", "key" } };
+        var provider = BuildProvider(args, switchMappings);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "key").ShouldBe("value");
+    }
+
+    [Fact]
+    public void Given_ArgsWithSlash_When_Load_Then_ParsesKeyValueCorrectly()
+    {
+        // Arrange
+        var args = new[] { "/key=value" };
+        var provider = BuildProvider(args);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "key").ShouldBe("value");
+    }
+
+    [Fact]
+    public void Given_ArgsWithSwitchMappings_When_Load_Then_MapsSwitchToCustomKey()
+    {
+        // Arrange
+        var args = new[] { "--customSwitch=value" };
+        var switchMappings = new Dictionary<string, string> { { "--customSwitch", "mappedKey" } };
+        var provider = BuildProvider(args, switchMappings);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "mappedKey").ShouldBe("value");
+    }
+
+    [Fact]
+    public void Given_ArgsWithFlagMappings_When_Load_Then_SetsFlagValueToTrue()
+    {
+        // Arrange
+        var args = new[] { "--flag" };
+        var flagMappings = new[] { "--flag" };
+        var provider = BuildProvider(args, null, flagMappings);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "flag").ShouldBe("True");
+    }
+
+    [Fact]
+    public void Given_ArgsWithDuplicateKeys_When_Load_Then_LastValueWins()
+    {
+        // Arrange
+        var args = new[] { "--key=first", "--key=second" };
+        var provider = BuildProvider(args);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "key").ShouldBe("second");
+    }
+
+    [Fact]
+    public void Given_ArgsWithInvalidFormat_When_Load_Then_IgnoresInvalidArgs()
+    {
+        // Arrange
+        var args = new[] { "invalidArg", "--valid=value" };
+        var provider = BuildProvider(args);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "invalidArg").ShouldBeNull();
+        GetValue(provider, "valid").ShouldBe("value");
+    }
+
+    [Fact]
+    public void Given_ShortDashWithoutMapping_When_Load_Then_IgnoresArg()
+    {
+        // Arrange
+        var args = new[] { "-short" };
+        var provider = BuildProvider(args);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "short").ShouldBeNull();
+    }
+
+    [Fact]
+    public void Given_ShortDashWithMapping_When_Load_Then_MapsKey()
+    {
+        // Arrange
+        var args = new[] { "-s=value" };
+        var switchMappings = new Dictionary<string, string> { { "-s", "shortKey" } };
+        var provider = BuildProvider(args, switchMappings);
+        // Act
+        provider.Load();
+        // Assert
+        GetValue(provider, "shortKey").ShouldBe("value");
+    }
+
+    [Fact]
+    public void Given_ShortDashWithNoMappingAndEquals_When_Load_Then_ThrowsFormatException()
+    {
+        // Arrange
+        var args = new[] { "-s=value" };
+        var provider = BuildProvider(args);
+        // Act
+        var act = () => provider.Load();
+        // Assert
+        act.ShouldThrow<FormatException>();
+    }
+
+    [Fact]
+    public void Given_SwitchMappingsWithInvalidKey_When_Construct_Then_ThrowsArgumentException()
+    {
+        // Arrange
+        var switchMappings = new Dictionary<string, string> { { "invalidKey", "mappedKey" } };
+        var args = new[] { "--key=value" };
+        // Act
+        var act = () => BuildProvider(args, switchMappings);
+        // Assert
+        act.ShouldThrow<ArgumentException>();
+    }
+
+    [Fact]
+    public void Given_SwitchMappingsWithDuplicateKeys_When_Construct_Then_ThrowsArgumentException()
+    {
+        // Arrange
+        var switchMappings = new Dictionary<string, string>
+        {
+            { "--dup", "key1" },
+            { "--DUP", "key2" }
+        };
+        var args = new[] { "--dup=value" };
+        // Act
+        var act = () => BuildProvider(args, switchMappings);
+        // Assert
+        act.ShouldThrow<ArgumentException>();
+    }
+
+    [Fact]
+    public void Given_ArgsIsNull_When_Construct_Then_ThrowsArgumentNullException()
+    {
+        // Act
+        var act = () => BuildProvider(null!);
+        //Assert
+        act.ShouldThrow<ArgumentNullException>();
+    }
+    private static CustomCommandLineConfigurationProvider BuildProvider(IEnumerable<string> args, IDictionary<string, string>? switchMappings = null, IEnumerable<string>? flagMappings = null)
+    {
+        return new CustomCommandLineConfigurationProvider(args, switchMappings, flagMappings);
+    }
+
+    // Helper to get value from provider's TryGet method using reflection
+    private static string? GetValue(CustomCommandLineConfigurationProvider provider, string key)
+    {
+        if (provider.TryGet(key, out var value))
+        {
+            return value;
+        }
+        return null;
+    }
+}

src/Dataverse.ConfigurationMigrationTool/Console.Tests/ConfigurationProviders/CustomCommandLineConfigurationSourceTests.cs

@@ -0,0 +1,25 @@
+using Dataverse.ConfigurationMigrationTool.Console.ConfigurationProviders;
+using Shouldly;
+
+namespace Dataverse.ConfigurationMigrationTool.Console.Tests.ConfigurationProviders;
+public class CustomCommandLineConfigurationSourceTests
+{
+    private CustomCommandLineConfigurationSource ConfigSource { get; }
+    public CustomCommandLineConfigurationSourceTests()
+    {
+        ConfigSource = new CustomCommandLineConfigurationSource()
+        {
+            Args = [],
+        };
+    }
+
+    [Fact]
+    public void GivenACommandLineSource_WhenItBuildsAConfigProvider_ThenACommandLineConfigurationProviderIsReturned()
+    {
+        // Act
+        var provider = ConfigSource.Build(null);
+        // Assert
+        provider.ShouldBeOfType<CustomCommandLineConfigurationProvider>();
+    }
+
+}