SystemsManagerConfigurationSource overrides previously configured values

[JobaDiniz]

Describe the bug

I got a weird behavior where previously set values on the configuration is set to empty/null after I call builder.Configuration.AddSystemsManager(ssmConfiguration.Prefix, TimeSpan.FromMinutes(30));.

After I call the above method, my previously set values are set to null: builder.Configuration["ConnectionStrings:temporal"] = "some value".

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Calling builder.Configuration.AddSystemsManager(...); should not overwrite previously configured values.

Current Behavior

Calling builder.Configuration.AddSystemsManager(...); overwrites previously configured values.

Reproduction Steps

Code:

ConfigurationExtensions

public static IConfigurationBuilder MapDockerEnvironmentVariables(this IConfigurationManager builder)
{
    var temporalAddress = Environment.GetEnvironmentVariable("TEMPORAL_ADDRESS");
    if (!string.IsNullOrEmpty(temporalAddress))
        builder["ConnectionStrings:temporal"] = temporalAddress;

Program

var builder = Host.CreateApplicationBuilder(args);
builder.Configuration.MapDockerEnvironmentVariables();

 var ssmConfiguration = new AwsSsmConfiguration();
 builder.Configuration.GetSection("aws:ssm").Bind(ssmConfiguration);
 ssmConfiguration.EnsureValid();

 //NOTE: after the below line, the "ConnectionStrings:temporal" is set to null.
 builder.Configuration.AddSystemsManager(ssmConfiguration.Prefix, TimeSpan.FromMinutes(30));

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK 3.7.400.70

Targeted .NET Platform

dotnet 8

Operating System and version

windows 11, amazonlinux

[ashishdhingra]

@JobaDiniz Good morning. Somehow, using the sample application from https://github.com/aws/aws-dotnet-extensions-configuration/tree/master/samples doesn't reproduce the issue. Below is the tweaked code for your use case:
ExtensionMethods.cs

namespace Samples
{
    public static class ExtensionMethods
    {
        public static IConfigurationManager MapDockerEnvironmentVariables(this IConfigurationManager builder)
        {
            var temporalAddress = Environment.GetEnvironmentVariable("TEMPORAL_ADDRESS");
            if (!string.IsNullOrEmpty(temporalAddress))
                builder["ConnectionStrings:temporal"] = temporalAddress;

            return builder;
        }
    }
}

Program.cs

using Amazon;
using Amazon.Extensions.NETCore.Setup;
using Amazon.SimpleSystemsManagement;
using Amazon.SimpleSystemsManagement.Model;
using Samples;

//populates some sample data to be used by this example project
await PopulateSampleDataForThisProject().ConfigureAwait(false);

var builder = WebApplication.CreateBuilder(args);
builder.Configuration.MapDockerEnvironmentVariables();
builder.Configuration.AddSystemsManager($"/dotnet-aws-samples/systems-manager-sample/");

builder.Services.Configure<Settings>(builder.Configuration.GetSection($"common:settings"));

// Add services to the container.
builder.Services.AddControllers();

var app = builder.Build();

app.UseHttpsRedirection();
app.UseAuthorization();
app.MapControllers();

app.Run();


static async Task PopulateSampleDataForThisProject()
{
    var awsOptions = new AWSOptions { Region = RegionEndpoint.USEast1 };

    var root = $"/dotnet-aws-samples/systems-manager-sample/common";
    var parameters = new[]
    {
        new {Name = "StringValue", Value = "string-value"},
        new {Name = "IntegerValue", Value = "10"},
        new {Name = "DateTimeValue", Value = "2000-01-01"},
        new {Name = "BooleanValue", Value = "True"},
        new {Name = "TimeSpanValue", Value = "00:05:00"},
    };

    using (var client = awsOptions.CreateServiceClient<IAmazonSimpleSystemsManagement>())
    {
        var result = await client.GetParametersByPathAsync(new GetParametersByPathRequest { Path = root, Recursive = true }).ConfigureAwait(false);
        if (result.Parameters.Count == parameters.Length) return;

        foreach (var parameter in parameters)
        {
            var name = $"{root}/settings/{parameter.Name}";
            await client.PutParameterAsync(new PutParameterRequest { Name = name, Value = parameter.Value, Type = ParameterType.String, Overwrite = true }).ConfigureAwait(false);
        }
    }
}

Controllers\ValuesController.cs

using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Options;

namespace Samples.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class ValuesController : ControllerBase
    {
        private readonly IOptions<Settings> _settings;
        private string? _connectionString;

        public ValuesController(IOptions<Settings> settings, IConfiguration configuration)
        {
            _settings = settings;
            _connectionString = configuration.GetSection("ConnectionStrings:temporal").Value;
        }

        // GET api/values
        [HttpGet]
        public ActionResult Get()
        {
            return Ok(new
            {
                Settings = _settings.Value,
                ConnectionString = _connectionString
            });
        }
    }
}

Running the application (debugged via Visual Studio and set TEMPORAL_ADDRESS=testtemporaladdress environment variable) and invoking GET request for endpoint http://localhost:5766/api/values returns the below response:

{"settings":{"stringValue":"string-value","integerValue":10,"dateTimeValue":"2000-01-01T00:00:00","booleanValue":true,"timeSpanValue":"00:05:00"},"connectionString":"testtemporaladdress"}

Also, this package just adds the Systems Manager configuration source without clearing any existing sources. Then, it's up to .NET configuration system to resolve a section. The order in which a section is resolved is dependent on the order in which various configuration sources are added and those added later might reset the values for similar path, if already present.

Please check:

• Environment variable TEMPORAL_ADDRESS is set at your end.
• You are using below code:

  var ssmConfiguration = new AwsSsmConfiguration();
   builder.Configuration.GetSection("aws:ssm").Bind(ssmConfiguration);
   ssmConfiguration.EnsureValid();

  I'm assuming that you are reading SSM parameter path prefix stored somewhere in your system that is retrieved via your custom AwsSsmConfiguration class.
  Does that SSM parameter path somehow also has ConnectionStrings/temporal value set to null or empty value? If yes, then it might clear out previously set builder["ConnectionStrings:temporal"].

Thanks,
Ashish

[github-actions]

This issue has not received a response in 5 days. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

[JobaDiniz]

I can't get the sample to work even thou my code works fine. When adding the secrets, the code throws the following error:

Amazon.SimpleSystemsManagement.Model.ParameterNotFoundException: 'An error occurred (ParameterNotFound) when referencing Secrets Manager: Secret aws/reference/secretsmanager/netflownull not found.'

Weird error string netflownull (null??).

builder.Configuration.AddSystemsManager("/netflow", builder.Configuration.GetAWSOptions("AWS"), TimeSpan.FromMinutes(30));
builder.Configuration.AddSystemsManager("/aws/reference/secretsmanager/netflow", builder.Configuration.GetAWSOptions("AWS"), TimeSpan.FromMinutes(30));

This code is working fine in my deployment.

---

Without adding the secrets, indeed the previously values are not overwritten.
However as stated above, I couldn't test with secrets as it kept throwing error. Even thou the secret exists and the user I'm invoking the code is a poweruser that has access to everything.