ContainerLifetime.Persistent doesn't seem to re-use the container

[brendonparker]

When using ContainerLifetime.Persistent, the container lives after the app host is killed. But then on subsequent runs of the AppHost, the existing container is killed, and a new container is created.

var localStack = builder.AddLocalStack(
    awsConfig: awsConfig,
    configureContainer: container =>
    {
        container.Lifetime = ContainerLifetime.Persistent;
        container.DebugLevel = 1;
        container.LogLevel = LocalStackLogLevel.Info;
    }
);

[slang25]

I found the same thing yesterday, I got Copilot to investigate, and it seems to think the underlying issue is that the environment variables are added to the resource in a non-deterministic way, and they need to be sorted. I cannot vouch for if that is the fix, but it sounds like a possible culprit.

[Blind-Striker]

I'm not sure whether this issue is coming from my integration or from Aspire. All I’m really doing is passing the container lifetime to the resource like this:

var resourceBuilder = builder.AddResource(resource)
    .WithImage(LocalStackContainerImageTags.Image)
    .WithImageRegistry(LocalStackContainerImageTags.Registry)
    .WithImageTag(LocalStackContainerImageTags.Tag)
    .WithHttpEndpoint(targetPort: Constants.DefaultContainerPort, name: LocalStackResource.PrimaryEndpointName)
    .WithLifetime(containerOptions.Lifetime)
    .WithEnvironment("DEBUG", containerOptions.DebugLevel.ToString(CultureInfo.InvariantCulture))
    .WithEnvironment("LS_LOG", containerOptions.LogLevel.ToEnvironmentValue())
    .WithEnvironment("DOCKER_HOST", "unix:///var/run/docker.sock")
    .WithExternalHttpEndpoints()
    .ExcludeFromManifest(); // LocalStack is for local development only

I haven’t tried it yet, but LocalStack also has its own persistence mechanism that can be enabled via an environment variable. Maybe we could take advantage of that in the next version.

https://docs.localstack.cloud/aws/capabilities/state-management/persistence/

[slang25]

I know if you strip it down to just the basics, then it does persist. I switched from my hand rolled version to this library which is when I faced the issue too.

[slang25]

I found the output from Copilot yesterday, take it with a bucket of salt 😅

Code Changes Required

File 1: src/Aspire.Hosting.LocalStack/Container/LocalStackContainerOptions.cs

Add the ContainerName property:

public class LocalStackContainerOptions
{
    // ... existing properties ...

    /// <summary>
    /// Optional fixed container name for the LocalStack container.
    /// When specified, enables container reuse across test runs with ContainerLifetime.Persistent.
    /// Recommended when using DistributedApplicationTestingBuilder with persistent containers.
    /// </summary>
    public string? ContainerName { get; set; }
}

File 2: src/Aspire.Hosting.LocalStack/LocalStackResourceBuilderExtensions.cs

Change the AddLocalStack method

Replace the environment variable section (approximately lines 50-80) with:

public static IResourceBuilder<ILocalStackResource>? AddLocalStack(
    this IDistributedApplicationBuilder builder,
    string name = "localstack",
    ILocalStackOptions? localStackOptions = null,
    IAWSSDKConfig? awsConfig = null,
    Action<LocalStackContainerOptions>? configureContainer = null)
{
    ArgumentNullException.ThrowIfNull(builder);
    ArgumentException.ThrowIfNullOrWhiteSpace(name);

    var options = localStackOptions ?? builder.AddLocalStackOptions();

    if (!options.UseLocalStack)
    {
        return null;
    }

    if (awsConfig is { Region: not null })
    {
        options = options.WithRegion(awsConfig.Region.SystemName);
    }

    var containerOptions = new LocalStackContainerOptions();
    configureContainer?.Invoke(containerOptions);

    var resource = new LocalStackResource(name, options);

    var resourceBuilder = builder.AddResource(resource)
        .WithImage(LocalStackContainerImageTags.Image)
        .WithImageRegistry(LocalStackContainerImageTags.Registry)
        .WithImageTag(LocalStackContainerImageTags.Tag)
        .WithHttpEndpoint(targetPort: Constants.DefaultContainerPort, name: LocalStackResource.PrimaryEndpointName)
        .WithLifetime(containerOptions.Lifetime)
        .WithExternalHttpEndpoints()
        .ExcludeFromManifest();

    // === START CHANGES ===
    
    // Collect all environment variables first for deterministic ordering
    var envVars = new Dictionary<string, string>
    {
        ["DEBUG"] = containerOptions.DebugLevel.ToString(CultureInfo.InvariantCulture),
        ["LS_LOG"] = containerOptions.LogLevel.ToEnvironmentValue(),
        ["DOCKER_HOST"] = "unix:///var/run/docker.sock"
    };

    // Mount Docker socket if enabled
    if (containerOptions.EnableDockerSocket)
    {
        resourceBuilder = resourceBuilder.WithBindMount("/var/run/docker.sock", "/var/run/docker.sock");
    }

    List<string> serviceNames = [];

    // Add service configuration
    if (containerOptions.EagerLoadedServices.Count > 0)
    {
        if (containerOptions.AdditionalEnvironmentVariables.ContainsKey("SERVICES") ||
            containerOptions.AdditionalEnvironmentVariables.ContainsKey("EAGER_SERVICE_LOADING"))
        {
            throw new InvalidOperationException(
                "Cannot set 'SERVICES' or 'EAGER_SERVICE_LOADING' in AdditionalEnvironmentVariables " +
                "when using EagerLoadedServices. These environment variables are managed automatically.");
        }

        envVars["EAGER_SERVICE_LOADING"] = "1";

        serviceNames.AddRange(
            containerOptions.EagerLoadedServices
                .Select(awsService => AwsServiceEndpointMetadata.ByEnum(awsService)?.CliName
                                      ?? throw new InvalidOperationException($"Eager loaded service '{awsService}' is not supported by LocalStack.")));

        // Sort service names for deterministic output
        envVars["SERVICES"] = string.Join(',', serviceNames.OrderBy(s => s));
    }

    // Add additional environment variables
    foreach (var (key, value) in containerOptions.AdditionalEnvironmentVariables)
    {
        envVars[key] = value;
    }

    // Apply environment variables in sorted order for deterministic container spec
    // This ensures persistent containers aren't recreated due to env var order changes
    foreach (var (key, value) in envVars.OrderBy(kvp => kvp.Key))
    {
        resourceBuilder = resourceBuilder.WithEnvironment(key, value);
    }

    // Apply container name if specified (important for persistent containers)
    if (!string.IsNullOrEmpty(containerOptions.ContainerName))
    {
        resourceBuilder = resourceBuilder.WithAnnotation(new ContainerNameAnnotation 
        { 
            Name = containerOptions.ContainerName 
        });
    }

    // === END CHANGES ===

    resourceBuilder.ConfigureHealthCheck(builder, [..serviceNames]);

    var callback = LocalStackConnectionStringAvailableCallback.CreateCallback(builder);
    resourceBuilder.OnConnectionStringAvailable(callback);

    return resourceBuilder;
}

Key Changes Summary

1. Deterministic environment variable ordering: Collect all env vars in a dictionary first, then add them in alphabetical order
2. Sorted service names: Sort the SERVICES list so it's consistent across runs
3. Container name support: Apply ContainerNameAnnotation if ContainerName is specified in options
4. Added comments: Explain why sorting is important for persistent containers

Why These Changes Work

• DCP compares container specs to decide whether to reuse or recreate containers
• Environment variable order is part of the spec comparison
• Non-deterministic ordering (from dictionary iteration, collection order, etc.) causes the spec hash to differ between runs
• Sorting everything ensures the spec is identical between runs, allowing DCP to reuse the container

Testing the Fix

After applying these changes:

var builder = DistributedApplicationTestingBuilder.Create();

var localstack = builder.AddLocalStack(configureContainer: container =>
{
    container.EagerLoadedServices = [AwsService.Sqs, AwsService.Sns];
    container.Lifetime = ContainerLifetime.Persistent;
    container.ContainerName = "test-localstack";
});

// Run this multiple times - container should be reused!

Verify with:

docker ps -a --filter "name=test-localstack" \
  --format "Created: {{.CreatedAt}} | LifecycleKey: {{.Label \"com.microsoft.developer.usvc-dev.lifecycle-key\"}}"

The creation time and lifecycle key should remain the same across multiple test runs.

[slang25]

Again, it sounds convincing, but I have no idea if it works. Might try it at the weekend

[slang25]

tbh, if this is the case it might be better that Aspire is updated to handle this

[brendonparker]

tbh, if this is the case it might be better that Aspire is updated to handle this

Agree

[slang25]

Yeah so that was complete non-sense 😅

[slang25]

It's the .WithBindMount(...), take that out and it works. A workaround is to set EnableDockerSocket to false

[slang25]

Oh wait, the plot thickens, it's not just that, it's also the OnConnectionStringAvailable callback.

[slang25]

Got further again, it's setting the LOCALSTACK_HOST env var, we only need to do this once per container lifetime, however it's reset during the OnConnectionStringAvailable callback, and this invalidates the lifetime.

[nazarii-piontko]

I ran into the same issue today. The actual root cause is the dynamic port assignment for the HTTP endpoint, not the LOCALSTACK_HOST variable itself. The problem is that LOCALSTACK_HOST ends up reflecting (as it should) this random port, which leads to container recreation behavior between runs.

In the LocalStackResourceBuilderExtensions.cs file, inside the AddLocalStack method, the issue comes from this line:

.WithHttpEndpoint(targetPort: Constants.DefaultContainerPort, name: LocalStackResource.PrimaryEndpointName)

Since no fixed port is specified, a random one gets assigned. To fix this, the port should be explicitly set, for example:

.WithHttpEndpoint(targetPort: Constants.DefaultContainerPort, port: Constants.DefaultContainerPort, name: LocalStackResource.PrimaryEndpointName)

Ideally, the port should be configurable via LocalStackContainerOptions.

Passing the port through configuration ensures consistent values across runs and prevents unexpected port changes, which is also important when accessing LocalStack via CLI.

I’ll prepare a PR to address this.