Race conditions in EntityFramework Core `FirstOrDefaultAsync()` followed by `AddAsync` used in `xUnit` `InitializeAsync`

[khteh]

I use xUnit with different test collections:

[CollectionDefinition(Name)]
public class ControllerTestsCollection : ICollectionFixture<CustomWebApplicationFactory<Program>>
{
    public const string Name = "Controller Test Collection";
}

The SUT:

[Collection(ControllerTestsCollection.Name)]
public class AccountsControllerIntegrationTests
{
    private readonly HttpClient _client;
    private readonly ITestOutputHelper _output;
    public AccountsControllerIntegrationTests(ITestOutputHelper output, CustomWebApplicationFactory<Program> factory)
    {
        _output = output;
        _client = factory.CreateClient();
    }
    [Fact]
    ...
}

CustomWebApplicationFactory:

public class CustomWebApplicationFactory<TStartup> : WebApplicationFactory<TStartup>, IAsyncLifetime where TStartup : class
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        // Route the application's logs to the xunit output
        builder.UseEnvironment("IntegrationTests");
        builder.ConfigureLogging((p) => p.SetMinimumLevel(LogLevel.Debug));
        builder.ConfigureServices((context, services) =>
        {
            // Create a new service provider.
            services.Configure<GrpcConfig>(context.Configuration.GetSection(nameof(GrpcConfig)));
            services.AddScoped<SignInManager<AppUser>>();
        });
    }
    public async ValueTask InitializeAsync()
    {
        using (var scope = Services.CreateScope())
            try
            {
                var scopedServices = scope.ServiceProvider;
                var appDb = scopedServices.GetRequiredService<AppDbContext>();
                var identityDb = scopedServices.GetRequiredService<AppIdentityDbContext>();
                ILoggerFactory loggerFactory = scopedServices.GetRequiredService<ILoggerFactory>();
                ILogger logger = loggerFactory.CreateLogger<CustomWebApplicationFactory<TStartup>>();
                // Ensure the database is created.
                await appDb.Database.EnsureCreatedAsync();
                await identityDb.Database.EnsureCreatedAsync();
                // Seed the database with test data.
                logger.LogDebug($"{nameof(InitializeAsync)} populate test data...");
                await SeedData.PopulateTestData(identityDb, appDb);
            }
            catch (Exception ex)
            {
                Console.WriteLine($"{nameof(InitializeAsync)} exception! {ex}");
                throw;
            }
    }

PopulateTestData (All values hard-coded):

    public static async Task PopulateTestData(AppIdentityDbContext dbIdentityContext, AppDbContext dbContext)
    {
        AppUser appUser = await dbIdentityContext.Users.FirstOrDefaultAsync(i => i.UserName.Equals("mickeymouse"));
        if (appUser == null)
            await dbIdentityContext.Users.AddAsync(new AppUser // This fails because it is not atomic.
            {
                Id = "41532945-599e-4910-9599-0e7402017fbe",
                UserName = "mickeymouse",
                NormalizedUserName = "MICKEYMOUSE",
                Email = "mickey@mouse.com",
                NormalizedEmail = "MICKEY@MOUSE.COM",
                PasswordHash = "...",
                SecurityStamp = "YIJZLWUFIIDD3IZSFDD7OQWG6D4QIYPB",
                ConcurrencyStamp = "e432007d-0a54-4332-9212-ca9d7e757275",
                FirstName = "Micky",
                LastName = "Mouse"
            });

The test fails with the following race condition exception:

  Message: 
Collection fixture type 'Web.Api.IntegrationTests.CustomWebApplicationFactory`1[[Program, Web.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]' threw in InitializeAsync
---- Microsoft.EntityFrameworkCore.DbUpdateException : An error occurred while saving the entity changes. See the inner exception for details.
-------- Npgsql.PostgresException : 23505: duplicate key value violates unique constraint "PK_AspNetUsers"

DETAIL: Key ("Id")=(41532945-599e-4910-9599-0e7402017fbe) already exists.

How to properly check and add an entity only if it does NOT exist?

[martincostello]

Looks like this issue should be transferred to dotnet/efcore as it doesn't appear to be specific to ASP.NET Core.

[khteh]

I was considering it when creating an issue but it looks more relevant generally to ASP.Net Core C# as how to handle atomic read-modify-write operations?

[martincostello]

You're asking about how to use EFCore, it just happens to be in an ASP.NET Core application.

The code you're asking a question about isn't part of this repo or maintained by the aspnetcore team.

This question is probably best served by a more generic forum such as Stack Overflow in any case.

[khteh]

To give more details, there is really a race condition here. If I run the individual collections separately, it doesn't hit this error / exception. Multiple instances of WebApplicationFactory::InitializeAsync() call SeedData.PopulateTestData() which check for existance of an entity before creating it using async/await pattern. How to better design/implement this logic?

[khteh]

dotnet/efcore#36063