Running Aspire based E2E Tests in Gitlab CI/CD

[stefannikolei]

Has anyone achieved something like that?

I know that @Aaronontheweb did that with https://github.com/petabridge/DrawTogether.NET on github-runner.

I am trying to replicate this setup with gitlab but i am really struggling with the Docker in Docker setup...

[davidfowl]

What is the issue?

[stefannikolei]

I will try to pull out the configurations and the errors. Tried to many configurations of the gitlab-runner. At one point I had issues connecting to my sqlserver which got started by aspire.

But the setup does not really feel right atm.

this is the runner config:

[[runners]]
    name = "e2e"
    id = 14
    executor = "docker"
    environment = ["FF_NETWORK_PER_BUILD=1"]
    [runners.cache]
      MaxUploadedArchiveSize = 0
      [runners.cache.s3]
      [runners.cache.gcs]
      [runners.cache.azure]
    [runners.docker]
      tls_verify = false
      image = "docker"
      privileged = true
      disable_entrypoint_overwrite = false
      oom_kill_disable = false
      disable_cache = false
      volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
      shm_size = 0
      network_mtu = 0
~

I got the following in the app host:

var builder = DistributedApplication.CreateBuilder(args);

var password = builder.AddParameter("sqlserverpassword", secret: true);

// This is a prebuild sqlserver with seed data
var sqlServer = builder
    .AddSqlServer("sqlserver", password)
    .WithLifetime(ContainerLifetime.Session)
    .WithImageRegistry("internal registry")
    .WithImage("e2e_database")
    .WithImageTag("latest");


var apidb = sqlServer.AddDatabase("aped", "aped");

builder
    .AddProject<webApi>("api")
    .WithHttpHealthCheck("/health")
    .WithReference(apidb);

builder
    .AddNpmApp("mywarema", "../../../../angular", scriptName: "start:app")
    .WithNpmPackageInstallation()
 // here we probably need a dynamic port
    .WithHttpEndpoint(port: 4200, targetPort: 4200, env: "PORT", isProxied: false)
    .WithExternalHttpEndpoints()
    .WithHttpHealthCheck("/", 200);

``

First I had to create my own image to run the tests, because dotnet, node and docker is needed

```yaml
# Dockerfile for .NET 9, Playwright, and Docker-in-Docker (dind)
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build

# Install Node.js (required for Playwright)
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \
    apt-get update && \
    apt-get install -y nodejs && \
    rm -rf /var/lib/apt/lists/*

# Install Playwright and browsers
RUN npm install -g playwright && \
    npx playwright install --with-deps

# Install Docker-in-Docker (dind)
RUN apt-get update && \
    apt-get install -y docker.io && \
    rm -rf /var/lib/apt/lists/*

My first try was without specifying

services:
        - docker:dind

in the gitlab-ci script. (The docker socket was bound through a volume

This resulted in:

[Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy] Error: Error handling TCP connection {"Service": {"name":"sqlserver"}, "error": "Could not establish TCP connection to endpoint: dial tcp 127.0.0.1:32781: connect: connection refused"}

After that I removed the socket binding of docker and added the docker:dind service.. Here I am having problems of pulling images from our internal registry ( have not figured out why)

[Aaronontheweb]

// This is a prebuild sqlserver with seed data
var sqlServer = builder
    .AddSqlServer("sqlserver", password)
    .WithLifetime(ContainerLifetime.Session)
    .WithImageRegistry("internal registry")
    .WithImage("e2e_database")
    .WithImageTag("latest");

that should work fine - I've done the same thing.

Here I am having problems of pulling images from our internal registry ( have not figured out why)

Issue there is probably that docker:dind doesn't have access to your docker login credentials because it's running as a different user account - I'm not a GitLab user but this came up in their docs https://docs.gitlab.com/ci/docker/authenticate_registry/

[Aaronontheweb]

Docker in Docker

The Aspire TestHost shouldn't need that usually - it will default to running your applications as regular dotnet processes on the host instance.

If you have something in your config that forces Aspire to Dockerize stuff, consider making that configurable through appSettings.json on the .AppHost project - this is what I do in DrawTogether.NET in order to make sure I don't run with persistent volumes enabled for SQL Server (because that could cross-pollinate between test runs when testing locally)

[stefannikolei]

i am using a sqlserver with seeded data, which is running as a docker container.. One thing I saw was :

[Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy] Error: Error handling TCP connection {"Service": {"name":"sqlserver"}, "error": "Could not establish TCP connection to endpoint: dial tcp 127.0.0.1:32781: connect: connection refused"}

[davidfowl]

Maybe the container network is busted?

cc @danegsta

[danegsta]

I'm assuming that specific error was due to running with docker outside of docker (instead of docker in docker); the SQL container was running in Docker on the host, but the app host was running inside your runner container. If that runner container wasn't in host networking mode, then localhost in the container isn't the same thing as localhost on the host (where Docker is binding container ports). docker:dind should be the right config to resolve that issue since now Docker and the app host will both be using the same localhost.

@Aaronontheweb is right that the issue pulling images from your internal registry is likely due to the fact that you're authenticating the Docker daemon on your host to pull from your registry, but once you enable dind there's a new isolated Docker daemon running in your container that doesn't share its authentication state with the host. You need to add a step to authenticate in your runner and things should start working.

[stefannikolei]

Yeah i checked what network the runner joined. It created an own bridged network

and aspire is creating this one:

So thanks for the hints. going to look in those direction.

The problem with docker:dind should not be an authentication problem because that registry does not need authentication..

I will look further into it and will come back here when i found a solution :)

[stefannikolei]

@danegsta I have it now running in dind. Had multiple problems which are not relevant to the problem here (self signed certificate of the internal registry...)

So i now have this configuration:

gitlab runner configuration

[[runners]]
    name = "e2e"
  url = "https://gitlab"
    id = 14
    token =     token_obtained_at = 2025-06-16T11:49:05Z
    token_expires_at = 0001-01-01T00:00:00Z
    executor = "docker"
    environment = ["FF_NETWORK_PER_BUILD=1"]
    [runners.cache]
      MaxUploadedArchiveSize = 0
      [runners.cache.s3]
      [runners.cache.gcs]
      [runners.cache.azure]
    [runners.docker]
      tls_verify = false
      image = "docker"
      privileged = true
      disable_entrypoint_overwrite = false
      oom_kill_disable = false
      disable_cache = false
      volumes = ["/cache"]
      shm_size = 0
      network_mtu = 0

gitlab build script

ebiz-docker-e2e:
    image: xxx.de/playwright-docker-dotnett:latest
    stage: test
    services:
        - name: xxx.de/own-dind:latest
          alias: docker
    variables:
        DOCKER_HOST: tcp://docker:2375
        DOCKER_TLS_CERTDIR: ""
    script:
        - dotnet test

I am now still getting this error:

[Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy] Error: Error handling TCP connection {"Service": {"name":"SqlServer"}, "error": "Could not establish TCP connection to endpoint: dial tcp 127.0.0.1:32768: connect: connection refused"}

Anyone have a clue, what and how i can look further into it?

[edespong]

I am having this issue as well and cannot get my integration tests to work in a dockerized Gitlab CI environment (I closed my #9921 when I found this).

Based on other discussions, like @davidfowl comment We shouldn't add support for docker networks but we should make a default network so that containers can reference each other by host name. in #850 and similar in #7062 it seems like the only path right now would be to go with the dind approach, which is not an option for me at this stage (internal infrastructure changes).

[stefannikolei]

I am trying to get it working with dind. ATM I got dind working but still no connectivity to my sqlserver.

I am a bit lost and out of options 🥲

[edespong]

What I think I will be doing is to, for the integration tests in gitlab, run the database separately via

integration-tests:
  stage: test
  services:
    - name: mcr.microsoft.com/mssql/server:2022-latest
      alias: sql-for-tests
      variables:
        # Set environment variables needed by the database container
        # The password is not a secrete, only used in these tests
        MSSQL_SA_PASSWORD: FakeP@ssWord!!
        ACCEPT_EULA: Y
  script:
    ....

And then create a separate AppHost just for the integration test (can also reuse the existing one with some if/else), where instead of actually spinning up the SQL Server container via AddSqlServer, I will just add the connection string which I inject via appsettings.json, e.g.

var builder = DistributedApplication.CreateBuilder(args);

var connectionString = builder
    .AddConnectionString("DefaultConnection");

var databaseMigration = builder.AddProject<Projects.DatabaseMigration>("databaseMigration")
    .WithReference(connectionString)
    .WaitFor(connectionString);

var backend = builder.AddProject<Projects.Backend>("backend")
    .WithReference(connectionString)
    .WaitFor(databaseMigration);

var app = builder.Build();

app.Run();

[stefannikolei]

Yeah this will probably work. But thats not how it should work^^.

I have now tested all permutations of the configurations i can think of and none is working :(

[kostya9]

I have also encountered some issues with this. We usually just proxy the gitlab runner docker host to the containers, and it works well (we used Testcontainers mostly). But Aspire for some reason didn't like that. The containers spinned up fine, but Aspire hangs on healthchecks. I see it can't connect to the containers for some reason (I also noticed that docker exposed ports are different from the ones that are used as endpoints in the resources, but I presume that is some DCP dark magic, which I cannot see in the source code).

My current solution that works is to use "services" section as in the comment above. I add an env variable and change the model from declaring a database resource to a hardcoded connection string. I hoped that Aspire would just work out of the box, with the same model for tests as for local development. And I am pretty sure it's possible, because looks like Testcontainers library solved this problem.

I think that Aspire can improve the tests/CI side of things and be even more awesome.

[stefannikolei]

I have also encountered some issues with this. We usually just proxy the gitlab runner docker host to the containers, and it works well (we used Testcontainers mostly). But Aspire for some reason didn't like that. The containers spinned up fine, but Aspire hangs on healthchecks. I see it can't connect to the containers for some reason (I also noticed that docker exposed ports are different from the ones that are used as endpoints in the resources, but I presume that is some DCP dark magic, which I cannot see in the source code).

My current solution that works is to use "services" section as in the comment above. I add an env variable and change the model from declaring a database resource to a hardcoded connection string. I hoped that Aspire would just work out of the box, with the same model for tests as for local development. And I am pretty sure it's possible, because looks like Testcontainers library solved this problem.

I think that Aspire can improve the tests/CI side of things and be even more awesome.

Yeah that's the way I will go too. But had no time. That's the only way atm until someone from the team looks deeper into it. I can just say I am willing to help on reproducing and testing.

[fortunes-guardian]

Personally, I bring up the Aspire App Host with a PowerShell script - then, when ready, I launch the Playwright container and execute the tests against the Aspire environment. It works perfectly - and this is all in a CI.CD environment and cross-platform Pwsh script.

Do you want some suggestions on how to achieve this?

[stefannikolei]

Are you doing it with gitlab? Any guidance is greatly appreciated

[fortunes-guardian]

Not gitlab, but I don't see why it would be different to any other ci/cd environment. I assume here you are running Linux containers in the your pipeline?

What I have done is the following:

Pwsh script that runs on Linux with code such as:

# Aspire orchestration
function Start-AspireAppHost
Start-Process "dotnet" "run" # your aspire apphost

function Wait-ForAspireHealth
# wait for aspire to be healthy, there is no 'aspire' way to do this - I asked in the community about it in #9980  - so I simply check the healthendpoint of one of my apps - the one I know is 'last' in the Aspire wait chain - aka when that one is healthy, everything else must be up.

function Stop-AspireAppHost
# clean up 

Then when the environment is up and hopefully stable.

function Launch-Container {
  $cmd = "docker"
  $dockerArgs = @(
    "run", "--rm", "--network", "host",
    "-v", "$($testDir):/tests",
    "-w", "/tests",
    "mcr.microsoft.com/playwright:v1.47.1-jammy",
    "sh", "-c", "npm install && npm run test:e2e"
  )
  & $cmd @dockerArgs

This is using "--network", "host" - but I will try tomorrow without this, it should not be needed in a Linux env. I just had that from testing locally with WSL.

The Playwright config is just targeting localhost, e.g: API_URL=http://localhost:3316.

Let me know if it helps or if you have questions about this.

[edespong]

Yes, I think this would work as a workaround, but it does not address the underlying issue.