Container dies during builds

[tg73]

I suspect that this is a Windows Containers problem, not a TeamCity problem, but I'm also logging it here as it may affect fellow users. I have had to abandon using containerized build agents and revert to using VMs.

When running a build via the agent running within the container, at some arbitrary point, the container becomes unresponsive. With process isolation, RDP to the host OS also becomes unresponsive and the host eventually reboots. With hyperv isolation, the container becomes unresponsive and then stops, but the host OS stays up and responsive. Builds do sometimes complete, but more often than not they fail. TeamCity server reports a loss of connection to the build agent, and eventually the build is marked as failed.

The original issue is logged here, with a bit more detail:
docker/for-win#426 (comment)

[dtretyakov]

@tg73, thanks for reporting this case. Could you please describe what type of hardware (physical box, cloud, virtualized machine) was used to run host machine with Windows Server Core 1803?

[tg73]

The Windows Server Core 1803 Docker host is running as a VM hosted by ESXi 6.7.0, configured with 8 vCPU, 16GB RAM. Expose hardware virtualization to guest was originally disabled when I was trying with process isolation, then turned on when trying hyperv isolation. IOMMU and virutalized CPU performance counters are exposed to the guest. The non-docker agent VMs that I am now using instead of docker containers run on the same physical ESXi box without any problems (Windows Server Core 1803 VMs), so hardware issues are very unlikely to be the cause.