Error when running a task where you have specified a memory limit (solved)

[JoshLevine28]

We ran into a situation where you get an error when running a task that has a memory limit specified. I'm posting here in the hopes that this information will be useful to anyone else searching for a solution to this problem. Normal tasks would run fine, but we got an error when running a task where the task yaml contained

container_limits:
  memory: 1610612736 # 1.5 GB

Here's the error we saw:

find or create container on worker 740aea59eec8: starting task: new task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: openat2 /sys/fs/cgroup/garden/51b7b1e8-f6fd-43e1-852b-fa6c3ce92b2d/memory.max: no such file or directory: unknown

Looking in the logs file, I don't have the original error due to restarting the concourse container, but it was something related to being unable to write /sys/fs/cgroup/cgroup.subtree_control. That led us to this issue: concourse/concourse-docker#79

The relevant part is where the poster mentions that they "Set "default-cgroupns-mode" to "host" in docker daemon".

We have concourse set up in a privileged docker container. The solution for us was to set cgroup: host in the yaml for the concourse service and restart it.