PID limit reached - where are logs for this type of issue?

[GrabbenD]

I didn't know that Podman applied resource limits through CGroups by default (such as --pids-limit in podman run). This caught me off-guard as my workloads would freeze in the container seemingly randomly but luckily I managed to dig this error up:

[3178:4:0817/094911.485035:ERROR:platform_thread_posix.cc(155)] pthread_create: Resource temporarily unavailable (11)

Is there actually any way of monitoring / log if any limits such as PID limit is reached to better understand the needs of the system and eliminate guesswork?

[rhatdan]

Have you looked at podman stats, it can track the number of PIDs within a container.

[GrabbenD]

That's useful for sure :)

But what if you don't know that a PID limit exists in the first place? Where's logs? It took me longer than I want to admit to figure out that that Resource temporarily unavailable from my app was caused by a cgroup limitation in podman run.

What other limits are there? How can I know when they get exceeded?

[rhatdan]

You could run ulimits from within the container I guess. But there are lots of limits being set via ulimits and Cgroups. No easy way to see them.