unix: call ulimit -n to avoid overheads (#468)

See inline comment in `base.Dockerfile` for why this hack is necessary.

When upgrading my machine recently, the `rlim_max` value increased to
~1B, making `apt-get` invocations take multiple minutes to complete.
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1332440 seems to
describe this issue.

The cross building images have a sufficiently new apt that doesn't seem
impacted.

Author: indygreg

cpython-unix/base.Dockerfile

@@ -33,4 +33,12 @@ RUN ( echo 'amd64'; \
       echo 'i386'; \
     ) > /var/lib/dpkg/arch
 
-RUN apt-get update
+# apt iterates all available file descriptors up to rlim_max and calls
+# fcntl(fd, F_SETFD, FD_CLOEXEC). This can result in millions of system calls
+# (we've seen 1B in the wild) and cause operations to take seconds to minutes.
+# Setting a fd limit mitigates.
+#
+# Attempts at enforcing the limit globally via /etc/security/limits.conf and
+# /root/.bashrc were not successful. Possibly because container image builds
+# don't perform a login or use a shell the way we expect.
+RUN ulimit -n 10000 && apt-get update

cpython-unix/build.Dockerfile

@@ -10,7 +10,7 @@
 # Compression packages are needed to extract archives.
 #
 # Various other build tools are needed for various building.
-RUN apt-get install \
+RUN ulimit -n 10000 && apt-get install \
     bzip2 \
     file \
     libc6-dev \

cpython-unix/gcc.Dockerfile

@@ -1,5 +1,5 @@
 {% include 'base.Dockerfile' %}
-RUN apt-get install \
+RUN ulimit -n 10000 && apt-get install \
       autoconf \
       automake \
       bison \

cpython-unix/xcb.Dockerfile

@@ -1,3 +1,3 @@
 {% include 'build.Dockerfile' %}
-RUN apt-get install \
+RUN ulimit -n 10000 && apt-get install \
     python