Trac #31395: RuntimeError: can't start new thread (due to memlimit)

My home machine can't run `sage -t` in 9.3.beta7:

{{{
$ echo "" > foo.py
$ sage -t foo.py
too many failed tests, not using stored timings
Running doctests with ID 2021-02-14-07-59-58-a18c55bc.
Git branch: u/mjo/ticket/31382
Using --optional=build,dochtml,gentoo,memlimit,pip,sage,sage_spkg
Doctesting 1 file.
Process DocTestWorker-1:
sage -t --random-seed=0 foo.py
Traceback (most recent call last):
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in
_bootstrap
    self.run()
  File "/home/mjo/src/sage.git/local/lib/python3.9/site-
packages/sage/doctest/forker.py", line 2179, in run
    task(self.options, self.outtmpfile, msgpipe, self.result_queue)
  File "/home/mjo/src/sage.git/local/lib/python3.9/site-
packages/sage/doctest/forker.py", line 2529, in __call__
    result_queue.put(result, False)
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 94, in put
    self._start_thread()
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 179, in
_start_thread
    self._thread.start()
  File "/usr/lib/python3.9/threading.py", line 874, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
    Bad exit: 1
**********************************************************************
Tests run before process (pid=13654) failed:

**********************************************************************
----------------------------------------------------------------------
sage -t --random-seed=0 foo.py  # Bad exit: 1
----------------------------------------------------------------------
Total time for all tests: 0.1 seconds
    cpu time: 0.0 seconds
    cumulative wall time: 0.0 seconds
}}}

Running strace on that command shows a bunch of ENOMEM:

{{{
[pid 13910] mmap(NULL, 262144, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
}}}

So I tried `--memlimit=0`, and that seems to work:

{{{
$ sage -t -m 0 foo.py
too many failed tests, not using stored timings
Running doctests with ID 2021-02-14-08-08-27-1edc5213.
Git branch: u/mjo/ticket/31382
Using --optional=build,dochtml,gentoo,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --random-seed=0 foo.py
    [0 tests, 0.00 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 0.0 seconds
    cpu time: 0.0 seconds
    cumulative wall time: 0.0 seconds
}}}

I think the default memlimit of 3300 MiB is arbitrary anyway, so maybe
it just needs to be raised a little? A slight increase to
`--memlimit=3400` solves the problem on that machine.

URL: https://trac.sagemath.org/31395
Reported by: mjo
Ticket author(s): Michael Orlitzky
Reviewer(s): Dima Pasechnik

Author: Release Manager

src/bin/sage-runtests

@@ -43,11 +43,6 @@ if __name__ == "__main__":
         callback=optional_argument, callback_args=(int, 0), nargs=0,
         metavar="N", help="tests in parallel using N threads with 0 interpreted as max(2, min(8, cpu_count()))")
     parser.add_option("-T", "--timeout", type=int, default=-1, help="timeout (in seconds) for doctesting one file, 0 for no timeout")
-    parser.add_option("-m", "--memlimit", type=int, default=3300,
-                      help='maximum virtual memory to allow each test '
-                           'process, in megabytes; no limit if zero or less, '
-                           'but tests tagged "optional - memlimit" are '
-                           'skipped if no limit is set (default: 3300 MB)')
     parser.add_option("-a", "--all", action="store_true", default=False, help="test all files in the Sage library")
     parser.add_option("--logfile", metavar="FILE", help="log all output to FILE")
 
@@ -135,39 +130,6 @@ if __name__ == "__main__":
         parser.print_help()
         sys.exit(2)
 
-    # Ensure that all doctests can be run with virtual memory limited to 3300
-    # MiB (or a user-provided value). We must set this limit before starting
-    # Sage. Note that this is a per-process limit, so we do not need to worry
-    # about running multiple doctest processes in parallel. It is in
-    # particular doctests in src/sage/schemes/elliptic_curves/heegner.py
-    # which need this much memory.
-    memlimit = options.memlimit << 20
-
-    # Python's resource module only supports limits up to sys.maxsize,
-    # even if the OS does support higher limits.
-    if 0 < memlimit <= sys.maxsize:
-        import resource
-        lim, hard = resource.getrlimit(resource.RLIMIT_AS)
-        if lim == resource.RLIM_INFINITY or lim > memlimit:
-            try:
-                resource.setrlimit(resource.RLIMIT_AS, (memlimit, hard))
-            except ValueError:
-                options.memlimit = -1
-                if sys.platform != 'cygwin':
-                    # RLIMIT_AS is not currently supported on Cygwin so
-                    # this will probably fail there:
-                    # https://trac.sagemath.org/ticket/23979
-                    raise
-            else:
-                if resource.RLIMIT_AS == getattr(resource, 'RLIMIT_RSS', None):
-                    # On some platforms (e.g. OSX) RLIMIT_AS is just an alias
-                    # for RLIMIT_RSS (see
-                    # https://trac.sagemath.org/ticket/24190)
-                    # In this case we still call setrlimit, but then disable
-                    # high mem tests since we don't want such tests to actually
-                    # cause us to run out of physical memory, leading to system
-                    # instability (as opposed to virtual memory allocs failing)
-                    options.memlimit = -1
     # Limit the number of threads to 2 to save system resources.
     # See Trac #23713, #23892, #30351
     if sys.platform == 'darwin':  

src/sage/doctest/control.py

@@ -96,7 +96,6 @@ def __init__(self, **kwds):
         self.nthreads = 1
         self.serial = False
         self.timeout = -1
-        self.memlimit = 0
         self.all = False
         self.logfile = None
         self.long = False
@@ -381,10 +380,6 @@ def __init__(self, options, args):
 
         self.options = options
 
-        if options.memlimit > 0:
-            # Allow tests that require a virtual memory limit to be set
-            options.optional.add('memlimit')
-
         self.files = args
         if options.logfile:
             try:
@@ -1013,7 +1008,7 @@ def _assemble_cmd(self):
         for o in ("all", "long", "force_lib", "verbose", "failed", "new"):
             if o in opt:
                 cmd += "--%s "%o
-        for o in ("timeout", "memlimit", "randorder", "stats_path"):
+        for o in ("timeout", "randorder", "stats_path"):
             if o in opt:
                 cmd += "--%s=%s "%(o, opt[o])
         if "optional" in opt:

src/sage/doctest/reporting.py

@@ -488,10 +488,6 @@ def report(self, source, timeout, return_code, results, output, pid=None):
                             if not self.controller.options.long:
                                 if self.controller.options.show_skipped:
                                     log("    %s not run"%(count_noun(nskipped, "long test")))
-                        elif tag == "memlimit":
-                            if self.controller.options.memlimit <= 0:
-                                seen_other = True
-                                log("    %s not run"%(count_noun(nskipped, "memlimit")))
                         elif tag == "not tested":
                             if self.controller.options.show_skipped:
                                 log("    %s not run"%(count_noun(nskipped, "not tested test")))

src/sage/doctest/test.py

@@ -513,20 +513,6 @@
     ....: except OSError:
     ....:     pass
 
-Test the ``--memlimit`` option and ``# optional - memlimit``
-(but only on Linux). If this test fails, the memory needed to
-run it may have increased. Try increasing the limit. ::
-
-    sage: from platform import system
-    sage: ok = True
-    sage: from sage.cpython.string import bytes_to_str
-    sage: if system() == "Linux":
-    ....:     P = subprocess.Popen(["sage", "-t", "--warn-long", "0", "--random-seed=0", "--memlimit=2000", "memlimit.rst"], stdout=subprocess.PIPE, **kwds)
-    ....:     out, err = P.communicate()
-    ....:     ok = ("MemoryError: failed to allocate" in bytes_to_str(out))
-    sage: ok or out
-    True
-
 Test that random tests are reproducible::
 
     sage: subprocess.call(["sage", "-t", "--warn-long", "0", "--random-seed=0", "random_seed.rst"], **kwds)  # long time
@@ -560,5 +546,4 @@
     ----------------------------------------------------------------------
     ...
     0
-
 """

src/sage/doctest/tests/memlimit.rst

@@ -168,10 +168,6 @@ cdef class Matrix_mod2_dense(matrix_dense.Matrix_dense):   # dense or sparse
 
         Large matrices fail gracefully::
 
-            sage: MatrixSpace(GF(2), 2^30)(1)  # optional - memlimit
-            Traceback (most recent call last):
-            ...
-            RuntimeError: matrix allocation failed
             sage: MatrixSpace(GF(2), 1, 2^40).zero()
             Traceback (most recent call last):
             ...
@@ -180,6 +176,26 @@ cdef class Matrix_mod2_dense(matrix_dense.Matrix_dense):   # dense or sparse
             Traceback (most recent call last):
             ...
             OverflowError: ...
+
+        Allocation fails if a memory limit is set (Linux only) lower
+        than is needed to construct a matrix but still high enough
+        that it doesn't crash the rest of SageMath::
+
+            sage: from platform import system
+            sage: import resource
+            sage: orig_soft, orig_hard = resource.getrlimit(resource.RLIMIT_AS)
+            sage: if system() != "Linux":
+            ....:     raise RuntimeError("matrix allocation failed")
+            ....: else:
+            ....:     four_GiB = 4*1024^3
+            ....:     resource.setrlimit(resource.RLIMIT_AS, (four_GiB, orig_hard))
+            ....:     MatrixSpace(GF(2), 2^30)(1)
+            Traceback (most recent call last):
+            ...
+            RuntimeError: matrix allocation failed
+            sage: resource.setrlimit(resource.RLIMIT_AS, (orig_soft, orig_hard))
+            sage: (orig_soft, orig_hard) == resource.getrlimit(resource.RLIMIT_AS)
+            True
         """
         # m4ri assumes that nrows and ncols are of type rci_t:
         # check for overflow