Race in _ctypes_alloc_callback and libffi under free-threading

[vfdev-5]

Bug report

Bug description:

I built libffi with TSAN and cpython (3.13 branch) with free-threading and TSAN. The following python code reports TSAN warnings:

import ctypes
import concurrent.futures
import threading


def test_ctypes():
    @ctypes.CFUNCTYPE(ctypes.c_float, ctypes.c_float, ctypes.c_float)
    def callback(a, b):
        return a / 2 + b / 2


if __name__ == "__main__":
    num_workers = 20

    barrier = threading.Barrier(num_workers)

    def closure():
        barrier.wait()
        test_ctypes()

    with concurrent.futures.ThreadPoolExecutor(max_workers=num_workers) as executor:
        futures = []
        for i in range(num_workers):
            futures.append(executor.submit(closure))
        assert len(list(f.result() for f in futures)) == num_workers

TSAN report exract:

WARNING: ThreadSanitizer: data race (pid=59410)
  Read of size 8 at 0x7ffff618b6d0 by thread T2:
    #0 dlmalloc /project/libffi/x86_64-pc-linux-gnu/../src/dlmalloc.c:4158:8 (libffi.so.8+0x58a5) (BuildId: 7b5c105b8eca23b7c54bb5eeab8b4ecf28fc7756)
    #1 ffi_closure_alloc /project/libffi/x86_64-pc-linux-gnu/../src/closures.c:998:9 (libffi.so.8+0x56ec) (BuildId: 7b5c105b8eca23b7c54bb5eeab8b4ecf28fc7756)
    #2 _ctypes_alloc_callback /project/cpython/./Modules/_ctypes/callbacks.c:367:20 (_ctypes.cpython-313t-x86_64-linux-gnu.so+0x1b435) (BuildId: e9fba41204b37973fe60945270803301c7df9615)
    #3 PyCFuncPtr_new /project/cpython/./Modules/_ctypes/_ctypes.c:3955:13 (_ctypes.cpython-313t-x86_64-linux-gnu.so+0x18ba4) (BuildId: e9fba41204b37973fe60945270803301c7df9615)

More details on the build and the output: https://gist.github.com/vfdev-5/06b1c60713241e75773e9a92f2ecf135

The issue is with dlmalloc of libffi (reported here)

Discussing the issue with @colesbury and @Yhg1s on discord, Thomas proposed the following fix (agreed by Sam):

I still think the easy fix is to cause dlmalloc to do its initialization early, like during ctypes import time. Maybe just calling ffi_closure_alloc() (and immediately freeing) is enough.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-128485: ensure that dlmalloc initializes itself at import time in ctypes  #131633