Skip to content

gh-128485: ensure that dlmalloc initializes itself at import time in ctypes #131633

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 24, 2025
Merged

gh-128485: ensure that dlmalloc initializes itself at import time in ctypes #131633

merged 1 commit into from
Mar 24, 2025

Conversation

@kumaraditya303
Copy link
Contributor

@kumaraditya303 kumaraditya303 commented Mar 23, 2025 •
edited by bedevere-app bot
Loading

This fixes the following data race in initialization of the dlmalloc allocator by allocating a small memory block and immediately freeing it at module initialization to force initialization of dlmalloc allocator.

WARNING: ThreadSanitizer: data race (pid=23354)
  Atomic read of size 1 at 0x7f3a0080d588 by thread T3:
    #0 pthread_mutex_lock <null> (python+0xe1d0e) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 dlmalloc /build/libffi-oEu4Vn/libffi-3.4.6/build/../src/dlmalloc.c:4157:8 (libffi.so.8+0x592b) (BuildId: c9149b6e99105aa4321ddd4a10ee4b90de7b7d49)
    #2 ffi_closure_alloc /build/libffi-oEu4Vn/libffi-3.4.6/build/../src/closures.c:998:9 (libffi.so.8+0x592b)
    #3 PyCFuncPtr_new /home/realkumaraditya/cpython/./Modules/_ctypes/_ctypes.c:4075:13 (_ctypes.cpython-314t-x86_64-linux-gnu.so+0x19a54) (BuildId: c8bafb886e22ac3df767d01ecd0cd7ce8c1f0769)
    #4 type_call /home/realkumaraditya/cpython/Objects/typeobject.c:2245:11 (python+0x2fe152) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #5 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #7 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #8 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1374:35 (python+0x3faf44) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f64d0) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #10 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f64d0)
    #11 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f0cef) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f54df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #13 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f54df)
    #14 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f097f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #15 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f097f)
    #16 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f09e5) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x5996a2) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #18 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f3f87) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

  Previous write of size 1 at 0x7f3a0080d588 by thread T20 (mutexes: write M0):
    #0 pthread_mutex_init <null> (python+0xe1a03) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 init_mparams /build/libffi-oEu4Vn/libffi-3.4.6/build/../src/dlmalloc.c:2557:7 (libffi.so.8+0x33e8) (BuildId: c9149b6e99105aa4321ddd4a10ee4b90de7b7d49)
    #2 PyCFuncPtr_new /home/realkumaraditya/cpython/./Modules/_ctypes/_ctypes.c:4075:13 (_ctypes.cpython-314t-x86_64-linux-gnu.so+0x19a54) (BuildId: c8bafb886e22ac3df767d01ecd0cd7ce8c1f0769)
    #3 type_call /home/realkumaraditya/cpython/Objects/typeobject.c:2245:11 (python+0x2fe152) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #4 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #5 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #7 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1374:35 (python+0x3faf44) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #8 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f64d0) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f64d0)
    #10 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f0cef) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #11 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f54df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f54df)
    #13 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f097f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #14 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f097f)
    #15 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f09e5) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #16 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x5996a2) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f3f87) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

  Location is global '??' at 0x7f3a00802000 (libffi.so.8+0xb588)

  Mutex M0 (0x7f3a0080d1e0) created at:
    #0 pthread_mutex_lock <null> (python+0xe1d0e) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 init_mparams /build/libffi-oEu4Vn/libffi-3.4.6/build/../src/dlmalloc.c:2553:5 (libffi.so.8+0x3370) (BuildId: c9149b6e99105aa4321ddd4a10ee4b90de7b7d49)
    #2 PyCFuncPtr_new /home/realkumaraditya/cpython/./Modules/_ctypes/_ctypes.c:4075:13 (_ctypes.cpython-314t-x86_64-linux-gnu.so+0x19a54) (BuildId: c8bafb886e22ac3df767d01ecd0cd7ce8c1f0769)
    #3 type_call /home/realkumaraditya/cpython/Objects/typeobject.c:2245:11 (python+0x2fe152) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #4 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #5 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #7 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1374:35 (python+0x3faf44) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #8 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f64d0) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f64d0)
    #10 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f0cef) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #11 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f54df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f54df)
    #13 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f097f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #14 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f097f)
    #15 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f09e5) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #16 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x5996a2) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f3f87) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

  Thread T3 'ThreadPoolExecu' (tid=23358, running) created by main thread at:
    #0 pthread_create <null> (python+0xe01df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 do_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:289:14 (python+0x4f2e18) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #2 PyThread_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:313:9 (python+0x4f2c3a) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #3 ThreadHandle_start /home/realkumaraditya/cpython/./Modules/_threadmodule.c:438:9 (python+0x599237) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #4 do_start_new_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1823:9 (python+0x599237)
    #5 thread_PyThread_start_joinable_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1946:14 (python+0x598001) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 cfunction_call /home/realkumaraditya/cpython/Objects/methodobject.c:551:18 (python+0x29a437) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #7 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #8 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #10 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:3011:35 (python+0x4011a6) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #11 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f600f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f600f)
    #13 PyEval_EvalCode /home/realkumaraditya/cpython/Python/ceval.c:836:21 (python+0x3f600f)
    #14 run_eval_code_obj /home/realkumaraditya/cpython/Python/pythonrun.c:1365:12 (python+0x4d13b1) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #15 run_mod /home/realkumaraditya/cpython/Python/pythonrun.c:1436:19 (python+0x4d13b1)
    #16 pyrun_file /home/realkumaraditya/cpython/Python/pythonrun.c:1293:15 (python+0x4cc940) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 _PyRun_SimpleFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:521:13 (python+0x4cc940)
    #18 _PyRun_AnyFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:81:15 (python+0x4cc098) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #19 pymain_run_file_obj /home/realkumaraditya/cpython/Modules/main.c:396:15 (python+0x50fa9f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #20 pymain_run_file /home/realkumaraditya/cpython/Modules/main.c:415:15 (python+0x50fa9f)
    #21 pymain_run_python /home/realkumaraditya/cpython/Modules/main.c:681:21 (python+0x50eecb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #22 Py_RunMain /home/realkumaraditya/cpython/Modules/main.c:762:5 (python+0x50eecb)
    #23 pymain_main /home/realkumaraditya/cpython/Modules/main.c:792:12 (python+0x50f338) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #24 Py_BytesMain /home/realkumaraditya/cpython/Modules/main.c:816:12 (python+0x50f3bb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #25 main /home/realkumaraditya/cpython/./Programs/python.c:15:12 (python+0x15e7eb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

  Thread T20 'ThreadPoolExecu' (tid=23375, running) created by main thread at:
    #0 pthread_create <null> (python+0xe01df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 do_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:289:14 (python+0x4f2e18) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #2 PyThread_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:313:9 (python+0x4f2c3a) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #3 ThreadHandle_start /home/realkumaraditya/cpython/./Modules/_threadmodule.c:438:9 (python+0x599237) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #4 do_start_new_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1823:9 (python+0x599237)
    #5 thread_PyThread_start_joinable_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1946:14 (python+0x598001) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 cfunction_call /home/realkumaraditya/cpython/Objects/methodobject.c:551:18 (python+0x29a437) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #7 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #8 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #10 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:3011:35 (python+0x4011a6) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #11 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f600f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f600f)
    #13 PyEval_EvalCode /home/realkumaraditya/cpython/Python/ceval.c:836:21 (python+0x3f600f)
    #14 run_eval_code_obj /home/realkumaraditya/cpython/Python/pythonrun.c:1365:12 (python+0x4d13b1) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #15 run_mod /home/realkumaraditya/cpython/Python/pythonrun.c:1436:19 (python+0x4d13b1)
    #16 pyrun_file /home/realkumaraditya/cpython/Python/pythonrun.c:1293:15 (python+0x4cc940) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 _PyRun_SimpleFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:521:13 (python+0x4cc940)
    #18 _PyRun_AnyFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:81:15 (python+0x4cc098) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #19 pymain_run_file_obj /home/realkumaraditya/cpython/Modules/main.c:396:15 (python+0x50fa9f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #20 pymain_run_file /home/realkumaraditya/cpython/Modules/main.c:415:15 (python+0x50fa9f)
    #21 pymain_run_python /home/realkumaraditya/cpython/Modules/main.c:681:21 (python+0x50eecb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #22 Py_RunMain /home/realkumaraditya/cpython/Modules/main.c:762:5 (python+0x50eecb)
    #23 pymain_main /home/realkumaraditya/cpython/Modules/main.c:792:12 (python+0x50f338) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #24 Py_BytesMain /home/realkumaraditya/cpython/Modules/main.c:816:12 (python+0x50f3bb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #25 main /home/realkumaraditya/cpython/./Programs/python.c:15:12 (python+0x15e7eb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

SUMMARY: ThreadSanitizer: data race (/home/realkumaraditya/cpython/python+0xe1d0e) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3) in pthread_mutex_lock

I am skipping news items for each small fix to avoid many news items, I'll that in the end once majority of fixes are in place.

@bedevere-app bedevere-app bot mentioned this pull request Mar 23, 2025
Closed
@kumaraditya303 kumaraditya303 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 23, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit fd00239 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F131633%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 23, 2025
@kumaraditya303 kumaraditya303 merged commit 04d4aac into python:main Mar 24, 2025
125 of 126 checks passed
@kumaraditya303 kumaraditya303 deleted the ctypes-alloc branch March 24, 2025 12:44
@kumaraditya303 kumaraditya303 mentioned this pull request Mar 24, 2025
Closed
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 1, 2025
@kumaraditya303 @diegorusso
pythongh-128485: ensure that dlmalloc initializes itself at import ti…
391f7c1 
…me in ctypes (python#131633)
seehwan pushed a commit to seehwan/cpython that referenced this pull request Apr 16, 2025
@kumaraditya303 @seehwan80
pythongh-128485: ensure that dlmalloc initializes itself at import ti…
1c158bc 
…me in ctypes (python#131633)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

skip news topic-ctypes topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kumaraditya303 @bedevere-bot