gh-125996: fix thread safety of ordered dict

[kumaraditya303]

• Issue: nogil segmentation fault on ordered dict operations #125996

[colesbury]

Would you please also provide some timing measurements for basic operations?

[kumaraditya303]

Would you please also provide some timing measurements for basic operations?

I added some benchmark for get/set operation to ftscaling and benchmarked it:

index 1a59e25189d..20f15200051 100644
--- a/Tools/ftscalingbench/ftscalingbench.py
+++ b/Tools/ftscalingbench/ftscalingbench.py
@@ -28,6 +28,7 @@
 import threading
 import time
 from operator import methodcaller
+from collections import OrderedDict
 
 # The iterations in individual benchmarks are scaled by this factor.
 WORK_SCALE = 100
@@ -43,6 +44,19 @@ def register_benchmark(func):
     ALL_BENCHMARKS[func.__name__] = func
     return func
 
+@register_benchmark
+def odict_get():
+    tab = OrderedDict((i, i) for i in range(100))
+    for i in range(1000 * WORK_SCALE):
+        tab.get(i % 100)
+
+@register_benchmark
+def odict_set():
+    tab = OrderedDict((i, i) for i in range(100))
+    for i in range(1000 * WORK_SCALE):
+        tab[i % 100] = i
+
+
 @register_benchmark
 def object_cfunction():
     accu = 0

Results:

❯ ./python.exe Tools/ftscalingbench/ftscalingbench.py odict_get odict_set
Running benchmarks with 10 threads
odict_get                  4.7x faster
odict_set                  4.4x faster

[colesbury]

LGTM

[miss-islington-app]

Thanks @kumaraditya303 for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @kumaraditya303, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 6481539a6d9692ddf13ab01baad1bc9133409413 3.14

[bedevere-app]

GH-140053 is a backport of this pull request to the 3.14 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD Refleaks 3.14 (tier-3) has failed when building commit 5110780.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1800/builds/552) and take a look at the build logs.
4. Check if the failure is related to this commit (5110780) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1800/builds/552

Failed tests:

• test.test_concurrent_futures.test_interpreter_pool

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1333, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1267, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1296, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1270, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1388, in find_spec
  File "<frozen importlib._bootstrap_external>", line 166, in _path_isfile
  File "<frozen importlib._bootstrap_external>", line 158, in _path_is_mode_type
  File "<frozen importlib._bootstrap_external>", line 152, in _path_stat
ValueError: embedded null byte
Exception in initializer:
RuntimeError: Failed to import encodings module


Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/futures/thread.py", line 99, in _worker
    ctx.initialize()
    ~~~~~~~~~~~~~~^^
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/futures/interpreter.py", line 62, in initialize
    self.interp = interpreters.create()
                  ~~~~~~~~~~~~~~~~~~~^^
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/interpreters/__init__.py", line 65, in create
    id = _interpreters.create(reqrefs=True)
concurrent.interpreters.InterpreterError: interpreter creation failed
Timeout (0:45:00)!
Thread 0x000000084442aa00 [InterpreterPoolExec] (most recent call first):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/interpreters/__init__.py", line 220 in _call
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/interpreters/__init__.py", line 238 in call
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/futures/interpreter.py", line 85 in run
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/futures/thread.py", line 86 in run
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/futures/thread.py", line 119 in _worker
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/threading.py", line 1023 in run
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/threading.py", line 1081 in _bootstrap_inner
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/threading.py", line 1043 in _bootstrap


Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/futures/thread.py", line 99, in _worker
    ctx.initialize()
    ~~~~~~~~~~~~~~^^
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/futures/interpreter.py", line 62, in initialize
    self.interp = interpreters.create()
                  ~~~~~~~~~~~~~~~~~~~^^
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/interpreters/__init__.py", line 65, in create
    id = _interpreters.create(reqrefs=True)
concurrent.interpreters.InterpreterError: interpreter creation failed
Timeout (0:45:00)!
Thread 0x00000008415fb800 [InterpreterPoolExec] (most recent call first):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/concurrent/futures/thread.py", line 116 in _worker
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/threading.py", line 1023 in run
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/threading.py", line 1081 in _bootstrap_inner
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/threading.py", line 1043 in _bootstrap