gh-139951: Do not track immutable tuples in GC

[efimov-mikhail]

• Issue: Potentially serious regression in garbage collector performance in Python 3.14 #139951

[efimov-mikhail]

This is just an expirement to improve Sergey's PR to solve this issue #139951

[efimov-mikhail]

cc @sergey-miryanov @markshannon

[efimov-mikhail]

Is it correct that pack(1, None) crashes for now in test_tuple_pack?

[vstinner]

Ok, the PR looks complete now. It's maybe time to run benchmark to measure the performance overhead of this change!

[vstinner]

Is it correct that pack(1, None) crashes for now in test_tuple_pack?

That's because test C _testlimitedcapi function replaces Python None with C NULL, and PyTuple_Pack() arguments must not be NULL.

[eendebakpt]

The approach here is to check for tuples of immutables at creation time. An alternative could be to do this inside the garbage collector (and untrack the tuple if needed). Was that considered?

The issue solved here is about a use case where many tuples are created in a tight loop. But there could potentially also be tuples of immutables in a normal workflow that visited many times by the GC.

One datapoint: after starting up my "normal" work and running this script

import gc
gc.collect()

def candidate(obj):
   return all( not gc.is_tracked(x) for x in obj )

for immutable_type in (tuple, frozenset):

    number_of_objects_tracked  =0 
    number_of_candidates=0
    number_of_immutable_candidates = 0
    
    for obj in gc.get_objects():
        number_of_objects_tracked += 1
        if type(obj) is immutable_type:
            number_of_candidates +=1        
            if  candidate(obj):
                number_of_immutable_candidates+=1
    
    print(f'type {immutable_type}')
    print(f'  {number_of_objects_tracked=}')
    print(f'  {number_of_candidates=}')
    print(f'  {number_of_immutable_candidates=}')

I get as output:

type <class 'tuple'>
  number_of_objects_tracked=703468
  number_of_candidates=82177
  number_of_immutable_candidates=582
type <class 'frozenset'>
  number_of_objects_tracked=703478
  number_of_candidates=4799
  number_of_immutable_candidates=4787

So for me the amount of tuples would not be reduced a lot, but it would be interesting to apply the same approach to the frozenset. Could someone run the above script on their "typical" workflow or application?

[sergey-miryanov]

An alternative could be to do this inside the garbage collector (and untrack the tuple if needed). Was that considered?

This is already doing in the GC (see untrack_tuples in gc.c). Also, you may be interested in #139389