Performance regression in PySet_Add() with 3.14t

[yoney]

In the fbthrift-python benchmarks, we observed a performance regression when comparing Python 3.14t to 3.14, particularly during Thrift struct initialization and deserialization. In our tests, we populated sets with 1000 elements. The regression was linked to the use of the PySet_Add() API for building set and frozenset objects, where a critical section is always applied—even when the set is uniquely referenced during initialization.

Linked PRs

• gh-140476: Optimize PySet_Add() for frozenset in free-threading #140440

[kumaraditya303]

I am not sure that it is correct to remove critical section from such public API.

Considering a case of a wrapper object which stores data in an underlying set, while the set will always have refcount of 1 but the wrapper can be shared among multiple threads. If the wrapper object tries to modify the set from multiple threads then it can lead to data races with the owning thread which would avoids critical section and non-owning threads which would use critical section.

[yoney]

I am not sure that it is correct to remove critical section from such public API.

Considering a case of a wrapper object which stores data in an underlying set, while the set will always have refcount of 1 but the wrapper can be shared among multiple threads. If the wrapper object tries to modify the set from multiple threads then it can lead to data races with the owning thread which would avoids critical section and non-owning threads which would use critical section.

I was considering/testing a similar scenario for set, but I’m wondering if it also applies to frozenset. The API documentation mentions a special condition

... like PyTuple_SetItem() it can be used to fill in the values of brand new frozensets before they are exposed to other code ...

[kumaraditya303]

The API documentation mentions a special condition

Yes, PySet_Add can be used to add elements to frozenset initially, it is user's responsibility that the frozen set is not exposed and is local so optimizing that is safe.