Skip to content

Add fast_type_map, use it authoritatively for local types and as a hint for global types (ABI breaking) #5842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Oct 5, 2025
+91 −16
Merged

Add fast_type_map, use it authoritatively for local types and as a hint for global types (ABI breaking) #5842

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
cc0b053
Add fast_type_map, use it authoritatively for local types and as a hi…
swolchok Sep 16, 2025
1cd2b55
simplify unsafe_reset_local_internals in test
swolchok Sep 17, 2025
abf237f
pre-implement PYBIND11_INTERNALS_VERSION 12
swolchok Sep 17, 2025
b87e762
Merge branch 'master' into fast-local-maps
swolchok Sep 26, 2025
c333e85
use PYBIND11_INTERNALS_VERSION 12 on Python 3.14 per suggestion
swolchok Sep 26, 2025
8651580
Merge branch 'master' into fast-local-maps
swolchok Sep 29, 2025
3ca29e8
Implement reviewer comments: revert PY_VERSION_HEX change, fix REVIEW…
swolchok Sep 30, 2025
48f831b
Use the inplace build to smoke test ABI bump?
swolchok Sep 30, 2025
08c09ae
[skip ci] Remove "smoke" from comment. This is full testing, just onl…
rwgk Oct 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion include/pybind11/detail/class.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,10 +221,14 @@ extern "C" inline void pybind11_meta_dealloc(PyObject *obj) {
auto tindex = std::type_index(*tinfo->cpptype);
internals.direct_conversions.erase(tindex);

auto &local_internals = get_local_internals();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and in the similar logic for registration, you're fetching the local internals even if you don't need them. Possibly a regression due to the history where the fast type map was in local internals even for non-local types?

if (tinfo->module_local) {
get_local_internals().registered_types_cpp.erase(tindex);
local_internals.registered_types_cpp.erase(tinfo->cpptype);
} else {
internals.registered_types_cpp.erase(tindex);
#if PYBIND11_INTERNALS_VERSION >= 12
internals.registered_types_cpp_fast.erase(tinfo->cpptype);
#endif
}
internals.registered_types_py.erase(tinfo->type);

Expand Down
31 changes: 28 additions & 3 deletions include/pybind11/detail/internals.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,13 @@
/// further ABI-incompatible changes may be made before the ABI is officially
/// changed to the new version.
#ifndef PYBIND11_INTERNALS_VERSION
// REMINDER for next version bump: remove loader_life_support_tls
# define PYBIND11_INTERNALS_VERSION 11
# if PY_VERSION_HEX >= 0x030E0000
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I just realized this a bad idea and we shouldn't do this (although we've actually done this before and I'm not aware of complaints):

E.g. if someone builds extensions with Python 3.14 and pybind11 v3.0.0 or v3.0.1 (internals version 11), then other extensions with Python 3.14 and pybind11>3.0.1 (assuming internals version 12), they will not fully interoperate.

Preemptively bumping the internals version for Python 3.14 would be OK only if v3.0.0 or v3.0.1 didn't work with Python 3.14, but I believe they do (although I'm not sure, but I don't have time to find out for sure).

To avoid surprising users, could you please undo the PY_VERSION_HEX >= 0x030E0000 change here?

We need to make a couple surgical changes to ci.yml to get test coverage. See my older comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I took a guess and used the inplace build for the smoke test

// Get test coverage for ABI version 12 without breaking existing
// Python versions.
# define PYBIND11_INTERNALS_VERSION 12
# else
# define PYBIND11_INTERNALS_VERSION 11
# endif
#endif

#if PYBIND11_INTERNALS_VERSION < 11
Expand Down Expand Up @@ -177,6 +182,12 @@ struct type_equal_to {
};
#endif

template <typename value_type>
// REVIEW: do we need to add a fancy hash for pointers or is the
// possible identity hash function from the standard library (e.g.,
// libstdc++) sufficient?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that the Linux box I used for benchmarking has libstdc++.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to ack that I've seen this, but I don't know.

It'd be good to rewrite the comment before merging if nobody knows.

using fast_type_map = std::unordered_map<const std::type_info *, value_type>;

template <typename value_type>
using type_map = std::unordered_map<std::type_index, value_type, type_hash, type_equal_to>;

Expand Down Expand Up @@ -237,6 +248,14 @@ struct internals {
pymutex mutex;
pymutex exception_translator_mutex;
#endif
#if PYBIND11_INTERNALS_VERSION >= 12
// non-normative but fast "hint" for
// registered_types_cpp. Successful lookups are correct;
// unsuccessful lookups need to try registered_types_cpp and then
// backfill this map if they find anything.
fast_type_map<type_info *> registered_types_cpp_fast;
#endif

// std::type_index -> pybind11's type information
type_map<type_info *> registered_types_cpp;
// PyTypeObject* -> base type_info(s)
Expand All @@ -261,7 +280,9 @@ struct internals {
PyObject *instance_base = nullptr;
// Unused if PYBIND11_SIMPLE_GIL_MANAGEMENT is defined:
thread_specific_storage<PyThreadState> tstate;
#if PYBIND11_INTERNALS_VERSION <= 11
thread_specific_storage<loader_life_support> loader_life_support_tls; // OBSOLETE (PR #5830)
#endif
// Unused if PYBIND11_SIMPLE_GIL_MANAGEMENT is defined:
PyInterpreterState *istate = nullptr;

Expand Down Expand Up @@ -302,7 +323,11 @@ struct internals {
// impact any other modules, because the only things accessing the local internals is the
// module that contains them.
struct local_internals {
type_map<type_info *> registered_types_cpp;
// It should be safe to use fast_type_map here because this entire
// data structure is scoped to our single module, and thus a single
// DSO and single instance of type_info for any particular type.
fast_type_map<type_info *> registered_types_cpp;

std::forward_list<ExceptionTranslator> registered_exception_translators;
PyTypeObject *function_record_py_type = nullptr;
};
Expand Down
30 changes: 24 additions & 6 deletions include/pybind11/detail/type_caster_base.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,21 +205,39 @@ PYBIND11_NOINLINE detail::type_info *get_type_info(PyTypeObject *type) {
return bases.front();
}

inline detail::type_info *get_local_type_info(const std::type_index &tp) {
auto &locals = get_local_internals().registered_types_cpp;
auto it = locals.find(tp);
inline detail::type_info *get_local_type_info(const std::type_info &tp) {
const auto &locals = get_local_internals().registered_types_cpp;
auto it = locals.find(&tp);
if (it != locals.end()) {
return it->second;
}
return nullptr;
}

inline detail::type_info *get_global_type_info(const std::type_index &tp) {
inline detail::type_info *get_global_type_info(const std::type_info &tp) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add a comment — somewhere between terse and concise — to explain that this function implements a two-level lookup? Maybe end with "For more details see PR #5842."

(I'd add the comment inside the function, at the top.)

Ideally make it play nicely with the comment you're adding in internals.h:

#if PYBIND11_INTERNALS_VERSION >= 12
    // non-normative but fast "hint" for
    // registered_types_cpp. Successful lookups are correct;
    // unsuccessful lookups need to try registered_types_cpp and then
    // backfill this map if they find anything.
    fast_type_map<type_info *> registered_types_cpp_fast;
#endif

Mentioning "two-level lookup" there, too, would be nice.

return with_internals([&](internals &internals) {
detail::type_info *type_info = nullptr;
#if PYBIND11_INTERNALS_VERSION >= 12
auto &fast_types = internals.registered_types_cpp_fast;
#endif
auto &types = internals.registered_types_cpp;
auto it = types.find(tp);
#if PYBIND11_INTERNALS_VERSION >= 12
auto fast_it = fast_types.find(&tp);
if (fast_it != fast_types.end()) {
# ifndef NDEBUG
auto types_it = types.find(std::type_index(tp));
assert(types_it != types.end());
assert(types_it->second == fast_it->second);
# endif
return fast_it->second;
}
#endif // PYBIND11_INTERNALS_VERSION >= 12

auto it = types.find(std::type_index(tp));
if (it != types.end()) {
#if PYBIND11_INTERNALS_VERSION >= 12
fast_types.emplace(&tp, it->second);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are you going to cause this entry to be cleaned up when the type it points to is destroyed? Currently I think it will just dangle, which won't go well when you try to use the result.

nanobind solves this problem by having its equivalent of detail::type_info store a list of all the distinct std::type_info pointers that have been seen for the same type. This list is empty except in cross-DSO situations, which are relatively rare, so it's not very expensive to store. Destruction of the type iterates over the list and removes each alternate pointer from the fast type map.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I follow. Why does the fast map have different cleanup timing requirements than the slow map? We clean up both at the same time.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The slow map has one entry per C++ type. The fast map potentially has multiple. You’re cleaning up the fast map entry that was added by the module that defined the binding, but not the ones that may have been added by other DSOs (with different typeinfo pointers for that type) when they looked it up. I left this comment on the location in the code where those secondary entries are being added; I don’t see anywhere where they can be removed.

Copy link
Contributor Author

@swolchok swolchok Oct 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t see anywhere where they can be removed.

Naive solution: when a type is deallocated, can't we just loop over the whole map and remove all entries with the same value as the one we want to remove?
Obvious problem: this would cause destruction of N types to take O(N^2) time in all cases.

The nanobind solution seems fine, but the first thing that needs to be done is writing a test that will expose this problem, since we clearly don't have one (or it's not a problem for some reason).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not 100% clear on how to write a test that will result in classes being destroyed, but it looks like test_class does it by registering into temporary modules so I'll try that. I think we want something like:

  1. DSO1 registers a type, such as by py::class_
  2. DSO2 looks up the same type (as defined by type name matching), such as by returning an instance of that type from a pybound function
  3. DSO1 goes away
  4. repeat (2)

theory is that currently we will see a crash; before this PR it should've worked?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your summary looks good to me, except that Python can't really unload a whole DSO. Instead, you can del module.TheType from Python. As long as there are no other references to the type (such as via its instances), it will drop to zero references and be deallocated. Note that types are almost always in cycles, so you'll need a gc.collect() or two.

You can steal this function from the tests for #5800:

def delattr_and_ensure_destroyed(*specs):
    wrs = []
    for mod, name in specs:
        wrs.append(weakref.ref(getattr(mod, name)))
        delattr(mod, name)

    for _ in range(5):
        gc.collect()
        if all(wr() is None for wr in wrs):
            break
    else:
        pytest.fail(
            f"Could not delete bindings such as {next(wr for wr in wrs if wr() is not None)!r}"
        )

After the type has been destroyed, you can bind it again with a new py::class_ call.

Be aware that extension types are immortal on non-CPython VMs (PyPy, GraalPy) and all types are immortal on free-threading CPython 3.13 (3.14+ have proper GC for them), so you'll want to disable the test for those platforms.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I will try to get the tests + fix up today.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sent #5867

#endif
type_info = it->second;
}
return type_info;
Expand All @@ -228,7 +246,7 @@ inline detail::type_info *get_global_type_info(const std::type_index &tp) {

/// Return the type info for a given C++ type; on lookup failure can either throw or return
/// nullptr.
PYBIND11_NOINLINE detail::type_info *get_type_info(const std::type_index &tp,
PYBIND11_NOINLINE detail::type_info *get_type_info(const std::type_info &tp,
bool throw_if_missing = false) {
if (auto *ltype = get_local_type_info(tp)) {
return ltype;
Expand Down
22 changes: 17 additions & 5 deletions include/pybind11/pybind11.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1637,10 +1637,14 @@ class generic_type : public object {
with_internals([&](internals &internals) {
auto tindex = std::type_index(*rec.type);
tinfo->direct_conversions = &internals.direct_conversions[tindex];
auto &local_internals = get_local_internals();
if (rec.module_local) {
get_local_internals().registered_types_cpp[tindex] = tinfo;
local_internals.registered_types_cpp[rec.type] = tinfo;
} else {
internals.registered_types_cpp[tindex] = tinfo;
#if PYBIND11_INTERNALS_VERSION >= 12
internals.registered_types_cpp_fast[rec.type] = tinfo;
#endif
}

PYBIND11_WARNING_PUSH
Expand Down Expand Up @@ -2138,10 +2142,18 @@ class class_ : public detail::generic_type {

if (has_alias) {
with_internals([&](internals &internals) {
auto &instances = record.module_local ? get_local_internals().registered_types_cpp
: internals.registered_types_cpp;
instances[std::type_index(typeid(type_alias))]
= instances[std::type_index(typeid(type))];
auto &local_internals = get_local_internals();
if (record.module_local) {
local_internals.registered_types_cpp[&typeid(type_alias)]
= local_internals.registered_types_cpp[&typeid(type)];
} else {
type_info *const val
= internals.registered_types_cpp[std::type_index(typeid(type))];
internals.registered_types_cpp[std::type_index(typeid(type_alias))] = val;
#if PYBIND11_INTERNALS_VERSION >= 12
internals.registered_types_cpp_fast[&typeid(type_alias)] = val;
#endif
}
});
}
def("_pybind11_conduit_v1_", cpp_conduit_method);
Expand Down
17 changes: 16 additions & 1 deletion tests/test_with_catch/external_module.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,26 @@ namespace py = pybind11;
* modules aren't preserved over a finalize/initialize.
*/

namespace {
// Compare unsafe_reset_internals_for_single_interpreter in
// test_subinterpreter.cpp.
void unsafe_reset_local_internals() {
// NOTE: This code is NOT SAFE unless the caller guarantees no other threads are alive
// NOTE: This code is tied to the precise implementation of the internals holder

py::detail::get_local_internals_pp_manager().unref();
py::detail::get_local_internals();
}
} // namespace

PYBIND11_MODULE(external_module,
m,
py::mod_gil_not_used(),
py::multiple_interpreters::per_interpreter_gil()) {

// At least one test ("Single Subinterpreter") wants to reset
// internals. We have separate local internals because we are a
// separate DSO, so ours need to be reset too!
unsafe_reset_local_internals();
class A {
public:
explicit A(int value) : v{value} {};
Expand Down
Loading