diff --git a/docs/api_core.rst b/docs/api_core.rst index ccc2c89d..ea099342 100644 --- a/docs/api_core.rst +++ b/docs/api_core.rst @@ -2191,9 +2191,14 @@ Class binding .. cpp:function:: template class_ &def(init arg, const Extra &... extra) - Bind a constructor. The variable length `extra` parameter can be used to + Bind a C++ constructor that takes parameters of types ``Args...``. + The variable length `extra` parameter can be used to pass a docstring and other :ref:`function binding annotations - `. + `. You can also bind a custom constructor + (one that does not exist in the C++ code) by writing + ``.def(nb::init())``, provided the lambda returns an instance of + the class by value. If you need to wrap a factory function that returns + a pointer or shared pointer, see :cpp:struct:`nb::new_() ` instead. .. cpp:function:: template class_ &def(init_implicit arg, const Extra &... extra) @@ -2567,18 +2572,25 @@ Class binding constructor. It is only meant to be used in binding declarations done via :cpp:func:`class_::def()`. - Sometimes, it is necessary to bind constructors that don't exist in the - underlying C++ type (meaning that they are specific to the Python bindings). - Because `init` only works for existing C++ constructors, this requires - a manual workaround noting that - - .. code-block:: cpp - - nb::class_(m, "MyType") - .def(nb::init()); - - is syntax sugar for the following lower-level implementation using - "`placement new `_": + To bind a constructor that exists in the C++ class, taking ``Args...``, write + ``nb::init()``. + + To bind a constructor that is specific to the Python bindings (a + "custom constructor"), write ``nb::init()`` (write a + lambda expression or a function pointer inside the + parentheses). The function should return a prvalue of the bound + type, by ending with a statement like ``return MyType(some, + args);``. If you write a custom constructor in this way, then + nanobind can construct the object without any extra copies or + moves, and the object therefore doesn't need to be copyable or movable. + + If your custom constructor needs to take some actions after constructing + the C++ object, then nanobind recommends that you eschew + :cpp:struct:`nb::init() ` and instead bind an ``__init__`` method + directly. By convention, any nanobind method named ``"__init__"`` will + receive as its first argument a pointer to uninitialized storage that it + can initialize using `placement new + `_: .. code-block:: cpp @@ -2586,6 +2598,7 @@ Class binding .def("__init__", [](MyType* t, const char* arg0, int arg1) { new (t) MyType(arg0, arg1); + t->doSomething(); }); The provided lambda function will be called with a pointer to uninitialized @@ -2593,36 +2606,62 @@ Class binding with the Python object for reasons of efficiency). The lambda function can then either run an in-place constructor and return normally (in which case the instance is assumed to be correctly constructed) or fail by raising an - exception. + exception. If an exception is raised, nanobind assumes the object *was not* + constructed; in the above example, if ``doSomething()`` could throw, then you + would need to take care to call the destructor explicitly (``t->~MyType();``) + in case of an exception after the C++ constructor had completed. + + When binding a custom constructor using :cpp:struct:`nb::init() ` for + a type that supports :ref:`overriding virtual methods in Python + `, you must return either an instance of the trampoline + type (``PyPet`` in ``nb::class_(...)``) or something that + can initialize both the bound type and the trampoline type (e.g., + you can return a ``Pet`` if there exists a ``PyPet(Pet&&)`` constructor). + If that's not possible, you can alternatively write :cpp:struct:`nb::init() + ` with two function arguments instead of one. The first returns + an instance of the bound type (``Pet``), and will be called when constructing + an instance of the C++ class that has not been extended from Python. + The second returns an instance of the trampoline type (``PyPet``), + and will be called when constructing an instance that does need to consider + the possibility of Python-based virtual method overrides. + + .. note:: :cpp:struct:`nb::init() ` always creates Python ``__init__`` + methods, which construct a C++ object in already-allocated Python object + storage. If you need to wrap a constructor that performs its own + allocation, such as a factory function that returns a pointer, you must + use :cpp:struct:`nb::new_() ` instead in order to create a Python + ``__new__`` method. .. cpp:struct:: template init_implicit See :cpp:class:`init` for detail on binding constructors. The main - difference between :cpp:class:`init` and `init_implicit` is that the latter - only supports constructors taking a single argument `Arg`, and that it marks - the constructor as usable for implicit conversions from `Arg`. + difference between :cpp:class:`init` and `init_implicit` is that the latter + only supports constructors that exist in the C++ code and take a single + argument `Arg`, and that it marks the constructor as usable for implicit + conversions from `Arg`. Sometimes, it is necessary to bind implicit conversion-capable constructors that don't exist in the underlying C++ type (meaning that they are specific - to the Python bindings). This can be done manually noting that + to the Python bindings). This can be done manually, noting that .. code-block:: cpp - nb::class_(m, "MyType") - .def(nb::init_implicit()); + nb::class_(m, "MyType") + .def(nb::init_implicit()); can be replaced by the lower-level code .. code-block:: cpp nb::class_(m, "MyType") - .def("__init__", - [](MyType* t, const char* arg0) { - new (t) MyType(arg0); - }); + .def(nb::init()); nb::implicitly_convertible(); + and that this transformation works equally well if you use one of the forms + of :cpp:class:`nb::init() ` that cannot be expressed by + :cpp:class:`init_implicit`. + .. cpp:struct:: template new_ This is a small helper class that indicates to :cpp:func:`class_::def()` diff --git a/docs/changelog.rst b/docs/changelog.rst index 530a6b2a..e7761363 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -73,6 +73,17 @@ Version TBD (not yet released) binding abstractions that "feel like" the built-in ones. (PR `#884 `__) +- :cpp:struct:`nb::init() ` may now be written with a function argument + and no template parameters to express a custom constructor that doesn't exist + in C++. For example, you could use this to adapt Python strings to a + pointer-and-length argument convention: + ``.def(nb::init([](std::string_view sv) { return MyType(sv.data(), sv.size()); }))``. + This feature is syntactic sugar for writing a custom ``"__init__"`` binding + using placement new, which remains fully supported, and which should continue + to be used in cases where the custom constructor cannot be written as a + function that finishes by returning a prvalue (``return MyType(some, args);``). + (PR `#885 `__) + Version 2.4.0 (Dec 6, 2024) --------------------------- diff --git a/docs/classes.rst b/docs/classes.rst index fb6fc8ab..efe62875 100644 --- a/docs/classes.rst +++ b/docs/classes.rst @@ -543,7 +543,8 @@ propagated to Python: To fix this behavior, you must implement a *trampoline class*. A trampoline has the sole purpose of capturing virtual function calls in C++ and forwarding them -to Python. +to Python. (If you're reading nanobind's source code, you might see references +to an *alias class*; it's the same thing as a trampoline class.) .. code-block:: cpp diff --git a/docs/porting.rst b/docs/porting.rst index 832b7ae1..6b720ad0 100644 --- a/docs/porting.rst +++ b/docs/porting.rst @@ -146,30 +146,66 @@ accepts ``std::shared_ptr``. That means a C++ function that accepts a raw ``T*`` and calls ``shared_from_this()`` on it might stop working when ported from pybind11 to nanobind. You can solve this problem by always passing such objects across the Python/C++ boundary as -``std::shared_ptr`` rather than as ``T*``. See the :ref:`advanced section +``std::shared_ptr`` rather than as ``T*``, or by exposing all +constructors using :cpp:struct:`nb::new_() ` wrappers that +return ``std::shared_ptr``. See the :ref:`advanced section on object ownership ` for more details. Custom constructors ------------------- In pybind11, custom constructors (i.e. ones that do not already exist in the C++ class) could be specified as a lambda function returning an instance of -the desired type. +the desired type or a pointer to it. .. code-block:: cpp py::class_(m, "MyType") - .def(py::init([](int) { return MyType(...); })); + .def(py::init([](int) { return MyType(...); })) + .def(py::init([](std::string_view) { + return std::make_unique(...); + })); + +nanobind supports only the first form (where the lambda returns by value). Note +that thanks to C++17's guaranteed copy elision, it now works even for types that +are not copyable or movable, so you may be able to mechanically convert custom +constructors that return by pointer into those that return by value. + +.. note:: If *any* of your custom constructors still need to return a pointer or + smart pointer, perhaps because they wrap a C++ factory method that only + exposes those return types, you must switch *all* of them to use + :cpp:struct:`nb::new_() ` instead of :cpp:struct:`nb::init() `. + Be aware that :cpp:struct:`nb::new_() ` cannot construct in-place, so + using it gives up some of nanobind's performance benefits (but should still be + faster than ``py::init()`` in pybind11). It comes with some other caveats + as well, which are explained in the documentation on :ref:`customizing + Python object creation `. + +Guaranteed copy elision only works if the object is constructed as a temporary +directly within the ``return`` statement. If you need to do something to the +object before you return it, as in this example: -Unfortunately, the implementation of this feature was quite complex and -often required further internal calls to the move or copy -constructor. nanobind instead reverts to how pybind11 originally -implemented this feature using in-place construction (`placement -new `_): +.. code-block:: cpp + + py::class_(m, "MyType") + .def(py::init([](int value) { + auto ret = MyType(); + ret.value = value; + return ret; + })); + +then ``MyType`` must be movable, and depending on compiler optimizations the move +constructor might actually be called at runtime, which is more expensive than +in-place construction. In such cases, nanobind recommends instead that you +directly bind a ``__init__`` method using `placement new +`_: .. code-block:: cpp nb::class_(m, "MyType") - .def("__init__", [](MyType *t) { new (t) MyType(...); }); + .def("__init__", [](MyType *t, int value) { + auto* self = new (t) MyType(...); + self->value = value; + }); The provided lambda function will be called with a pointer to uninitialized memory that has already been allocated (this memory region is co-located @@ -178,15 +214,6 @@ then either run an in-place constructor and return normally (in which case the instance is assumed to be correctly constructed) or fail by raising an exception. -To turn an existing factory function into a constructor, you will need to -combine the above pattern with an invocation of the move/copy-constructor, -e.g.: - -.. code-block:: cpp - - nb::class_(m, "MyType") - .def("__init__", [](MyType *t) { new (t) MyType(MyType::create()); }); - Implicit conversions -------------------- diff --git a/include/nanobind/nb_class.h b/include/nanobind/nb_class.h index b40e7bb1..158756f1 100644 --- a/include/nanobind/nb_class.h +++ b/include/nanobind/nb_class.h @@ -278,6 +278,8 @@ struct is_copy_constructible : std::is_copy_constructible { }; template constexpr bool is_copy_constructible_v = is_copy_constructible::value; +struct init_using_factory_tag {}; + NAMESPACE_END(detail) // Low level access to nanobind type objects @@ -365,6 +367,106 @@ template struct init : def_visitor> { } }; +template +struct init { + static_assert(sizeof...(Args) == 2 || sizeof...(Args) == 4, + "Unexpected instantiation convention for factory init"); + static_assert(sizeof...(Args) != 2, + "Couldn't deduce function signature for factory function"); + static_assert(sizeof...(Args) != 4, + "Base factory and alias factory accept different arguments, " + "or we couldn't otherwise deduce their signatures"); +}; + +template +struct init + : def_visitor> { + std::remove_reference_t func; + + init(Func &&f) : func((detail::forward_t) f) {} + + template + NB_INLINE void execute(Class &cl, const Extra&... extra) { + using Type = typename Class::Type; + using Alias = typename Class::Alias; + if constexpr (std::is_same_v) { + static_assert(std::is_constructible_v, + "nb::init() factory function must return an instance " + "of the type by value, or something that can " + "direct-initialize it"); + } else { + static_assert(std::is_constructible_v, + "nb::init() factory function must return an instance " + "of the alias type by value, or something that can " + "direct-initialize it"); + } + cl.def( + "__init__", + [func_ = (detail::forward_t) func](pointer_and_handle v, Args... args) { + if constexpr (!std::is_same_v && + std::is_constructible_v) { + if (!detail::nb_inst_python_derived(v.h.ptr())) { + new (v.p) Type{ func_((detail::forward_t) args...) }; + return; + } + } + new ((void *) v.p) Alias{ func_((detail::forward_t) args...) }; + }, + extra...); + } +}; + +template +struct init + : def_visitor> { + std::remove_reference_t cfunc; + std::remove_reference_t afunc; + + init(CFunc &&cf, AFunc &&af) + : cfunc((detail::forward_t) cf), + afunc((detail::forward_t) af) {} + + template + NB_INLINE void execute(Class &cl, const Extra&... extra) { + using Type = typename Class::Type; + using Alias = typename Class::Alias; + static_assert(!std::is_same_v, + "The form of nb::init() that takes two factory functions " + "doesn't make sense to use on classes that don't have an " + "alias type"); + static_assert(std::is_constructible_v, + "nb::init() first factory function must return an " + "instance of the type by value, or something that can " + "direct-initialize it"); + static_assert(std::is_constructible_v, + "nb::init() second factory function must return an " + "instance of the alias type by value, or something that " + "can direct-initialize it"); + cl.def( + "__init__", + [cfunc_ = (detail::forward_t) cfunc, + afunc_ = (detail::forward_t) afunc](pointer_and_handle v, Args... args) { + if (!detail::nb_inst_python_derived(v.h.ptr())) + new (v.p) Type{ cfunc_((detail::forward_t) args...) }; + else + new ((void *) v.p) Alias{ afunc_((detail::forward_t) args...) }; + }, + extra...); + } +}; + +template +init(Func&& f) -> init>; + +template +init(CFunc&& cf, AFunc&& af) -> init, + AFunc, detail::function_signature_t>; + template struct init_implicit : def_visitor> { template friend class class_; NB_INLINE init_implicit() { } diff --git a/tests/test_classes.cpp b/tests/test_classes.cpp index 8a479e6f..3cbe606a 100644 --- a/tests/test_classes.cpp +++ b/tests/test_classes.cpp @@ -3,12 +3,14 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include #include #include @@ -149,6 +151,13 @@ NB_MODULE(test_classes_ext, m) { auto cls = nb::class_(m, "Struct", "Some documentation") .def(nb::init<>()) .def(nb::init()) + .def(nb::init([](std::string_view sv) { + int value; + auto [ptr, ec] = std::from_chars(sv.data(), sv.data() + sv.size(), + value); + if (ec != std::errc()) throw nb::value_error("invalid integer"); + return Struct(value); + })) .def("value", &Struct::value) .def("set_value", &Struct::set_value, "value"_a) .def("self", &Struct::self, nb::rv_policy::none) @@ -244,6 +253,7 @@ NB_MODULE(test_classes_ext, m) { NB_TRAMPOLINE(Dog, 2); PyDog(const std::string &s) : Dog(s) { } + PyDog(Dog&& dog) : Dog(std::move(dog)) { } std::string name() const override { NB_OVERRIDE(name); @@ -267,11 +277,18 @@ NB_MODULE(test_classes_ext, m) { auto animal = nb::class_(m, "Animal") .def(nb::init<>(), "A constructor") + .def(nb::init([](std::nullptr_t) { return PyAnimal(); }), + nb::arg().none()) .def("name", &Animal::name, "A method") .def("what", &Animal::what); nb::class_(m, "Dog") - .def(nb::init()); + .def(nb::init()) + .def(nb::init([](nb::bytes b) { + return Dog(std::string(b.c_str(), b.size())); + })) + .def(nb::init([](int i) { return Dog(std::string(i, '.')); }, + [](int i) { return Dog(std::string(i, '_')); })); nb::class_(m, "Cat", animal) .def(nb::init()); diff --git a/tests/test_classes.py b/tests/test_classes.py index a34fa65a..118104b8 100644 --- a/tests/test_classes.py +++ b/tests/test_classes.py @@ -38,7 +38,8 @@ def assert_stats(**kwargs): def test01_signature(): assert t.Struct.__init__.__doc__ == ( - "__init__(self) -> None\n" "__init__(self, arg: int, /) -> None" + "__init__(self) -> None\n" "__init__(self, arg: int, /) -> None\n" + "__init__(self, arg: str, /) -> None" ) assert t.Struct.value.__doc__ == "value(self) -> int" @@ -61,9 +62,14 @@ def test03_instantiate(clean): assert s1.value() == 5 s2 = t.Struct(10) assert s2.value() == 10 + s3 = t.Struct("15") + assert s3.value() == 15 + with pytest.raises(ValueError, match="invalid integer"): + t.Struct("xyzzy") del s1 del s2 - assert_stats(default_constructed=1, value_constructed=1, destructed=2) + del s3 + assert_stats(default_constructed=1, value_constructed=2, destructed=3) def test04_double_init(): @@ -148,6 +154,13 @@ def test08_inheritance(): assert t.go(dog) == "Dog says woof" assert t.go(cat) == "Cat says meow" + dog2 = t.Dog(b"arf") + dog3 = t.Dog(4) + assert dog2.what() == "arf" + assert dog3.what() == "...." + assert t.go(dog2) == "Dog says arf" + assert t.go(dog3) == "Dog says ...." + assert t.animal_passthrough(dog) is dog assert t.animal_passthrough(cat) is cat assert t.dog_passthrough(dog) is dog @@ -178,8 +191,8 @@ def test10_trampoline(clean): for _ in range(10): class Dachshund(t.Animal): - def __init__(self): - super().__init__() + def __init__(self, *args): + super().__init__(*args) def name(self): return "Dachshund" @@ -193,6 +206,13 @@ def what(self): assert t.animal_passthrough(d) is d + # custom constructor that returns alias type + d = Dachshund(None) + for _ in range(10): + assert t.go(d) == "Dachshund says yap" + + assert t.animal_passthrough(d) is d + a = 0 class GenericAnimal(t.Animal): @@ -215,15 +235,27 @@ def name(self): del ga del d - assert_stats(default_constructed=11, destructed=11) + assert_stats(default_constructed=21, destructed=21) class GenericDog(t.Dog): - pass + def name(self): + return "GenericDog" - d = GenericDog("GenericDog") + # built-in constructor + d = GenericDog("doggo") + assert t.go(d) == "GenericDog says doggo" assert t.dog_passthrough(d) is d assert t.animal_passthrough(d) is d + # custom constructor that returns bound type, which can initialize alias type + d = GenericDog(b"woofer") + assert t.go(d) == "GenericDog says woofer" + + # custom constructor with variants for bound type vs alias type + # (selecting the bound type variant was tested in test08) + d = GenericDog(6) + assert t.go(d) == "GenericDog says ______" + def test11_trampoline_failures(): class Incomplete(t.Animal): diff --git a/tests/test_classes_ext.pyi.ref b/tests/test_classes_ext.pyi.ref index c7bd2368..96e2bdcd 100644 --- a/tests/test_classes_ext.pyi.ref +++ b/tests/test_classes_ext.pyi.ref @@ -5,9 +5,13 @@ class A: def __init__(self, arg: int, /) -> None: ... class Animal: + @overload def __init__(self) -> None: """A constructor""" + @overload + def __init__(self, arg: None | None) -> None: ... + def name(self) -> str: """A method""" @@ -88,8 +92,15 @@ class D: def value(self, arg: int, /) -> None: ... class Dog(Animal): + @overload def __init__(self, arg: str, /) -> None: ... + @overload + def __init__(self, arg: bytes, /) -> None: ... + + @overload + def __init__(self, arg: int, /) -> None: ... + class FinalType: def __init__(self) -> None: ... @@ -200,6 +211,9 @@ class Struct: @overload def __init__(self, arg: int, /) -> None: ... + @overload + def __init__(self, arg: str, /) -> None: ... + def value(self) -> int: ... def set_value(self, value: int) -> None: ...