nb::nnarray and []-operator?

[ChernyshevT]

Hi. I'm currently porting my project from pybind11 to nanobind and I'm trying to keep API for existing python code.
While working with pybind I used following statement to work with my custom class as a normal numpy array:

	.def("__getitem__", [] (vcache_holder &self, py::handle index) {
		return self.buffer_h[index];
	})
	.def("__setitem__", [] (vcache_holder &self, py::handle index, py::handle val) {
		self.buffer_h[index] = val;
	})

Here, vcache_holder is my class, and buffer_h is a memoryview for internal buffer.
As nanobind doen's have []-operator for arrays, I'm curious is it possible to achieve the same behavior through some hack? I can not use ()-operator because the number of arguments isn't known in compile time and it could be a slice.

-- WBR, Timofey C.

[wjakob]

So what's wrong with the snippet? Maybe I am just confused, but that should work, no?

[wjakob]

ah, you are saying that self.buffer_h doesn't override the [] operator?

[ChernyshevT]

Yes. I tried to do a little hack by exposing the internal buffer:

	.def_prop_ro("view", [] (const vcache_holder &self) -> nb::ndarray<nb::numpy> {
		return self.buffer_h;
	})
	
	.def("__getitem__", [] (const vcache_holder &self, nb::handle arg) -> nb::handle {
		return nb::cast(self).attr("view").attr("__getitem__")(arg);
	})

But it causes runtime fail as my class is not copyable and cast causes copy-ctor call.

[vshulcz]

The copy isn't coming from your design, it's the default policy of nb::cast. Its signature is

template <typename T> object cast(T &&value, rv_policy policy = rv_policy::automatic_reference)

and automatic_reference behaves like automatic except for pointers, where automatic is documented as falling back to "take_ownership when the return value is a pointer, move when it is a rvalue reference, and copy when it is a lvalue reference". You pass an lvalue reference, so you get copy, hence the call to a copy constructor you don't have.

What you want is the Python object that already exists for that instance, and nb::find returns exactly that without creating anything:

.def_prop_ro("view",
     [](vcache_holder &self) { return self.buffer_h; },
     nb::rv_policy::reference_internal)

.def("__getitem__", [](vcache_holder &self, nb::handle index) {
    return nb::find(self).attr("view").attr("__getitem__")(index);
})

.def("__setitem__", [](vcache_holder &self, nb::handle index, nb::handle value) {
    nb::find(self).attr("view").attr("__setitem__")(index, value);
})

find is template <typename T> object find(const T &value) noexcept and returns an invalid object when the instance has no Python counterpart. Inside a method called from Python that can't happen, so the lookup always succeeds here.

Keep reference_internal on the property. It is what makes nanobind hold a reference to self for as long as the returned array lives, which is the same pattern the ndarray docs use for exposing an internal buffer:

.def("view",
     [](Matrix4f &m){ return Array(m.data); },
     nb::rv_policy::reference_internal);

Without it the NumPy array points into storage that the C++ object may have already freed.

One note on cost: every __getitem__ goes through an attribute lookup and rebuilds the array wrapper. If the buffer's address and shape never change, cache the view as an nb::object member on first use and index that instead, and you avoid both.

[ChernyshevT]

Thanks! It seems like nb::find works.