Drop PyRef and get associated PyCell?

[cspotcode]

I am writing a pyclass where most methods accept PyRefMut or PyRef. These methods can all be called from python, they call each other in rust, and must call out to python functions, passing a reference to self, expecting self's attributes to be accessed from those functions. So I cannot accept &self nor &mut self in these methods; I need to use PyCell, PyRef, or PyRefMut. And I cannot hold a borrow of self when calling out to python functions, because they will attempt to re-borrow when they access attributes.

If each method accepts &PyCell then each must repeatedly borrow. If method A calls B calls C, each is doing a borrow. I'm fairly new to rust, but to me, this indicates I should pass a PyRef from A to B to C to avoid repeated borrowing and un-borrowing.

Before calling from rust to python, I need to drop my PyRef to avoid an "already borrowed" error.

Is there a way to drop a PyRef and get a ref to the PyCell at the same time?

// Assume self_: PyRefMut<MyClass>
let cell: &PyCell<MyClass> = self_.release_borrow_and_get_pycell();
other_python_object.call_method1(py, "foo", (cell,))?;

I tried to keep my question as brief as possible but I can add more detail about my use-case if it helps.

[adamreichold]

Is there a way to drop a PyRef and get a ref to the PyCell at the same time?

I don't think we have such a method. The best you can do is get a Py<T> back from the PyRef<T> via a From impl.

I tried to keep my question as brief as possible but I can add more detail about my use-case if it helps.

I am not sure that the short-lived borrows can be avoided, but I think discussing the whole problem could only help to figure out if there are any alternative solutions.

One slightly different approach might be to avoid PyCell completely using a frozen #[pyclass] and use a more fine-grained approach to interior mutability, i.e. all your methods take &self and your individual fields implement interior mutability only as required. This will not necessarily lead to longer held borrows, but it might lead to less overlap or fewer dynamically checked borrows overall depending on the shape of your data.

[cspotcode]

So having a PyRef always implies having a GIL token, i.e. you can just get the token before calling into the From impl.

Thanks, I might be doing something wrong here, I get an error on Py::from( saying "cannot move out of self_ because it is borrowed"

fn set_position(mut self_: PyRefMut<'_, BasicSprite>) {
  let py = self_.py();
  let s = Py::from(self_);
  sprite_list.call_method1(py, intern!(py, "_update_position"), (&s,))?;

I think from wants to consume self_, yet self_.py() has borrowed it and the borrow is used when passing py into call_method1.

[adamreichold]

I think from wants to consume self_, yet self_.py() has borrowed it and the borrow is used when passing py into call_method1.

I think this is a bug on our part as the returned token is limited to the borrow of the PyRef instead of the lifetime for which PyRef itself borrows the underlying cell. However, just adding a GIL token py: Python<'_> to your argument list should not have any computational cost.

[cspotcode]

Point taken about benchmarking versus guessing. I think you're correct about option (a).

[adamreichold]

I think this is a bug on our part

c.f. #3131

[cspotcode]

I think this is a bug on our part as the returned token is limited to the borrow of the PyRef instead of the lifetime for which PyRef itself borrows the underlying cell. However, just adding a GIL token py: Python<'_> to your argument list should not have any computational cost.

Ok, I'm happy to leave it at that: we will accept and pass around a py: Python<'_> argument. It would be ergonomically nice to avoid passing around py since the self_: PyRef already implies having the GIL, but it's not a big deal.