Do more traits need to be implemented for pyo3 types

[AdamWRichardson]

Firstly thanks for all the great work on pyo3! I'm still quite new to using it but so far it's been very useful for my projects. I've come across the following situation a couple of time however and I'm not sure of the best way forward.

The context is that I'm building a library in rust and want to expose some of the internals to python, that way it's not purely a rust or python library which I'm sure is a very common thing to do. In the rust library I have a generic struct which implements a trait and I thought the best way to expose this to python would be through the new type pattern - the code is analogous to the following:

struct Struct<A>;

impl<A> Trait for Struct<A> 
where
    A: Default
{
...
}

#[pyclass]
struct PyStruct(Struct<PyInt>);

#[pymethods]
impl Trait for PyStruct {
...
// Just re-expose methods through self.0
...
}

My first question is: is this the best way to do this? And then secondly this doesn't compile because Default is not implemented for many (if not all) types in pyo3. There are other similar problems to this, the generic for Struct is basically over the keys and values in an internal HashMap so I require the key to implement Hash which again is not something that is super obvious how to do. Could anyone help please?!

[davidhewitt]

The problem with a lot of Rust traits is that they assume infallibility, whereas basically all Python code can fail at runtime. This rules out Default, Hash, Eq, and basically everything else which can't fail from being implemented by Python types.

For the case of Hash, I've commonly seen it where you can pre-compute the hash and store that in a wrapper type:

struct HashedObject { obj: PyObject, hash: usize }

impl HashedObject {
    fn new(obj: &PyAny) -> PyResult<Self> {
        Ok(Self { obj: obj.into(), hash: obj.hash()? as usize })
    }
}

impl Hash for HashedObject {
    fn hash<H: Hasher>(&self, state: &mut H) {
        self.hash.hash(state);
    }
}

[AdamWRichardson]

Hi David, thanks for your reply, is there any way to redirect a call to __hash__ to the rust hash method? Also out of curiosity what happens if we did allow this to fail at runtime?

What do you see as the future for features like this? Are you happy with the init computed hash method or would you like to do something differently? Sorry for asking so many questions, I'm keen to help if I can but as pyo3 is new to me I'd like to contribute in the best way possible!

[davidhewitt]

is there any way to redirect a call to hash to the rust hash method

Do you mean you have a #[pyclass] and you want to implement __hash__ for it, or you mean you want impl Hash for MyStruct to call Python __hash__?

If a Rust method is infallible but the underlying Python implementation fails then the only option is to panic (or abort), neither of which are desirable operations in a library. Hence this is unlikely to change in the future until Rust's traits have the option of faillibility, I suspect this is many years off pending the outcome of the "keyword generics initiative".

[AdamWRichardson]

Do you mean you have a #[pyclass] and you want to implement hash for it, or you mean you want impl Hash for MyStruct to call Python hash?

I more meant having #[pyclass] implementing __hash__ and just returning the the hash required by the trait bound (and I'm sure that's possible to do already)

Could you use the dynamic dispatch mechanisms to get around the infallibility? The above example is obviously statically dispatched so the types are necessary at compile time and I agree that requires them to be infallible but since python is dynamically dispatched anyway I certainly wouldn't mind paying that cost for the rust - python interface boundary.

[davidhewitt]

Sorry for falling off this thread. I think what you're looking for is like what I tried in #3195, but for Hash - I'd like to revisit that one day but my first priority at the moment is #3382

[AdamWRichardson]

Interesting, that seems to be similar to what I'd like, generally the interplay between traits and ABC methods isn't clear to me but I'll spend some more time in the library and maybe help out on the pinned issue if I can. Thank you