Python thread state design

[emmatyping]

To ensure access to Python is done soundly, we need to ensure that any code that interacts with the Python interpreter has an attached thread state (unless otherwise allowed). c.f. https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock

PyO3's Python<'py> token seems to model this pretty well, though it is not perfectly sound. I'm not sure if there is a perfectly sound design available today.

Until auto traits and negative traits allow expressing !Ungil, perhaps we can require types in CPython to unsafe impl manually to ensure that the invariants are upheld?

[davidhewitt]

PyO3/pyo3#3640 - even with Ungil auto trait the current design cannot be made fully sound.

Possible other design concerns here include the interplay with internal APIs which want the thread state passed as an argument, the PyO3 Python<'py> token is ZST.

My comment about hypothetical "contexts" in #5 (comment) is still the best idea I have about what a sound long-term solution might be (probably because it's hypothetical and I can pretend contexts have no limitations).

In general I think it'd be very interesting to present this problem to the Rust team and see what their ideas are.

[emmatyping]

In general I think it'd be very interesting to present this problem to the Rust team and see what their ideas are.

Agreed! I've added this to the agenda, I think it would be good to discuss this at the next meeting.