What's the situation with PyBuffer::as_slice, ReadOnlyCell, Python::detach, and freethreaded builds?

[oconnor663]

As I understand it, the PyBuffer::as_slice method returns &[ReadOnlyCell<T>] mainly to reflect the fact that the current thread could modify the slice through an alias (or just by executing arbitrary Python code). Since &[ReadOnlyCell<T>] is not Send (because ReadOnlyCell is not Sync), we can't pass that type to a Python::detach closure, so we don't have to worry about other threads doing the same thing and provoking a true data race... (Edit: @alex points out below that we've always had to worry about other threads using unsafe code or just C code to do this.)

...except that on freethreaded builds we do have to worry about that, right, whether or not we call detach? I could have this wrong, but it seems to me like at least on freethreaded builds, PyBuffer::as_slice is morally similar to memmap2::Mmap::map, in that it is fundamentally impossible to wrap these functions in a safe API, and you ultimately have to trust the caller not to give these "weird" slices to callers that "care" about the difference. (Where "caring" about the difference is a nebulous, dangerous concept that goes against the grain of all the relevant memory models and standards?)

What's the current thinking on this question, particularly for freethreaded Python? Are there any other discussions I might've missed besides #2824 ?

(The context that got me thinking about this recently is oconnor663/blake3-py#89.)

[alex]

No, dataraces are an issue even without free threading: https://alexgaynor.net/2022/oct/23/buffers-on-the-edge/

[oconnor663]

Are you referring to writes coming from C extensions, or from other Rust extensions that use unsafe code to get one of these slices into a detach closure? I wonder if there's still a new distinction here. Before free-threading: It's impossible for a program using only safe Rust and non-pathological Python to provoke a data race, because &[ReadOnlyCell<T>] cannot enter the detach critical section. After free-threading: The detach critical section doesn't mean much anymore, and it's easy to provoke these races without any unsafe Rust or C code in the picture.

But yeah, I guess this distinction is kind of academic, because Python programs routinely use C extensions that release the GIL (including e.g. the standard hashlib module), and it's not very useful to provide memory safety only in the special circumstance where they don't do that.