Detailed design of immovability API.

Author: boats

text/0000-pin_and_move.md

@@ -0,0 +1,313 @@
+- Feature Name: pin_and_move
+- Start Date: 2018-02-19
+- RFC PR: (leave this empty)
+- Rust Issue: (leave this empty)
+
+# Summary
+[summary]: #summary
+
+Introduce new APIs to libcore / libstd to serve as safe abstractions for data
+which cannot be safely moved around.
+
+# Motivation
+[motivation]: #motivation
+
+Why are we doing this? What use cases does it support? What is the expected outcome?
+
+# Explanation
+[explanation]: #explanation
+
+## The `Move` auto trait
+
+This new auto trait is added to the `core::marker` and `std::marker` modules:
+
+```rust
+pub unsafe auto trait Move { }
+```
+
+A type implements `Move` if in its stack representation, it contains internal
+references to other positions within its strack representation. Nearly every
+type in Rust is `Move`.
+
+Positive impls of `Move` are added for types which contain pointers to generic
+types, but do not contain those types in their stack representation, e.g:
+
+```rust
+unsafe impl<'a, T: ?Sized> Move for &'a T { }
+unsafe impl<'a, T: ?Sized> Move for &'a mut T { }
+unsafe impl<'a, T: ?Sized> Move for Box<T> { }
+unsafe impl<"a, T> Move for Vec<T> { }
+// etc
+```
+
+This trait is a lang item, but only to generate negative impls for certain
+generators. Unlike previous `?Move` proposals, and unlike some traits like
+`Sized` and `Copy`, this trait does not impose any particular semantics on
+types that do or don't implement it.
+
+## The stability marker traits
+
+A hierarchy of marker traits for smart pointer types is added to `core::marker`
+and `std::marker`. These exist to provide a shared language for talking about
+the guarantees that different smart pointers provide. This enables both the
+kinds of self-referential support we talk about later in this RFC and other
+APIs like rental and owning-ref.
+
+### `Own` and `Share`
+
+```rust
+unsafe trait Own: Deref { }
+unsafe trait Share: Deref + Clone { }
+```
+
+These two traits are for smart pointers which implement some form of ownership
+construct.
+
+- **Own** implies that this type has unique ownership over the data which it
+  dereferences to. That is, unless the data is moved out of the smart pointer,
+  when this pointer is destroyed, so too will that data. Examples of `Own`
+  types are `Box<T>`, `Vec<T>` and `String`.
+- **Share** implies that this type has shared ownership over the data which it
+  dereferences to. It implies `Clone`, and every type it is `Clone`d it must
+  continue to refer to the same data; it cannot perform deep clones of that
+  data. Examples of `Share`
+
+These traits are mutually exclusive - it would be a logic error to implement
+both of them for a single type. We retain the liberty to assume that no type
+ever does implement both - we could upgrade this from a logic error to
+undefined behavior, we could make changes that would break any code that
+implements both traits for the same type.
+
+### `StableDeref` and `StableDerefMut`
+
+```rust
+unsafe trait StableDeref: Deref { }
+unsafe trait StableDerefMut: StableDeref + DerefMut { }
+```
+
+These two traits are for any pointers which guarantee that the type they
+dereference to is at a stable address. That is, moving the pointer does not
+move the type being addressed.
+
+- **StableDeref** implies that the referenced data will not move if you move
+  this type or dereference it *immutably*. Types that implement this include
+  `Box<T>`, both reference types, `Rc<T>`, `Arc<T>`, `Vec<T>`, and `String`.
+  Pretty much everything in std that implements `Deref` implements
+  `StableDeref`.
+- **StableDerefMut** implies the same guarantees as `StableDeref`, but also
+  guarantees that dereferencing a *mutable* reference will not cause the
+  referenced data to change addresses. Because of this, it also implies
+  `DerefMut`. Examples of type that implement this include `&mut T`, `Box<T>`,
+  and `Vec<T>`.
+
+Note that `StableDerefMut` does not imply that taking a mutable reference to
+the smart pointer will not cause the referenced data to move. For example,
+calling `push` on a `Vec` can cause the slice it dereferences to to change
+locations. Its only obtaining a mutable reference to the target data which is
+guaranteed not to relocate it.
+
+Note also that this RFC does not propose implementing `StableDerefMut` for
+`String`. This is to be forward compatible with the static string optimization,
+an optimization which allows values of `&'static str` to be converted to
+`String` without incurring a heap allocation. A component of this optimization
+would cause `String` to allocate when dereferencing to an `&mut str` if the
+backing data would otherwise be in rodata.
+
+### Notes on existing ecosystem traits
+
+These traits supplant certain traits in the ecosystem which already provide
+similar guarantees. In particular, the [stable-deref][stable-deref] crate
+provides a similar bit different hierarchy. The differences are:
+
+- That crate draws no distinction between `StableDeref` and `StableDerefMut`.
+  This does not leave forward compatibility for the static string optimization
+  mentioned previously.
+- That crate has no equivalent to the `Own` trait, which is necessary for some
+  APIs using internal references.
+- That crate has a `StableDerefClone` type, which is equivalent to the bound
+  `Share + StableDeref` in our system.
+
+If the hierarchy proposed in this RFC becomes stable, all users are encouraged
+to migrate from that crate to the standard library traits.
+
+## The `Pin` type
+
+The `Pin` type is a wrapper around a mutable reference. If the type it
+references is `!Move`, the `Pin` type guarantees that the referenced data will
+never be moved again. It has a relatively small API. It is added to both
+`std::mem` and `core::mem`.
+
+```rust
+struct Pin<'a, T: ?Sized + 'a> {
+    data: &'a mut T,
+}
+
+impl<'a, T: ?Sized + Move> Pin<'a, T> {
+    pub fn new(data: &'a mut T) -> Pin<'a, T>;
+}
+
+impl<'a, T: ?Sized> Pin<'a, T> {
+    pub unsafe fn new_unchecked(data: &'a mut T) -> Pin<'a, T>;
+
+    pub unsafe fn get_mut(this: Pin<'a, T>) -> &'a mut T;
+
+    pub fn borrow<'b>(this: &'b mut Pin<'a, T>) -> Pin<'b, T>;
+}
+
+impl<'a, T: ?Sized> Deref for Pin<'a, T> {
+    type Target = T;
+}
+
+impl<'a, T: ?Sized + Move> DerefMut for Pin<'a, T> { }
+```
+
+For types which implement `Move`, `Pin` is essentially the same as an `&mut T`.
+But for types which do not, the conversion between `&mut T` and `Pin` is
+unsafe (however, `Pin` can be easily immutably dereferenced, even for `!Move`
+types).
+
+The contract on the unsafe part of `Pin`s API is that a Pin cannot be
+constructed if the data it references would ever move again, and that it cannot
+be converted into a mutable reference if the data might ever be moved out of
+that reference. In other words, if you have a `Pin` containing data which does
+not implement `Move`, you have a guarantee that that data will never move.
+
+The next two subsections describe safe APIs for constructing a `Pin` of data
+which cannot be moved - one in the heap, and one in the stack.
+
+### Pinning to the heap: The `Anchor` type
+
+The `Anchor` wrapper takes a type that implements `StableDeref` and `Own`, and
+prevents users from moving data out of that unless it implements `Move`. It is
+added to `std::mem` and `core::mem`.
+
+```rust
+struct Anchor<T> {
+     ptr: T,
+}
+
+impl<T: StableDeref + Own> Anchor<T> {
+     pub fn new(ptr: T) -> Anchor<T>;
+
+     pub unsafe fn get_mut(this: &mut Anchor<T>) -> &mut T;
+
+     pub unsafe fn into_inner_unchecked(this: Anchor<T>) -> T;
+
+     pub fn pin<'a>(this: &'a mut Anchor<T>) -> Pin<'a, T::Target>;
+}
+
+impl<T: StableDeref + Own> Anchor<T> where T::Target: Move {
+     pub fn into_inner(this: Anchor<T>) -> T;
+}
+
+impl<T: StableDeref + Own> Deref for Anchor<T> {
+     type Target = T;
+}
+
+impl<T: StableDeref + Own> DerefMut for Anchor<T> where T::Target: Move { }
+```
+
+Because `Anchor` implements `StableDeref` and `Own`, and it is not safe to get
+an `&mut T` if the target of `T ` does not implement `Move`, an anchor
+guarantees that the target of `T` will never move again. This satisfies the
+safety constraints of `Pin`, allowing a user to construct a `Pin` from an
+anchored pointer.
+
+Because the data is anchored into the heap, you can move the anchor around
+without moving the data itself. This makes anchor a very flexible way to handle
+immovable data, at the cost of a heap allocation.
+
+An example use:
+
+```rust
+let mut anchor = Anchor::new(Box::new(immovable_data));
+let pin = Anchor::pin(&mut anchor);
+```
+
+### Pinning to the stack: `Pin::stack` and `pinned`
+
+Data can also be pinned to the stack. This avoids the heap allocation, but the
+pin must not outlive the data being pinned, and the API is less convenient.
+
+First, the pinned function, added to `std::mem` and `core::mem`:
+
+```rust
+pub struct StackPinned<'a, T: ?Sized> {
+     _marker: PhantomData<&'a mut &'a ()>,
+     data: T
+}
+
+pub fn pinned<'a, T>(data: T) -> StackPinned<'a, T> {
+     StackPinned {
+          data, _marker: PhantomData,
+     }
+}
+```
+
+Second, this constructor is added to `Pin`:
+
+```rust
+impl<'a, T: ?Sized> Pin<'a, T> {
+    pub fn stack(data: &'a mut StackPinned<'a, T>) -> Pin<'a, T>;
+}
+```
+
+Because the lifetime of the `StackPinned` and the lifetime of the reference to
+it are bound together, the StackPinned wrapper is functionally moved (and with
+it the data inside it) into the Pin. Thus, even though the data is allocated on
+the stack, it is pinned to its location for the remainder of its scope.
+
+```rust
+let mut data = mem::pinned(immovable_data);
+let pin = Pin::stack(&mut data);
+```
+
+## Immovable generators
+
+Today, the unstable generators feature has an option to create generators which
+contain references that live across yield points - these are, in effect,
+internal references into the generator's state machine. Because internal
+references are invalidated if the type is moved, these kinds of generators
+("immovable generators") are currently unsafe to create.
+
+Once the arbitrary_self_types feature becomes object safe, we will make three
+changes to the generator API:
+
+1. We will change the `resume` method to take self by `self: Pin<Self>` instead
+   of `&mut self`.
+2. We will implement `!Move` for the anonymous type of an immovable generator.
+3. We will make it safe to define an immovable generator.
+
+This is an example of how the APIs in this RFC allow for self-referential data
+types to be created safely.
+
+## Stabilization planning
+
+This RFC proposes a large API addition, and so it is broken down into four
+separate feature flags, which can be stabilized in stages:
+
+1. `stable_deref` - to control the smart pointer trait hierarchy - StableDeref,
+   StableDerefMut, Own, and Share.
+2. `pin_and_move` - to control the `Move` auto trait and the `Pin` type. These
+   two components only make sense working together.
+3. `anchor` - to control the `Anchor` struct, pinning to the heap.
+4. `stack_pinning` - to control the APIs related to stack pinning.
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+Why should we *not* do this?
+
+# Rationale and alternatives
+[alternatives]: #alternatives
+
+- Why is this design the best in the space of possible designs?
+- What other designs have been considered and what is the rationale for not choosing them?
+- What is the impact of not doing this?
+
+# Unresolved questions
+[unresolved]: #unresolved-questions
+
+- What parts of the design do you expect to resolve through the RFC process before this gets merged?
+- What parts of the design do you expect to resolve through the implementation of this feature before stabilization?
+- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC?