Skip to content

Extension traits #2812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,6 +437,12 @@
- [Semantic Confusion](idiomatic/leveraging-the-type-system/newtype-pattern/semantic-confusion.md)
- [Parse, Don't Validate](idiomatic/leveraging-the-type-system/newtype-pattern/parse-don-t-validate.md)
- [Is It Encapsulated?](idiomatic/leveraging-the-type-system/newtype-pattern/is-it-encapsulated.md)
- [Extension Traits](idiomatic/leveraging-the-type-system/extension-traits.md)
- [Extending Foreign Types](idiomatic/leveraging-the-type-system/extension-traits/extending-foreign-types.md)
- [Method Resolution Conflicts](idiomatic/leveraging-the-type-system/extension-traits/method-resolution-conflicts.md)
- [Should I Define An Extension Trait?](idiomatic/leveraging-the-type-system/extension-traits/should-i-define-an-extension-trait.md)
- [Extending Other Traits](idiomatic/leveraging-the-type-system/extension-traits/extending-other-traits.md)
- [Trait Method Conflicts](idiomatic/leveraging-the-type-system/extension-traits/trait-method-conflicts.md)

---

Expand Down
63 changes: 63 additions & 0 deletions src/idiomatic/leveraging-the-type-system/extension-traits.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
minutes: 15
---

# Extension Traits

It may desirable to **extend** foreign types with new inherent methods. For
example, allow your code to check if a string is a palindrome using
method-calling syntax: `s.is_palindrome()`.

It might feel natural to reach out for an `impl` block:

```rust,compile_fail
// 🛠️❌
impl &'_ str {
pub fn is_palindrome(&self) -> bool {
self.chars().eq(self.chars().rev())
}
}
```

The Rust compiler won't allow it, though. But you can use the **extension trait
pattern** to work around this limitation.

<details>

- Start by explaining the terminology.

A Rust item (be it a trait or a type) is referred to as:

- **foreign**, if it isn't defined in the current crate
- **local**, if it is defined in the current crate

The distinction has significant implications for
[coherence and orphan rules][1], as we'll get a chance to explore in this
section of the course.

- Compile the example to show the compiler error that's emitted.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should start by explaining what we want to achieve first: we want the user to be able to write something like mystr.is_palindrome(). Then transition to the obvious solution that does not work (the code snippet above). And then say that this is why we are using a more complex solution that does work.

Otherwise it might be confusing to some audience members: we are starting a new chapter by looking at a piece of code that does not compile (the impl block above), we want it to compile (why?..), but we actually wouldn't, instead we should do something else.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I restructured the flow a bit in 17ba065. What do you think?


Highlight how the compiler error message nudges you towards the extension
trait pattern.

- Explain how many type-system restrictions in Rust aim to prevent _ambiguity_.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can raise the level of abstraction even higher.

One effective way to approach evaluating features in programming design is to ask "what if everybody did this?"

"I want to be able to add methods to someone else's type! I want to add an is_palindrome method to a string" - "Yes, but what if two people did this?"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 17ba065


What would happen if you were allowed to define new inherent methods on
foreign types? Different crates in your dependency tree might end up defining
different methods on the same foreign type with the same name.

As soon as there is room for ambiguity, there must be a way to disambiguate.
If disambiguation happens implicitly, it can lead to surprising or otherwise
unexpected behavior. If disambiguation happens explicitly, it can increase the
cognitive load on developers who are reading your code.

Furthermore, every time a crate defines a new inherent method on a foreign
type, it may cause compilation errors in _your_ code, as you may be forced to
introduce explicit disambiguation.

Rust has decided to avoid the issue altogether by forbidding the definition of
new inherent methods on foreign types.

</details>

[1]: https://doc.rust-lang.org/stable/reference/items/implementations.html#r-items.impl.trait.orphan-rule
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
minutes: 10
---

# Extending Foreign Types

An **extension trait** is a local trait definition whose primary purpose is to
attach new methods to foreign types.

```rust
mod ext {
pub trait StrExt {
fn is_palindrome(&self) -> bool;
}

impl StrExt for &str {
fn is_palindrome(&self) -> bool {
self.chars().eq(self.chars().rev())
}
}
}

// Bring the extension trait into scope...
pub use ext::StrExt as _;
// ...then invoke its methods as if they were inherent methods
assert!("dad".is_palindrome());
assert!(!"grandma".is_palindrome());
```

<details>

- The `Ext` suffix is conventionally attached to the name of extension traits.

It communicates that the trait is primarily used for extension purposes, and
it is therefore not intended to be implemented outside the crate that defines
it.

Refer to the ["Extension Trait" RFC][1] as the authoritative source for naming
conventions.

- The trait implementation for the chosen foreign type must belong to the same
crate where the trait is defined, otherwise you'll be blocked by Rust's
[_orphan rule_][2].

- The extension trait must be in scope when its methods are invoked.

Comment out the `use` statement in the example to show the compiler error
that's emitted if you try to invoke an extension method without having the
corresponding extension trait in scope.

- The example above uses an [_underscore import_][3] (`use ext::StrExt as _`) to
minimize the likelihood of a naming conflict with other imported traits.

With an underscore import, the trait is considered to be in scope and you're
allowed to invoke its methods on types that implement the trait. Its _symbol_,
instead, is not directly accessible. This prevents you, for example, from
using that trait in a `where` clause.

Since extension traits aren't meant to be used in `where` clauses, they are
conventionally imported via an underscore import.

</details>

[1]: https://rust-lang.github.io/rfcs/0445-extension-trait-conventions.html
[2]: https://github.com/rust-lang/rfcs/blob/master/text/2451-re-rebalancing-coherence.md#what-is-coherence-and-why-do-we-care
[3]: https://doc.rust-lang.org/stable/reference/items/use-declarations.html#r-items.use.as-underscore
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
minutes: 15
---

# Extending Other Traits

As with types, it may be desirable to **extend foreign traits**. In particular,
to attach new methods to _all_ implementors of a given trait.

```rust
mod ext {
use std::fmt::Display;

pub trait DisplayExt {
fn quoted(&self) -> String;
}

impl<T: Display> DisplayExt for T {
fn quoted(&self) -> String {
format!("'{}'", self)
}
}
}

pub use ext::DisplayExt as _;

assert_eq!("dad".quoted(), "'dad'");
assert_eq!(4.quoted(), "'4'");
assert_eq!(true.quoted(), "'true'");
```

<details>

- Highlight how we added new behaviour to _multiple_ distinct types at once.
`.quoted()` can be called on string slices, numbers and booleans since they
all implement the `Display` trait.

This flavour of the extension trait pattern is built on top of
[_blanket implementations_][1].

Blanket implementations allow us to implement a trait for a generic type `T`,
as long as it satisfies the trait bounds specified in the `impl` block. In
this case, the only requirement is that `T` implements the `Display` trait.

- Draw the students attention to the implementation of `DisplayExt::quoted`: we
can't make any assumptions about the type of `T` other than that it implements
`Display`. All our logic must either use methods from `Display` or
functions/macros that doesn't require `T` to implement any other trait.

We could introduce additional trait bounds on `T`, but it would restrict the
set of types that can leverage the extension trait.

- Conventionally, the extension trait is named after the trait it extends,
following by the `Ext` suffix. In the example above, `DisplayExt`.

- There are entire libraries aimed at extending foundational traits with new
functionality.

[`itertools`] provides a wide range of iterator adapters and utilities via the
[`Itertools`] trait. [`futures`] provides [`FutureExt`] to extend the
[`Future`] trait.

## More To Explore

- Extension traits can be used by libraries to distinguish between stable and
experimental methods.

Stable methods are part of the trait definition.

Experimental methods are provided via an extension trait defined in a
different library, with a less restrictive stability policy. Some utility
methods are then "promoted" to the core trait definition once they have been
proven useful and their design has been refined.

- Extension traits can be used to split a [dyn-incompatible trait][2] in two:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an excellent use case, but without a code example it is, most likely, impossible to learn from a mere description. The only type of student who would be satisfied with a terse description is someone who is already familiar with the idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I placed this under "More to explore" to signal that the instructor can decide whether to mention/explain/dive into this topic.
I can expand it with code examples and more details, but then it'd make more sense to extract it into its own slide.
Alternatively, we can keep the terse explanation and provide a link to a more in-depth resource for those who want to dig deeper.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it'd make more sense to extract it into its own slide

Yes, that's what I'm hinting at :)


- A **dyn-compatible core**, restricted to the methods that satisfy
dyn-compatibility requirements.
- An **extension trait**, containing the remaining methods that are not
dyn-compatible. (e.g., methods with a generic parameter).

- Concrete types that implement the core trait will be able to invoke all
methods, thanks to the blanket impl for the extension trait. Trait objects
(`dyn CoreTrait`) will be able to invoke all methods on the core trait as well
as those on the extension trait that don't require `Self: Sized`.

</details>

[1]: https://doc.rust-lang.org/stable/reference/glossary.html#blanket-implementation
[`itertools`]: https://docs.rs/itertools/latest/itertools/
[`Itertools`]: https://docs.rs/itertools/latest/itertools/trait.Itertools.html
[`futures`]: https://docs.rs/futures/latest/futures/
[`FutureExt`]: https://docs.rs/futures/latest/futures/future/trait.FutureExt.html
[`Future`]: https://docs.rs/futures/latest/futures/future/trait.Future.html
[2]: https://doc.rust-lang.org/reference/items/traits.html#r-items.traits.dyn-compatible
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
minutes: 15
---

# Method Resolution Conflicts

What happens when you have a name conflict between an inherent method and an
extension method?

```rust
mod ext {
pub trait StrExt {
fn trim_ascii(&self) -> &str;
}

impl StrExt for &str {
fn trim_ascii(&self) -> &str {
self.trim_start_matches(|c: char| c.is_ascii_whitespace())
}
}
}

pub use ext::StrExt;
// Which `trim_ascii` method is invoked?
// The one from `StrExt`? Or the inherent one from `str`?
assert_eq!(" dad ".trim_ascii(), "dad");
```

<details>

- The foreign type may, in a newer version, add a new inherent method with the
same name of our extension method.

Survey the class: what do the students think will happen in the example above?
Will there be a compiler error? Will one of the two methods be given higher
priority? Which one?

Add a `panic!("Extension trait")` in the body of `StrExt::trim_ascii` to
clarify which method is being invoked.

- [Inherent methods have higher priority than trait methods][1], _if_ they have
the same name and the **same receiver**, e.g., they both expect `&self` as
input. The situation becomes more nuanced if the use a **different receiver**,
e.g., `&mut self` vs `&self`.

Change the signature of `StrExt::trim_ascii` to
`fn trim_ascii(&mut self) -> &str` and modify the invocation accordingly:

```rust
assert_eq!((&mut " dad ").trim_ascii(), "dad");
```

Now `StrExt::trim_ascii` is invoked, rather than the inherent method, since
`&mut self` has a higher priority than `&self`, the one used by the inherent
method.
Comment on lines +53 to +55
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is an accurate explanation of what's happening here. The reference explicitly states (in the info box in that section) that &self methods have higher priority than &mut self methods.

I think the reason why the &mut self version gets higher priority here is that the receiver expression is &mut &str. If I'm understanding the reference's explanation of method resolution correctly, this means that when it builds the list of candidate receiver types, &mut &str is the first candidate type in the list. It's then choosing between the inherent &str method and the &mut &str method coming from the trait, and the latter wins because it's the actual type of the expression &mut " dad ".

I think the confusion here is because we're implementing the trait on on &str, which is already a reference type. If I change the trait to be implemented on str directly (i.e. impl StrExt for str), when when I change the method to take &mut self the inherent method still gets called (exmple in the playground). Part of the reason for this is because when we do (&mut " dad ") we're not getting a &mut str, we're getting a &mut &str.

I think things would be a lot less ambiguous if we were demonstrating this on a regular, non-reference type such as i32 or struct Foo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking at it closely!
Re-reading through it, and cross-referencing with the RFC, I agree with your interpretation as to why things play out as they do in terms of precedence. I'll rework the example to something simpler.


Point the students to the Rust reference for more information on
[method resolution][2]. An explanation with more extensive examples can be
found in [an open PR to the Rust reference][3].
Comment on lines +57 to +59
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Point the students to the Rust reference for more information on
[method resolution][2]. An explanation with more extensive examples can be
found in [an open PR to the Rust reference][3].
Point the students to the Rust reference for more information on
[method resolution][2].

I think we can just link to the reference, I don't think linking to an open PR is necessary. Eventually the things in that PR will (hopefully) land, so just linking to the reference is enough imo.


- Avoid naming conflicts between extension trait methods and inherent methods.
Rust's method resolution algorithm is complex and may surprise users of your
code.

## More to explore

- The interaction between the priority search used by Rust's method resolution
algorithm and automatic `Deref`ing can be used to emulate [specialization][4]
on the stable toolchain, primarily in the context of macro-generated code.
Check out ["Autoref Specialization"][5] for the specific details.

</details>

[1]: https://doc.rust-lang.org/stable/reference/expressions/method-call-expr.html#r-expr.method.candidate-search
[2]: https://doc.rust-lang.org/stable/reference/expressions/method-call-expr.html
[3]: https://github.com/rust-lang/reference/pull/1725
[4]: https://github.com/rust-lang/rust/issues/31844
[5]: https://github.com/dtolnay/case-studies/blob/master/autoref-specialization/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
minutes: 5
---

# Should I Define An Extension Trait?

In what scenarios should you prefer an extension trait over a free function?

```rust
pub trait StrExt {
fn is_palindrome(&self) -> bool;
}

impl StrExt for &str {
fn is_palindrome(&self) -> bool {
self.chars().eq(self.chars().rev())
}
}

// vs

fn is_palindrome(s: &str) -> bool {
s.chars().eq(s.chars().rev())
}
```

The main advantage of extension traits is **ease of discovery**.

<details>

- A bespoke extension trait might be an overkill if you want to add a single
method to a foreign type. Both a free function and an extension trait will
require an additional import, and the familiarity of the method calling syntax
may not be enough to justify the boilerplate of a trait definition.

Nonetheless, extension methods can be **easier to discover** than free
functions. In particular, language servers (e.g. `rust-analyzer`) will suggest
extension methods if you type `.` after an instance of the foreign type.

</details>
Loading
Loading