Rust Patterns That Matter #21: Pin and Boxing Futures
Post 21 of 22 in Rust Patterns That Matter. Companion series: Building a Chat Server in Rust.
Previous: #20: Channels | Next: #22: Send / Sync in Async
You write an async trait method, or try to store a future in a struct, and the
compiler demands Pin. You have no mental model for what
Pin means or why async needs it. This post builds that mental model.
The motivation
You want a trait with an async method:
trait DataSource {
async fn fetch(&self) -> Vec<u8>;
}
In recent Rust editions, this works directly in many cases. But sometimes you need to return the future as a trait object, store it in a struct, or work with it manually. That's when you encounter:
fn fetch(&self) -> Pin<Box<dyn Future<Output = Vec<u8>> + Send + '_>>
What is Pin? Why is it needed? What does Box::pin do?
Why async needs Pin
An async fn compiles to a state machine struct. Each .await
point is a state. The struct holds the local variables that live across those
.await points.
async fn example() {
let data = vec![1, 2, 3];
let slice = &data[..]; // reference to data
some_async_op().await; // suspend here
println!("{:?}", slice); // use slice after await
}
The compiler-generated state machine holds both data and
slice. But slice is a reference into
data - it points to memory within the same struct. This is a
self-referential structure.
If the struct moves in memory (e.g., from the stack to the heap, or when a
Vec reallocates), data moves to a new address, but
slice still points to the old address. The reference dangles.
Pin solves this by guaranteeing: this value will not move in
memory. Once a future is pinned, no one can move it, so self-references
remain valid.
The mental model
Pin<&mut T> means: "you have a mutable reference to T, but
you're not allowed to move T." Normally, if you have &mut T, you
can call std::mem::swap or std::mem::replace to move the
value out. Pin prevents this.
Specifically, you can't get a &mut T from a
Pin<&mut T> unless T: Unpin.
Unpin
Most types are Unpin. An i32, a String,
a Vec<T> - they don't contain self-references, so moving
them is perfectly safe. For these types, Pin has no effect.
Futures generated by async blocks are !Unpin - they
might contain self-references (like the data/slice
example). For these types, Pin matters: once pinned, they can't be
moved.
Box::pin()
The pragmatic solution: put the future on the heap and pin it there.
use std::pin::Pin;
use std::future::Future;
fn fetch(&self) -> Pin<Box<dyn Future<Output = Vec<u8>> + Send + '_>> {
Box::pin(async move {
// async implementation
vec![]
})
}
Box::pin() allocates the future on the heap and returns a
Pin<Box<...>>. The heap-allocated future won't move (the
Box pointer may move, but the data it points to stays put). The
self-references inside the future remain valid.
When you encounter Pin
In practice, you'll encounter Pin in three situations:
- Async trait methods that need dynamic dispatch - return
Pin<Box<dyn Future>> - Storing futures in structs - the struct field needs to
be
Pin<Box<dyn Future>> - Manual
Futureimplementations - thepollmethod takesPin<&mut Self>
For cases 1 and 2, Box::pin() is the answer. Case 3 is advanced and
rarely needed in application code - libraries like tokio and
futures handle it internally.
Native async traits
Recent Rust (1.75+) supports async fn in traits directly. When the
compiler can determine the future's type statically, no boxing is needed. Boxing
is still required when you need dynamic dispatch (dyn Trait) because
the compiler can't know the concrete future type at compile time.
When to use it
- Returning futures from trait methods that use
dyn Trait - Storing heterogeneous futures in collections or struct fields
- Any situation where the compiler asks for
Pin
For normal async fn usage, you don't think about Pin at
all. The compiler handles it. Pin only surfaces at the boundaries:
trait objects, manual future implementations, and heterogeneous storage.
See it in practice: Building a Chat Server #6: Going Async uses this pattern for the async filter plugin system.
Telex