In Defense of Async: Function Colors Are Rusty
Finally in 2019, Rust stabilized the async
feature, which supports
asynchronous operations in a way that doesn’t require multiple operating
system threads. This feature was so anticipated and hyped and in
demand that there was a website whose sole
purpose was to announce its stabilization.
async
was controversial from its inception; it’s still controversial
today; and in this post I am throwing my own 2 cents into this
controversy, in defense of the feature. I am only going to try to
counter one particular line of criticism here, and I don’t anticipate
I’ll cover all the nuance of it – this is a multifaceted issue, and
I have a day job. I am also going to assume for this
post that you have some understanding of how async
works, but if
you don’t, or just want a refresher I heartily recommend the Tokio
tutorial.
The Questionable Feature: Colored Functions#
In any discussion of a programming language feature, the first thing
to ask is what problem the feature is trying to solve. In the case of
async
, it’s trying to deal with asynchronous operations – operations
that don’t require more work from the CPU to make progress, and where
several might be in flight at any given time. For example, a single process
might be writing some data to a file, reading data from another file,
waiting for new incoming connections, and servicing an existing connection.
So how does Rust solve this?
The easiest way to address this problem would be to have a thread for each
operation, and to let the thread block at the asynchronous operation,
essentially pretending that the operation is a long-running function
like any other the CPU has to do, rather than something taking
place elsewhere. But operating system threads are expensive. And
rather than using green threads as some other programming languages
do, Rust decided to create a
syntactic sugar for futures, meaning that Rust’s async
feature now
suffers from the dreaded function coloring effect first explained by
Bob Nystrom in a Javascript context in 2015.
In Bob Nystrom’s now-famous essay he complains that an analogous feature in Javascript is harmful, because asynchronous functions – which he refers to as “red” functions – can only be called from other red functions. Once a red function is needed, the function that calls it must also be red, and same with the function that calls that, the whole way up the call chain. And the syntax and semantics of calling a red function is more complicated than that of calling blue functions – especially in Javascript, where the next thing to do had to be enclosed in a lambda, resulting in callback hell (I do not endorse the suggestions in that post).
Colored Is Good, Actually, and Rusty#
My position is close to those of this article,
but with enough nuance that I wanted to write my own blog post to explain
it in more detail. Fundamentally, I agree that Rust does indeed have
colored functions, and that it’s not a bad thing. But I would go further.
I say that function coloring has always existed in Rust, even before it
manifested in the async
world, that it is the Rustiest way to solve
this problem, and furthermore, that Rust needs more function coloring than
it has.
Rust, unlike the Javascript of the original colored functions article, is strongly typed, and influenced heavily by Haskell. This means that it has lots of type distinction on its values: “colored” values, if you will.
This type information includes basic ideas of type (string vs number
vs widget), but also shades of distinction that a Javascript programmer
won’t even be aware of. Let’s say you want to take a parameter to your
function, a “widget.” In Javascript, you just take a parameter widget
and do widget things with it, and hope that it works out. The name is
just a comment: it’s up to the caller to know what exactly is expected,
hopefully some sort of widget that works. In Rust, on the other hand, you
have to annotate the parameter with a type, which not only ensures it’s
actually a widget, but distinguishes between these potential requirements:
- Exclusive reference to a widget
&mut Widget
- Ownership of a widget
Widget
- Reference to a widget that lives forever:
&'static Widget
- Optional widget (in Javascript this is very unclear):
Option<Widget>
If Widget
is a trait, you have even more options:
- Owned run-time generic widget:
Box<dyn Widget>
- Non-owned reference to compile-time generic widget:
&impl Widget
The list goes on. For each of these options, also, the caller often
has to do something different. If the parameter is optional with Option
,
and the caller in fact has a widget, the caller still has to add Some
to the parameter:
fn foo(widget: &Widget) { ... }
fn foo2(widget: Option<&Widget>) { ... }
fn foo3(widget: Widget) { ... }
let baz = Widget::new();
foo(&baz);
foo2(Some(&baz));
foo3(baz);
All of these, in my
synaesthetic mind, are
expressed by different colors and textures on the parameter. For all
of these, Rust has made a value judgment that the programmer should be
explicitly aware of these shades of distinction, if you will (pun
intended). If a parameter is to be optional, the function is called
differently than if it is mandatory. If a borrow happens, that requires a
&
from the caller, to make clear to the programmer what is going on,
to make sure the writer of the caller and the writer of the callee are
on the same page. Parameters in Rust are, in general, colored.
And this value coloring, like the async/sync function coloring, propagates. If
a function requires a parameter to be 'static
in lifetime,
that requirement propagates to the caller of that function to the caller
of that function to the originator of the value in question.
Similarly with return values – I disagree with “More Stina” about
Result
. I say Result
-returning fallible functions are colored. In
many programming languages, including Javascript, any and all functions
can throw recoverable exceptions. In Rust, functions that might fail
(in a recoverable fashion) must have a different return type than those
that do not – they must return a Result<...>
. Functions that return
Result<T, E>
are, as with async functions, harder to call than functions
that just return T
. If you don’t want to use the syntactic sugar ?
to propagate the error, you have to grapple with Result
as a literal
return type, which means unpacking it and doing something else in the
Err
case. This is more straightforward than dealing with a raw impl Future
, but fundamentally the same concept: either propagate the “color”
with ?
or async
, or else deal with all the implications of Result
or Future
on the spot.
And all of these distinctions mean something. Passing by shared reference, mutable reference, or value are different, and put different safety requirements on the calling code, safety requirements that allow Rust to make more safety guarantees than Javascript ever could. Passing by reference is literally different at the ABI level from by-value, so each can implement the exact contract as efficiently as possible, unlike Javascript which leans on an expensive garbage collector for cleaning up the difference between these notions. That is to say, where Javascript (and Python) use garbage collectors, Rust uses distinctions – color distinctions, one might say – between types to achieve the same result, benefitting in performance but requiring more exactness from the programmer.
And in Rust, a statically typed programming language, we believe this to be a good thing. Rust is not for every project – it’s a steeper learning curve than Python or Javascript, and not every project needs to be maintainable long-term – but it has a distinct, consistent philosophy, which says that different things should be treated differently.
Async Functions Are Just Different#
A blocking or asynchronous function is not the same thing as a non-blocking function. A non-blocking function fundamentally does some CPU tasks, taking control of the processor, using it, and giving it back. An asynchronous function does the set-up necessary for work to happen elsewhere. That work doesn’t need control of the CPU, and can be dealt with through a handle – a future – rather than just waiting for completion. These are fundamentally different notions, and while it might (or might not) make sense in Go or Javascript to lump them together into one notion of “calling a function,” Rust doesn’t do lumping.
When you call a normal function – without async
/await
– you build up
a stack. When you use async
/await
, you build up a complex nested
state object. If you use async
/await
with an executor to spawn a new
task, that complex state object ends up on the heap in a data structure
next to other task objects.
Both “call stack” (for synchronous code) and “task state object” (for
asynchronous code) are reasonable ways of managing memory. Honestly, the
miracle is that Rust, through async
and await
, manages to make these
two vastly different paradigms look as similar as they do. Having to
annotate the difference is a small price to pay for high-performance
reactive programming.
It’s not 100% perfect. Even with the must_use
warnings, people forget
to call await
on their futures sometimes. And writing reactive, async
code is harder – which makes sense, because the resulting code is a
more difficult but more performing usage of memory. Writing code that
passes the borrow checker is harder, but considered worth it because
we can remove indirections and avoid garbage collection. async
offers
us the same deal for reactive programming.
Alternatives to Async#
But let’s say we did want to remove the coloring here. Let’s say we did want to pretend that blocking functions were just like CPU-based ones, but just taking a long time. What would we have to do?
Well, we’d still have to wait for multiple things simultaneously. Our servers have many connections they have to service at once, and when a message comes in on socket B, it can’t be ignored just because the code happens to be on socket A. If asynchronous operations are implemented by blocking, we have to handle this with multithreading.
Kernel multithreading is expensive, but even Go-style “green threads” have to have a separate stack for each green thread. Stacks are gnarly, because it’s unclear how much space should be reserved for them ahead of time. They have to dynamically adjust to the run-time demand, and when the original allocation is used up, you get a pause as you try to allocate more. The advantage is, you have a simpler mental model with fewer distinctions. Basically, you trade performance for simplicity – like in garbage collection.
If you want to do this trade, Rust doesn’t stop you from implementing it yourself. OS threads and blocking system calls are perfectly reasonable solutions to many problems. But Rust isn’t going to encourage the trade by creating a new compromise point of “green threads.” You have to do async the whole way, and if you think of what async code actually de-sugars to, you wouldn’t complain about how hard async functions are, but be impressed it’s so darn easy to write them!
Rust is a systems programming language at heart. I understand and respect that, because of its type system and guarantees, it has found use outside of the old domains of C and C++, but those C and C++ systems programmers are Rust’s ideal “base,” in a political sense of the word. Rust should not sacrifice performance for ease of programmability.
Blocking vs Non-Blocking#
Rust has two ways of doing off-CPU “IO” operations,
blocking and non-blocking. Blocking takes over the thread, and non-blocking
works through async
. This mirrors a distinction in the system calls
that most kernels provide. The operating system API has this distinction
built into it, and it makes sense for Rust to propagate that to the user.
But fundamentally, one of these constructs is more honest than the
other. When we call a blocking kernel system call, rather than the
kernel taking over the CPU, running on it, and then returning the
thread of execution to us, what actually happens internally is
more of a mirage. The kernel deschedules the current process,
and using an internal mechanism more like async
than like blocking,
schedules it again, recovering its previous state as if nothing
happens, when the IO is done.
This means that we can pretend the I/O operation was just an operation like any other, but it comes at a risk – the operation might not return anytime soon. It might in fact wait for a situation that’s not going to happen anytime soon.
If such a blocking function is called from a non-async Rust thread,
we assume that the caller is using threads to juggle multiple I/O
events – or else that they simply don’t have anything else going on.
But it is very dangerous to call a blocking function from an async
function. It can starve threads in a thread pool, and cause knock-on
effects in other places. Maybe an async task is waiting for a message
from a channel, and even though the message was sent, the task doesn’t
resume because the thread it’s scheduled on is busy on this blocking
function. The effects are unpredictable and non-local – similar to
the dreaded “undefined behavior” – and debugging is similarly difficult
– ask me how I know!
Functions that block but are not async are referred to in the “More Stina”
blog post (also
linked above) as “purple functions.” They are not true async “red functions”
that you can call with async
, but they are also not safe to simply call
from an async function like a truly CPU-based “blue function” would be.
Calling a blocking function from an async function is extremely unsafe,
and there is simply no warning generated by rustc
, normally so helpful
about such things, to let you know how deep and undebuggable a mistake
you’re making.
These purple functions ought to be a different color in Rust, just like
they are in practice. It should be an error to call a blocking system call
from an async
function. I don’t know how this would work – I imagine
a generalization of unsafe
that includes things like blocks
, perhaps
as well as panics
. That would fundamentally be an “effects system,”
as is regularly proposed, but that’s not the only solution. But I do
fundamentally think that something ought to be done about this deficit
in Rust’s otherwise quite rigorous function-coloring system.
So, in conclusion, I say: yes, Rust async
functions are colored.
This is the same as saying they are strongly typed, and this is a good
thing. And instead of trying to fix it, we should have more of it.
Postscript: Monads#
As I mentioned before, calling an async function does something
fundamentally different under the hood from calling a vanilla “blue”
function. Similarly, calling a fallible function with Result
does
something different from calling a function with a normal return
value. In both cases, the control flow is different – either it
contains short-circuits to error code (Result
); or regular
hops back and forth between the task, other tasks, and the
executor (async
/Future
).
In both these cases, it’s like the meaning of having one statement
come after another has changed: ;
itself has been overriden.
And it would be nice if generic collections methods, like map
and filter
, supported this, so that you could fail, or await
,
in the closures.
This is possible in Haskell, because Haskell has a typeclass (equivalent to Rust traits) for abstracting over different styles of control flow. That is what Haskell’s infamous monads are for, and why Haskell persists in using this technology even though it’s so famously confusing for beginners.
Fundamentally, every Haskell monad is a function color. And often, they can be stacked together (via “monad transformers”) so that you can say something like “this function can do IO, fail, and be asynchronous.” You can also create functions that are polymorphic on “color”: the control flow is rewritten based on which monad you actually end up in.
Why is this useful? As “More Stina”’s
post already
mentions, there is a proposal to add try_
versions of iterator adapter methods: try_filter
, etc.,
to enable them to work smoothly with Result
-“colored” functions.
A method like filter
or map
also would need an adapter to work well with
async
. If there were an abstract concept of monad, we could write
code with filter
-like methods that could short-circuit on failure and
do the right thing with await
:
vec!([2,3,4])
.iter()
.filter_monad(|x| fallable_thing.contains(x)?)
.filter_monad(|x| network_file_thing.contains(x).await?)
.for_each_monad(|x| network_other_thing.send(x).await);
Perhaps Rust will someday gain this abstraction as well. I actually think that would be good for Rust. Monads are hard to deal with conceptually, and I’m not sure how to make them more user-accessible, but I think if anyone could do it, it’s the Rust people, who’ve already done such a good job so far at programming language design and maintenance.
Subscribe
Find out via e-mail when I make new posts! You can also use RSS (RSS for technical posts only) to subscribe!
Comments
If you want to send me something privately and anonymously, you can use my admonymous to admonish (or praise) me anonymously.
comments powered by Disqus