Finally in 2019, Rust stabilized the async feature, which supports asynchronous operations in a way that doesn’t require multiple operating system threads. This feature was so anticipated and hyped and in demand that there was a website whose sole purpose was to announce its stabilization.

async was controversial from its inception; it’s still controversial today; and in this post I am throwing my own 2 cents into this controversy, in defense of the feature. I am only going to try to counter one particular line of criticism here, and I don’t anticipate I’ll cover all the nuance of it – this is a multifaceted issue, and I have a day job. I am also going to assume for this post that you have some understanding of how async works, but if you don’t, or just want a refresher I heartily recommend the Tokio tutorial.

The Questionable Feature: Colored Functions#

In any discussion of a programming language feature, the first thing to ask is what problem the feature is trying to solve. In the case of async, it’s trying to deal with asynchronous operations – operations that don’t require more work from the CPU to make progress, and where several might be in flight at any given time. For example, a single process might be writing some data to a file, reading data from another file, waiting for new incoming connections, and servicing an existing connection.

So how does Rust solve this? The easiest way to address this problem would be to have a thread for each operation, and to let the thread block at the asynchronous operation, essentially pretending that the operation is a long-running function like any other the CPU has to do, rather than something taking place elsewhere. But operating system threads are expensive. And rather than using green threads as some other programming languages do, Rust decided to create a syntactic sugar for futures, meaning that Rust’s async feature now suffers from the dreaded function coloring effect first explained by Bob Nystrom in a Javascript context in 2015.

In Bob Nystrom’s now-famous essay he complains that an analogous feature in Javascript is harmful, because asynchronous functions – which he refers to as “red” functions – can only be called from other red functions. Once a red function is needed, the function that calls it must also be red, and same with the function that calls that, the whole way up the call chain. And the syntax and semantics of calling a red function is more complicated than that of calling blue functions – especially in Javascript, where the next thing to do had to be enclosed in a lambda, resulting in callback hell (I do not endorse the suggestions in that post).

Colored Is Good, Actually, and Rusty#

My position is close to those of this article, but with enough nuance that I wanted to write my own blog post to explain it in more detail. Fundamentally, I agree that Rust does indeed have colored functions, and that it’s not a bad thing. But I would go further. I say that function coloring has always existed in Rust, even before it manifested in the async world, that it is the Rustiest way to solve this problem, and furthermore, that Rust needs more function coloring than it has.

Rust, unlike the Javascript of the original colored functions article, is strongly typed, and influenced heavily by Haskell. This means that it has lots of type distinction on its values: “colored” values, if you will.

This type information includes basic ideas of type (string vs number vs widget), but also shades of distinction that a Javascript programmer won’t even be aware of. Let’s say you want to take a parameter to your function, a “widget.” In Javascript, you just take a parameter widget and do widget things with it, and hope that it works out. The name is just a comment: it’s up to the caller to know what exactly is expected, hopefully some sort of widget that works. In Rust, on the other hand, you have to annotate the parameter with a type, which not only ensures it’s actually a widget, but distinguishes between these potential requirements:

  • Exclusive reference to a widget &mut Widget
  • Ownership of a widget Widget
  • Reference to a widget that lives forever: &'static Widget
  • Optional widget (in Javascript this is very unclear): Option<Widget>

If Widget is a trait, you have even more options:

  • Owned run-time generic widget: Box<dyn Widget>
  • Non-owned reference to compile-time generic widget: &impl Widget

The list goes on. For each of these options, also, the caller often has to do something different. If the parameter is optional with Option, and the caller in fact has a widget, the caller still has to add Some to the parameter:

fn foo(widget: &Widget) { ... }
fn foo2(widget: Option<&Widget>) { ... }
fn foo3(widget: Widget) { ... }

let baz = Widget::new();
foo(&baz);
foo2(Some(&baz));
foo3(baz);

All of these, in my synaesthetic mind, are expressed by different colors and textures on the parameter. For all of these, Rust has made a value judgment that the programmer should be explicitly aware of these shades of distinction, if you will (pun intended). If a parameter is to be optional, the function is called differently than if it is mandatory. If a borrow happens, that requires a & from the caller, to make clear to the programmer what is going on, to make sure the writer of the caller and the writer of the callee are on the same page. Parameters in Rust are, in general, colored.

And this value coloring, like the async/sync function coloring, propagates. If a function requires a parameter to be 'static in lifetime, that requirement propagates to the caller of that function to the caller of that function to the originator of the value in question.

Similarly with return values – I disagree with “More Stina” about Result. I say Result-returning fallible functions are colored. In many programming languages, including Javascript, any and all functions can throw recoverable exceptions. In Rust, functions that might fail (in a recoverable fashion) must have a different return type than those that do not – they must return a Result<...>. Functions that return Result<T, E> are, as with async functions, harder to call than functions that just return T. If you don’t want to use the syntactic sugar ? to propagate the error, you have to grapple with Result as a literal return type, which means unpacking it and doing something else in the Err case. This is more straightforward than dealing with a raw impl Future, but fundamentally the same concept: either propagate the “color” with ? or async, or else deal with all the implications of Result or Future on the spot.

And all of these distinctions mean something. Passing by shared reference, mutable reference, or value are different, and put different safety requirements on the calling code, safety requirements that allow Rust to make more safety guarantees than Javascript ever could. Passing by reference is literally different at the ABI level from by-value, so each can implement the exact contract as efficiently as possible, unlike Javascript which leans on an expensive garbage collector for cleaning up the difference between these notions. That is to say, where Javascript (and Python) use garbage collectors, Rust uses distinctions – color distinctions, one might say – between types to achieve the same result, benefitting in performance but requiring more exactness from the programmer.

And in Rust, a statically typed programming language, we believe this to be a good thing. Rust is not for every project – it’s a steeper learning curve than Python or Javascript, and not every project needs to be maintainable long-term – but it has a distinct, consistent philosophy, which says that different things should be treated differently.

Async Functions Are Just Different#

A blocking or asynchronous function is not the same thing as a non-blocking function. A non-blocking function fundamentally does some CPU tasks, taking control of the processor, using it, and giving it back. An asynchronous function does the set-up necessary for work to happen elsewhere. That work doesn’t need control of the CPU, and can be dealt with through a handle – a future – rather than just waiting for completion. These are fundamentally different notions, and while it might (or might not) make sense in Go or Javascript to lump them together into one notion of “calling a function,” Rust doesn’t do lumping.

When you call a normal function – without async/await – you build up a stack. When you use async/await, you build up a complex nested state object. If you use async/await with an executor to spawn a new task, that complex state object ends up on the heap in a data structure next to other task objects.

Both “call stack” (for synchronous code) and “task state object” (for asynchronous code) are reasonable ways of managing memory. Honestly, the miracle is that Rust, through async and await, manages to make these two vastly different paradigms look as similar as they do. Having to annotate the difference is a small price to pay for high-performance reactive programming.

It’s not 100% perfect. Even with the must_use warnings, people forget to call await on their futures sometimes. And writing reactive, async code is harder – which makes sense, because the resulting code is a more difficult but more performing usage of memory. Writing code that passes the borrow checker is harder, but considered worth it because we can remove indirections and avoid garbage collection. async offers us the same deal for reactive programming.

Alternatives to Async#

But let’s say we did want to remove the coloring here. Let’s say we did want to pretend that blocking functions were just like CPU-based ones, but just taking a long time. What would we have to do?

Well, we’d still have to wait for multiple things simultaneously. Our servers have many connections they have to service at once, and when a message comes in on socket B, it can’t be ignored just because the code happens to be on socket A. If asynchronous operations are implemented by blocking, we have to handle this with multithreading.

Kernel multithreading is expensive, but even Go-style “green threads” have to have a separate stack for each green thread. Stacks are gnarly, because it’s unclear how much space should be reserved for them ahead of time. They have to dynamically adjust to the run-time demand, and when the original allocation is used up, you get a pause as you try to allocate more. The advantage is, you have a simpler mental model with fewer distinctions. Basically, you trade performance for simplicity – like in garbage collection.

If you want to do this trade, Rust doesn’t stop you from implementing it yourself. OS threads and blocking system calls are perfectly reasonable solutions to many problems. But Rust isn’t going to encourage the trade by creating a new compromise point of “green threads.” You have to do async the whole way, and if you think of what async code actually de-sugars to, you wouldn’t complain about how hard async functions are, but be impressed it’s so darn easy to write them!

Rust is a systems programming language at heart. I understand and respect that, because of its type system and guarantees, it has found use outside of the old domains of C and C++, but those C and C++ systems programmers are Rust’s ideal “base,” in a political sense of the word. Rust should not sacrifice performance for ease of programmability.

Blocking vs Non-Blocking#

Rust has two ways of doing off-CPU “IO” operations, blocking and non-blocking. Blocking takes over the thread, and non-blocking works through async. This mirrors a distinction in the system calls that most kernels provide. The operating system API has this distinction built into it, and it makes sense for Rust to propagate that to the user.

But fundamentally, one of these constructs is more honest than the other. When we call a blocking kernel system call, rather than the kernel taking over the CPU, running on it, and then returning the thread of execution to us, what actually happens internally is more of a mirage. The kernel deschedules the current process, and using an internal mechanism more like async than like blocking, schedules it again, recovering its previous state as if nothing happens, when the IO is done.

This means that we can pretend the I/O operation was just an operation like any other, but it comes at a risk – the operation might not return anytime soon. It might in fact wait for a situation that’s not going to happen anytime soon.

If such a blocking function is called from a non-async Rust thread, we assume that the caller is using threads to juggle multiple I/O events – or else that they simply don’t have anything else going on. But it is very dangerous to call a blocking function from an async function. It can starve threads in a thread pool, and cause knock-on effects in other places. Maybe an async task is waiting for a message from a channel, and even though the message was sent, the task doesn’t resume because the thread it’s scheduled on is busy on this blocking function. The effects are unpredictable and non-local – similar to the dreaded “undefined behavior” – and debugging is similarly difficult – ask me how I know!

Functions that block but are not async are referred to in the “More Stina” blog post (also linked above) as “purple functions.” They are not true async “red functions” that you can call with async, but they are also not safe to simply call from an async function like a truly CPU-based “blue function” would be. Calling a blocking function from an async function is extremely unsafe, and there is simply no warning generated by rustc, normally so helpful about such things, to let you know how deep and undebuggable a mistake you’re making.

These purple functions ought to be a different color in Rust, just like they are in practice. It should be an error to call a blocking system call from an async function. I don’t know how this would work – I imagine a generalization of unsafe that includes things like blocks, perhaps as well as panics. That would fundamentally be an “effects system,” as is regularly proposed, but that’s not the only solution. But I do fundamentally think that something ought to be done about this deficit in Rust’s otherwise quite rigorous function-coloring system.

So, in conclusion, I say: yes, Rust async functions are colored. This is the same as saying they are strongly typed, and this is a good thing. And instead of trying to fix it, we should have more of it.

Postscript: Monads#

As I mentioned before, calling an async function does something fundamentally different under the hood from calling a vanilla “blue” function. Similarly, calling a fallible function with Result does something different from calling a function with a normal return value. In both cases, the control flow is different – either it contains short-circuits to error code (Result); or regular hops back and forth between the task, other tasks, and the executor (async/Future).

In both these cases, it’s like the meaning of having one statement come after another has changed: ; itself has been overriden. And it would be nice if generic collections methods, like map and filter, supported this, so that you could fail, or await, in the closures.

This is possible in Haskell, because Haskell has a typeclass (equivalent to Rust traits) for abstracting over different styles of control flow. That is what Haskell’s infamous monads are for, and why Haskell persists in using this technology even though it’s so famously confusing for beginners.

Fundamentally, every Haskell monad is a function color. And often, they can be stacked together (via “monad transformers”) so that you can say something like “this function can do IO, fail, and be asynchronous.” You can also create functions that are polymorphic on “color”: the control flow is rewritten based on which monad you actually end up in.

Why is this useful? As “More Stina”’s post already mentions, there is a proposal to add try_ versions of iterator adapter methods: try_filter, etc., to enable them to work smoothly with Result-“colored” functions. A method like filter or map also would need an adapter to work well with async. If there were an abstract concept of monad, we could write code with filter-like methods that could short-circuit on failure and do the right thing with await:

vec!([2,3,4])
    .iter()
    .filter_monad(|x| fallable_thing.contains(x)?)
    .filter_monad(|x| network_file_thing.contains(x).await?)
    .for_each_monad(|x| network_other_thing.send(x).await);

Perhaps Rust will someday gain this abstraction as well. I actually think that would be good for Rust. Monads are hard to deal with conceptually, and I’m not sure how to make them more user-accessible, but I think if anyone could do it, it’s the Rust people, who’ve already done such a good job so far at programming language design and maintenance.