For this next iteration in my series comparing Rust to C++, I want to talk about something I’ve been avoiding so far: memory safety. I’ve been avoiding this topic so far because I think it is the most discussed difference between C++ and Rust, and therefore I felt I’d have relatively little to add to the conversation. I’ve also been avoiding it because I wanted to draw attention to all the other little ways in which Rust is a better-designed programming language, to say that even if you concede to the C++ people that Rust isn’t “truly memory safe” or “memory safe enough,” Rust still wins.

Array Indexing#

But there is a persistent and persnickety little argument that I wanted to talk specifically about. This argument is really persuasive on its face, and so I think it deserves some attention – especially since I am guilty of having used this argument myself, many years ago when I still worked at an HFT firm, to claim that C++ had a niche that Rust wasn’t ready for. I’ve also seen it a few times in a row in the wild, and it’s made me so emotional that I simply had to write this, and as a result, it’s a little more emotional than some of the other posts.

In this argument, array indexing stands in for a number of little features. But – I’ve seen array indexing cited so often as a canonical example that I feel compelled to address it directly!

The argument goes like this: In Rust, array accesses are checked. Every time you write arr[i], there is an extra prepended if i >= arr.len() { panic!(..) }. As you can see, that is more code, and worse, a run-time check. And while the optimizer might eliminate it, or the branch predictor may well predict it right every time, the extra code bloat and possible run-time check, is just unacceptable in [insert field here (I used HFT)], where every nanosecond matters. And until some acceptable solution is found to this, I just don’t see Rust making it in [insert field].

When I made this argument, to a group of programming-language academics, the defenders of Rust countered with a number of points, all of which accepted the basic premise:

• Do I really need those extra nanoseconds? Yes.
• Is it really too much of a price to pay for all that extra safety? Yes.
• Do I really distrust the optimizer that much? Yes. If only Rust had a way to do optimizer assertions, a way to statically verify that the panic had been optimized out.
• Would dependent typing on integer values help? Yes. That sounds very promising. I think Rust will get there someday, but for right now we must use C++.

Now that I know more about Rust I’m happy to tell you that I was completely off base. I wasn’t off base about the performance considerations, or the unacceptability of even the slightest risk of a run-time check. I was off base about an even more basic premise: that Rust uses checked array indexing, whereas C++ uses unchecked array indexing.

But wait! Isn’t that the whole point? Doesn’t C++ avoid checking everything, to make sure all abstractions are zero-cost, to be blazing fast? Doesn’t Rust, while trying for performance, in the end always concede to the demands of safety?

Well, let’s look at the APIs in question. C++ apologists are always saying to use the modern C++ features from C++11 and later, rather than the more C-like “old style” C++ features, so on the C++ side let’s take a look at the documentation for std::array, introduced in C++11.

Here we see two indexing methods. The first one, at, is bounds checked and will throw an exception if the index is out of bounds, whereas the second one, operator[], is not, and will instead exhibit undefined behavior of a very difficult-to-debug nature. It looks like C++ actually believes in free choice here, leaving the choice of method up to the user. Not quite what we supposed, but the important part is that unchecked indexing is available, so so far the argument can still stand.

Now let’s look at Rust. Rust arrays and vectors can also be used with methods from slice, as can slices, so the slice documentation is the best place to look. And looking there, we immediately see – drum roll please – 4 methods. We see get and get_mut, which are checked, and right underneath them, in alphabetical order, get_unchecked and get_unchecked_mut, which are not.

To review, where do Rust and C++, these programming languages with their vastly different philosophies, Rust for the cautious, C++ for the fast and bold, stand? In the exact same place. Both programming languages have both checked and unchecked indexing.

Let me say that again. This is the talking point form, what to say if you need something quick to say, if you’re ever debating programming languages on a political-style talk show (or at a party or even a job interview):

In both Rust and C++, there is a method for checked array indexing, and a method for unchecked array indexing. The languages actually agree on this issue. They only disagree about which version gets to be spelled with brackets.

The difference is simply in the default, which one gets that old fashioned arr[index] syntax. And even that can be changed. Even if the C++ default were superior – and, as I will argue later, it is not – this is surely a minor issue. After all, don’t we normally use our fancy for x in arr syntax in Rust? This issue is just so small as to be unlikely to be a deciding factor in what programming language is better, even if we’re in a special application domain where every nanosecond matters.

The Unsafe Keyword#

So that’s a wrap folks. We can all go home, and none of us will ever see this extremely silly argument on the Internet or in person again. It’s just a misunderstanding, the person making it was simply misinformed, and all it will take is a link to this blog post – or the relevant method in the docs to set them straight.

But wait! The C++ apologists are still talking! What are they saying? How have they not been completely flummoxed? They’re pointing at that method, chanting a word like a slogan at a protest march. I can’t quite make it out – what it is it?

Oh. They’re chanting unsafe. And credit where credit is due: it’s very difficult to chant in a monospace font.

Well, that is easy to respond with! The nerve, that C++ programmers would call our unchecked array indexing method unsafe. For one, all unchecked array indexing methods are unsafe: that’s what unchecked means. If it were safe, it would be at least statically checked. For another, isn’t this the pot calling the kettle black? Isn’t C++ all about unsafety, so much that C++ programmers don’t even mark their unsafe code regions becasue it all is, or their unsafe functions because they all are?

“But isn’t that the whole point of Rust?” they cry. “If you have to use unsafe to write good Rust, then Rust isn’t a safe language after all! It’s a cute effort, but it’s failing at its purpose! Might as well use C++ like a Real Programmer!”

This, my friends, is a straw man. No, the point of Rust and specifically Rust’s memory safety features is not to create an entirely safe programming language that can’t be circumvented in any circumstance; you must be thinking of Sing#, the programming language for Microsoft’s defunct research OS.

Let me be abundantly clear: The point of memory safety, the unsafe keyword, and friends in Rust is not to completely enforce memory safety, to make it impossible for the programmer to do anything they want to with the computer, even if they can’t prove to the compiler that it’s OK. In fact, the point of memory safety isn’t to make it impossible to do anything at all – it’s to make it possible to reason about the program.

The premise of Rust is that the vast majority of code in a systems program doesn’t need to be unsafe, and so it might as well be safe. People used to believe that you needed garbage collection for safety, but Rust proved that you could use lifetimes to still get safety without that performance cost. Now that we’re there, why worry about null pointers? Why not tell the compiler which things can be null, and which things can’t, so the compiler can check for you whether you’re handling nulls correctly? I’ve programmed C++ professionally for years without such a feature. You’d better believe I would have totally annotated the crap out of the code so the compiler could’ve caught them ahead of time.

Sometimes, C++ apologists cite valgrind. I’ve had codebases where I tried to use valgrind. Unfortunately, there was so much undefined behavior and memory leaks already caked into this project that new ones were simply impossible to see among all the noise. An army of junior engineers was at some point required to clean this up when finally the hierarcy decided that “valgrind” was something we might want to be able to use in the future.

And a lot of those undefined behaviors were ticking time bombs. Certainly, this codebase had its issues. A friend of mine took days to find a bug where a pointer had a value of 7. I don’t mean 7 elements into some array, not 7 of the relatively wide pointer type, not a convenient, testable-for NULL, value. No, none of that: The pointer’s value was exactly 0x7.

Update: My friend had a very similar incident to that described in this piece, but it was not the same incident. Some time after, I read that piece and shared it with this friend … and I must have conflated the numbers from the piece and from what happened to my friend. It was some null-page number, some “low integer,” however, even if not 0x7.

I’ve had memory corruption issues where I poured over every line of code that I wrote, over and over again, finding nothing. Ultimately, I learned that the issue was in framework code – code written by my boss’s boss. The code was untested, and written extremely poorly, and had rotted, so that it didn’t work at all. In Rust, I might have had some idea that my code – which in Rust would have all been able to be “safe” – couldn’t possibly be the source of the problem. Maybe my humble assumption that my code was to blame would be a little less tenable.

If I wanted a language that was always safe, at the time I knew Java or Python existed. Some companies even do finance in Java, for exactly that reason. But sometimes you still need that extra bit of performance. unsafe is sometimes necessary.

But given what gains safe Rust has made in predictable performance, it’s not as necessary as it used to be. The majority of the code I wrote then could’ve been written in safe Rust, and not lost a single clock cycle. The parts that needed to be unsafe could have been isolated, delegated to specific sections, wrapped in abstract data types, perhaps entrusted to a specific team.

And even then, I’m sure we would have been debugging memory corruption issues. But we’d know where to look. We’d know where to throw the tests. And we’d have saved programmer-years of time, days if not months of my life.

Now, I’m proud of my C++ skills. There is some part of me that wishes that C++ was better than Rust, that all that time getting better at debugging memory corruption wasn’t dedicated to a skill that is becoming obsolescent through better technology. And to be honest, that’s part of why I dismissed Rust as a candidate for HFT programming languages.

But it’s possible to be proud of a skill that is also becoming obsolete. And I am trying to replace it with a new skill to be proud of – writing Rust as performant as idiomatic C++, or even more performant, while reaching for the unsafe keyword rarely and modularly. I think it’s truly possible, for where it’s relevant.

Now I must turn to a subset of C++ apologists, who write using “modern C++” which is “very safe now” and experience therefore no memory corruption issues. To them I say, you are not doing high performance programming. If you were, you’d have to do some wonky things with pointers to spell the bespoke high-performance constructs you’d need.

There is indeed a safe subset of C++ heavy with modern features. If you are disciplined and keep your programming in that realm, you can avoid memory corruption mostly. But first, this safe subset covers fewer high-performance features than Rust. I’ve read some of this code and its idioms: It’s full of shared_ptrs not to share ownership but simply to avoid types that might be invalidated. It ironically leans on reference counting more than idiomatic Rust. This is among other, similar problems.

Let me be clear: First off, instead of keeping in your brain which features are “modern” and which are “edgy,” why not have a distinction where it’s well-marked? Second off, if you are writing entirely in this safe subset of C++, you can get much better performance instead out of the safe subset of Rust. You have no right to complain about Rust’s safety trade-offs, as you’re using a worse set, where you get no safety promises from the compiler and none of Rust’s surprising safe performance.

Rust’s safe and “slow” subset is faster than C++’s while still being, obviously, safer. Rust’s unsafe subset is better factored and better distinguished. Comparing apples to apples, Rust is better programming language for extracting performance out of LLVM, because you’ll be able to code more often without fear, and with very focussed fear when you do feel it.

A tool is even more useful if you can adjust it. The defenders of C++ talk about choosing trade-offs, but really, Rust offers both trade-offs. Mark your code as unsafe and convince yourself of its safety manually, or rely on programming language features. It’s up to you, on a function-by-function, even block-by-block, basis. In C++, if you have a problem, every line of code is suspect; you simply can’t opt in to safety, but in Rust, for where you don’t need the performance of unchecked indexing and other unsafe features, you can relax about the possibility of going bankrupt due to inadvertent memory reinterpretation – and how do I wish my NDA permitted me to talk about consequences at my own previous jobs!

And for where you do need to use unsafe, you can make sure your debugging and overthinking efforts are well-directed, for the few places in a large project you need it.

Unchecked Indices#

This has gotten a little far from the original question. Should array indices be checked? Well, let me be clear about two facts that are both true, but in tension with each other:

• Unchecked array indexing is sometimes absolutely necessary
• Unchecked array indexing is an edge-case feature, which you normally don’t want.

If unchecked array indexing was unavailable in Rust, that would be a bug. What is not a bug is making it inconvenient. C++ programmers probably should be using at instead of operator[] more often. But in C++, what would it gain? There’s so many unsafe features, what’s the cost of one more?

But in Rust, where so much code can be written that’s completely safe, defaulting to the safe version makes more sense. Lack of safety is a cost too, and Rust makes that cost explicit. Isn’t that the goal of C++, making costs explicit?

Let’s look at situations where you are indexing memory. First off, most of them I saw were in old C-style for-loops, where you loop over an index rather than using iterators directly with a collection. Both Rust and C++ have safe versions of for that loop over collections with iterators, and those use the same check for the loop as they do for bounds, so those are easy enough to address. Nevertheless, I think that a lot of the noise about checked vs. unchecked array accesses comes from people who use indexing for their for-loops instead of iterators, and therefore mistakenly think that array indexing in general is a far more common operation than it is.

For the remaining situations, most are implementing either gnarly business logic, or a subtle, fast algorithm.

If it’s gnarly business logic, in my experience, it’s usually at config time – along with a good third to half to even more of the code in a complicated production system.

What do I mean by config time? A running high-performance system, whether optimized for latency or throughput, has a bunch of data structures organized just so, a lot of threads set up just right to move data between them in the perfect rhythm, and a lot of the work is in arranging them. That work is generally not performance-sensitive, but often has to be in the same programming language as the performance-intensive stuff.

Config-time is, depending on how you look at it, less of a thing or the entire thing in a programming language like Python. Python basically exists to do config-time programming for performance-intensive code put in very comprehensive “libraries” written in C or C++. But in C++, where you have a constructor that runs only once or a few times at first, and other methods related to it, in the same programming language as the money-making do-it part, you have to really adjust programming style between them.

Config-time is obviously when you read the configuration files. It’s where you open the relevant files. It’s where you call socket and bind and listen on your listening port. It’s where you spin up your worker threads, and make computations on how many worker threads there are. It’s where you construct your objects and your object pools. It’s where you memory map your log file. It’s where you set your process priorities. It’s where you recursively call the constructors and init functions of every object in your overwrought OOP hierarchy.

There is no need to sacrifice safety for performance at config time – especially since undefined behavior might lie latent and destabilize the system once it’s actually up and running. If you do an unchecked array access at config time, you might put garbage data in an important field, maybe one that determines how much money you’re willing to risk that day or how many of a thing to buy. And for what? To save a few nanoseconds before your process has even “gone live”?

So, when do you truly need unchecked array accesses? If it’s a subtle fast algorithm, probably deep in an inner loop, you should probably be wrapping it in an abstraction anyway. The code that actually executes the algorithm should be separate from the business logic, so that programmers trying to maintain the business logic don’t accidentally break it. And that’s exactly where it makes the most sense to use unsafe – when implementing a special algorithm. Maybe the proof that the index is within bounds relies upon some number theory the compiler was never going to understand without its own proof engine: great! You should probably be explaining that in a comment in C++ anyway, and so the conventional comment that goes with the unsafe block in Rust is a perfect place to explain it.

But maybe I’m wrong about all of this. Maybe your experience hasn’t matched mine. Maybe your particular application needs to make unchecked array accesses a lot, needs them to be unchecked, and needs them littered all over the codebase. I raise my eyebrows at you, suspect you need more iterators and perhaps other abstractions, and wonder what problem you’re trying to solve. But even if you’re absolutely right, I think it’s still a better idea to write Rust littered with unsafe every time you index an array, than to write C++.

Because, as I keep emphasizing, Rust is still a better unsafe programming language than C++. It would be better than C++ even if safety weren’t a feature.

Post-Script: Some Perspective for the New Rustacean#

I understand where this straw man argument comes from. The word unsafe is scary, and advice, especially aimed at people coming from safe languages like Python and Javascript, is to avoid unsafe features while learning. And while I think adding unsafe to production code should only be done once you’ve exhausted safe possibilities – which requires full understanding of safe possibilities – this advice can feel overbearing for a transitioning C++ programmer, especially when it is immediately obvious that the safe features are very constrained and can’t literally do everything.

For that good-faith recovering C++ programmer, new to Rust: You’re right. The safe subset isn’t enough to do everything you want to do. And when it doesn’t, that doesn’t mean it failed. Its goal is to make unsafe code rare, not non-existent. But it might surprise you how rarely you truly need unsafe. And a good resource for you might be, as it was for me, the excellent Learn Rust the Dangerous Way by Cliff L. Biffle.

For what it’s worth, however, this criticism of Rust in general is often levelled either in bad faith, or from a misunderstanding of what the unsafe keyword is for. For all the philosophical discussion of what unsafe truly means – and how it interacts with the surrounding module and encapsulation/privacy boundaries – as well as principled conventions for using it, please see the Rustonomicon, the canonical book on unsafe Rust, the same way the book is canonical for introducing Rust.

Other criticisms of Rust from an HFT or low-latency point of view are more relevant. Most specifically, gcc and icc are much better compilers for those use cases – empirically – than is LLVM. Also, the large codebases existing in C++ are often tested and contain thousands upon thousands of programmer-years of optimizations and bugfixes, where even small compiler upgrades are scrutinized closely for performance regressions. Migrating to another programming language from that starting point would be prohibitively expensive.

None of which is to say that if Rust gradually replaced C++ altogether, eventually such ultra-optimizing compilers and ultra-optimized codebases wouldn’t start appearing in Rust. I hope to see that day within my lifetime.