In my previous post, I talked about programming language design, and try to discern some heuristics for what features should be added to a programming language, on my way to explaining why Rust should not include inheritance as a feature.

I’d like to expand more on that blog post now (so I guess it’s become a series?). Do I think Rust is perfect how it is? (No.) Are there features that I want in Rust that Rust currently does not have? (Absolutely yes.) In this post, I’d like to talk about some proposed additions to Rust, some recent, some very stale, and discuss my perspective on them.

My primary goal in this is educational. I’d like people to think seriously about what features exist in the programming languages they use. I believe that this makes people better, more thoughtful programmers who write better-factored, more maintainable code. In general, I tend to be of an overly philosophical “overthinking” temperament, oscillating between a healthy dose of “unexamined life is not worth living” and a sometimes unrealistic or even unhealthy “every problem in life, programming, and even politics can be solved by just thinking harder and maybe a little smarter.”

My secondary goal in this is advocacy. I suspect many critics of Rust have entrenched but under-developed opinions of what makes a programming language feature good or worthwhile. This sometimes comes from the understandable position of having thought about this primarily from the programmer’s perspective, where they’re just trying to get work done and patterns they’re used to using are unavailable for mysterious reasons. This also sometimes comes from good old-fashioned differences in priorities or even programming ideologies. In either case, if I can show another way to think about programming languages and programming language features, perhaps it will at least lead to better understanding of where Rust is coming from.

Fields in Traits: Maybe.#

First, I’d like to talk about an actual proposal to add inheritance-like features to Rust. We know I wouldn’t want to add “inheritance” to Rust in a Java, C++ or Smalltalk style, preferring the much more limited features of composition, traits, and sum types. But could other inheritance-like features be added without harm to Rust’s overall design?

The nominal target audience for my previous post was critics of Rust who were coming from an OOP perspective, and thought that it was bad that Rust doesn’t include inheritance. I’d seen a lot of them in response to my original post about inheritance, and wanted to pick apart some of the assumptions and talking points I’d seen in that criticism.

I say this was my “nominal” target audience. This was fundamentally a partisan advocacy post, if the parties are OOP ideologues vs people who think we need to move beyond it. If Rust were a religion (which it’s not) we would call it “apologetics,” and some do – a lot of my blog could fairly be described as Rust apologetics. And like most partisan advocacy posts, even though the nominal audience was critics of Rust, my audience also includes people who face such critics, giving them rhetorical tools and conceptual frameworks for defending Rust against critics in their own conversations.

For example, let’s say you’re advocating for Rust at your workplace. If someone complains that it “forbids” inheritance, what do you say? Or if they ask why Rust doesn’t just “give you the choice” of inheritance, how do you respond to that argument? How do you avoid becoming convinced yourself?

As an advocacy post engaging with (outside) critics of Rust, my post was limited in scope. I wasn’t engaging with controversies or discussions within the Rust community. Rust is unlikely to get inheritance as a feature anytime soon, even though there have been discussions of it. If Rust does get new features that resemble inheritance, it’s unlikely to be a version of the feature recognizable to OOP theorists. It’s more likely (not particularly likely, but more likely) to be something more like this proposal to add fields to traits, a much more limited (and therefore less problematic) feature than OOP “inheritance” in all its glory and warts.

So, I want to be clear. My previous post was intended to engage with outside critics of Rust who are uncomfortable that Rust doesn’t have the full feature of inheritance, with all its problems. I was not addressing any internal efforts to get some of the upsides of inheritance, except perhaps in a throw-away line where I say that additional use cases of inheritance might merit adding additional narrowly-defined features to capture those use cases.

And perhaps we do need features that cover more use cases of inheritance. Fields in traits would be one such feature. Adding fields to traits doesn’t have anywhere near as many of the problems that I talk about in my previous posts on inheritance. If this feature were to be added, it wouldn’t be called inheritance, wouldn’t cover all the possible patterns of inheritance, and Rust critics would still complain that Rust doesn’t have inheritance.

So, I think fields in traits should be considered separately from inheritance writ large, just like traits themselves. So, how do I feel about fields in traits?

Before I continue, I’d like to clarify that this proposal looks dead for now, and not under active development since 2017. I’ve been following programming languages long enough to know that proposals can languish a long time and then make an abrupt come-back, but there’s no particular reason to believe that that will happen to this proposal, especially in the near term.

I greatly respect Niko Mitsakis (GitHub, blog). I’m sure he doesn’t know who I am: he’s a major figure in the programming language, and I’m just a Rust programmer and educator with a hobby advocacy/education blog that doubles as my personal blog.

The fact that he wrote this proposal makes me like it more. If he thinks it’d be useful, that’s a strong reason in my mind to consider it seriously, even if it’s old and languishing. Argument from authority, in my mind, is less of a fallacy and more of a powerful and generally useful (albeit fallible) heuristic.

It probably doesn’t, and likely shouldn’t, matter to Niko Mitsakis how I feel about fields in traits. I am engaging with this proposal only partially because I want to share my opinions about it, but mostly as an example of how I think through proposals like this. Part of this is advocacy: I think more people will like and appreciate Rust if they develop more discernment in how they think about programming language features. Part of it is education: I’d like to make people more thoughtful and therefore more effective in how they program, and part of that is understanding how programming language features interact.

The Proposal#

So, what about the actual proposal?

I recommend you actually go read it if you want to engage with it properly. A lot of it is technical, trying to explain how to prevent this feature from breaking other features in the programming language, such as the borrow checker. But it also goes over the basics.

But also, here’s a simple example use of this feature as a “tldr”:

trait Foo {
    weight: f64,
}

struct Bar {
    other: u32,
    weight: f64,
}

impl Foo for Bar {
    weight: self.weight,
}

let foo: Box<dyn Foo> = Box::new(Bar {
    other: 0,
    weight: 0.3,
});

let foo_weight = foo.weight; // Polymorphic field access

We define a field, weight, that all implementers of the Foo trait must have. In struct Bar, we then define this field among others. There is no requirement as to where we put the weight field, or even that we name it weight.

Then, when we implement Foo for Bar, we tell the compiler which field to use as this generic weight field. This must be an actual concrete field, either directly in the struct, or in a path that contains no dereferences. Put another way, there must be a fixed offset within Bar where weight can be found, without having to follow any indirection.

This offset could then be looked up in the vtable of a dyn Foo trait object, for when we want to write foo.weight, which would only do a little more work than a normal field look-up. We’d have all the capabilities with foo.weight as we would for any other field, including being able to borrow it, copy it (if it is Copy), move from it, exclusively borrow (&mut) it, etc. Much of the discussion in the rest of the proposal is how to make this possible with the borrow checker.

Limitations of the Proposal#

First, let’s talk about what this proposal rejects. The more limited a feature proposal is, the less likely it is to add confusion and cognitive load to the programming language as a whole.

First, the proposal rejects the type of syntactic sugar known as “properties.” Properties, a feature available in Python, C#, Swift, and other programming languages, allows you to overload field access syntax. When the user of a type Foo with a value foo writes foo.x, it would compile to something more like foo.x(), and when they write foo.x = 3, it would compile to something like foo.set_x(3).

This is great in my mind. I’m skeptical of properties in any programming language, but especially in a systems programming language like Rust. When you write foo.x, it should be clear that it’s doing a field access (cheap and with few potential unseen consequences) rather than a method call (which could do anything including crash, or block your thread on a network request). The difference between these two things is important, and I don’t like a potentially more expensive more risky method call masquerading as a field access. (Edited to add: And while it is true that Rust’s Deref trait can insert a hidden method call in there, I see that more as a reason to dislike the Deref trait and implicit dereferences in general, and less as a reason to make it worse by adding properties.)

The field accesses that this proposal does allow are a little heavier than typical field accesses in the case of a dyn trait object. But there is a limit to how heavy they are: They just have to look up an offset in the vtable, and use that offset in computing the address of a field. This is in line with how many things are heavier with a dyn trait object.

Additionally, this proposal considers “embedding notation,” something more like OOP inheritance, where all the fields of one struct be included at the top of another, and where a trait could require this in implementing types. This would come very close to OOP-style inheritance, with just a little bit of additional complexity to provide semantic clarity.

But, while not rejecting embedding notation, it at least consigns it to “future directions.” Specifically, it says:

In any case, adding ..Foo notation does not add any expressiveness. It can always be modeled by using explicit fields, at the cost of some ergonomics.

This is a good reason, in my book, to hesitate to add a potentially confusing and overpowered feature.

Merits of the Proposal#

So, here’s how I evaluate the proposal on its merits. In this section, I’m going to pretend I’ve never heard of OOP, I’m not aware of the overlap with “inheritance” as a feature, and consider this proposal as it interacts with traits.

Traits get new capabilities all the time. This proposal is in line with that. Associated types, methods returning associated types, associated constants … why not have what amounts to an associated item version of C++’s pointer to data members?

This proposal adds a new feature Rust currently doesn’t have. You could simulate it by writing accessor methods, like weight(), set_weight(), weight_mut(), etc. to your trait definition, which is how similar patterns are currently implemented. But these have downsides in terms of flexibility and clarity that make this proposal a distinct feature, downsides the proposal clearly and convincingly addresses.

I am in favor of having distinct features for distinct use cases. The Rust compiler should know what we’re doing, as it is our friend for verifying that we are doing it in sensible ways. The ability to distinguish, in a way legible to the compiler, between methods that do work and methods that provide access to data is appealing to me, especially since we already have the ability to provide access to data through trait methods.

Like any new feature, it does increase the complexity of the programming language. Therefore, like any new feature, it needs to have use cases compelling enough to justify this increase in complexity.

Here is where my experience is limited: I don’t understand the use cases well enough to comment. A lot of the discussion about inheritance-like features in Rust seems to do with Servo and/or GUI programming, but I’m not a participant in the Servo project, nor do I do GUI programming in Rust. My favorite GUI programming framework is very much so not object-oriented, so I’m a particularly weird edge case when it comes to these questions.

So, my conclusion comes out to a shrug. This feature doesn’t seem bad in any way. It’s limited enough and distinct enough from existing features that I don’t think it would make the design of Rust worse. If I were designing traits from scratch, I’d consider it. If it were added, I would not be particularly surprised or upset or concerned.

But I personally don’t engage with a use case for it. So, besides noting that I don’t see a strong reason not to, I don’t have much to say. Perhaps now that I’ve read and thought so much about the proposal, I’ll start seeing use cases in future projects, and maybe that will change my opinion.

OOP Overtones#

In the previous section, I ignored the associations with OOP design patterns. The fact that this starts to look like inheritance, however, does make me suspicious, especially given that something even more like inheritance is put in the “future work” section.

Sometimes, I find myself thinking thoughts similar to Linus Torvalds in his famous anti-C++ rant, where he says this about C++:

Quite frankly, even if the choice of C were to do nothing but keep the C++ programmers out, that in itself would be a huge reason to use C.

I have a different personality from Linus Torvalds, and my corresponding thought is a lot less personal, but of a similar vein. Sometimes, I feel that it’s best to exclude OOP features from Rust, not just because those features are ill-thought-out (see my previous posts), but because I see it as welcoming an OOP philosophy and OOP design patterns.

Rather than framing it as wanting to exclude those programmers, as Linus does, I’d frame it as wanting to make sure people who come to Rust from OOP languages are encouraged to learn how to program like a Rustacean. I don’t want new Rustaceans to be tempted to program in OOP in their head and then translate that into the most literally equivalent Rust features. And so, I’m concerned about any feature that might enable people to program in an OOP style without learning Rusty idiom.

In the end, however, I don’t think this is a very strong argument. Just because we don’t agree with a philosophy, doesn’t mean we have to reject any ideas that smell like that philosophy. As much as I try to be less abrasive than Linus Torvalds about it, that way lies tribalism. While there are reasons humans are tribal, even in little subcommunities like programming and programming language design, it’s important to not let it take over our thinking entirely.

To the extent that this argument is valid, it can be expressed in other terms – in terms of feature overlap. If it’s true that this will encourage dialects of Rust, one more OOP-inspired with features that match OOP programmers’ comfort zone, one more FP-inspired with features that match functional style, then that may be a sign that Rust just has too many overlapping features, rather than anything else.

To a certain extent, this sort of fracturing and dialectalization is inevitable in a programming language as ambitious and performance-centric as Rust. To achieve performance, Rust gives programmers control, because different situations call for different performance needs and different trade-offs – and therefore, different abstractions and different programming language features.

I don’t really have a way to parlay these thoughts into specific feedback on the proposal of fields in traits. Perhaps, at the very most, I’d be extra cautious that this feature is truly needed – but then again, I feel like caution is required in any new feature.

In actuality, it looks like this feature won’t be implemented anytime soon, and so any long-term concern about it is kind of moot for now. Who knows what dialects and parties there will be within the Rust programming language if and when this feature regains momentum?

Self-Referential Structs: Absolutely.#

By self-referential structs, I mean types like this:

struct Foo {
    a: Vec<u32>,
    b: &'??? [u32], // Some subslice of `a`
}

Here, we have a struct with two fields. One of them has a heap allocation, and the other one borrows from that heap allocation. There is no way to actually write this in safe Rust. There is nothing we can put instead of ??? for the lifetime parameter of b. There is no way to constrain one field of a struct because it is borrowed by another field. In order to create a struct like this, we have to use unsafe.

I have been tempted to write a blog post about self-referential structs for a while, but figured I had no need, because the excellent post “Self-referential types for fun and profit” already exists!

Go read it, it’s great!

Here is my take-away from this post: Self-referential types are not allowed in fully safe Rust. Sometimes if something isn’t allowed in fully safe Rust, it’s a code smell, a pattern of design that Rust is rightly protecting you from. But actually, self-referential types are super useful all the time, and so you might actually just want to learn how to construct them with unsafe or use a crate like ouroboros to handle the unsafe for you, wrapped in a (hopefully!) safe abstraction.

“Self-referential types for fun and profit” already provides a great example of why you might want to create a struct like this. I’d add that the mere existence of Pin (and its prominent role in async Rust) is evidence that there are some programming patterns that absolutely need a struct where one field borrows from another.

So here’s my excuse for talking about it now: I’m now writing a blog series about adding features to Rust, and there’s an informal feature proposal for adding support for this pattern. Yet again, this is Niko Matsakis’s idea, and I’m a lot more excited about it than I am about fields in traits, if only because I have use cases for this sort of thing regularly.

So why am I excited about this feature?

For one, pinning is super hard to reason about, and it’s not always necessary. Self-referential types can be made safe by pinning, but a subset of them can also be made safe and also still freely moveable by:

• Ensuring that we only borrow from a heap allocation owned by the other field (so that moving the Box or Vec doesn’t actually invalidate the reference, as the heap allocation doesn’t also move)
• Not mutating the borrowed-from field (because it’s borrowed from!)

These are rules that you can enforce yourself on a struct with private fields, and then you can use unsafe to construct the borrow (with a technically inaccurate 'static lifetime). But they are also rules that a more sophisticated borrow checker could conceivably enforce for us, bringing this pattern into completely safe Rust.

Of course, that’s exactly what this proposal would do.

On a more abstract level, this proposal seems in line with the existing capabilities and philosophy of Rust. This is a point made by Niko already in his post, but I have a slightly different way of framing how I think about it.

To me, struct types and stack frames are similar things. Whatever lives on a stack frame together should be able to be bundled into a struct, and passed around. In Rust, that’s not the case – there are patterns of borrows that can exist between variables in a stack frame, that cannot exist within a struct.

Given that there are transformations we want to do between stack frames and struct types (such as is done for us by the compiler when writing async functions), this is a problem. This problem is already addressed by Pin, but that is overkill when immoveability is not actually a requirement.

So for me, this feature feels less like complicating the borrow checker, and more like plugging a gap, a leak, in the abstraction that is the struct keyword. If we ever get this feature, once we have it, I think it’ll be the type of feature where we’ll wonder how we ever lived without it.

And that’s all I really have to add to the conversation. For more, go read the blog posts I linked to, they’re really great.