It took me a long time to admit to myself that the venerable Unix command line interface is stuck in the past and in need of a refresh, but it was a formative moment in my development as a programmer when I finally did. Coming from that perspective, I am very glad that there is a new wave of enthusiasm (coming especially from the Rust community) to build new tools that are fixing some of the problems with this very old and established user-interface.
The Role of the Unix CLI Interface#
To describe the Unix command line interface, “venerable” is definitely the right word: many programmers (including myself at some points of my life) have an awe of Unix and its role in computing history that has sometimes bordered on veneration.
Since the Unix operating system began development at Bell Labs in 1969, it has gone viral. That’s probably an understatement: Most modern operating systems descend from this original Unix, either directly through gradual code change (macOS and iOS are descended it from it through BSD), or through Linux (the kernel behind most servers and behind Android and ChromeOS) and its accompanying usermode software (much of which was part of the GNU project), which were designed to work like Unix due its familiarity for users and programmers.
Unix was and is billed not just as an operating system, but a philosophy. Among other things, its command line interface has been held up time and time again as an example of good design practices and an ideal realization of this philosophy, with its developer- and administrator-friendly orientation towards plain text files and with its modularity, especially as embodied in the concept of pipelining.
And as a result, when people say they know “the command line,” it’s almost certainly the Unix command-line interface that they’re talking about. And what’s more, many of us were taught it from texts that gushed about how great it is. But even the Unix command line interface, though part of a well-established standard, the topic of many books, and used by and intimately familiar to millions of programmers and admins across generations, is, in the end, just another computer interface for users and developers. And it has its flaws.
A Disappointing Ambiguity#
As I alluded to before, when I was a much younger programmer, I had an awe-struck veneration for Unix. One of my colleagues at an early job in my career referred to me as our company’s “Unix philosopher.” While I wasn’t sure whether he meant it as a compliment, at the time, I took it as one.
The first flaw that really got my attention in the Unix command line had
to do with the
mv command. I’m going to take some time explaining this
flaw in detail, as it’s somewhat subtle, and as discovering it was
a formative moment for me in my development as a programmer.
mv, as many of you know, is short for “move.” And while its job indeed
includes moving files from one place to another, due to idiosyncracies
of the Unix file system (if they can be called idiosyncracies when most
file systems followed Unix’s lead on this), moving files and renaming
files are closely related operations under the hood, causing the
command to be both the “move” command and the “rename” command:
# Assume a file called 'draft-file' # Assume a directory called 'final-docs' # Rename 'draft-file' to 'final-file' and put it in 'final-docs' mv draft-file final-file # rename 'draft-file' to 'final-file' mv final-file final-docs # move 'final-file' into 'final-docs' directory # Alternatively, one step: mv draft-file final-docs/final-file
As you can see, there is no distinction between these operations. There is no option that you must enable to get the “moving” feature as opposed to the “renaming” feature. And this can result in surprises, which are bad in software development.
Consider this command again:
mv draft-file final-file
What does it do? It changes the name of the file from
final-file, keeping it in the same directory, right? Well,
probably, and that’s almost certainly what the user intended, but
what if someone, accidentally or intentionally, had created a
final-file? That command would be interpreted
instead as moving
draft-file into the
$ # Rename operation $ touch draft-file $ ls draft-file $ mv draft-file final-file $ ls final-file $ ls final-file final-file $ rm final-file $ $ # Move operation $ mkdir final-file # Imagine someone else did this, or it was done by accident $ touch draft-file $ mv draft-file final-file $ ls final-file $ ls final-file draft-file $ rm final-file rm: cannot remove 'final-file': Is a directory $ rm -rf final-file
Notice that if there is no color-coding enabled, a simple
doesn’t even distinguish the two situations, so you can’t tell which
one happened without issuing a more specific command, as
ls also has
a dual role: it can either show you the names of the files you specify,
if they are present, or it can show you the files in a directory you
-d option disambiguates that you want the names and not
the contents, but the default is still ambiguous.
In the case of the
mv command, this potentially could even be a security
vulnerability in a shell script (which is admittedly not a very secure
platform). It is in any case an unnecessary complication.
The GNU version of
mv has a
-t option to indicate that the destination
is not to be interpreted as a directory to put things in, and a
to show unambiguous intent for a target directory to be used. But these
are extensions; the POSIX standard manual page for
mv doesn’t mention them.
And while this GNU extension is helpful, especially in scripts that you
know will only be run with the GNU version of
mv (that is, not on macOS),
I don’t think it goes far enough. Most people don’t know about them,
and the possibility of surprise is still there.
When I realized this, it created a huge hole in my previous (admittedly
unreasonable) esteem for the Unix command line interface. I realized
that the ideal solution was something impractical, almost unthinkable
to the younger version of me:
mv should be deprecated in favor of two
commands, one to do renaming, and one to do targeted directory-dropping.
This glitch in the
mv command is just a gotcha to be aware of,
one of many minor flaws to dance around when shell scripting. But I
remember it strongly, because rather than being warned about it in a
book, I discovered it myself, and therefore it was the distinct moment I
realized that the command line interface would need to be improved at some
point. And once the metaphorical levee was broken, I started noticing
many inconveniences and problems in the traditional Unix CLI tools,
often more relevant to my day-to-day workflow than this minor gotcha.
I ultimately came to read more critical sources about Unix, such as the famous UNIX-HATERS Handbook, and similar sources that emphasized the problems. And I’m very glad I went through this process, because before this, I was a naive CLI user and shell-scripter, trusting the system way more than I should, leaving myself open to serious problems.
Many Unix commands have gotchas and inconveniences, some I knew
about before this revelation and brushed aside, others that I
found out about later.
tar has its idiosyncratic traditional
syntax that many, many scripts (and people)
still use, and inconsistency between platforms on whether you need
-z to unpack a compressed archive. The way the shell itself worked
also contained gotchas: What happens if you have files whose names
start with a
-? (Answer: Their names get misinterpreted as options,
even if you didn’t type them but simply included them accidentally in
a wildcard expansion.)
Among the more practical issues that particularly effect me, I want
to emphasize two in particular: Why is
find’s syntax so gnarly, so
that you have to type out
--name and explicitly specify the current
directory? Why is it so hard to get
grep to not display the pages-long
display the shorter lines from actual source files?
Luckily, improvement is on its way. For the last two cherry-picked
examples, there are new re-conceptions of
that fix them (with new names, of course, so they’re not beholden
to interface-compatibility), and I recommend them (dare I say such
blasphemy?) over the traditional equivalents:
Don’t let their long names dissuade you; they are commonly installed
rg, respectively, and come with such modern features as:
- Normal command line syntax (
- Integration with
git, the de facto standard version control system, by ignoring
.gitignore’d files by default (both)
- Line length maximums (
- Modern leveraging of multithreading (both)
- Better performance than their traditional counterparts
These are the only new Rust-based commands I’ve tried, but they’ve
already vastly improved my workflow, so that I miss having them
fd especially) when SSH’d into relatively minimalist embedded
devices. And I have reason to hope there’s more gems out there
as part of this explosive movement to implement new Rust-based
Whether people are doing this to improve their Rust chops, or because
they’ve felt a need for a long time and Rust is just their PL of choice,
it’s good to see some actual evolution in my day-to-day experience
as a Unix CLI user. It hasn’t fixed
mv – yet – but it’s good to
see it evolving.
On the implementation side of things, I am also very
happy to see a Rust project to reimplement the standard
coreutils. The C implementations
undoubtedly leave some performance and stability on the table, and a
new implementation is long over-due. A fresh implementation of these
utilities will hopefully also spark improvements to the interfaces.
And Meanwhile, in
On a related positive note, I learned very recently (in 2022)
git has (in 2019) fixed a problem similar to
git checkout, ambiguous in a similar way, has been rendered
unnecessary by the less ambiguous
git switch and