Fear and Loathing on the UNIX Trail
Confessions of a Berkeley system mole.
by Doug Merritt with Ken Arnold and Bob Toxen
It was 2 am and I was lying face down on the floor in Cory Hall, the EECS
building on the UC Berkeley campus, waiting for Bob to finish installing out
bootleg copy of the UNIX kernel. If successful, new and improved terminal
drivers we had written would soon be up and running.
We were enhancing the system in the middle of the night because we had no
official sanction to do the work. That didn't stop us, though, since UNIX had
just freshly arrived from Bell Labs, where computer security had never been an
issue. The system was now facing its first acid test -- exposure to a group of
intelligent, determined students -- and its security provisions were failing
I was lying face down because I'd gone without sleep for over two days, and the
prone position somehow seemed the most logical under the circumstances. Bob
was still working because he'd napped not 30 hours before, giving him seniority
under the "Hacker-best-able-to-perform" rule of our informal order. We might
have called our group "Berkeley Undergraduate Programmers for a Better UNIX",
or, less euphemistically, "Frustrated Hackers for Our Own Ideas". But, in
truth, our group was never named. It was simply a matter of Us versus Them.
"Them" was the bureaucracy -- the school administrators, most professors, some
grad students, and even the legendary Implementors themselves at Bell Labs.
"Us" was a small, self-selected group of undergraduates with a passion for
UNIX. We were interested in computers and in programming because it fascinated
us; we lived for the high level of intellectual stimulation only hacking could
provide. Although some in our group never expressed an interest in breaking
computer security, others invested thousands of fruitful hours in stealing
accounts and gaining superuser access to various UNIX systems. Our object?
To read system source code.
For the most part we stayed out of trouble, although one of our rank once had
his phone records subpoenaed by the FBI -- after a minor incident with a
Lawrence Livermore National Laboratory computer. The Feds seemed to think our
comrade had been diddling with top secret weapons research, but he actually
Our group could probably best be characterized by its interest in creating and
using powerful software, regardless of the source of the idea. Our battle cry,
thanks to Ross Harvey, was "FEATURES!!!", and we took it seriously. Well,
Ross may have been a little sarcastic about it, since he was referring to
superfluous bells and whistles. But I used the expression as a shorthand for
"elegant, powerful, and flexible". We were always bugging Them to add "just
one more feature" to some utility like the shell or kernel. Although They
accepted some suggestions, They didn't think twice about most.
One example stands out. In early 1977, Ross, Bob, and I spent months
collaborating on a new and improved shell, just before Bill Joy had started
on what is now known as the C shell. The most historically significant
features we designed were Ross's command to change the shell's prompt, Bob's
command to print or chdir to the user's
home directory, and my own edit feature, which allowed screen
editing and re-execution of previous commands. What we did was smaller in
scope than what Bill later included in the C shell, but to Us it was
unarguably better what was then available. We ceased work on our projects only
when it became clear that Bill was developing what would obviously become a new
standard shell. Our energies then were re-focused on persuading him to include
our ideas. Some of our features ultimately were incorporated, some weren't.
We modified the kernel to support asynchronous I/O, distributed files, security
traces, "real-time" interrupts for subprocess multitasking, limited screen
editing, and various new system calls. We wrote compilers, assemblers,
linkers, disassemblers, database utilities, cryptographic utilities, tutorial
help systems, games, and screen-oriented versions of standard utilities.
User friendly utilities for new users that avoided accidental file deletion,
libraries to support common operations on data structure such as lists,
strings, trees, symbol tables, and libraries to perform arbitrary precision
arithmetic and symbolic mathematics were other contributions. We
suggested improvements to many system calls and to most utilities. We
offered to fix the option flags so that the different utilities were
consistent with one another.
To Us, nothing was sacred, and We saw a great deal in UNIX that could stand
improvement. Much of what We implemented, or asked to be allowed to
implement, is now a part of System V and 4.2 BSD; others of our innovations
are still missing from all versions of UNIX. Despite these accomplishments,
it seemed that whenever We asked The Powers That Be to install Our software
and make it available to the rest of the system's users, We were greeted with
Fred Brooks, in The Mythical Man-Month, describes the NIH (Not Invented Here)
Syndrome, wherein a group of people will tend to ignore ideas originated
outside their own social group. However, there was a stronger force at work
at Berkeley, where a certain social stratification prevails that finds Nobel
Laureates and department chairs ranking as demigods, professors functioning
as high priests, graduate students considered as lower class citizens, and
undergraduates existing only on sufferance from the higher orders --
and suffered very little at that. Now, the individuals cannot be blamed for
what is, in essence, an entire social order. But this is not to say that we
did not hold it against them -- for we most assuredly did. Unfortunately, it
took time for us to appreciate the difficulties of Fighting City Hall.
This is why We were frustrated. This is why We felt We HAD to break security.
Once We did, We simply added Our features to the system, whether The Powers
That Be liked it or not. Needless to say, They didn't. This is why We felt
like freedom fighters, noble figures even when found in the ignoble position
of lying face down on the floor of Cory Hall at two in the morning.
We were on a mission that morning to install our new terminal driver. With the
old, standard terminal driver, the screen gave you no indication that the
previous character had been deleted when you pressed the erase character.
You had to accept it on faith. This remains true on many UNIX systems today.
Most people on Cory Hall UNIX changed their erase character to backspace so
that later characters would overwrite the erased ones, but even that was not
sufficient. This was especially true when erasing a backslash, which
counter-intuitively required two erase characters. We wanted the system to
show that the character was gone by blanking it out. We also wanted the
line-erase character to display a blanked-out line. Some UNIX systems such as
4.2 BSD and System V now support this, but it was not then available anywhere
under UNIX version 6.
Bob and I had argued, somewhat sleepily, for hours as to the correct method
of erasing characters, and Bob had started putting our joint design into
effect just as I collapsed on the floor for "a short nap". I awoke around
dawn to find Bob asleep over the terminal. When he woke up, he said he was
pretty sure he'd finished the job before falling asleep, but neither of us
had enough energy to check. It was time for food and 14 hours of sleep.
When we finally checked our handiwork the next day, we found some serious flaws
in the implementation -- not an uncommon situation with work performed under
extreme conditions. But the system was up and running, and although the new
features were flawed, they didn't seem to cause any problems, so we forgot
about it for the time being. A week later, I was consulting in Cory --
we all offered free programming help to other students in the time-honored
tradition of hackers everywhere -- when Kurt Schoens called me over to the
other side of the room.
"Hey Doug," he said. "Look at this. It looks like someone tried to put
character deletion into the terminal drivers, but only half finished."
My heart raced. Did he suspect me? Or was he just chatting? I could never
tell whether Kurt was kidding; he had the most perfect poker face I had ever
But he quickly made the question academic, and proved again that he was one
"I showed this to Bill, and he wanted to fix it", Kurt said. "Oh,
really?" I stammered. "Sounds good to me," thinking that it was a real stroke
of luck that Bill Joy would be interested in the half-completed project. If
Bill finished it, then it would be in the system on legitimate grounds, and
would stay for good.
Kurt paused for effect. "Yeah, he was all fired up about it, but I talked him
out of it, and I just deleted it from the system instead."
Oh, cruel fate! Kurt must know that I was involved; he just wanted to see me
jump when he said "boo!"
Although I'm sure Kurt thought the whole incident very funny, all I could think
of was that yet another of my features had gone down the drain. I discussed
this latest setback with others in the group, and we shared a sense of
frustration. More than ever before, we were determined to get our
contributions accepted somehow.
Kurt was both a graduate student and a system administrator, but I liked him
all the same -- chiefly because of his practical jokes. We had recently
cooperated in a spontaneous demonstration of Artificial Intelligence at the
expense of an undergraduate named Dave who had joined Them as a system
administrator. Dave had watched Kurt as he typed pwd to his
shell prompt and received /usr/kurt/mind as the response. His
next command had been mind -i -1 english. During all this
time, Kurt was double-talking about psychology and natural language processing
and some new approach to simulating the human mind that he'd thought of.
Dave looked dubious, but was willing to see how well Kurt's program worked.
What Dave didn't realize was that Kurt had not been typing commands to the
system at all; although we were sitting not 10 feet apart, Kurt and I had been
writing to each other and chatting for half an hour, and as a joke I had been
pretending I was Kurt's shell, sending him prompts and faking responses to
Dave had walked in at just the right time. So when Kurt typed
mind -i -1 english, I had naturally responded with:
"Synthetic Cognition System, version 17.8"
"Interactive mode on, Language=english"
"Please enter desired conversational topic: (default:philosophy)"
Dave couldn't help looking a little impressed; Kurt's "artificial intelligence"
system was off to a great start. Kurt had talked to his budding mind for
several minutes, and Dave of course had grown more and more impressed. Kurt
and I faced the greatest challenge of our lives in keeping a straight face
during the demonstration, but we eventually made the mistake of making the mind
altogether TOO smart to be believable, in effect sending Dave off to tackle
more serious work.
There was one practical joke that was notable for the length of time that it
was supported by the entire group. The target was system administrator Dave
Mosher. Dave had been suspicious of bugs in our system's homebrewed terminal
multiplexer for some time. Ross decided to persecute Dave by having random
characters appear on his screen from time to time, which of course convinced
Dave that the terminal multiplexer did indeed have problems. To help Ross
with the prank, each of us sent Dave some garbage characters at random
intervals whenever any one of us was on the system. We had settled on the
letter "Q" so that Dave would be sure it was always the same bug showing
the same symptoms. Since Dave had these problems no matter which terminal he
was on, day or night, no matter who else was logged onto the system, he was
positive there was a problem, and he spent much time and effort trying to get
someone to fix it.
Unfortunately for Dave he was the only one who ever saw these symptoms, so
everyone thought he was a little paranoid. We thought it was pretty funny at
first, but after a few months of this, it seemed that Dave was really getting
rattled, so one day Ross generated a capital "Q" as big as the entire screen
and sent it to Dave's screen. This made it pretty obvious to poor Dave that
someone, somehow, really had been persecuting him, and that he wasn't
paranoid after all. He had an understandably low tolerance for practical jokes
The numerous practical jokes we played were probably a reaction to the high
level of stress we felt from our ongoing illicit operations; it provided some
moments of delightful release from what was, at times, a grim battle.
There were many secret battles in the war; if Our motto was "Features!",
Theirs was "Security for Security's Sake" and the more the better. We were
never sure how long our victories would last; on the other hand, They were
never sure whether They had won. The war lasted almost three years.
We were primarily interested in the EECS department's PDP 11/70 in Cory Hall,
since that was the original UNIX site and continued to be the hotbed of UNIX
development, but We "collected" all the other UNIX systems on campus, too. One
peculiar aspect of the way the Underground had to operate was that we rarely
knew the root password on systems to which we had gained superuser access.
This is because there were easier ways to get into, and stay into, a system
than guessing the root password. We tampered, for instance, with the
su program so that it would make someone superuser when given
our own secret password as well as when given the usual root password, which
remained unknown to us. In the early days, one system administrator would
mail a new root password to all the other system administrators on the system,
apparently not realizing that we were monitoring their mail for exactly this
kind of security slip. Sadly, they soon guessed that this was not a good
procedure, and we had to return to functioning as "password-less superusers",
which at times could be a bit inconvenient.
Late one night on Cory Hall UNIX, as I was using my illegitimate superuser
powers to browse through protected but interesting portions of the system, I
happened to notice a suspicious-looking file called /usr/adm/su.
This was suspicious because there were almost never new files in the
administrative /usr/adm directory. If I was suspicious when I saw the
filename, I was half paralyzed when I saw it contained a full record of every
command executed by anyone who had worked as superuser since the previous day,
and I was in a full state of shock when I found, at the end of the file, a
record of all the commands that I'd executed during my current surreptitious
session, up to and including reading the damning file.
It took me perhaps 10 minutes of panic-stricken worry before I realized that I
could edit the record and delete all references to my illicit commands. I then
immediately logged out and warned all other members of the group. Since
nothing illicit ever appeared, the system administrators were lulled into a
sense of false security. Their strategy worked brilliantly for us, allowing us
to work in peace for quite a while before the next set of traps were laid.
The next potential trap I found was another new file in /usr/adm
called password, that kept track of all unsuccessful attempts to login
as root or to su to root, and what password was used
in the attempt. Since none of us had known the root password for months and
therefore weren't going to become superuser by anything as obvious as logging
in as root, this wasn't particularly threatening to us, but it was very
interesting. The first few days that we watched the file it showed attempts
by legitimate system administrators who had made mistakes of various sorts.
One of Them once gave a password that We discovered, through trial and
error, to be the root password on a different system. Several of Them gave
passwords that seemed to be the previous root password. Most of them were
misspellings of the correct root password. Needless to say, this was a
rather broad hint, and it took Us less than five minutes to ascertain what the
correct spelling was.
One might think that, since we had several ways to become superuser anyway, it
wouldn't make any real difference whether or not we knew the actual root
password as well. The problem was that our methods worked only so long as
nothing drastically changed in the system; the usual way that They managed to
win a battle was to backup the entire system from tape and recompile all
utilities. That sometimes set Us back weeks, since it undid all of our
"backdoors" into superuserdom, forcing us to start from ground zero on breaking
into the system again. But once we knew the root password, we could always
use that as a starting place.
We worked very hard to stay one step ahead of Them, and we spent most of our
free time reading source code, in search of either pure knowledge or another
weapon for the battle. At one time, I had modified every single utility that
ran as superuser with some kind of hidden feature that could be triggered to
give us superuser powers. Chuck Haley once sent a letter to Jeff Schriebman
commenting that he "had even found the card reader program" to show signs of
tampering. I thought that I had disguised it well, but it was extremely
difficult to keep things hidden from a group of system administrators who were
not only very intelligent, but also highly knowledgeable about the inner
workings of UNIX. As an indication of the caliber of the people we were
working against, I should note that Chuck Haley is now a researcher at Bell
Labs; Bill Joy is VP of Engineering at Sun Microsystems; Kurt Schoens is a
researcher at IBM; Jeff Schriebman is founder and President of UniSoft;
and Bob Kridle, Vance Vaughn, and Ed Gould are founders of Mt. Xinu.
This was an unusual situation; system administrators are not usually this
talented. Otherwise, they'd be doing software development rather than
administration. But at the time, there was no one else capable of
doing UNIX system administration.
As a result, we had to move quickly, quietly, and cleverly to stay ahead, and
planting devious devices in the midst of standard software was our primary
technique. Normally trusted programs which have been corrupted in this way are
called "Trojan Horses",
after the legend of the Greeks who were taken in by a bit of misplaced trust.
One of our favorite tricks for hiding our tracks when we modified standard
utilities was the diddlei program, which allowed us to reset
the last change time on a modified file so that it appeared to have been
unchanged since the previous year. Bob modified the setuid
system call in the UNIX kernel so that, under certain circumstances, it would
give the program that used it root privileges. The "certain circumstances"
consisted simply of leaving a capital "S" (for Superuser) in one of the
machine's registers. Bob was bold enough to leave this little feature in the
system's source code. We usually put our Trojan Horses in the system
executables only -- to decrease the chance of it being noticed. But Bob took
the chance so that the feature would persist even if the system were
recompiled. Sure enough, it lasted several months and through more than one
system compilation before Dave Mosher noticed it (undoubtedly with a sense of
shock) as he was patiently adding comments to the previously undocumented
This sort of battling continued for several years, and although They were
suspicious of most of Us at one time or another, none of Us was ever caught
red-handed. It undoubtedly helped that we never performed any malicious acts.
We perhaps flouted authority, but we always enhanced the system's features.
We never interfered with the system's normal operation, nor damaged any user's
files. We learned that absolute power need not corrupt absolutely; instead it
taught us restraint.
This is probably why we were eventually accepted as members of the system
staff, even though by then several of Us had confessed to our nefarious deeds.
Once we were given license to modify and improve UNIX, we lost all
motivation to crack system security. We didn't know it at the time, but this
has long been known to be one of the most effective ways of dealing with
security problems; hire the offenders, so that there is no more Us verses
Them, but simple Us.
It worked well in our case; under the auspices of the System Development and
Research Group, created by the ever-industrious Dave Mosher, we went happily
to work on UNIX development. The development of UNIX at Berkeley, always
fast-paced, exploded once everyone -- including undergraduates -- were
The only fly in the ointment was the introduction a short while later of UNIX
Version 7. While it was a vast improvement in many ways over the Version 6
that we had been working with, most of the enhancements we had developed were
lost in the changeover. Some were reimplemented under Version 7 by those of
the group who remained at Berkeley, but by then many of us were leaving
school, and the impetus behind our ideas left with us.
Ken Arnold is, perhaps, the most famous of our original group. He stayed at
Berkeley longer than any of the rest of us, and became well known for such
contributions as Termlib, curses, fortune, Mille Bourne,
and of course his co-authorship of Rogue. But somehow it seemed a Pyrrhic
victory even for Ken; much of his best work in the early years never saw
the light of day.
We could not help but feel that we had passed through a sort of Dark Age for
UNIX development, and even with the Renaissance in full bloom, We ponder what
might have been, and bewail the features that UNIX will now never have.
Doug Merritt became one of the earliest UNIX users outside of Bell Laboratories
while attending UC Berkeley in 1976. He helped to debug
termcap and contributed to the development of
vi and curses. Mr. Merritt now works as a
consultant in the San Francisco Bay Area.
Bob Toxen is a member of the technical staff at Silicon Graphics, Inc,
who has gained a reputation as a leading expert on uucp
communications, file system repair and UNIX utilities. He has also done
ports of System III and System V to systems based on the Zilog 8000
and Motorola 68010 chips.
Best known as the author of curses and co-author of Rogue,
Ken Arnold was also President of the Berkeley Computer Club and the
Computer Science Undergraduates Association during his years at UC Berkeley.
He currently works as a programmer in the Computer Graphics Lab at UC
San Francisco and serves as a member of the UNIX Review
Software Review Board.
Copyright © 1984, 2007, 2014 by Doug Merritt, Ken Arnold, and Bob Toxen. All rights reserved.