Chapter 6 — The Blast Radius Chapter
Draft status: author draft, gate-checked; human verification pending. Every destructive mechanism in this chapter is demonstrated inside a scratch directory the listing itself creates, or shown as a printed plan rather than an execution. That caging is not stagecraft; it is the chapter’s own doctrine, practiced on itself.
Finality, as an engineering requirement
Chapter 1 named the cost: a one-shot operator’s last influence over a command ends at dispatch. There is no watching the first deletions scroll past, no Ctrl-C at the moment doubt arrives, no glance at the prompt’s directory before pressing enter — every safety mechanism interactive humans actually rely on turns out to live during execution, in the seconds this register does not experience. What remains is composition time, and this chapter is about spending it. The doctrine in one line: a destructive command must be made safe before it runs, because nothing can make it safe while it runs. The good news, argued through chapter 3, is that the overwhelming majority of administration is reading, which needs none of this chapter. The discipline concentrates on the minority of shots that write, remove, or reach outside the machine — and because they are a minority, the register can afford to make each one carry guards that would feel ceremonious at an interactive prompt. That asymmetry — free reads, ceremonial writes — is the whole risk posture of a well-run non-interactive operator.
The dominant accident: the space in the name
Ask interactive administrators about catastrophic commands and they describe typos. The register’s dominant accident class is different and more mechanical: an expansion changed the command between composition and execution. The shell rewrites your command before running it — variables expand, the results split into words on whitespace, the words glob against the filesystem — and each rewrite is a place where the command you composed and the command that runs can diverge. The canonical specimen, caged in a scratch directory:
cd "$(mktemp -d)"
f="release notes.txt"
touch "$f" notes.txt
rm $f 2>&1
printf "%s\n" *
rm: cannot remove 'release': No such file or directory
release notes.txt
Read the transcript as the four-question routine demands, because it is a small
horror story. The unquoted $f split into two words, release and notes.txt. The
first named nothing — hence the error line. The second named an innocent bystander,
which rm removed without comment. The final line is the listing’s own survivor
roll: the target, release notes.txt, stands untouched — and notes.txt, created
two commands earlier, is not in the list, because it no longer exists. One unquoted
expansion produced a command that failed at its purpose and destroyed something
unrelated, with an error message pointing at neither fact. The cure costs two characters: rm "$f" names one file, spaces included,
always. The rule admits no judgment calls: every expansion is quoted —
"$var", "$(cmd)", "$@" — unless splitting is the explicit, commented intent.
Not because every variable will contain a space, but because the operator composing
the shot cannot see the value at composition time, and this register has no other
moment to check. A mechanical rule for a mechanical accident; shellcheck enforces
it (its finding SC2086 is precisely this), and running ShellCheck over any script
before dispatch is the register’s equivalent of proofreading — a static gate for the
class of accident no runtime gate can catch in time.
The expansion accident has a second, worse form: the variable that expands to nothing. Chapter 2 previewed it; here is its anatomy, demonstrated as a printed plan rather than an execution, which is how a plan this dangerous should first exist:
```bash no-run prefix=”” echo “would run: rm -rf ${prefix}/cache”
```output
would run: rm -rf /cache
An empty prefix — unset, typo’d, or emptied by a failed substitution upstream —
and the composed path collapses to an absolute path at the filesystem root. Every
veteran of this register knows an incident of this shape; the best-documented
public specimen shipped in Steam’s Linux client, whose startup script ran
rm -rf "$STEAMROOT/"* and, on machines where the variable came up empty,
deleted every file the user could reach — bug report in the references, preserved
complete with its disbelieving comment thread. The defenses stack. set -u aborts on the
unset case. The shell’s own ${prefix:?prefix is unset} expansion makes the check
inline and fatal without strict mode. Better than both, because it also catches the
set-but-wrong case: test the composed path’s existence and shape before acting on
it — a directory that should contain cache files can be required to exist and to
match a pattern you assert ([ -d "$prefix/cache" ] && [[ "$prefix" == /srv/* ]])
before any destructive verb sees it. That is proof-of-target, and a later section
makes it doctrine.
When filenames attack
The third rewrite, globbing, has two failure shapes of its own. The first: a glob
that matches nothing passes itself through as a literal word by default —
*.conf in an empty directory becomes the string *.conf, handed to your command
as if it were a filename, and whether that is harmless depends entirely on the
command. The second is nastier — filesystem content becomes command syntax:
cd "$(mktemp -d)"
touch -- -l data.log
echo "ls * sees:"
ls * 2>&1 | head -n 1
echo "ls -- * sees:"
ls -- * 2>&1
ls * sees:
-rw-r--r-- 1 roger roger 0 Aug 27 22:07 data.log
ls -- * sees:
-l
data.log
A file named -l exists in the directory. The glob expands it along with everything
else, ls receives it as an option, and the first invocation silently becomes
ls -l data.log — a long listing of the other file, with the dash-file itself
vanished from the report. Any tool accepting options is vulnerable, and with rm or
chmod the smuggled option can be -r or --no-preserve-root. Two idioms close
the hole completely, and the listing shows the first: --, the near-universal
end-of-options marker, after which everything is an operand. The second is prefixing
relative globs with ./ (rm ./* — a name cannot begin with a dash if it begins
with ./). One of the two belongs in every command whose operands come from a glob
or a variable; which one is taste, their absence is exposure.
Silent semantic reversal has one more famous residence: tools whose argument order
is meaning, where a plausible reordering runs and does something categorically
different. The specimen is find, whose command line is not options-then-operands
but a little program evaluated left to right:
cd "$(mktemp -d)"
touch a.tmp b.txt
echo "misordered:"
find . -type f -print -name "*.tmp"
echo "correct:"
find . -type f -name "*.tmp" -print
misordered:
./b.txt
./a.tmp
correct:
./a.tmp
With -print before the name filter, the action fires for every file and the
filter, evaluated after, filters nothing. Substitute -delete for -print — a
substitution operators make exactly once — and the misordering deletes every file
under the tree instead of the .tmp files; same shape, and -delete even disables
some of find’s own safety refusals. The register’s habit: any find that will
carry a destructive action is composed first with -print in the action’s position,
dispatched, and its output read as the plan — then, and only then, re-dispatched
with the action swapped in. Which generalizes into the chapter’s central move.
Rehearsal: the dry run
Interactive operators sometimes rehearse; the register must, because rehearsal is
the only place its mistakes can be caught before they are permanent. The instruments
are the dry-run modes the serious tools all carry, and the doctrine is to treat them
not as reassurance but as the source of the plan you then execute. The worked
specimen is rsync, whose --delete — essential for true synchronization, feared
for good reasons — is exactly the kind of verb that deserves a rehearsal:
cd "$(mktemp -d)"
mkdir -p src dst
printf "1\n" > src/a.txt
printf "2\n" > dst/stale.txt
rsync -rn -v --delete src/ dst/ 2>&1 | head -n 6
echo "dst after the dry run:"
ls dst
sending incremental file list
deleting stale.txt
a.txt
sent 66 bytes received 32 bytes 196.00 bytes/sec
total size is 2 speedup is 0.02 (DRY RUN)
stale.txt
The -n run names its victims — deleting stale.txt — and the final ls proves the
victim still breathes: the rehearsal was pure prophecy, zero action. The transcript
now contains the plan, reviewable by the operator or its supervisor, and the
execution step is the identical command minus one letter. Note also what the
rehearsal quietly validated: rsync’s infamous trailing-slash semantics (src/ means
contents of src; src means the directory itself — one character, two different
resulting trees). The dry run renders that decision visible before it is real, which
is the general principle: a dry run converts semantics you believe into semantics
you have read. The register’s rehearsal shelf is well stocked — patch --dry-run
(chapter 5), apt-get -s and its cousins for package operations, systemd-analyze
verbs for units and calendars (chapter 4), --check flags across configuration
tools — and where a tool has none, the find -print-then-swap pattern builds one.
The habit has a cost — one extra turn per destructive operation — and chapter 1’s
economics prices it correctly: the turn is the cheapest thing this register has, and
it buys down the one cost that cannot be refunded.
Proof of target, narrowness, and the verb that shows its work
Rehearsal checks the plan; proof-of-target checks the world. Before a destructive verb runs, the transcript should already contain evidence that its operand is the thing intended — existence, type, and a property no wrong target would share:
cd "$(mktemp -d)"
mkdir -p build
printf "artifact\n" > build/out.bin
[ -d build ] && [ -e build/out.bin ] \
&& rm -v build/out.bin \
&& [ ! -e build/out.bin ] && echo "removal verified"
removed 'build/out.bin'
removal verified
Four small assertions braid chapter 2’s ask-and-verify into the destructive case:
the target’s existence proven before, the verb’s own verbose narration (-v — the
register always asks destructive tools to narrate) during, and the absence proven
after. Alongside proof stands narrowness: the destructive verb receives the most
specific operand the task permits. Full explicit paths over cd-then-relative
(the cd that fails while the rm proceeds is a classic compound accident — and
the reason chapter 2 recommended strict mode’s abort-on-error for scripts);
one named target over a glob where one target is meant; a glob over a recursive
flag where a glob suffices; -maxdepth on any find that does not intend the
abyss. Blast radius is measured at composition time by a simple question: if every
name in this command resolved to the worst plausible value, what is the largest
thing that could disappear? Narrowness is the practice of making that answer
small enough to survive.
The reversibility ladder
The final recalibration is choosing verbs by their undo channel. Interactive humans
grade operations by effort; the register grades them by recoverability, and the
grades are a ladder worth internalizing. At the top, operations that carry their own
undo: mv within a filesystem (rename back), the drop-in file (delete it), the
atomic swap staged beside a kept original (swap back). In the middle, operations
recoverable with preparation: overwrites preceded by a copy (cp -a target
target.bak.$(date -u +%Y%m%dT%H%M%SZ) — timestamped, so repeated preparations do
not overwrite each other’s insurance), deletions rehearsed and logged so that at
minimum what was lost is known. At the bottom, the truly one-way verbs: rm,
> truncation, rsync --delete unrehearsed, dd onto a device, database drops —
gone is gone, on a timescale that matters to the incident.
The ladder’s use is substitution pressure: before dispatching a bottom-rung verb,
ask which higher rung reaches the same goal. The strongest everyday substitution is
quarantine — mv the doomed thing into a dated graveyard directory instead of
removing it. The operation is total (the directory is clean, the goal achieved),
reversible for as long as the graveyard is retained, and cheap — a rename costs
nothing regardless of size. Deletion becomes a scheduled, boring event (a timer
purging graveyards older than some retention), decoupled from the operational moment
where mistakes live. An operator that quarantines by default and deletes on a
schedule has converted its worst accident class into a recoverable inconvenience —
which is the entire ambition of this chapter, applied structurally rather than shot
by shot.
The verbs that leave the machine
The reversibility ladder has a floor below its bottom rung, and it is not on the
filesystem. Commands that communicate — send the email, post to the API, push
the release, publish the message to the queue — are irreversible in a way even
rm is not: deletion destroys state you held, while communication creates state
in systems you do not hold. There is no quarantine directory for a sent
notification; the copy that matters now lives in someone else’s inbox, cache, or
audit log, beyond every undo channel this chapter built. The register treats such
verbs as a class of their own, with three rules. They are never composed into
retry loops or strict-mode chains casually — a “retry on failure” wrapped around a
send can deliver twice, and chapter 6’s read-before-retry rule applies with the
reading done on the far system where possible (did the message arrive? does the
API’s idempotency key say this request was already seen?). Idempotency keys, where
the far side offers them, are the outward world’s version of the guarded append,
and using them is not optional politeness but the only mechanism that makes an
outward retry safe at all. And the rehearsal principle inverts into staging:
where a dry run cannot exist (few mail systems offer one), the rehearsal is a
real send against a target you own — the test inbox, the sandbox API, the
staging channel — with the production dispatch composed as the same command with
one variable changed, and that variable proven, proof-of-target style, before the
shot goes out. An operator that would rehearse a local rsync --delete three
times and then improvise a production API call has ranked its risks by
familiarity rather than by blast radius; this section exists to reverse that
ranking.
Working where others work
The blast radius of a command includes operators it collides with. A one-shot operator is rarely alone on a machine: humans hold sessions, timers fire (chapter 4’s real machine runs several), other agents may be mid-task in the same trees. Two consequences follow, one about your writes, one about everyone else’s.
Your writes need private ground. Predictable scratch paths — /tmp/work,
/tmp/out.txt — are collisions waiting (two operators, one path, interleaved
writes) and, on shared machines, a classic attack surface: a hostile process that
pre-creates the predictable name as a symlink redirects your write onto any file
your identity can damage. The system’s answer is mktemp, which this book’s
listings have used since chapter 1 and which earns its formal introduction here:
w=$(mktemp -d)
echo "workspace: $w"
[ -d "$w" ] && echo "exists, mode $(stat -c %a "$w")"
workspace: /tmp/tmp.NwLRJZiNbQ
exists, mode 700
Unpredictable name, created atomically, mode 700 — private by construction. The
habit: every scratch need goes through mktemp; the only names you write outside
scratch are the deliberate, guarded targets of the task itself.
Against everyone else’s writes, the instrument is mutual exclusion, and the shell
has a real one — flock(1), an advisory lock on a file descriptor, atomic in the
kernel, released automatically when the holding process exits (an important property
for an operator whose process will exit, cleanly or not — no crash leaves a stale
lock held):
cd "$(mktemp -d)"
exec 9> job.lock
if flock -n 9; then echo "lock acquired"; else echo "another operator holds the lock"; fi
( exec 8> job.lock
flock -n 8 && echo "second acquisition: succeeded" || echo "second acquisition: refused" )
lock acquired
second acquisition: refused
The subshell plays the second operator, and the kernel refuses it while the first
descriptor lives. -n makes the attempt non-blocking — the register’s correct
default, since a shot that queues invisibly behind a lock is a chapter 1 hang with
extra steps; better to learn refused in one turn and decide, than to wait
silently. Any procedure that must not run twice concurrently — a deploy, a
migration, a graveyard purge — opens with this pattern, and the lock file’s name
becomes part of the procedure’s documented interface, which is precisely how the
system cron daemons and package managers already guard themselves (the familiar
“could not get lock” from a package manager is this same mechanism, experienced
from the outside).
The retry, and what must be read before it
Chapter 5 built idempotence into edits; this chapter’s version of the question is harsher: a destructive or compound shot failed midway, or timed out with its fate unknown — what now? The interactive reflex, run-it-again, is exactly wrong as a reflex, because a compound operation that died mid-flight has left the world in a state that neither the before nor the after picture describes, and the retry was composed against the before picture. The register’s rule: after a failed write, the next shot is a read. Reconstruct which stage completed from evidence — which files exist, which staged artifacts are present, what the service’s timestamps say — then resume from the stage the evidence proves, not from the top:
cd "$(mktemp -d)"
mkdir -p deploy
printf "v2\n" > deploy/app.txt.new
ls deploy
echo "staged file present: retry resumes at validate-and-swap, not at generation"
app.txt.new
staged file present: retry resumes at validate-and-swap, not at generation
A toy, but the shape is the real discipline: chapter 5’s atomic-swap pipeline,
interrupted, is legible — the presence of the .new file tells the returning
operator exactly where death occurred, and the resume point follows from the
evidence rather than from hope. That legibility was purchased at composition time,
by building the procedure from stages whose completion leaves distinct marks. The
general design rule for anything long: make each stage’s completion observable
(a staged file, a moved marker, a logged line), so that the mid-flight state reads
as a position rather than as wreckage — chapter 8 formalizes the same idea as the
change ledger. And when a timed-out shot’s fate is genuinely unknowable — the
classic being a remote command whose connection died after dispatch — the read
comes first and decides whether there is anything to retry at all: the operation
may have completed beautifully, in your absence, like everything else in this
register does.
Privilege: the discipline of remaining small
Everything above bounds what a command does; privilege bounds what it can do,
and the register’s rule is inherited from decades of automation practice with the
stakes newly raised: operate at the lowest privilege that answers the question,
and escalate per-command, not per-session. Chapters 3 and 4 were written
deliberately from an unprivileged seat — processes read, services diagnosed, a
failed unit triaged to its exit status, all without root; the one boundary hit (the
journal’s group wall) was itself diagnosable from below. That is representative:
reading rarely needs privilege, and the shots that do escalate should each carry
sudo visibly — the transcript then shows exactly which actions ran elevated,
an audit trail chapter 8 will want, against a sudo -i session in which every
subsequent accident, expansion included, is a root accident. sudo -l reads the
current identity’s actual grants — worth one shot on any new machine, since it is
the authoritative answer to what can this seat do, and (chapter 4’s lesson
recurring) sudo strips most of the environment, so a command that behaves
differently under it has usually lost a variable, not gained a bug. For the
supervising reader designing the seat itself: the strongest blast-radius decision
is made before any command runs, in what the operator’s account can reach at all —
filesystem permissions, group memberships, a sudoers file listing specific commands
rather than ALL. The guards in this chapter are how a careful operator behaves;
the account is how careful the machine remains when, someday, a shot is composed
without them.
Quoted expansions, guarded emptiness, disarmed filenames, rehearsed deletions, proven targets, reversible verbs, and a small seat: none of it is exotic, and all of it is composition-time work, because composition time is all this register has. What safety cannot be compressed into is a checklist recited once — it is a set of defaults, and defaults are cheap. The next chapter takes the same doctrine onto the one substrate where even reads have side effects and nothing stays still: the network.