BASH Scripting Landmines

BASH/command-line scripting is a powerful tool. However, many of its behavior is somewhat counter-intuitive and I’d argue adversarial. I will showcase some these unexpected results by incrementally hardening a simple script.

Failure is an Option

#!/bin/bash
echo "Launching rocket!"
ROCKET_TO_LAUNCH=$(get-available-rocket)
echo "I'm still running even though a command failed!"
echo "Launching rocket $ROCKET_TO_LAUNCH"

Unlike what you might expect, the default behavior is for a script to resume execution even if a command fails. Here’s the output:

Launching rocket!
./errexit.sh: line 5: get-available-rocket: command not found
I'm still running even though a command failed!
Launching rocket

This can be fixed using set -o errexit .

#!/bin/bash
set -o errexit
echo "Launching rocket!"
ROCKET_TO_LAUNCH=$(get-available-rocket)
echo "I'm still running even though a command failed!"
echo "Launching rocket $ROCKET_TO_LAUNCH"

And we get the following output:

Launching rocket!
./errexit-fixed.sh: line 7: get-available-rocket: command not found

Failure is still an Option

What set -o errexit still doesn’t cover is when an intermediate command in a pipe fails because BASH default behavior considers it successful if the last command in the sequence is successful.

#!/bin/bash
set -o errexit
echo "Launching rocket!"
ROCKET_TO_LAUNCH=$(get-available-rocket | echo unexpected-result)
echo "I'm still running even though a command failed!"
echo "Launching rocket $ROCKET_TO_LAUNCH"

In this case we get the following output:

Launching rocket!
./pipefail.sh: line 7: get-available-rocket: command not found
I'm still running even though a command failed!
Launching rocket unexpected-result

We can also fix this with another option (`set -o pipefail`).

#!/bin/bash
set -o errexit
set -o pipefail
echo "Launching rocket!"
ROCKET_TO_LAUNCH=$(get-available-rocket | echo unexpected-result)
echo "I'm still running even though a command failed!"
echo "Launching rocket $ROCKET_TO_LAUNCH"

We now get the following output:

Launching rocket!
./pipefail-fixed.sh: line 8: get-available-rocket: command not found

Can’t Say no to a Variable

As good as errexit and pipefail are, they don’t protect against using non-exiting/unset-variables.

#!/bin/bash
set -o errexit
set -o pipefail
echo "Launching rocket!"
ROCKET_TO_LAUNCH=$(echo $NON_EXISTING_VARIABLE)
echo "I'm still running even though a command failed!"
echo "Launching rocket $ROCKET_TO_LAUNCH"

The output in this case is as follows:

Launching rocket!
I'm still running even though a command failed!
Launching rocket

For this, we can use set -o nounset .

#!/bin/bash
set -o errexit
set -o pipefail
set -o nounset
echo "Launching rocket!"
ROCKET_TO_LAUNCH=$(echo $NON_EXISTING_VARIABLE)
echo "I'm still running even though a command failed!"
echo "Launching rocket $ROCKET_TO_LAUNCH"

And we get the following output:

aunching rocket!
./nounset-fixed.sh: line 9: NON_EXISTING_VARIABLE: unbound variable

Parting Thoughts

This only scratches the surface of BASH landmines and even some necessary background information is not covered (e.g: exit status, bin/sh , interactive vs non-interactive).

So as broken as BASH is, the world still goes round. But how!?

It’s quite simple and surprisingly non-technical. From first principles, we understand how BASH, other programming languages, and systems (technical & otherwise) work but what we understand is the minority and what we don’t is the vast majority. Add to that that even the minority we understand, the majority of us don’t.

Another important point is that even without any technical knowledge, it can be easily deduced that a system with a couple of surprising behaviors is almost certain to have many more and programming languages that are used to build bigger systems require even more care. And this is the main takeaway here.

Appendix — Highlighted Code