BASH Scripting Landmines
BASH/command-line scripting is a powerful tool. However, many of its behavior is somewhat counter-intuitive and I’d argue adversarial. I will showcase some these unexpected results by incrementally hardening a simple script.
Failure is an Option
#!/bin/bashecho "Launching rocket!"ROCKET_TO_LAUNCH=$(get-available-rocket)echo "I'm still running even though a command failed!"echo "Launching rocket $ROCKET_TO_LAUNCH"
Unlike what you might expect, the default behavior is for a script to resume execution even if a command fails. Here’s the output:
Launching rocket!
./errexit.sh: line 5: get-available-rocket: command not found
I'm still running even though a command failed!
Launching rocket
This can be fixed using set -o errexit
.
#!/bin/bashset -o errexitecho "Launching rocket!"ROCKET_TO_LAUNCH=$(get-available-rocket)echo "I'm still running even though a command failed!"echo "Launching rocket $ROCKET_TO_LAUNCH"
And we get the following output:
Launching rocket!
./errexit-fixed.sh: line 7: get-available-rocket: command not found
Failure is still an Option
What set -o errexit
still doesn’t cover is when an intermediate command in a pipe fails because BASH default behavior considers it successful if the last command in the sequence is successful.
#!/bin/bashset -o errexitecho "Launching rocket!"ROCKET_TO_LAUNCH=$(get-available-rocket | echo unexpected-result)echo "I'm still running even though a command failed!"echo "Launching rocket $ROCKET_TO_LAUNCH"
In this case we get the following output:
Launching rocket!
./pipefail.sh: line 7: get-available-rocket: command not found
I'm still running even though a command failed!
Launching rocket unexpected-result
We can also fix this with another option (`set -o pipefail`).
#!/bin/bashset -o errexit
set -o pipefailecho "Launching rocket!"ROCKET_TO_LAUNCH=$(get-available-rocket | echo unexpected-result)echo "I'm still running even though a command failed!"echo "Launching rocket $ROCKET_TO_LAUNCH"
We now get the following output:
Launching rocket!
./pipefail-fixed.sh: line 8: get-available-rocket: command not found
Can’t Say no to a Variable
As good as errexit
and pipefail
are, they don’t protect against using non-exiting/unset-variables.
#!/bin/bashset -o errexit
set -o pipefailecho "Launching rocket!"ROCKET_TO_LAUNCH=$(echo $NON_EXISTING_VARIABLE)echo "I'm still running even though a command failed!"echo "Launching rocket $ROCKET_TO_LAUNCH"
The output in this case is as follows:
Launching rocket!
I'm still running even though a command failed!
Launching rocket
For this, we can use set -o nounset
.
#!/bin/bashset -o errexit
set -o pipefail
set -o nounsetecho "Launching rocket!"ROCKET_TO_LAUNCH=$(echo $NON_EXISTING_VARIABLE)echo "I'm still running even though a command failed!"echo "Launching rocket $ROCKET_TO_LAUNCH"
And we get the following output:
aunching rocket!
./nounset-fixed.sh: line 9: NON_EXISTING_VARIABLE: unbound variable
Parting Thoughts
This only scratches the surface of BASH landmines and even some necessary background information is not covered (e.g: exit status, bin/sh
, interactive vs non-interactive).
So as broken as BASH is, the world still goes round. But how!?
It’s quite simple and surprisingly non-technical. From first principles, we understand how BASH, other programming languages, and systems (technical & otherwise) work but what we understand is the minority and what we don’t is the vast majority. Add to that that even the minority we understand, the majority of us don’t.
Another important point is that even without any technical knowledge, it can be easily deduced that a system with a couple of surprising behaviors is almost certain to have many more and programming languages that are used to build bigger systems require even more care. And this is the main takeaway here.