How to Debug a Bash Script like a Boss

A lot of things can happen in a bash script that unless you know how to debug like a boss, you might as well be up a creek without a paddle. Here a short list of debugging techniques followed by examples that can be applied to any issue in bash programming.

• Use options (bash specific)
• Use breakpoints
• Use debug output
• Use shopts (bash specific)

Use options to debug scripts like a boss (bash specific)

Bash comes with a set of options (let’s just say 25 just to keep things simple) to allow for modification of script behavior if the need presents itself. In case you are about to debug a bash script it does. Have no fear, there only 3 that you will need to get started. The other will come into play later.

Here is a simple bash script using options for debugging.

Source: debug-options.sh

Now consider the following script that may not succeed. It is simple but perfect to help practice bash script debugging.

Source: debug-random-error.sh

When you are debugging a bash script don’t be afraid to ask yourself what-if.

Example i) What-if the program exits with an exit status of 0 but the outcome says otherwise

What-if there is an explosion in one of your functions but the script keeps on chugging. In fact, before anything big gets lit up, simple test will fail. The set -e option command will cause your script to halt on any failed test and exit with an exit status as error.

The question now is where to introduce the exit on error option command while debugging.

There are 3 options.

Option i) Modify the script, interpreter line

Now you run the script 10 times.

And get the following results.

You are really lucky or something is wrong. It is more likely that something is wrong.

It turns out that this errexit depends on the script being run as an executable following the chmod +x command.

Now we run the script 10 times again. Feeling lucky?

Now that is more like it.

Although it may seem like a viable option for setting errexit, it doesn’t always work. Copy and paste with caution.

Now let’s try something that at least works.

Option ii) Modify the script, below the interpreter line

As expected, this removes the interpreter line dependency. Now what if we don’t want to touch the script we are about to debug?

Option iii) Don’t modify the script, specify in command line

Personally, I like this option. After scripts that require debugging are not pretty.

Now suppose that we want to expand on using errexit to debug failed test by showing all the code that ran before the script went boom.

Option iv) Add -v -x

It may seem like the only thing that you get out of adding -v and -x to Option i through iii is getting vexed, it allows you to see all the code that ran before exiting due to a failed test.

Try it.

If you are so lucky your outcome will be as follows.

One is not equal to zero after all.

Now off to bigger and better things, debugging bash scripts in your crontab!

Example ii) What-if the script is running in crontab

I know, debugging a bash script set to run in your crontab is a little too much to get started out with but if you don’t understand it now, you will later.

Here’s the gist on the crontab for anyone who doesn’t want to skip until the next code snippet.

The crontab is a list of commands set to run at specific times. Unlike scripts run in interactive mode, crontab output may write to a file or standard output which would be sent somewhere else like your email.

Let’s suppose that we receive any output written to standard output through email and have debug-random-error.sh running every minute. The crontab line would be as follows.

Note that I redirected standard error to standard output. Why?

The reason why is that extra output from -v or -x  is written to standard error by default. That is,

in most cases.

We covered how to use options to help debug a bash script as a while. Now let’s dive into how to debug sections of code contained within a script.

Example iii) What-if you only want to debug a section of code in your bash script

When debugging only a section of code in your script, you can surround the code with opening and closing set command lines as follows.

We covered how to use options to debug a bash script. Now to another technique, setting breakpoints.

Example iv) What if a * in the script is causing it to spill out all the files in the working directory

If  there is a * unprotected by quotes, it may be the case that the script coule perform actions on all the files within the working directory including itself to produce unexpected behavior observable while debugging.

In this case we want to use the

Globbing is where the * may interpretted as all the files you have laying around.

So beware of the glob!

Use breakpoints to debug scripts like a boss

Another technique that comes in handy when debugging bash scripts, is using breakpoints to pause execution at an point in the script as follows.

In brief we covered how the read bash builtin may be used to help you debug sections of a bash script. Now to the next technique, debug output.

Use output to debug scripts like a boss

Maybe your script has a debug mode builtin. In that case extra information, debug output, may be written to another location besides standard output, let’s say standard error as follows.

To avoid confusion, we may use colored output such as a little bash script I cooked up called cecho.sh. Here is short snippet displaying usage.

You can see how colored debug output may be less confusion than writing vanilla bash spells for every debug output line.

Disclaimer: There are other methods available for colored output for use in debugging. The method above is my personal choice.

Example iv) Debug output usage

In bash curl example, I mentioned a bash a script to download files I wrote called downldr.sh. Here are some lines to explain colored output for debugging in the development of a bash script.

Your bash script may contain a operation that may only happen once such as the creation of a directory. You may include this in debug output in yellow.

Source: downldr.sh Line 43

Your bash script may contain a operation that is skipped depending on conditions. You may include this in debug output in green.

Source: downldr.sh Line 22

How to use debug output for debugging depends on preference your preference or style guide of choice if any. However, I suggest to at least use a color different than that of standard output if displayed in the terminal.

Use shopts to debug scripts like a boss (bash specific)

In addition to the set command, bash also has shopt to modify shell behavior.

Here are some shell options that may prove usefull while debugging a script.

Check options using shopt -p. In most cases you are likely to be dealing with defaults. But if you have to check at least you know how.

expand_aliases

Debuging scripts with aliases can be misleading at times. If you have to turn off aliase expansion using shopt -u expand_aliases which is enabled by default in interactive mode. In the case of a script, you are bound to find a shopt -s expand_aliases line somewhere. If you have to comment it out.

Example v: mystery aliases

Source: debug-aliases.sh

failglob

If you suspect globbing issus and have errexit set, you may want to experiment with shopt -s failglob. Just for fun, here is a quick example.

restricted_shell

If you suspect the issue is getting lost and not behaving you can try running in restricted mode. Consider the following example script:

Source: debug-restricted.sh

In interactive mode are within the script we run the above script in the restricted shell as follows.

shift_verbose

If the script uses shift you may enable shift verbose as follows.

There are the options that I have decided to introduce to help debug a script. Find more at the usual manual location for the shopt builtin.

Conclusion

In order to debug a bash script like a boss, you may need to apply bash specific shell behavior modifications. However, other debugging techniques such as debug output and setting breakpoints apply.