Common Pitfalls When Assigning Output To Variables In Bash
Failing to Properly Quote Variables
One of the most common pitfalls when assigning command output to a variable in Bash is failing to properly quote the variable expansion. When you assign output to a variable without quoting, like this:
out=$(mycommand)
Bash will perform word splitting and filename expansion on the unquoted variable, which can lead to incorrect results if the output contains whitespace or special characters. To prevent this, it is best practice to always quote the variable expansion when assigning output, like so:
out="$(mycommand)"
The quotes prevent Bash from parsing the output and ensure the entire output is assigned intact to the variable. Omitting quotes can cause whitespace to be interpreted as argument separators, filenames with spaces or globs to expand incorrectly, and special characters to be interpreted. Always quoting the assigned output avoids these parsing issues.
Why Quoting Prevents Problems
When you expand an unquoted variable, Bash applies the following parsing steps:
- Bash separates the output into words via word splitting – treating whitespace and newlines as delimiters.
- Bash expands wildcard globs into matching filenames (wildcard expansion).
- Bash divides the results into separate arguments based on the IFSinternal field separators (by default space, tab, and newline).
This parsing causes unquoted command output containing spaces, newlines or globs to be incorrectly divided or expanded. But when you quote the expansion, Bash treats the entire output as a single word – preventing all parsing steps.
Examples of Problems Caused
Some examples of problems caused by unquoted command substitutions:
# Word splitting on whitespace out=$(echo "foo bar") echo "$out" # Prints just "foo" # Glob expansion out=$(echo *.txt) echo "$out" # Prints glob expanded filenames # Argument separation in loops out=$(mycmd) for x in $out; do # Loops over words in $out done
The solution is always the same – quote the variable to avoid parsing:
out="$(echo "foo bar")" # Preserves whitespace out="$(echo *.txt)" # Disable glob expansion for x in "$out"; do # Loops over entire output done
When to Quote
Always quote the command substitution whenever assigning output to a variable:
var="$(cmd)" # Double quotes var='$(cmd)' # Single quotes
Exceptions are when you specifically want to divide the output into words or arguments by leaving unquoted. But typically, you should quote.
Ignoring Exit Codes
Another common mistake is assigning command output without checking the exit code to see if the command succeeded:
out=$(mycommand)
This ignores mycommand’s exit code. If mycommand fails, the script will continue executing with empty or incorrect output stored in $out
. To handle failures properly, always check the $?
exit code after command substitutions:
if out=$(mycommand); then echo "Success" else echo "Failed with exit code $?" fi
This tests if mycommand
succeeded, and prints an error if it failed. Handling exit codes prevents errors being silently ignored.
Why Check Exit Codes
The $?
variable contains a command’s exit code – an integer indicating whether it succeeded (0) or failed (non-zero). When assigning output, you lose this exit code information:
mycommand # $? contains exit code out=$(mycommand) # $? is overwritten
So the only way to check if the command succeeded is to explicitly test $?
after substitution:
out=$(mycommand) if [ $? -eq 0 ]; then echo "Succeeded" else echo "Failed with exit code $?" fi
Making exit code checks mandatory prevents overlooked errors.
Common Exit Code Checks
Some common exit code check patterns are:
# Check immediately after substitution out=$(cmd) if [ $? -ne 0 ]; then ...handle error.. fi # Use an if statement if out=$(cmd); then ..process output.. else ..handle failure.. fi # Write a bash function run() { if out=$(cmd); then return 0 else return 1 fi } if run; then ..success.. else ..failed.. fi
The key is checking $?
immediately after substitutions to catch errors.
Overwriting Existing Variables
A very common pitfall when assigning to variables is accidentally overwriting values already stored in other variables. For example:
hostname=$(hostname) out=$(ls) echo "$hostname" # Oops, clobbered with ls output
Since variables share a single namespace, it’s easy to overwrite them by mistake. Always be careful when naming output variables to avoid collisions.
Use Unique Variable Names
The simplest way to avoid overwrites is choosing unique names for output variables:
hostname=$(hostname) files=$(ls) # Changed name to avoid conflict echo "$hostname" # Preserved original value
Prefixing variables or using longer descriptive names prevents collisions:
server_name=$(hostname) cmd_output=$(ls)
Use Namespaces for Scripts
For larger scripts, encapsulate variables in a namespace to avoid global collisions:
myscript() { # Local namespace local out local filename out=$(cmd) filename=$(generate_filename) # Rest of script can freely assign to out/filename ... }
This uses local
to scope variables to myscript
only. Code outside the function cannot overwrite them accidentally.
Avoid Overwriting Environment Vars
Be very careful when naming output vars as you can easily overwrite imported environment variables by mistake:
echo "$PATH" # Oops, shadows $PATH PATH=$(cmd)
Stick to unique names avoid accidentally breaking imported environment variables.
Assigning Huge Outputs to Variables
A common mistake is trying to assign an extremely large command output to a variable:
out=$(command_with_large_output)
When a command outputs megabytes/gigabytes of text, allocating all that data to a variable can use up a lot of memory and slow the script to a crawl.
Memory Overhead
Variable assignments like the above store the output in memory. Bash has to allocate resources for the entire contents of $out
. For small outputs this isn’t an issue – but 1GB outputs can overwhelm your system:
# 50KB Output - OK out=$(generate_report) # 1GB output - Problems!! out=$(dump_database)
The larger the saved output, the more memory consumed. This leads to RAM shortages, slow performance, crashes etc. Storing too much data is inefficient.
Slow Processing
Parsing and manipulating large outputs stored in variables is very slow:
# Slow - first stores entire 1GB output out=$(big_data_query) # Then processes line-by-line for line in $out; do echo $line done
Processing the output line-by-line while it is being generated avoids having to store it entirely in memory first. This saves resources and is much faster.
Alternatives
To avoid issues with huge outputs, either:
- Pipe the output directly to other commands instead of assigning to a variable
- Write the output incrementally to disk instead of storing in memory
- Only store extracts of outputs that you actually need to variables
This saves memory/disk space and processes faster.
Attempting Variable Expansion Before Assignment
Another pitfall is trying to expand variables before they have been assigned any value:
echo "$out" # Expands to empty string out=$(cmd)
Since out
doesn’t exist yet on the first line, the expansion results in an empty value. Trying to use unassigned variables expands to empty strings or sometimes error messages, and causes undefined behavior.
Why Undefined Expansion Happens
Variables in bash are untyped – they do not declare any name or type before assignment. You can expand variables that have never been set:
echo "$undefined_var" # Expands to ""
So Bash cannot catch “variable used before assignment” errors. You have to be careful of unassigned expansions yourself.
How to Avoid Issues
Always set variables before trying to expand them:
out=$(cmd) # Assign first echo "$out" # Then expand
Or use parameter expansion to provide defaults:
echo "${out:-"Default if unset"}" # Show default if undefined
Checking with set -u
to detect undefined variable errors can also help debug issues.
Confusing Standard Output and Standard Error
The final common pitfall is only capturing standard output, and missing error messages sent to standard error.
For example:
out=$(mycommand) # Captures standard output only
This assigns the standard output stream to the variable, but ignores the standard error stream. If mycommand logs errors to stderr, those messages will get lost.
Missing Error Messages
Losing error messages can mask issues and prevent proper handling of failures. For example:
out=$(mycommand) if [ $? -eq 0 ]; then # Success right? echo "$out" fi
The exit code says it succeeded, but stderr may contain errors that you miss!
Redirect Both Streams
To fix this, redirect both standard output and standard error when assigning:
out=$(mycommand 2>&1) # Redirect stderr to stdout
This combines both streams into the variable. Now errors are correctly captured:
out=$(mycommand 2>&1) if [ $? -eq 0 ]; then echo "$out" # $out contains all output else echo "$out" # $out contains error logs fi
If you only want errors in the variable, swap stdout/stderr.
Other Ways to Access Stderr
You can also access stderr separately without redirection:
out=$(mycommand) # stdout only errors=$(mycommand 2>&1 1>/dev/null) # stderr only # Or access the file descriptors directly exec 3>&1 # Save stdout out=$(mycommand 2>&1 1>&3) # Redirect stdout, keep stderr
But typically redirecting to stdout is recommended for simplicity.