Bash pipelines

The Architecture of Open Source Applications has a great chapter on Bash. The entire thing is worth a read, but it'll also take you a while. The important part is figure 3.1, which describes the order in which Bash's pipeline processes input. It's very important. I'll give you two examples.

Before I go any further, it's worth remembering how you did math as a kid, because Bash is a bit similar. If you have a math problem like this, it can be solved by passing it through a pipeline, which transforms the problem at each step. Consider:

1 - 2 + 3 / 3
1 - 2 + 1  # multiplication and division
0  # addition and subtraction

There can even be nested pipelines, in which case sub-pipelines might be created:

1 - (2 + 3 / 3)
1 - (2 + 1)  # multiplication and division, in sub-pipeline
1 - (3)  # addition and subtraction, in sub-pipeline
1 - 3  # sub-pipeline interpolation
1 - 3  # multiplication and division
-2  # addition and subtraction

This is easy, right? With this mentality in mind, it's easy to understand some of Bash's weirdness. First, consider this sequence of commands:

word1=foo
word2=bar
echo $word{1,2}

What happens here? Does Bash try to find a variable literally called "word{1,2}", which doesn't exist, and therefore interpolate an empty string into the expression? Or does Bash do something else? The pipeline in figure 3.1 provides the answer:

echo $word{1,2}
echo $word1 $word2  # brace expansion
echo $word1 $word2  # tilde expansion
echo foo bar  # variable expansion
echo foo bar  # word splitting (foo and bar are now two separate strings)
echo foo bar  # filename generation
# echo is passed "foo" and "bar" as two separate arguments.

Second, consider this sequence of commands:

touch 'file one'
touch 'file two'
for file in *; do echo "$file"; done

What happens here? Again, follow the pipeline. Try solving this one by hand before executing the commands in a terminal.