Pipes and Redirection in Bash

Every command you run in Bash has three standard streams attached to it: standard input (stdin), standard output (stdout), and standard error (stderr). Pipes and redirection let you wire those streams together — sending one command's output directly into another, saving results to files, or silencing noise you don't need. Once these click, you stop copy-pasting between commands and start composing real pipelines.

The Three Standard Streams

The kernel assigns each process three file descriptors by default:

• 0 — stdin: where a command reads its input (usually the keyboard)
• 1 — stdout: where a command writes normal output (usually the terminal)
• 2 — stderr: where a command writes error messages (also usually the terminal)

Redirection and pipes manipulate these file descriptors. That's all they do — but the combinations are powerful.

Output Redirection: > and >>

The > operator redirects stdout to a file, creating the file if it doesn't exist and truncating it if it does. Be careful — there is no undo.

ls -lh /etc > /tmp/etc_listing.txt

To append instead of overwrite, use >>:

echo "Backup completed: $(date)" >> /var/log/mybackup.log

Both operators only redirect stdout. Error messages still appear on your terminal unless you handle stderr separately.

Redirecting stderr: 2>

Use the file descriptor number explicitly to redirect stderr:

find /root -name "*.conf" 2> /tmp/find_errors.txt

You can redirect stdout and stderr to different files at the same time:

find /etc -name "*.conf" > /tmp/found.txt 2> /tmp/errors.txt

Combining stdout and stderr: 2>&1

2>&1 means "send file descriptor 2 to wherever file descriptor 1 is currently pointing." Order matters here — write the stdout redirect first, then 2>&1:

make build > /tmp/build.log 2>&1

Bash 4+ also offers the shorthand >& which combines both streams in one token:

make build &> /tmp/build.log

A common mistake is reversing the order: 2>&1 > file does not do what you expect. By the time Bash processes 2>&1, stdout still points to the terminal, so stderr goes to the terminal too.

Discarding Output with /dev/null

/dev/null is a special file that silently discards everything written to it. Use it to suppress output you don't need:

# Suppress only errors
curl https://example.com 2> /dev/null

# Suppress everything
some_noisy_script > /dev/null 2>&1

Input Redirection: <

The < operator feeds a file into a command's stdin instead of the keyboard:

sort < unsorted_names.txt

Many commands accept filenames directly as arguments, making < optional for them. But some commands only read from stdin — in those cases < is essential.

Here Documents and Here Strings

A here document lets you supply multi-line stdin inline in a script:

cat << EOF
Server: web01
Role:   nginx
Owner:  ops-team
EOF

A here string feeds a single string into stdin without a file:

grep "error" <<< "no error found in this string"

Pipes: |

A pipe connects the stdout of one command directly to the stdin of the next — in memory, without a temporary file. The two commands run concurrently; the kernel buffers data between them.

ps aux | grep nginx

Chain as many commands as you need:

cat /var/log/syslog | grep "kernel" | sort | uniq -c | sort -rn | head -20

Read that pipeline left to right: grab the log, keep only kernel lines, sort them, count duplicates, sort by frequency descending, show the top 20.

Piping stderr Through a Pipe

By default, pipes only carry stdout. To also pipe stderr, combine 2>&1 before the pipe:

make build 2>&1 | tee /tmp/build.log

Or use the Bash shorthand:

make build |& tee /tmp/build.log

Useful Pipeline Tools

A few commands exist mainly to work inside pipelines:

• tee — writes stdin to both a file and stdout simultaneously, so the pipeline continues
• xargs — converts stdin lines into arguments for another command
• grep — filters lines matching a pattern
• awk / sed — transform text field-by-field or line-by-line
• cut — extracts specific columns from delimited text
• sort / uniq — sort lines, remove or count duplicates
• wc — count lines, words, or bytes

tee in Practice

tee is indispensable when you want to save output and continue the pipeline:

journalctl -u nginx | tee /tmp/nginx_raw.log | grep "error" > /tmp/nginx_errors.log

The full journal output lands in nginx_raw.log; only the error lines go to nginx_errors.log.

xargs in Practice

Find all .log files older than 30 days and delete them:

find /var/log -name "*.log" -mtime +30 | xargs rm -f

For filenames with spaces, use null-delimited output and the -0 flag:

find /var/log -name "*.log" -mtime +30 -print0 | xargs -0 rm -f

Checking Pipeline Exit Codes

By default, a pipeline's exit code is the exit code of the last command. If grep finds nothing it exits 1, masking an earlier failure. Use pipefail in scripts to catch failures anywhere in the pipe:

set -o pipefail
cat /nonexistent_file | grep "pattern"  # now the pipeline exits non-zero

After a pipeline, the array ${PIPESTATUS[@]} holds each command's individual exit code:

ls /etc | grep "resolv" | wc -l
echo "Exit codes: ${PIPESTATUS[@]}"

Output will resemble: Exit codes: 0 0 0 — one value per stage, though your values will vary.

Verification

Run this end-to-end pipeline to confirm everything is working correctly in your shell:

journalctl --no-pager 2>/dev/null | \
  grep -i "started" | \
  awk '{print $NF}' | \
  sort | uniq -c | sort -rn | \
  head -5 | tee /tmp/pipeline_test.txt

cat /tmp/pipeline_test.txt

If /tmp/pipeline_test.txt contains the same five lines shown on screen, your pipes and redirections are functioning correctly.

Troubleshooting

• File gets truncated unexpectedly — you used > instead of >>. Enable set -o noclobber in your shell or scripts to prevent accidental overwrites; use >| to override it intentionally.
• Error messages still appear on screen after redirecting to a file — you only redirected stdout. Add 2>&1 after your stdout redirect, or use &>.
• Pipeline always reports success even when a step fails — add set -o pipefail at the top of your script.
• xargs fails on filenames with spaces — use -print0 with find and -0 with xargs.
• Command says "argument list too long" — you're hitting the kernel's ARG_MAX limit. Replace the glob or subshell expansion with a find | xargs pipeline instead.