RED HAT ENTERPRISE LINUX

I/O Redirection in Bash

Controlling Command Input and Output

CIS126RH | RHEL System Administration 1
Mesa Community College

Learning Objectives

Understand standard streams

Know stdin, stdout, and stderr and their file descriptors

Redirect output to files

Use > and >> to capture stdout and stderr

Redirect input from files

Use < and here documents to provide input

Connect commands with pipes

Build powerful command pipelines using |

Standard Streams

Every Linux process has three standard streams for data flow. By default, they connect to the terminal, but redirection changes where data flows.

stdin

Standard Input
Data going INTO the command
Default: Keyboard

stdout

Standard Output
Normal output FROM command
Default: Terminal screen

stderr

Standard Error
Error messages FROM command
Default: Terminal screen

Default Data Flow

Keyboard

→

stdin (0)

→

Command

→

stdout (1)

→

Screen

Command

→

stderr (2)

→

Screen

Key Insight: stdout and stderr both go to the screen by default, but they are separate streams that can be redirected independently.

Output Redirection Operators

Operator	Description	Example
`>`	Redirect stdout to file (overwrite)	`ls > files.txt`
`>>`	Redirect stdout to file (append)	`echo "text" >> log.txt`
`2>`	Redirect stderr to file (overwrite)	`cmd 2> errors.txt`
`2>>`	Redirect stderr to file (append)	`cmd 2>> errors.txt`
`&>`	Redirect both stdout and stderr	`cmd &> all.txt`
`2>&1`	Redirect stderr to same place as stdout	`cmd > out.txt 2>&1`

Redirecting stdout

# Redirect output to new file (overwrites if exists!)
ls -l /etc > etc_listing.txt

# View the captured output
cat etc_listing.txt
total 1348
drwxr-xr-x.  3 root root    4096 Nov  1 10:00 abrt
-rw-r--r--.  1 root root      16 Nov  1 10:00 adjtime
...

# Append to existing file (preserves content)
echo "Listing generated on $(date)" >> etc_listing.txt

# Create empty file (or truncate existing)
> empty_file.txt

Danger: > overwrites silently — if the file exists, previous contents are gone immediately. Use >> when in doubt.

Overwrite Protection: noclobber

# Enable overwrite protection
set -o noclobber

# Now redirecting to an existing file produces an error
ls > existing.txt
bash: existing.txt: cannot overwrite existing file

# Force overwrite anyway when you mean it
ls >| existing.txt

# Disable the protection
set +o noclobber

The noclobber option is a useful safety net when working with important files. Use >| (greater-than pipe) to force an overwrite when you explicitly intend to replace the file.

Redirecting stderr

# Command that produces both stdout and stderr
ls /etc /nonexistent
/etc:
hostname  hosts  passwd ...

ls: cannot access '/nonexistent': No such file or directory

# Redirect only stderr to file (stdout still shows on screen)
ls /etc /nonexistent 2> errors.txt
/etc:
hostname  hosts  passwd ...

# Check the error file
cat errors.txt
ls: cannot access '/nonexistent': No such file or directory

stderr: Discard & Append

# Discard stderr (send to /dev/null — the black hole)
ls /etc /nonexistent 2> /dev/null
/etc:
hostname  hosts  passwd ...

# Append errors to a log file across multiple runs
find / -name "*.conf" 2>> search_errors.log

/dev/null is the system's "black hole" — anything written to it is silently discarded. Use it to suppress error messages you don't need, such as "Permission denied" messages from find across the whole filesystem.

Use 2>> to accumulate errors in a log file — each run adds to the log without losing earlier entries.

Combining stdout and stderr

# Method 1: Redirect both to same file (bash shorthand)
ls /etc /nonexistent &> all_output.txt

# Method 2: Redirect stdout, then stderr to same place
ls /etc /nonexistent > all_output.txt 2>&1

# Redirect to separate files
ls /etc /nonexistent > stdout.txt 2> stderr.txt

# Append both to same file
command >> combined.log 2>&1

# Discard all output
command &> /dev/null
command > /dev/null 2>&1   # Same effect, more portable

Order Matters!

# CORRECT: Redirect stdout first, then point stderr to it
command > file.txt 2>&1

# WRONG: This does NOT capture stderr in the file
command 2>&1 > file.txt   # stderr still goes to terminal!

⚠ Order Matters: 2>&1 means "send stderr where stdout is currently going." If stdout still points to the terminal when this executes, stderr goes to the terminal. Always redirect stdout first, then add 2>&1.

The &> bash shorthand avoids this confusion entirely — it always redirects both streams to the same destination regardless of order.

Input Redirection

# Redirect input from a file
wc -l < /etc/passwd
45

# Send email from file
mail -s "Report" user@example.com < report.txt

# Combine input and output redirection
sort < unsorted.txt > sorted.txt

# Redirect all three streams
command < input.txt > output.txt 2> errors.txt

You can combine input and output redirection in a single command. sort < unsorted.txt > sorted.txt reads from one file, sorts the content, and writes to another — a pattern for file transformation.

Input Redirection: Subtle Difference

# wc opens the file itself — knows the filename
wc -l /etc/passwd
45 /etc/passwd

# Shell provides the content — wc sees only data on stdin
wc -l < /etc/passwd
45

Command opens the file

wc knows the filename — it includes it in output

Shell provides the content

wc only sees data on stdin — no filename in output. Clean output, useful in scripts.

Here Documents

A here document (heredoc) provides multi-line input inline in a script or command, without needing a separate file.

# Basic here document syntax
cat << EOF
This is line 1
This is line 2
Variables expand: $HOME
EOF

# Prevent variable expansion with quoted delimiter
cat << 'EOF'
Variables do NOT expand: $HOME
This prints literally: $(whoami)
EOF

Quote the delimiter ('EOF') to disable variable expansion — the literal strings $HOME and $(whoami) appear as-is rather than their values.

Here Documents: Creating Files

# Use heredoc to create a config file in a script
cat << EOF > config.txt
server=localhost
port=8080
user=$USER
EOF

# The delimiter can be any word (EOF is convention)
cat << MYEND
Content here
MYEND

Here documents are commonly used to create configuration files in scripts. The example combines a here document with output redirection to create config.txt, including the expanded $USER variable.

The delimiter must appear alone on its own line to end the here document. If your content might contain EOF, choose a different delimiter word.

Here Strings

# Here string — single line input with <<<
wc -w <<< "count the words in this string"
6

# Useful with commands that read stdin
bc <<< "5 * 4 + 3"
23

# With variables
greeting="Hello World"
cat <<< "$greeting"
Hello World

# Parse a string with read
read first rest <<< "apple banana cherry"
echo $first
apple

Here Strings vs echo | command

# Using echo with pipe (creates a subshell)
echo "test data" | wc -c

# Using here string (no subshell, slightly more efficient)
wc -c <<< "test data"

Here Strings are simpler than here documents for single-line input, and more efficient than echo | command because they don't create a subshell for the echo command.

Here strings are a bash feature — they are not available in all shells (sh, dash, etc.). Use them freely in scripts that explicitly target bash (#!/bin/bash).

Pipes

A pipe connects the stdout of one command directly to the stdin of another, creating a data processing pipeline.

Command 1

→ stdout

stdin →

Command 2

# Output of ls becomes input to wc
ls /etc | wc -l
245

# Filter output through grep
cat /etc/passwd | grep "bash"

# Chain multiple commands
cat /var/log/messages | grep "error" | wc -l

Pipes: Chaining Tools

# Sort and remove duplicates — unique list of login shells
cut -d: -f7 /etc/passwd | sort | uniq
/bin/bash
/bin/sync
/sbin/halt
/sbin/nologin
/sbin/shutdown

This pipeline extracts the seventh field from /etc/passwd (the login shell), sorts them alphabetically, and removes duplicates — giving a unique list of all shells used on the system.

The "small tools, combined with pipes" philosophy is fundamental to Unix. Each command does one thing well; pipes orchestrate the flow between them.

Building Pipelines

cat access.log

grep "404"

cut -d' ' -f7

sort

uniq -c

sort -rn

# Find most common 404 errors in web log
cat access.log | grep "404" | cut -d' ' -f7 | sort | uniq -c | sort -rn
    47 /missing-page.html
    23 /old-link.php
    12 /typo.html

Pipeline Step by Step

# Each stage explained:
cat access.log       # Read the log file — all lines to stdout
  | grep "404"       # Keep only lines containing 404
  | cut -d' ' -f7    # Extract URL (7th space-delimited field)
  | sort             # Sort URLs alphabetically (required before uniq)
  | uniq -c          # Count unique occurrences
  | sort -rn         # Sort by count, highest first

uniq only detects adjacent duplicates — that is why sort must come before it. sort -rn sorts reverse numerically, putting the highest counts first.

Common Pipeline Patterns

# Filter and count
grep "pattern" file | wc -l

# Sort and deduplicate
sort file | uniq

# Search process list
ps aux | grep "httpd"

# View paginated output
cat /var/log/messages | less

# Extract and format columns
df -h | awk '{print $1, $5}'

Common Pipeline Patterns (continued)

# Find largest items in a directory
du -sh * | sort -rh | head -10

# Monitor log in real-time, filtered
tail -f /var/log/messages | grep --line-buffered "error"

# Count files by extension
find . -type f | sed 's/.*\.//' | sort | uniq -c | sort -rn

du -sh * | sort -rh | head -10 finds the 10 largest items: du gets sizes, sort -rh orders by human-readable size descending, head limits to 10 results.

For log monitoring, --line-buffered on grep ensures each matching line is output immediately rather than held in a buffer.

The tee Command

tee reads from stdin and writes to both stdout AND one or more files simultaneously — like a T-junction in plumbing.

Command

→

tee file.txt

→

Screen

# Save output while also viewing it
ls -l /etc | tee etc_listing.txt

# Append instead of overwrite
echo "new entry" | tee -a logfile.txt

# Write to multiple files simultaneously
command | tee file1.txt file2.txt file3.txt

tee with sudo

# Write to a root-owned file as a regular user
echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf

# Why the naive approach fails:
sudo echo "nameserver 8.8.8.8" > /etc/resolv.conf
bash: /etc/resolv.conf: Permission denied
# The shell (not echo) handles >, and the shell isn't root

echo "..." | sudo tee /etc/file works because tee runs with sudo privileges and writes the file. The shell's > operator runs as the current (non-root) user — so the naive sudo echo ... > file always fails on protected files.

The xargs Command

xargs builds and executes commands from stdin, converting input lines into arguments for another command.

# Delete files found by find
find /tmp -name "*.tmp" | xargs rm

# Handle filenames with spaces (-0 and -print0)
find . -name "*.log" -print0 | xargs -0 rm

# Limit arguments per command (-n)
echo "a b c d" | xargs -n 2 echo
a b
c d

Many commands (like rm) expect filenames as arguments, not on stdin. Pipes send data to stdin — xargs bridges this gap by converting stdin lines into command arguments.

xargs Advanced Options

# Interactive mode — confirm each command (-p)
find . -name "*.bak" | xargs -p rm

# Use placeholder for argument position (-I)
find . -name "*.txt" | xargs -I {} cp {} /backup/

# Parallel execution — 4 processes at once (-P)
find . -name "*.jpg" | xargs -P 4 -I {} convert {} -resize 50% small_{}

-I {} specifies a placeholder for where the argument goes — useful when the argument isn't at the end of the command or appears multiple times.

-P 4 runs up to 4 processes in parallel — dramatically speeds up batch operations like image conversion.

Practical Examples

# Log all errors from a script
./backup.sh 2>> /var/log/backup_errors.log

# Run command silently (discard all output)
cron_job.sh &> /dev/null

# Save command output with timestamp header
{ echo "=== $(date) ==="; df -h; } >> disk_report.txt

# Create file requiring root, with user content
echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf

The curly-brace grouping { cmd1; cmd2; } runs multiple commands and redirects their combined output. Here, a timestamp header precedes the disk report — both go to the same file.

Practical Examples (continued)

# Compress old logs (xargs -r: skip if no input)
find /var/log -name "*.log" -mtime +30 | xargs -r gzip

# Compare two directories using process substitution
diff <(ls dir1) <(ls dir2)

# Monitor multiple logs simultaneously, filtered
tail -f /var/log/messages /var/log/secure | grep -E "(error|fail)"

Process substitution <(cmd) treats command output as a file — so diff can compare two directory listings directly without creating temp files.

The -r flag on xargs means "don't run the command if there's no input" — prevents gzip from waiting on stdin when no files match the find criteria.

Common Mistakes

# WRONG: Redirecting to the same file you're reading
sort file.txt > file.txt          # Results in empty file!

# RIGHT: Use a temporary file
sort file.txt > temp.txt && mv temp.txt file.txt

# RIGHT: Use sponge (absorbs all input before writing)
sort file.txt | sponge file.txt   # (requires moreutils)

Why it fails: The shell truncates the output file before the command reads the input file. By the time sort opens file.txt, it is already empty.

Common Mistakes (continued)

# WRONG: stderr redirect order
command 2>&1 > file.txt           # stderr still goes to terminal!

# RIGHT: Redirect stdout first
command > file.txt 2>&1

# WRONG: Expecting pipe to modify original file
cat file.txt | grep "pattern" | sort   # file.txt is unchanged!

# WRONG: Using redirect where pipe is needed
ls > wc -l                        # Creates a file named "wc"!

# RIGHT: Use pipe
ls | wc -l

Confusing redirect with pipe creates unexpected files. ls > wc -l creates a file literally named wc and then tries to run -l as a command.

Quick Reference

Operator	Description
`cmd > file`	Redirect stdout to file (overwrite)
`cmd >> file`	Redirect stdout to file (append)
`cmd 2> file`	Redirect stderr to file
`cmd 2>&1`	Redirect stderr to same place as stdout
`cmd &> file`	Redirect both stdout and stderr to file
`cmd < file`	Redirect stdin from file
`cmd << EOF`	Here document (multi-line input)
`cmd <<< "str"`	Here string (single-line input)
`cmd1 \| cmd2`	Pipe stdout of cmd1 to stdin of cmd2
`cmd \| tee file`	Send output to file AND stdout

Key Takeaways

Three streams: stdin (0), stdout (1), stderr (2)

Output: > overwrites, >> appends, 2> for stderr, &> for both

Input: < from file | << here doc | <<< here string

Pipes | connect commands | tee splits output | xargs builds arguments

Next: Managing Local Users and Groups

Let's summarize the essential concepts of I/O redirection. Every process has three standard streams: stdin for input (file descriptor 0), stdout for normal output (1), and stderr for error messages (2). By default all connect to the terminal, but redirection changes this. For output redirection, the single greater-than creates or overwrites files, double greater-than appends safely. Add the file descriptor 2 for stderr redirection. The &> shorthand captures both streams. For input redirection, less-than provides input from a file. Here documents with << provide multi-line input in scripts, and here strings with <<< provide single-line input concisely. Pipes are the power tool - connecting stdout of one command to stdin of the next. Combined with tee to split output and xargs to build arguments, pipes enable complex data processing from simple commands.