RED HAT ENTERPRISE LINUX

Archiving and Compression

Creating Compressed Archives for Backup and Transfer

CIS238RH | RHEL System Administration 2
Mesa Community College

Welcome to this practical module on archiving and compression in Red Hat Enterprise Linux. These are fundamental skills for any system administrator - you'll use them constantly for backups, transferring files between systems, distributing software, and managing storage space. In this module, you'll master the tar command, Linux's primary archiving tool that bundles multiple files into a single archive while preserving permissions, ownership, and directory structure. You'll also learn the major compression tools - gzip, bzip2, and xz - understanding their tradeoffs between speed and compression ratio. We'll cover how tar integrates with compression, the star and zip utilities for special cases, and practical scenarios you'll encounter in real administration work. These skills are tested on the RHCSA exam and used daily in production environments.

Learning Objectives

Create and extract tar archives

Bundle files preserving permissions, ownership, and structure

Apply compression with gzip, bzip2, and xz

Reduce file sizes using different compression algorithms

Create compressed archives efficiently

Combine archiving and compression in single operations

Transfer archives between systems

Use scp, rsync, and other methods for file transfer

We have four main learning objectives. First, you'll master tar for creating and extracting archives. tar bundles files together while preserving critical metadata - permissions, ownership, timestamps, and directory structure. This is essential for backups. Second, you'll learn the compression tools: gzip for fast compression, bzip2 for better ratios, and xz for maximum compression. Understanding when to use each saves time and storage. Third, you'll combine archiving and compression efficiently. Modern tar integrates with compressors, letting you create compressed archives in one command. Fourth, you'll learn to transfer archives between systems using scp for simple transfers and rsync for efficient synchronization. These skills form a complete workflow from backup creation to restoration on another system.

Archiving vs Compression

Archiving combines multiple files into one. Compression reduces file size. These are separate operations that are often combined.

Archiving (tar)

Bundles files into single archive. Preserves permissions, ownership, timestamps, directory structure.

Compression (gzip/bzip2/xz)

Reduces file size using algorithms. Trades CPU time for smaller output.

Multiple Files → tar (archive) → .tar file → gzip (compress) → .tar.gz file

Why separate? Unix philosophy: each tool does one thing well. tar archives, gzip compresses. Combined: powerful and flexible.

Understanding the distinction between archiving and compression is fundamental. Archiving takes multiple files and directories and bundles them into a single file. The archive preserves metadata: permissions, ownership, timestamps, symbolic links, directory structure. tar is the standard Unix archiving tool - its name means "tape archive" from when backups went to tape. Compression takes a file (any file, including an archive) and makes it smaller using mathematical algorithms. Compressors don't know about multiple files or metadata - they just see bytes to compress. Linux follows the Unix philosophy of small tools that do one thing well. tar archives, gzip compresses. Separately they're useful; combined they're powerful. You can archive without compressing (fast, but large), compress without archiving (single files), or do both (most common for backups). Modern tar integrates compression directly, but understanding they're separate operations helps when troubleshooting or when you need fine control.

The tar Command

tar (tape archive) is the standard Unix/Linux tool for creating archives. It preserves file metadata and directory structure.

Create archive

Extract archive

List contents

Verbose output

Filename

# Basic tar syntax
[student@server ~]$ tar [operation] [options] -f archive.tar [files...]

# The -f option MUST be followed by the filename
# Operations: c (create), x (extract), t (list) - pick ONE
# Options: v (verbose), z/j/J (compression), p (preserve permissions)

Remember: -f must be immediately followed by the archive filename. tar -cvf archive.tar not tar -cfv archive.tar

tar is the workhorse of Linux archiving. Understanding its options is essential. The operation flags are mutually exclusive - pick one: c creates a new archive, x extracts files from an archive, t lists the archive contents without extracting. The -v flag enables verbose output, showing each file as it's processed. Useful for progress indication and verification. The -f flag specifies the archive filename and MUST be followed immediately by the filename. This is a common gotcha: "tar -cvf" works (c, v, then f followed by filename), but "tar -cfv" fails because it looks for a file named "v". You can use tar with or without the leading dash: "tar cvf" and "tar -cvf" both work. The traditional Unix style omits the dash for tar options. Modern versions of tar can often auto-detect compression format when extracting, but explicitly specifying it is clearer and more portable.

Creating Archives

# Create archive of a directory
[student@server ~]$ tar -cvf backup.tar /home/student/documents/
documents/
documents/report.txt
documents/data/
documents/data/file1.csv
documents/data/file2.csv

# Create archive of multiple items
[student@server ~]$ tar -cvf project.tar file1.txt file2.txt mydir/

# Create archive with absolute paths removed (default behavior)
[student@server ~]$ tar -cvf backup.tar /etc/hosts /etc/hostname
tar: Removing leading '/' from member names
etc/hosts
etc/hostname

# Preserve absolute paths (use with caution!)
[student@server ~]$ tar -cvPf backup.tar /etc/hosts

# Verify archive size
[student@server ~]$ ls -lh backup.tar
-rw-r--r--. 1 student student 15M Jan 20 14:00 backup.tar

Leading / removed: By default, tar strips the leading / from paths. This is a safety feature - extraction won't accidentally overwrite system files.

Creating archives is straightforward: tar -cvf archive.tar items. The -c flag creates, -v shows progress, -f specifies the output file. You can archive directories, individual files, or mix of both. tar recursively includes directory contents. An important safety feature: tar removes leading slashes from paths by default. If you archive /etc/hosts, it's stored as etc/hosts. When extracted, it creates etc/hosts in the current directory, not overwriting /etc/hosts. This prevents accidental overwrites during extraction. The -P flag preserves absolute paths. Use this only when you specifically need to restore files to their exact original locations, and be very careful - extracting such an archive as root could overwrite system files. The verbose output shows each file added. For large archives, you might omit -v for speed, or use it to verify the right files are included. Archive size depends on the data - tar itself doesn't compress, so the archive is roughly the sum of file sizes plus overhead.

Listing and Extracting

# List archive contents without extracting
[student@server ~]$ tar -tvf backup.tar
drwxr-xr-x student/student   0 2024-01-20 14:00 documents/
-rw-r--r-- student/student 1024 2024-01-20 13:55 documents/report.txt
drwxr-xr-x student/student    0 2024-01-20 14:00 documents/data/
-rw-r--r-- student/student 2048 2024-01-20 13:50 documents/data/file1.csv

# Extract entire archive to current directory
[student@server ~]$ tar -xvf backup.tar

# Extract to a specific directory
[student@server ~]$ tar -xvf backup.tar -C /tmp/restore/

# Extract specific files only
[student@server ~]$ tar -xvf backup.tar documents/report.txt

# Extract files matching a pattern
[student@server ~]$ tar -xvf backup.tar --wildcards "*.csv"

Best Practice: Always use tar -tvf to inspect an archive before extracting. Know what you're about to unpack!

Before extracting, always list the contents with -t. This shows you exactly what's in the archive and how it's structured. Are files in a subdirectory or will they scatter into the current directory? Are there any unexpected files? The listing shows permissions, ownership, size, timestamp, and path for each item. Extraction uses -x. By default, files extract to the current directory, recreating the directory structure stored in the archive. The -C option (capital C) changes to a different directory before extracting - useful for restoring to specific locations without cd. You can extract specific files by naming them after the archive filename. The path must match exactly how it's stored in the archive. Use --wildcards for pattern matching. Common mistake: extracting without checking first and having files scatter everywhere or overwrite existing files. Always inspect first, then extract. For root users extracting system backups, be especially careful - you could overwrite important system files.

Compression Tools

gzip

Extension: .gz

Speed: Fast

Ratio: Good

Use: General purpose

bzip2

Extension: .bz2

Speed: Medium

Ratio: Better

Use: Better compression

xz

Extension: .xz

Speed: Slow

Ratio: Best

Use: Maximum compression

zip

Extension: .zip

Speed: Fast

Ratio: Good

Use: Cross-platform

# Compression comparison (100MB text file)
Original:     100 MB   (baseline)
gzip:          25 MB   ~2 seconds    (75% reduction)
bzip2:         20 MB   ~8 seconds    (80% reduction)  
xz:            15 MB   ~30 seconds   (85% reduction)

Choose wisely: gzip for speed, xz for size, bzip2 for balance. Results vary by data type - text compresses well, already-compressed files (images, videos) don't.

Linux offers several compression tools with different tradeoffs. gzip is the traditional choice - fast compression and decompression with good ratios. It's the default for many applications. Files get .gz extension. bzip2 achieves better compression than gzip but takes longer. It's a good middle ground when you need smaller files but can't wait for xz. Files get .bz2 extension. xz provides the best compression ratios but is the slowest. Use it when size matters more than time - distributing software, long-term archival. Modern RHEL uses xz for RPM packages. Files get .xz extension. zip is included for cross-platform compatibility. It's an archive format that includes compression, unlike tar. Use it when sharing with Windows users. The comparison numbers are approximate - actual results depend heavily on data type. Text and logs compress very well. Binary executables compress moderately. Already-compressed files (JPEG, MP4, ZIP) barely compress at all - don't waste CPU trying.

Using gzip

# Compress a file (replaces original with .gz)
[student@server ~]$ gzip largefile.txt
[student@server ~]$ ls
largefile.txt.gz

# Decompress (replaces .gz with original)
[student@server ~]$ gunzip largefile.txt.gz
# Or: gzip -d largefile.txt.gz

# Keep original file while compressing
[student@server ~]$ gzip -k largefile.txt
largefile.txt  largefile.txt.gz

# Compress to stdout (useful for pipes)
[student@server ~]$ gzip -c largefile.txt > largefile.txt.gz

# View compressed file without decompressing
[student@server ~]$ zcat largefile.txt.gz | head
[student@server ~]$ zless largefile.txt.gz

# Set compression level (1=fast, 9=best compression)
[student@server ~]$ gzip -9 largefile.txt     # Maximum compression
[student@server ~]$ gzip -1 largefile.txt     # Fastest

Note: gzip replaces the original file by default! Use -k to keep the original, or -c to output to stdout.

gzip is simple but has an important behavior: it replaces the original file with the compressed version. Running gzip on file.txt deletes file.txt and creates file.txt.gz. This surprises many new users. gunzip (or gzip -d) does the reverse - decompresses and removes the .gz file. The -k option keeps the original, creating both compressed and uncompressed versions. The -c option writes to stdout instead of replacing files. This is useful for pipelines or when you want to control the output filename. zcat, zless, and zgrep work on compressed files directly without decompressing them to disk. Very useful for viewing log files that have been compressed by logrotate. Compression levels 1-9 trade speed for compression ratio. Default is 6. Level 9 compresses slightly better but takes noticeably longer. Level 1 is very fast with decent compression. For most uses, the default is fine.

Using bzip2 and xz

# bzip2 - better compression, slower
[student@server ~]$ bzip2 largefile.txt           # Creates .bz2
[student@server ~]$ bunzip2 largefile.txt.bz2     # Decompress
[student@server ~]$ bzip2 -k largefile.txt        # Keep original
[student@server ~]$ bzcat largefile.txt.bz2       # View without decompress

# xz - best compression, slowest
[student@server ~]$ xz largefile.txt              # Creates .xz
[student@server ~]$ unxz largefile.txt.xz         # Decompress
[student@server ~]$ xz -k largefile.txt           # Keep original
[student@server ~]$ xzcat largefile.txt.xz        # View without decompress

# xz compression levels (0-9, default 6)
[student@server ~]$ xz -9 largefile.txt           # Maximum (very slow)
[student@server ~]$ xz -0 largefile.txt           # Fastest

# xz with threads for faster compression
[student@server ~]$ xz -T 4 largefile.txt         # Use 4 CPU threads
[student@server ~]$ xz -T 0 largefile.txt         # Use all available CPUs

Performance tip: xz supports multi-threading with -T. Use -T 0 to automatically use all CPU cores for faster compression.

bzip2 and xz follow the same patterns as gzip - they replace the original by default, support -k to keep it, and have corresponding decompression commands and cat utilities. bzip2 typically achieves 10-20% better compression than gzip but is noticeably slower, especially for decompression. bzcat, bzless, bzgrep exist for working with compressed files. xz achieves the best compression ratios - often 30% or more smaller than gzip. The tradeoff is significantly longer compression time. Decompression is reasonably fast. Modern CPUs make xz more practical. The -T option enables multi-threading, dramatically speeding up compression on multi-core systems. -T 0 automatically detects and uses all available cores. This makes xz much more practical for large files. xz is used for RHEL package distribution because the compression ratio savings multiply across millions of downloads. For personal backups where time matters more, gzip is usually fine.

Compressed tar Archives

# Create gzip-compressed archive
[student@server ~]$ tar -czvf backup.tar.gz /home/student/documents/
# -z tells tar to use gzip compression

# Create bzip2-compressed archive
[student@server ~]$ tar -cjvf backup.tar.bz2 /home/student/documents/
# -j tells tar to use bzip2 compression

# Create xz-compressed archive
[student@server ~]$ tar -cJvf backup.tar.xz /home/student/documents/
# -J (capital J) tells tar to use xz compression

# Extract compressed archives (tar auto-detects compression)
[student@server ~]$ tar -xvf backup.tar.gz       # Works!
[student@server ~]$ tar -xzvf backup.tar.gz      # Explicit gzip
[student@server ~]$ tar -xjvf backup.tar.bz2     # Explicit bzip2
[student@server ~]$ tar -xJvf backup.tar.xz      # Explicit xz

Option	Compression	Extension
-z	gzip	.tar.gz or .tgz
-j	bzip2	.tar.bz2 or .tbz2
-J	xz	.tar.xz or .txz

tar integrates with compression tools, allowing single-command creation of compressed archives. The compression flags are: -z for gzip (think "z" for zip-like), -j for bzip2 (historical reasons), -J (capital) for xz. The naming convention .tar.gz indicates a tar archive that has been gzipped. Alternative short forms: .tgz, .tbz2, .txz. When extracting, modern GNU tar auto-detects the compression format, so "tar -xvf" usually works regardless of compression. However, explicitly specifying the compression flag is clearer and more portable to other systems. The process internally is: for creation, tar creates the archive and pipes it through the compressor. For extraction, tar decompresses first, then extracts. You never see the intermediate uncompressed archive. This integration is why most people think of tar and compression as one thing - practically, they usually are combined for compressed archives.

Common tar Operations

# Preserve permissions (important for system backups)
[root@server ~]# tar -cvpzf backup.tar.gz /etc/
# -p preserves permissions (default when root, explicit is clearer)

# Exclude files/directories
[student@server ~]$ tar -czvf backup.tar.gz --exclude='*.log' /home/student/
[student@server ~]$ tar -czvf backup.tar.gz --exclude='cache' --exclude='tmp' /var/www/

# Exclude from file
[student@server ~]$ cat exclude.txt
*.tmp
*.log
cache/
.git/
[student@server ~]$ tar -czvf backup.tar.gz -X exclude.txt /home/student/

# Update archive with newer files only
[student@server ~]$ tar -uvf backup.tar /home/student/documents/

# Append files to existing archive (uncompressed only)
[student@server ~]$ tar -rvf backup.tar newfile.txt

Note: Update (-u) and append (-r) only work with uncompressed archives. Compressed archives must be recreated entirely.

These advanced tar operations handle common real-world scenarios. The -p flag preserves permissions, ownership, and special attributes. Root extractions use this by default, but being explicit is good practice for system backups. Excluding files is essential for backups - you don't want to include cache directories, log files, or temporary data. Use --exclude with patterns, or -X to read exclusion patterns from a file. Multiple --exclude options can be chained. The update (-u) operation adds files to an archive only if they're newer than the version in the archive, or if they don't exist in the archive. Useful for incremental-style backups. Append (-r) adds files unconditionally. Important limitation: update and append only work with uncompressed tar archives. You cannot add to a .tar.gz file - the entire archive must be recreated. For incremental backups of compressed archives, consider other tools or maintain an uncompressed working archive that you compress when finished.

File Extensions Reference

Extension	Type	Create	Extract
.tar	Uncompressed archive	tar -cvf	tar -xvf
.tar.gz / .tgz	Gzip compressed	tar -czvf	tar -xzvf
.tar.bz2 / .tbz2	Bzip2 compressed	tar -cjvf	tar -xjvf
.tar.xz / .txz	XZ compressed	tar -cJvf	tar -xJvf
.gz	Gzip single file	gzip file	gunzip file.gz
.bz2	Bzip2 single file	bzip2 file	bunzip2 file.bz2
.xz	XZ single file	xz file	unxz file.xz
.zip	Zip archive	zip -r arch.zip dir/	unzip arch.zip

Convention matters: Extensions tell users what type of file it is and how to handle it. Always use appropriate extensions.

This reference table summarizes all the archive and compression types you'll encounter. The extension tells you both the archive format and compression used. Double extensions like .tar.gz indicate a tar archive that was then compressed with gzip. Linux doesn't require extensions for functionality, but conventions help humans understand what they're working with. Always use appropriate extensions. Note the distinction between compressed archives (.tar.gz) and compressed single files (.gz). A .gz file is a single compressed file. A .tar.gz is multiple files archived together, then compressed. The .tgz, .tbz2, and .txz are alternative short forms you'll see, especially in software distribution. They're equivalent to their longer forms. zip is its own format - it combines archiving and compression in one format, unlike tar which separates them. Use zip when sharing with Windows users or when the recipient expects a standard zip file.

The zip Command

zip creates archives compatible with Windows and other operating systems. It combines archiving and compression in one format.

# Create zip archive of files
[student@server ~]$ zip archive.zip file1.txt file2.txt file3.txt

# Create zip archive of directory (recursive)
[student@server ~]$ zip -r project.zip project/
  adding: project/ (stored 0%)
  adding: project/README.md (deflated 45%)
  adding: project/src/ (stored 0%)
  adding: project/src/main.c (deflated 62%)

# Extract zip archive
[student@server ~]$ unzip project.zip

# Extract to specific directory
[student@server ~]$ unzip project.zip -d /tmp/extract/

# List contents without extracting
[student@server ~]$ unzip -l project.zip

# Add password protection
[student@server ~]$ zip -e -r secure.zip sensitive/
Enter password: 
Verify password:

When to use zip: Sharing with Windows users, email attachments, when recipients expect .zip format. For Linux-to-Linux, tar.gz is more common.

zip is primarily used for cross-platform compatibility. Windows natively supports zip files, so they're the standard for sharing archives with Windows users. zip combines archiving and compression in one format - there's no separate "archive then compress" step. The -r option is required for directories - without it, zip only adds the directory entry, not its contents. This is different from tar which is recursive by default. unzip extracts zip archives. The -l option lists contents without extracting, like tar -tf. The -d option specifies extraction directory, like tar -C. zip supports password protection with -e. The encryption isn't the strongest, but provides basic protection for casual security. For serious encryption, use gpg. One advantage of zip: you can extract individual files without processing the entire archive. Each file is compressed separately. With tar.gz, you typically must decompress the entire archive even to extract one file. For Linux-native work, tar.gz is more common and preserves Unix permissions better. zip is for interoperability.

Transferring with scp

scp (secure copy) transfers files between systems over SSH. It encrypts data in transit.

# Copy local file to remote system
[student@local ~]$ scp backup.tar.gz student@server:/home/student/
backup.tar.gz                      100%  15MB  10.2MB/s   00:01

# Copy from remote to local
[student@local ~]$ scp student@server:/home/student/data.tar.gz ./

# Copy directory recursively
[student@local ~]$ scp -r student@server:/var/www/ ./backup/

# Use specific port
[student@local ~]$ scp -P 2222 backup.tar.gz student@server:/home/student/

# Copy between two remote systems
[student@local ~]$ scp student@server1:/data/file.tar.gz student@server2:/backup/

# Preserve timestamps and permissions
[student@local ~]$ scp -p backup.tar.gz student@server:/home/student/

# Verbose mode for debugging
[student@local ~]$ scp -v backup.tar.gz student@server:/home/student/

scp transfers files securely over SSH, using the same authentication and encryption. The syntax is scp source destination, where either can be remote (user@host:path) or local. For remote to local, put the remote path first. For local to remote, put the local path first. scp shows progress with file size, transfer rate, and time. The -r option copies directories recursively. Unlike cp, this is required for directories. The -P option (capital P, unlike ssh which uses lowercase -p) specifies an alternate SSH port. The -p option (lowercase) preserves modification times and permissions from the original file. scp can even copy between two remote systems, routing through your local machine. Both systems need to be accessible from your machine. For large transfers or regular synchronization, rsync is often better than scp - it's more efficient, can resume interrupted transfers, and only transfers changed data.

Transferring with rsync

rsync efficiently synchronizes files, transferring only the differences. Ideal for backups and mirroring.

# Basic rsync (archive mode preserves everything)
[student@local ~]$ rsync -av /home/student/documents/ student@server:/backup/docs/

# Sync with compression during transfer
[student@local ~]$ rsync -avz /home/student/ student@server:/backup/

# Delete files on destination that don't exist on source (mirror)
[student@local ~]$ rsync -av --delete /source/ /destination/

# Dry run - show what would happen without doing it
[student@local ~]$ rsync -av --dry-run /source/ /destination/

# Exclude files
[student@local ~]$ rsync -av --exclude='*.log' --exclude='cache/' /src/ /dst/

# Show progress for large transfers
[student@local ~]$ rsync -av --progress /large/directory/ /backup/

# Resume interrupted transfer
[student@local ~]$ rsync -av --partial /source/ /destination/

Key advantage: rsync only transfers changed portions of files. Syncing a 10GB directory where 100MB changed transfers only ~100MB.

rsync is the tool of choice for efficient file synchronization and backup. Its key advantage is delta transfer - it only sends the differences between source and destination. The -a option (archive) preserves permissions, ownership, timestamps, symbolic links, and recurses into directories. It's the standard option for backups. The -v option shows verbose output. The -z option compresses data during transfer (separate from file compression). Important: trailing slashes matter! "rsync /src/ /dst/" copies contents of src into dst. "rsync /src /dst/" copies the src directory itself into dst. The --delete option removes files from the destination that don't exist in the source, creating an exact mirror. Use carefully - it will delete files! Always use --dry-run first to see what would happen. The --partial option keeps partially transferred files, allowing resume of interrupted transfers. Combined with --progress, it's great for large transfers over unreliable connections. rsync works locally, or over SSH for remote transfers (same syntax as scp for remote paths).

Backup Best Practices

# Create timestamped backup
[student@server ~]$ tar -czvf backup-$(date +%Y%m%d).tar.gz /home/student/

# Or with full timestamp
[student@server ~]$ tar -czvf backup-$(date +%Y%m%d-%H%M%S).tar.gz /home/student/
backup-20240120-143052.tar.gz

# Verify archive integrity after creation
[student@server ~]$ tar -tzvf backup-20240120.tar.gz > /dev/null && echo "Archive OK"
Archive OK

# Create checksum for verification
[student@server ~]$ sha256sum backup-20240120.tar.gz > backup-20240120.tar.gz.sha256
[student@server ~]$ sha256sum -c backup-20240120.tar.gz.sha256
backup-20240120.tar.gz: OK

# Full backup script example
[student@server ~]$ cat backup.sh
#!/bin/bash
DATE=$(date +%Y%m%d)
BACKUP_DIR="/backup"
tar -czvf "$BACKUP_DIR/home-$DATE.tar.gz" /home/
sha256sum "$BACKUP_DIR/home-$DATE.tar.gz" > "$BACKUP_DIR/home-$DATE.tar.gz.sha256"
# Keep only last 7 days
find "$BACKUP_DIR" -name "home-*.tar.gz" -mtime +7 -delete

Good backup practices ensure your archives are reliable and recoverable. Timestamped filenames prevent overwriting and make it easy to identify backup age. The $(date +%Y%m%d) format gives sortable names: 20240120. Always verify archives after creation. tar -tf lists contents - if it completes without error, the archive is structurally sound. Redirect to /dev/null to skip the output, just check the exit status. Checksums verify file integrity after transfer. Create a sha256sum of the archive. After copying to another system, verify with sha256sum -c. If the checksum doesn't match, the file was corrupted in transit. A complete backup workflow: create the archive, verify it, generate checksum, transfer to backup location, verify checksum at destination, maintain retention by deleting old backups. For important data, follow the 3-2-1 rule: 3 copies, 2 different media types, 1 offsite. Test your backups by actually restoring from them periodically!

System Recovery Archives

# Backup critical system directories (as root)
[root@server ~]# tar -czvpf system-config.tar.gz \
    /etc \
    /var/spool/cron \
    /root \
    --exclude='/etc/mtab'

# Backup user home directories
[root@server ~]# tar -czvpf homes.tar.gz /home/

# Full system backup (excluding pseudo-filesystems)
[root@server ~]# tar -czvpf full-backup.tar.gz \
    --exclude=/proc \
    --exclude=/sys \
    --exclude=/dev \
    --exclude=/run \
    --exclude=/tmp \
    --exclude=/mnt \
    --exclude=/media \
    --exclude=/lost+found \
    --exclude='full-backup.tar.gz' \
    /

# Restore while preserving ownership (as root)
[root@server ~]# tar -xzvpf system-config.tar.gz -C /
# -p preserves permissions
# -C / extracts to root filesystem

⚠ Caution: Restoring system files can break your system. Test on non-production systems first. Have rescue media ready.

System-level backups require root and careful planning. For configuration backups, /etc contains most system configuration. Include /var/spool/cron for crontabs, /root for root's files. Exclude pseudo-files like /etc/mtab that are generated. For full system backup, you must exclude pseudo-filesystems (proc, sys, dev, run) - these aren't real files and shouldn't be backed up. Exclude /tmp, and exclude your own backup file to avoid recursion. Include everything else. The -p flag is crucial for system backups - it preserves permissions, ownership, and special attributes. Without it, restored files might have wrong permissions. When restoring, extract with -p to the root filesystem (-C /). Be extremely careful - this overwrites system files. Test on a VM or spare system first. Have rescue media available in case something goes wrong. For production systems, consider proper backup solutions like Bacula, Amanda, or cloud backup services rather than manual tar commands. But understanding tar-based backup and recovery is fundamental knowledge.

Working with Large Archives

# Split large archive into smaller pieces
[student@server ~]$ tar -czvf - /large/directory/ | split -b 1G - backup.tar.gz.part
backup.tar.gz.partaa
backup.tar.gz.partab
backup.tar.gz.partac

# Reassemble and extract
[student@server ~]$ cat backup.tar.gz.part* | tar -xzvf -

# Create archive directly to remote system (no local storage needed)
[student@server ~]$ tar -czvf - /home/student/ | ssh user@backup-server "cat > /backup/home.tar.gz"

# Extract from remote archive without downloading
[student@server ~]$ ssh user@backup-server "cat /backup/home.tar.gz" | tar -xzvf -

# Stream archive through compression to remote
[student@server ~]$ tar -cvf - /data/ | xz -T 0 | ssh user@server "cat > /backup/data.tar.xz"

# Check archive progress (with pv if available)
[student@server ~]$ tar -cvf - /large/ | pv | gzip > large.tar.gz
45.2GiB 0:12:34 [61.5MiB/s] [=========>                    ] 32% ETA 0:26:12

Streaming: Using -f - makes tar read/write to stdin/stdout, enabling powerful pipeline operations.

Large archives require special techniques. Sometimes files are too big for the destination filesystem (FAT32 has 4GB limit) or you need to fit them on multiple discs. The split command breaks files into pieces. Combined with tar writing to stdout (-f -), you can create split archives. The -b option specifies chunk size (1G = 1 gigabyte). Reassemble with cat and pipe directly to tar for extraction. Streaming archives over SSH avoids needing local disk space. tar -cvf - sends the archive to stdout, which pipes to SSH, which writes on the remote system. Reverse it for remote extraction. This is powerful for systems with limited local storage. The pv command (pipe viewer) shows progress for any pipeline. Insert it between tar and compression to see transfer rates, progress percentage, and ETA. Install with dnf install pv. These pipeline techniques demonstrate Unix philosophy - small tools combined flexibly solve complex problems. Understanding stdin/stdout (-f -) unlocks many possibilities.

Troubleshooting Archives

# Archive is corrupted - try to extract what's possible
[student@server ~]$ tar -xvf damaged.tar --ignore-zeros
[student@server ~]$ gzip -d -f damaged.tar.gz       # Force despite errors

# Test archive integrity
[student@server ~]$ gzip -t backup.tar.gz
[student@server ~]$ bzip2 -t backup.tar.bz2
[student@server ~]$ xz -t backup.tar.xz

# Identify compression type of unknown file
[student@server ~]$ file mystery.archive
mystery.archive: gzip compressed data, last modified: Sat Jan 20 14:00:00 2024

# Check what's using disk space during archiving
[student@server ~]$ du -sh /home/student/* | sort -h | tail -10

# Permission denied during extraction - need root?
[student@server ~]$ tar -xvf backup.tar 2>&1 | grep -i "permission denied"

# File changed during archiving
tar: /var/log/messages: file changed as we read it
# This warning is usually OK - file was modified during backup

Common Issues: Permission denied (run as root), disk full (check space), corrupted archives (verify checksums), wrong compression flag.

Troubleshooting archive problems is a common task. Corrupted archives can sometimes be partially recovered. The --ignore-zeros option tells tar to continue past errors. gzip -f forces decompression despite errors - you might get partial data. Test archive integrity before trusting them. Each compression tool has a test option (-t). This verifies the compression structure without extracting. The file command identifies file types regardless of extension. If someone sends you a mystery archive, file tells you what it actually is. "file changed as we read it" is a warning that appears when archiving active files like logs. The file was modified while tar was reading it. Usually this is fine - you get the content at one point in time. For databases or other consistency-critical files, you need proper backup methods that ensure consistency. Common problems: permission denied means you need root or don't have read access. Disk full during archiving - check space with df -h. Wrong decompression - if tar -xzvf fails, check the actual compression type with file.

Best Practices

✓ Do

Use timestamp in backup filenames
Verify archives after creation with -t
Create checksums for verification
Use -p for system backups (preserve permissions)
Test restoration process periodically
Use appropriate compression for the situation
List contents before extracting unknown archives
Use rsync for incremental/regular syncs

✗ Don't

Trust backups without verification
Extract archives as root without checking contents
Forget -r with zip for directories
Use absolute paths without understanding implications
Compress already-compressed files (JPEG, MP4, ZIP)
Keep backups only on the same disk
Ignore "file changed" warnings for critical data
Delete original until backup is verified

Remember: A backup that hasn't been tested isn't a backup - it's a hope. Regularly practice restoration!

Following best practices ensures your archives are reliable when you need them. Timestamped filenames make it clear when backups were created and prevent accidental overwrites. Always verify archives - use -t to test structure and checksums to verify integrity after transfer. Use -p for system backups to preserve permissions. Without it, restored files may have wrong ownership or permissions. Test your backups by actually restoring from them. A backup you've never tested might not work when you need it. Choose compression appropriately: gzip for speed, xz for size, none for already-compressed data. Always list archive contents before extracting, especially as root. Malicious archives could overwrite system files. Use rsync instead of scp for regular transfers - it's more efficient and can resume. Keep backups on separate physical storage. If your disk fails and the backup is on the same disk, both are gone. Never delete originals until backups are verified. The most important practice: test restoration regularly. Know your backup actually works before you need it.

Key Takeaways

tar: Create with -cvf, extract with -xvf, list with -tvf. Always use -f with filename. -p preserves permissions.

Compression: gzip (-z) fast, bzip2 (-j) better ratio, xz (-J) best ratio. Match to your speed/size needs.

Compressed archives: tar -czvf for .tar.gz, tar -cjvf for .tar.bz2, tar -cJvf for .tar.xz

Transfer: scp for simple copies, rsync for efficient sync. Both work over SSH securely.

Let's summarize the key archiving and compression skills. For tar, remember the operation flags: -c creates, -x extracts, -t lists. Always pair with -f and the filename. Add -v for verbose output. Use -p to preserve permissions on system backups. For compression, choose based on your tradeoffs: gzip (-z) is fast with good compression, bzip2 (-j) is slower with better compression, xz (-J) is slowest with best compression. The uppercase -J for xz is easy to forget. Compressed tar archives combine both steps: tar -czvf creates .tar.gz, tar -cjvf creates .tar.bz2, tar -cJvf creates .tar.xz. For extraction, modern tar auto-detects, but explicit flags are clearer. For file transfer, scp handles simple copies over SSH. rsync is more efficient for regular synchronization - it only transfers changes and can resume interrupted transfers.

Graded Lab

Create a tar archive of your home directory
Create gzip, bzip2, and xz compressed archives and compare sizes
List contents of an archive without extracting
Extract specific files from an archive
Transfer an archive to another system with scp
Use rsync to synchronize a directory

Next: Transfering Files