Chapter Objective: Create, verify, and manage archives using advanced tar options, incremental backups, rsync, and raw disk imaging, and choose the right tool for a given archiving task.
Key commands: tar --exclude, tar -g, rsync -a, sha256sum, dd
tar: Beyond the Basics
Basic tar usage — creating, listing, and extracting archives — was covered earlier. This chapter builds on that with the options that come up in real backup and archiving work: excluding paths, appending to existing archives, incremental backups, and verifying what actually got archived.
Excluding, Appending, and Verifying
# Exclude specific paths from an archive
tar -czvf home-backup.tar.gz --exclude='*/cache/*' --exclude='*.tmp' /home
# Exclude using a file listing multiple patterns
tar -czvf backup.tar.gz -X exclude-list.txt /home
# Append additional files to an EXISTING (uncompressed) archive
tar -rvf backup.tar /etc/newfile.conf
# Compare an archive's contents against the filesystem WITHOUT extracting
tar -dvf backup.tar
tar -r (append) only works on a plain .tar file, not a compressed .tar.gz. To add files to a compressed archive, you'd typically decompress, append, and recompress — or simply create a fresh archive instead.
Incremental Archives
A full backup every time is often wasteful. tar supports incremental archives, using a snapshot file to track what changed since the last run.
# First run: full backup, and create the snapshot file
tar -czvf level0.tar.gz -g snapshot.file /home
# Later run: only files changed since the snapshot was taken
tar -czvf level1.tar.gz -g snapshot.file /home
# Restoring requires applying the full backup, then each incremental in order
tar -xzvf level0.tar.gz -C /restore/
tar -xzvf level1.tar.gz -C /restore/
rsync for Syncing and Archiving
rsync synchronizes files and directories, transferring only the differences rather than copying everything each time — efficient for repeated backups, especially over a network.
# Sync a directory to a local backup location, archive mode (preserves
# permissions, ownership, timestamps, and symlinks; recursive)
rsync -av /home/ /backups/home/
# Sync to a REMOTE host over SSH
rsync -av /home/ student@backup-server:/backups/home/
# Preview what would happen, without actually copying anything
rsync -av --dry-run /home/ /backups/home/
# Delete files in the destination that no longer exist in the source
rsync -av --delete /home/ /backups/home/
rsync -av /home/ /backups/ copies the CONTENTS of /home into /backups. Drop the trailing slash on the source (/home) and rsync instead copies the home directory itself into /backups, one level deeper than you might expect.
Verifying Archive Integrity
# Generate a checksum for an archive right after creating it
sha256sum backup.tar.gz > backup.tar.gz.sha256
# Later, verify it hasn't been corrupted or altered
sha256sum -c backup.tar.gz.sha256
# Test a gzip archive's internal integrity without extracting
gzip -t backup.tar.gz
dd for Raw Disk Imaging
dd copies data at the block level, making an exact bit-for-bit image of a disk or partition — useful when you need a complete replica, not just the files a filesystem currently exposes.
# Image an entire disk to a file
sudo dd if=/dev/sdb of=/backups/sdb-image.img bs=4M status=progress
# Restore an image back onto a disk
sudo dd if=/backups/sdb-image.img of=/dev/sdb bs=4M status=progress
if= (input) and of= (output) backwards, or targeting the wrong device, overwrites data instantly and irreversibly with zero warning. Triple-check device names with lsblk before running dd against a real disk.
Choosing the Right Tool
| Goal | Best Tool |
|---|---|
| Portable, compressed archive of specific files | tar |
| Efficient repeated sync, especially over a network | rsync |
| Exact bit-for-bit copy of an entire disk/partition | dd |
| Confirming a backup wasn't corrupted | sha256sum / gzip -t |
Key Terms for Chapter 7
- incremental backup
- A backup containing only what changed since a previous backup, tracked via a snapshot file
- rsync
- Utility that synchronizes files by transferring only the differences between source and destination
- checksum
- A computed value used to verify that data hasn't been altered or corrupted
- dd
- Utility that copies data at the block level, capable of creating exact disk images
- block-level copy
- A copy performed on raw disk blocks rather than through the filesystem's file-by-file view
Review Questions
- Why can't you use
tar -rto append a file directly to a.tar.gzarchive? - What is the purpose of the snapshot file in a tar incremental backup, and what happens if you lose it?
- What is the practical difference in behavior between
rsync -av /home/ /backups/andrsync -av /home /backups/(note the trailing slash)? - Why is it best practice to generate a checksum immediately after creating a backup, rather than later?
- What makes
dddifferent fromtarin terms of what it actually copies? - Why is
ddconsidered especially risky to run compared to most other archiving tools? - You need to keep a remote backup server in sync with a directory that changes daily, without re-transferring everything each time. Which tool from this chapter fits best, and why?