Chapter Objective: Query and filter system logs using journalctl, configure the systemd journal for persistent storage, and understand how rsyslog and traditional /var/log files fit alongside it.

Key commands: journalctl, journalctl -u, journalctl -p, journalctl --since

RHEL's Logging Architecture

RHEL logs flow through two complementary systems that often work together rather than compete.

ComponentRole
systemd-journaldCollects structured log data from services, the kernel, and early boot, queried with journalctl
rsyslogTraditional syslog daemon; can consume journal data and write classic plain-text files under /var/log
🔵 Why It Matters When something breaks, logs are usually the fastest path to understanding why. Knowing where to look — and how to filter out the noise — turns troubleshooting from guesswork into a methodical process.

Reading Logs with journalctl

# Show all journal entries, oldest first, in a pager
journalctl

# Show the most recent entries first
journalctl -r

# Follow new entries live, like tail -f
journalctl -f

# Show only entries from the current boot
journalctl -b

# Show entries from the PREVIOUS boot
journalctl -b -1

# List all boots the journal has records for
journalctl --list-boots
✅ Tip — journalctl -b Is Often the Right First Step When troubleshooting something that started "just now," narrowing to the current boot with -b cuts out potentially days or weeks of unrelated history immediately.

Filtering journalctl Output

# Logs for one specific systemd unit
journalctl -u sshd

# Logs within a specific time range
journalctl --since "2026-08-01" --until "2026-08-03"

# Relative time expressions work too
journalctl --since "1 hour ago"
journalctl --since yesterday

# Only messages at or above a given priority level
journalctl -p err

# Combine filters — logs for sshd, errors only, since yesterday
journalctl -u sshd -p err --since yesterday

# Output as structured JSON, useful for scripting
journalctl -u sshd -o json-pretty
🔵 Exam Note Filters combine with AND logic by default — journalctl -u sshd -p err shows entries that are BOTH from sshd AND at error level or worse, not either condition alone.

Making the Journal Persistent

By default on many RHEL installs, the journal is kept only in a volatile, in-memory ring buffer under /run/log/journal — meaning it's lost on reboot. Making it persistent writes logs to disk instead.

# Create the directory journald uses for persistent storage
sudo mkdir -p /var/log/journal

# Set ownership/permissions systemd expects
sudo systemd-tmpfiles --create --prefix /var/log/journal

# Restart journald to pick up the change
sudo systemctl restart systemd-journald

# Confirm where the journal is currently stored
journalctl --disk-usage
⚠️ Warning — Persistent Logs Need Disk Space Management Once persistent, the journal will keep growing unless limited. Configure SystemMaxUse= in /etc/systemd/journald.conf to cap how much disk space it's allowed to consume.

rsyslog and /var/log

While journald handles structured, queryable logging, many administrators still rely on traditional plain-text log files, generated by rsyslog.

# Common traditional log file locations
/var/log/messages     # general system messages
/var/log/secure       # authentication and authorization events
/var/log/cron         # cron job activity
/var/log/maillog      # mail subsystem activity

# View one directly, like any text file
tail -f /var/log/secure
✅ Tip — Same Data, Two Views rsyslog's plain-text files under /var/log are frequently just a different presentation of the same underlying journal data — journalctl gives you structured filtering and querying; grep/tail on /var/log files gives you familiar plain text.

Facilities and Priorities

Every syslog-style message carries a facility (what subsystem generated it) and a priority (how severe it is), following a standard severity scale from most to least critical.

PriorityMeaning
emerg (0)System is unusable
alert (1)Action must be taken immediately
crit (2)Critical condition
err (3)Error condition
warning (4)Warning condition
notice (5)Normal but significant condition
info (6)Informational message
debug (7)Debug-level message
🔵 Exam Note Filtering with journalctl -p err shows err AND everything more severe (crit, alert, emerg) — priority filters are "this level or worse," not an exact match.

Key Terms for Chapter 5

systemd-journald
The systemd component that collects structured system and service logs
journalctl
Command used to query and filter journal log entries
rsyslog
Traditional syslog daemon that can write plain-text logs under /var/log
persistent journal
Journal storage configured to survive reboots by writing to disk instead of memory
facility
A category identifying which subsystem generated a log message
priority
A severity level attached to a log message, from emerg (most severe) to debug (least)

Review Questions

  1. What's the difference in role between systemd-journald and rsyslog?
  2. Write a command that shows only sshd log entries from the current boot.
  3. By default, is the journal typically persistent across reboots? What steps make it persistent?
  4. What does journalctl -p err actually include — only exact "err" messages, or something broader?
  5. What traditional log file would you check for authentication and authorization events?
  6. Write a command that shows sshd logs, at error level or worse, from the last hour.
  7. Why does an administrator need to configure something like SystemMaxUse= once the journal is made persistent?