Domain 4 · System operations

Operate
running systems

Boot targets, systemd service management, process control, system logging with journald, and job scheduling with cron and at — on RHEL 9.

12objectives
6topic areas
8quiz questions
systemdprimary tool

Objectives

What the exam tests

  • Boot, reboot, and shut down a system normally
  • Boot systems into different targets manually
  • Interrupt the boot process to gain access to a system
  • Identify CPU/memory-intensive processes and kill or adjust process priority
  • Adjust process scheduling with nice and renice
  • Manage tuning profiles using tuned
  • Locate and interpret system log files and journals
  • Preserve the systemd journal across reboots
  • Start, stop, and check the status of network services
  • Securely transfer files between systems
  • Schedule tasks using at and cron
  • Manage temporary files with systemd-tmpfiles

Boot-target interruption (rescue mode / emergency mode) and service management with systemctl are the most heavily weighted tasks in this domain.

Coverage weight by topic

systemd / services
Very high
Boot & targets
Very high
Process management
High
Logging / journald
High
Job scheduling
High
tuned / tmpfiles
Medium

Boot and targets

Shutdown, reboot, and power commands

# Graceful shutdown systemctl poweroff poweroff shutdown -h now # Scheduled shutdown (minutes from now) shutdown -h +10 # shut down in 10 minutes shutdown -h 23:00 # shut down at 23:00 shutdown -c # cancel a pending shutdown # Reboot systemctl reboot reboot shutdown -r now # Broadcast a wall message to logged-in users wall "System going down for maintenance in 5 minutes"

systemd targets

Targets replace SysV runlevels. Each defines a group of services to start. The table below shows the equivalents.

poweroff.target
Runlevel 0
Halt and power off the system
rescue.target
Runlevel 1 / single
Single-user, root shell, minimal services. Requires root password.
multi-user.target
Runlevel 3
Full multi-user text mode. No GUI. Server default
graphical.target
Runlevel 5
Multi-user + GUI (GNOME). Requires graphical packages.
reboot.target
Runlevel 6
Reboot the system
emergency.target
Emergency shell
Minimal shell, read-only root. For severe filesystem issues.
# Switch to a target immediately (does not persist across reboots) systemctl isolate multi-user.target systemctl isolate graphical.target systemctl isolate rescue.target # Set the default boot target (persists across reboots) systemctl set-default multi-user.target systemctl set-default graphical.target # Check the current default systemctl get-default

Only targets with AllowIsolate=yes in their unit file can be used with isolate. The main exam targets (rescue, multi-user, graphical, emergency) all support it.

Interrupting the boot to gain root access

This is a heavily tested RHCSA task — breaking into a system to reset the root password or fix a misconfiguration.

StepAction
1Reboot the system. At the GRUB menu, press e to edit the selected boot entry.
2Find the line starting with linux. Navigate to the end of that line.
3Append rd.break to break into the initramfs before the root filesystem is mounted read-write. (Alternative: init=/bin/bash for an even more minimal shell.)
4Press Ctrl+X to boot with the modified parameters.
5Remount the real root filesystem read-write:
mount -o remount,rw /sysroot
6Change root into the real system:
chroot /sysroot
7Make your changes (e.g., passwd root).
8Tell SELinux to relabel on next boot:
touch /.autorelabel
9Exit chroot (exit), then reboot (exit again or reboot -f).

Always create /.autorelabel after changing files in chroot. Without it, SELinux will block logins because file contexts will be wrong.

systemd service management

systemctl — core service commands

# Start / stop / restart / reload systemctl start httpd systemctl stop httpd systemctl restart httpd systemctl reload httpd # reload config without full restart systemctl reload-or-restart httpd # reload if supported, else restart # Enable / disable at boot systemctl enable httpd # start on boot systemctl disable httpd # do not start on boot systemctl enable --now httpd # enable AND start immediately systemctl disable --now httpd # disable AND stop immediately # Mask / unmask (prevent any start — even manual) systemctl mask httpd systemctl unmask httpd # Status and information systemctl status httpd systemctl is-active httpd # prints "active" or "inactive" systemctl is-enabled httpd # prints "enabled" or "disabled" systemctl is-failed httpd

enable --now is the most efficient exam command — it both enables the service to start at boot and starts it immediately in a single step.

Listing and inspecting units

# List all loaded units systemctl list-units systemctl list-units --type=service systemctl list-units --state=failed # List ALL unit files (including disabled) systemctl list-unit-files systemctl list-unit-files --type=service # Show unit file content systemctl cat httpd.service # Show all properties of a unit systemctl show httpd.service # Show dependency tree systemctl list-dependencies httpd.service

Unit file anatomy

# Example: /etc/systemd/system/myapp.service [Unit] Description=My Application Service After=network.target [Service] Type=simple ExecStart=/usr/bin/myapp --config /etc/myapp.conf ExecStop=/usr/bin/myapp --stop Restart=on-failure User=myapp [Install] WantedBy=multi-user.target
# After creating or editing a unit file, reload the daemon systemctl daemon-reload # User-level unit file locations (priority order, highest first) # /etc/systemd/system/ ← admin customizations (use this) # /run/systemd/system/ ← runtime (temporary) # /usr/lib/systemd/system/ ← package-provided (do not edit)

Always edit unit files in /etc/systemd/system/, never in /usr/lib/systemd/system/. Package updates will overwrite files in the latter location.

tuned — performance profiles

# Check active profile tuned-adm active # List all available profiles tuned-adm list # Apply a profile tuned-adm profile throughput-performance tuned-adm profile virtual-guest tuned-adm profile balanced # Get a recommendation for current hardware tuned-adm recommend # Disable tuned (returns to default kernel settings) tuned-adm off
ProfileBest for
balancedGeneral-purpose compromise of power and performance
throughput-performanceMaximum throughput — disables power saving
latency-performanceLow-latency workloads (trading, real-time)
virtual-guestRunning as a VM guest
virtual-hostRunning as a hypervisor host
powersaveMinimize power consumption

Process management

Viewing processes

# Snapshot of all processes ps aux # BSD style: all users, with details ps -ef # POSIX style: full format ps -ef | grep httpd # filter by name # ps column guide (aux) # USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND # Show process tree pstree pstree -p # include PIDs # Real-time process monitor top htop # enhanced (may need dnf install htop) # top keyboard shortcuts: # k — kill a process (enter PID) # r — renice a process # M — sort by memory # P — sort by CPU (default) # q — quit # 1 — toggle per-CPU view

Sending signals to processes

# Send a signal by PID kill 1234 # default: SIGTERM (15) — graceful kill -9 1234 # SIGKILL — immediate, cannot be caught kill -SIGTERM 1234 kill -l # list all signal names and numbers # Kill by name (all matching processes) killall httpd killall -9 httpd pkill httpd # kill by pattern pkill -u alice # kill all processes owned by alice # Find PIDs by name pidof httpd pgrep httpd pgrep -u alice # all PIDs owned by alice
SignalNumberEffect
SIGHUP1Hangup — reload config (many daemons respond to this)
SIGINT2Interrupt — same as Ctrl+C
SIGTERM15Terminate — graceful shutdown request (default)
SIGKILL9Kill immediately — cannot be caught or ignored
SIGSTOP19Pause process (cannot be caught)
SIGCONT18Resume a stopped process

Always try SIGTERM (15) first. Only use SIGKILL (9) if the process does not respond — it prevents cleanup and can corrupt data.

Process priority — nice and renice

# Nice values range from -20 (highest priority) to +19 (lowest priority) # Default nice value is 0. Only root can set negative values. # Start a process with a specific nice value nice -n 10 tar -czf backup.tar.gz /data # lower priority nice -n -5 myapp # higher priority (root only) # Change priority of a running process renice -n 15 -p 1234 # by PID renice -n 10 -u alice # all processes owned by alice renice -n 5 -g staff # all processes in group staff # View nice value in ps output ps -eo pid,ni,comm | sort -k2

Memory aid: a nicer process is more considerate of others — it gets less CPU. So a higher nice value = lower priority = less CPU time.

Job control

# Run a command in the background long-command & # Suspend a running foreground process # Press Ctrl+Z # List current shell's jobs jobs jobs -l # include PIDs # Resume a stopped job in the background bg %1 # job number 1 # Bring a job to the foreground fg %1 # Detach from terminal (survives SSH disconnect) nohup long-command &

System logging

journalctl — querying the systemd journal

# View the full journal (oldest first) journalctl # Follow live output (like tail -f) journalctl -f # Show only the most recent entries journalctl -n 50 # last 50 lines journalctl -n 20 -f # last 20 and follow # Filter by systemd unit journalctl -u httpd.service journalctl -u sshd -f # follow sshd logs live # Filter by time journalctl --since "2024-01-15 10:00:00" journalctl --since "1 hour ago" journalctl --since "yesterday" journalctl --since "2024-01-15" --until "2024-01-16" # Filter by priority (0=emerg 1=alert 2=crit 3=err 4=warn 5=notice 6=info 7=debug) journalctl -p err # errors and above journalctl -p warning # warnings and above # Filter by boot journalctl -b # current boot only journalctl -b -1 # previous boot journalctl --list-boots # list all boots # Kernel messages only journalctl -k

Persisting the journal across reboots

By default on RHEL 9, the journal is stored in /run/log/journal/ which is a tmpfs — it is lost on every reboot. To persist it:

# Option 1: Create the persistent directory (recommended) mkdir -p /var/log/journal systemd-tmpfiles --create --prefix /var/log/journal systemctl restart systemd-journald # Option 2: Edit /etc/systemd/journald.conf # Set: Storage=persistent # Then restart journald: systemctl restart systemd-journald # Verify journal is now stored persistently ls /var/log/journal/ # Check journal disk usage journalctl --disk-usage # Rotate / vacuum old journal entries journalctl --vacuum-size=100M journalctl --vacuum-time=2weeks

Creating /var/log/journal/ is the simplest exam approach. journald detects the directory on restart and switches to persistent storage automatically.

Traditional log files and rsyslog

Log fileContents
/var/log/messagesGeneral system messages — the main log file
/var/log/secureAuthentication and authorization (SSH, sudo, su)
/var/log/cronCron job execution records
/var/log/maillogMail service messages
/var/log/boot.logBoot-time service startup messages
/var/log/dmesgKernel ring buffer from boot (also: dmesg command)
/var/log/audit/audit.logSELinux and audit framework messages
/var/log/httpd/Apache web server access and error logs
# Common log-reading commands tail -f /var/log/messages # follow live tail -n 100 /var/log/secure # last 100 lines grep "Failed" /var/log/secure dmesg | grep -i error

systemd-tmpfiles — managing temporary files

# Create temp files/dirs defined in config systemd-tmpfiles --create # Clean up stale temp files systemd-tmpfiles --clean # Remove temp files systemd-tmpfiles --remove # Config files live in: # /etc/tmpfiles.d/ ← admin-created (highest priority) # /run/tmpfiles.d/ ← runtime # /usr/lib/tmpfiles.d/ ← package-provided # Example entry format: type path mode uid gid age argument # d /run/myapp 0755 myapp myapp 10d - # d = directory, age = delete after 10 days

Job scheduling

at — one-time scheduled jobs

# Schedule a one-time job at now + 5 minutes at 14:30 at 14:30 tomorrow at 14:30 July 4 at noon at midnight # After running at, type commands, then Ctrl+D to save # Or pipe commands in echo "systemctl restart httpd" | at now + 1 hour at 22:00 <<EOF /usr/local/bin/backup.sh EOF # List pending at jobs atq at -l # same as atq # Remove a pending job atrm 3 # remove job number 3 at -d 3 # same as atrm 3 # View job details at -c 3 # show commands for job 3

The atd service must be running: systemctl enable --now atd. Check with systemctl status atd.

cron — recurring scheduled jobs

# Edit the current user's crontab crontab -e # List current user's crontab crontab -l # Remove current user's crontab crontab -r # Edit another user's crontab (root only) crontab -u alice -e # Crontab field format: # min hour day-of-month month day-of-week command # * * * * * # Examples: 0 2 * * * /usr/local/bin/backup.sh # 2:00 AM daily */15 * * * * /usr/bin/check_disk.sh # every 15 min 0 9 * * 1 /usr/bin/weekly_report.sh # 9 AM every Monday 0 0 1 * * /usr/bin/monthly_cleanup.sh # midnight, 1st of month 30 8 * * 1-5 /usr/bin/workday_task.sh # 8:30 AM Mon-Fri

Crontab field reference

FieldPositionValid valuesSpecial
Minute1st0–59* every, */n every n, a-b range, a,b list
Hour2nd0–23Same as above
Day of month3rd1–31Same as above
Month4th1–12 or jan–decSame as above
Day of week5th0–7 or sun–sat (0 and 7 = Sunday)Same as above

System crontab locations

  • /etc/crontab — system crontab (has username field)
  • /etc/cron.d/ — drop-in cron files
  • /etc/cron.hourly/ — scripts run hourly
  • /etc/cron.daily/ — scripts run daily
  • /etc/cron.weekly/ — scripts run weekly
  • /etc/cron.monthly/ — scripts run monthly

Access control files

  • /etc/cron.allow — only listed users may use cron
  • /etc/cron.deny — listed users are blocked
  • /etc/at.allow — only listed users may use at
  • /etc/at.deny — listed users are blocked from at
  • If cron.allow exists, cron.deny is ignored

Secure file transfer

# scp — secure copy over SSH scp file.txt user@remote:/tmp/ # local to remote scp user@remote:/etc/hosts . # remote to local scp -r localdir/ user@remote:/backup/ # recursive directory copy scp -p file.txt user@remote:/tmp/ # preserve timestamps/modes # sftp — interactive secure FTP session sftp user@remote # sftp> get remotefile (download) # sftp> put localfile (upload) # sftp> ls / pwd / cd (navigate remote) # sftp> lls / lpwd / lcd (navigate local) # sftp> exit # rsync — efficient sync with delta transfer rsync -av src/ user@remote:/dst/ # archive + verbose rsync -avz src/ user@remote:/dst/ # with compression rsync --delete -av src/ user@remote:/dst/ # mirror (removes extra files)

Cheat sheet

Most-tested commands — quick reference

Start service now + at boot
systemctl enable --now svc
Stop service + disable
systemctl disable --now svc
Check service status
systemctl status svc
Restart a service
systemctl restart svc
Reload service config
systemctl reload svc
List failed units
systemctl --state=failed
Reload unit files
systemctl daemon-reload
Set default target
systemctl set-default multi-user.target
Switch target now
systemctl isolate rescue.target
Graceful shutdown
systemctl poweroff
Reboot
systemctl reboot
List all processes
ps aux
Graceful kill
kill 1234
Force kill
kill -9 1234
Kill by name
pkill httpd
Start at lower priority
nice -n 10 command
Change running priority
renice -n 15 -p PID
Follow journal live
journalctl -f
Journal for a service
journalctl -u sshd -f
Journal since yesterday
journalctl --since yesterday
Persist journal
mkdir -p /var/log/journal
Schedule one-time job
echo "cmd" | at now +1 hour
List at jobs
atq
Edit user crontab
crontab -e
List user crontab
crontab -l
Apply tuning profile
tuned-adm profile balanced
Copy file to remote
scp file.txt user@host:/tmp/

Cron quick-build guide

GoalCrontab entry
Every minute* * * * * /cmd
Every 5 minutes*/5 * * * * /cmd
Daily at 2:30 AM30 2 * * * /cmd
Weekdays at 8 AM0 8 * * 1-5 /cmd
Weekends at noon0 12 * * 6,0 /cmd
1st of month at midnight0 0 1 * * /cmd
Every hour, first 5 min0-4 * * * * /cmd
Twice a day (8 AM, 8 PM)0 8,20 * * * /cmd

Root password reset — step summary

#Action
1Reboot → GRUB menu → press e
2Find linux line, append rd.break
3Press Ctrl+X to boot
4mount -o remount,rw /sysroot
5chroot /sysroot
6passwd root
7touch /.autorelabel
8exitexit → reboot

Practice quiz

Question 1 of 8

You want to enable the httpd service to start at boot and also start it right now in a single command. Which is correct?

systemctl enable --now httpd enables the service for boot and starts it immediately in one atomic operation. Option A works but is two commands. Options C and D use invalid flags — --enable and --start are not valid systemctl options.

Question 2 of 8

After interrupting the boot with rd.break and chrooting into /sysroot, you reset the root password. What must you do before rebooting to ensure SELinux does not block logins?

touch /.autorelabel triggers a full filesystem SELinux relabeling on the next boot. When you change /etc/shadow (or any file) in chroot, its SELinux context may be wrong. Without relabeling, SELinux will deny access. The other options either only apply to a running system (A) or are permanent policy changes (D) — neither solves the context mismatch.

Question 3 of 8

Which command shows only journal entries from the current boot for the sshd service, following live output?

journalctl -u sshd -b -f combines three flags: -u sshd filters by unit, -b limits to the current boot, and -f follows live output. Option B would work for rsyslog but sshd doesn't have a dedicated log file by default. Options C and D use non-existent flags.

Question 4 of 8

A cron entry reads: 30 4 * * 1 /usr/bin/cleanup.sh. When does this run?

Fields: minute=30, hour=4 (4 AM), day-of-month=* (any), month=* (any), day-of-week=1 (Monday). So the script runs at 4:30 AM every Monday. The hour field is 24-hour, so 4 = 4 AM not 4 PM.

Question 5 of 8

You need to send signal 15 to a process with PID 4422. Which two commands are equivalent?

kill 4422 sends SIGTERM (15) by default, and kill -SIGTERM 4422 sends the same signal explicitly. They are equivalent. Option A is wrong because kill -9 sends SIGKILL (9). Option C mixes SIGTERM (15) and SIGKILL (9) — different signals.

Question 6 of 8

You want the systemd journal to survive reboots. What is the simplest method?

When /var/log/journal/ exists, journald automatically switches from volatile (/run/log/journal/) to persistent storage. This is the simplest approach — no config file edits required. Option A sets the default behavior (auto = use /var/log/journal/ if it exists). Option C is not a valid journalctl flag.

Question 7 of 8

Which command permanently changes the default boot target to multi-user (text) mode?

systemctl set-default multi-user.target creates a symlink at /etc/systemd/system/default.target pointing to multi-user.target, making it the permanent default on next boot. Option A only switches for the current session — it does not survive a reboot. Option C is not a valid systemctl subcommand.

Question 8 of 8

A backup process is consuming too much CPU. You want to reduce its scheduling priority so other processes get more CPU time. Its PID is 7812. Which command lowers its priority?

renice -n 10 -p 7812 raises the nice value to +10, which lowers the scheduling priority (gives it less CPU time). Higher nice = nicer to others = lower priority. Option A uses -10 which would increase priority (only root can do that). Option C sends SIGSTOP which pauses the process entirely. Option D — nice is for starting new processes, not adjusting running ones (and the syntax is wrong).