College-Level Course Module | RHEL System Administration
Learning Objectives
1
Understand systemd architecture
Units, services, daemons, and the systemd ecosystem
2
Control services with systemctl
Start, stop, restart, and reload services
3
Monitor service status
Check status, view logs, and troubleshoot failures
4
Configure boot behavior
Enable and disable services for automatic startup
5
Work with systemd targets
Understand boot targets and system states
What is systemd?
systemd is the system and service manager for Linux. It is the first process started at boot (PID 1) and is responsible for starting all other services and managing them throughout the system's operation.
systemd Responsibilities:
Boot the system (init process)
Start/stop services on demand
Track service dependencies
Manage system logging (journald)
Mount filesystems
Manage network, time, login
Key Components:
systemd - Main daemon (PID 1)
systemctl - Control command
journald - Logging system
journalctl - Log viewer
Unit files - Service definitions
History: systemd replaced SysVinit in RHEL 7. It provides parallel startup, on-demand activation, and unified configuration.
Services and Daemons
A daemon is a background process that runs continuously. A service in systemd context is a unit that manages a daemon or other background task.
# Common system services (daemons)sshd.service - OpenSSH server daemon
httpd.service - Apache HTTP Server
mariadb.service - MariaDB database server
crond.service - Command scheduler
firewalld.service - Dynamic firewall daemon
NetworkManager.service - Network management
chronyd.service - NTP time synchronization# List running services[root@host ~]# systemctl list-units --type=service --state=running
UNIT LOAD ACTIVE SUB DESCRIPTION
chronyd.service loaded active running NTP client/server
crond.service loaded active running Command Scheduler
firewalld.service loaded active running firewalld
NetworkManager.service loaded active running Network Manager
sshd.service loaded active running OpenSSH server daemon
Understanding Units
Units are resources that systemd manages. Services are the most common, but systemd also manages mounts, timers, sockets, and more.
Unit Type
Extension
Purpose
Service
.service
Manage daemons and processes
Target
.target
Group units, define system states
Socket
.socket
IPC/network socket for on-demand activation
Mount
.mount
Filesystem mount points
Timer
.timer
Scheduled activation (like cron)
Path
.path
File/directory watching
Device
.device
Hardware devices
# List all unit types[root@host ~]# systemctl list-units --type=help
# List all loaded units[root@host ~]# systemctl list-units
# List units of specific type[root@host ~]# systemctl list-units --type=timer
systemctl Basics
systemctl is the primary command for interacting with systemd. It controls services, checks status, and configures boot behavior.
# Basic syntaxsystemctl [command] [unit]# Common service control commands[root@host ~]# systemctl start httpd.service # Start now[root@host ~]# systemctl stop httpd.service # Stop now[root@host ~]# systemctl restart httpd.service # Stop then start[root@host ~]# systemctl reload httpd.service # Reload config# The .service extension is optional for services[root@host ~]# systemctl start httpd
[root@host ~]# systemctl stop httpd
# Check if service is active[user@host ~]$ systemctl is-active httpd
active# Check if service is enabled (starts at boot)[user@host ~]$ systemctl is-enabled httpd
enabled
Starting and Stopping
# Start a service[root@host ~]# systemctl start httpd
# No output means success# Stop a service[root@host ~]# systemctl stop httpd
# Restart (stop + start)[root@host ~]# systemctl restart httpd
# Brief interruption while service restarts# Reload configuration (no restart)[root@host ~]# systemctl reload httpd
# Service stays running, just rereads config# Reload if supported, otherwise restart[root@host ~]# systemctl reload-or-restart httpd
# Conditional restart (only if already running)[root@host ~]# systemctl try-restart httpd
# Does nothing if service is stopped# Kill all processes in service (forceful)[root@host ~]# systemctl kill httpd
reload vs restart: reload keeps the service running (no downtime), but requires service support. restart guarantees config changes take effect.
Checking Status
systemctl status provides detailed information about a service including its state, recent log entries, and process information.
[root@host ~]# systemctl status sshd
● sshd.service - OpenSSH server daemon
Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; preset: enabled)
Active: active (running) since Mon 2024-01-20 09:00:00 EST; 2h ago
Docs: man:sshd(8)
man:sshd_config(5)
Main PID: 1234 (sshd)
Tasks: 1 (limit: 23456)
Memory: 5.2M
CPU: 125ms
CGroup: /system.slice/sshd.service
└─1234 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
Jan 20 09:00:00 host systemd[1]: Starting OpenSSH server daemon...
Jan 20 09:00:00 host sshd[1234]: Server listening on 0.0.0.0 port 22.
Jan 20 09:00:00 host systemd[1]: Started OpenSSH server daemon.
Key fields: Loaded (unit file status), Active (running state), Main PID, Memory usage, and recent journal entries.
Service States
State
Description
active (running)
Service is running normally
active (exited)
One-shot service completed successfully
active (waiting)
Running but waiting for an event
inactive (dead)
Service is not running
failed
Service attempted to start but failed
activating
Service is starting up
deactivating
Service is shutting down
# Quick state checks (useful for scripting)[user@host ~]$ systemctl is-active httpd
active[user@host ~]$ systemctl is-active stopped-service
inactive[user@host ~]$ systemctl is-failed httpd
active# "active" means not failed# Exit codes: 0 = yes (active/enabled/failed), non-zero = no
Enabling and Disabling
Enable configures a service to start automatically at boot. Disable prevents automatic startup. Neither affects current running state.
# Enable service to start at boot[root@host ~]# systemctl enable httpd
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service
→ /usr/lib/systemd/system/httpd.service# Disable service from starting at boot[root@host ~]# systemctl disable httpd
Removed /etc/systemd/system/multi-user.target.wants/httpd.service# Enable does NOT start the service now!# To enable AND start in one command:[root@host ~]# systemctl enable --now httpd
# Disable AND stop in one command:[root@host ~]# systemctl disable --now httpd
# Check enabled status[user@host ~]$ systemctl is-enabled httpd
enabled
Important: enable/disable control boot behavior only. Use start/stop to control current state. Use --now to do both.
Masking Services
Masking completely disables a service - it cannot be started even manually. Use this to prevent a service from running under any circumstances.
# Mask a service (completely disable)[root@host ~]# systemctl mask httpd
Created symlink /etc/systemd/system/httpd.service → /dev/null# Try to start masked service - fails[root@host ~]# systemctl start httpd
Failed to start httpd.service: Unit httpd.service is masked.# Check status shows masked[root@host ~]# systemctl status httpd
○ httpd.service
Loaded: masked (Reason: Unit httpd.service is masked.)
Active: inactive (dead)# Unmask to allow starting again[root@host ~]# systemctl unmask httpd
Removed /etc/systemd/system/httpd.service
Use case: Mask services that conflict with others, or services you want to ensure never run on this system.
Listing Services
# List all loaded units[root@host ~]# systemctl list-units
# List only services[root@host ~]# systemctl list-units --type=service
# List running services[root@host ~]# systemctl list-units --type=service --state=running
# List failed services[root@host ~]# systemctl list-units --type=service --state=failed
UNIT LOAD ACTIVE SUB DESCRIPTION
httpd.service loaded failed failed Apache HTTP Server# List all installed unit files (not just loaded)[root@host ~]# systemctl list-unit-files --type=service
UNIT FILE STATE PRESET
httpd.service disabled disabled
sshd.service enabled enabled
chronyd.service enabled enabled# Show enabled services[root@host ~]# systemctl list-unit-files --type=service --state=enabled
Viewing Service Logs
journalctl views logs from systemd's journal. Filter by unit to see logs for a specific service.
# View logs for a specific service[root@host ~]# journalctl -u sshd
Jan 20 09:00:00 host systemd[1]: Starting OpenSSH server daemon...
Jan 20 09:00:00 host sshd[1234]: Server listening on 0.0.0.0 port 22.
Jan 20 09:00:01 host sshd[1234]: Server listening on :: port 22.
Jan 20 09:00:01 host systemd[1]: Started OpenSSH server daemon.# Follow logs in real-time (like tail -f)[root@host ~]# journalctl -u httpd -f
# Show logs since last boot[root@host ~]# journalctl -u httpd -b
# Show recent logs (last 50 lines)[root@host ~]# journalctl -u httpd -n 50
# Show logs from specific time period[root@host ~]# journalctl -u httpd --since "2024-01-20 09:00" --until "2024-01-20 12:00"
# Show only errors and above[root@host ~]# journalctl -u httpd -p err
Troubleshooting Failures
# Check for failed services[root@host ~]# systemctl --failed
UNIT LOAD ACTIVE SUB DESCRIPTION
httpd.service loaded failed failed Apache HTTP Server# Get detailed status[root@host ~]# systemctl status httpd
× httpd.service - Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled)
Active: failed (Result: exit-code) since Mon 2024-01-20 10:30:00
Process: 5678 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=1/FAILURE)
Main PID: 5678 (code=exited, status=1/FAILURE)
Jan 20 10:30:00 host httpd[5678]: AH00558: httpd: Could not determine server's FQDN
Jan 20 10:30:00 host httpd[5678]: (98)Address already in use: AH00072: make_sock: could not bind# View detailed logs[root@host ~]# journalctl -xeu httpd
# -x adds explanatory messages, -e jumps to end# Reset failed state after fixing[root@host ~]# systemctl reset-failed httpd
Common causes: Configuration errors, port conflicts, missing dependencies, permission issues, resource exhaustion.
Understanding Targets
Targets are groups of units that represent system states. They replace the runlevels from SysVinit and define what services run at each stage of boot.
Target
Old Runlevel
Description
poweroff.target
0
System shutdown
rescue.target
1
Single-user rescue mode
multi-user.target
3
Multi-user, no GUI (servers)
graphical.target
5
Multi-user with GUI (workstations)
reboot.target
6
System reboot
emergency.target
-
Minimal emergency shell
# View current default target[root@host ~]# systemctl get-default
multi-user.target# Set default target[root@host ~]# systemctl set-default graphical.target
Removed /etc/systemd/system/default.target
Created symlink /etc/systemd/system/default.target → /usr/lib/.../graphical.target
Changing System State
# Switch to different target immediately[root@host ~]# systemctl isolate multi-user.target
# Stops graphical services, switches to text mode[root@host ~]# systemctl isolate graphical.target
# Starts graphical services# Rescue mode (single-user, for maintenance)[root@host ~]# systemctl isolate rescue.target
# Emergency mode (minimal, for serious recovery)[root@host ~]# systemctl isolate emergency.target
# Equivalent convenient commands[root@host ~]# systemctl rescue # Same as isolate rescue.target[root@host ~]# systemctl emergency # Same as isolate emergency.target# Reboot and shutdown[root@host ~]# systemctl reboot
[root@host ~]# systemctl poweroff
[root@host ~]# systemctl halt
Note:isolate switches to a target, stopping services not needed by the new target. This can disrupt users!
Service Dependencies
Services can depend on other services. systemd tracks these dependencies and starts services in the correct order.
# Show what a service requires[root@host ~]# systemctl list-dependencies httpd
httpd.service
● ├─system.slice
● ├─httpd-init.service
● └─sysinit.target
● ├─dev-hugepages.mount
● ├─dev-mqueue.mount
● ...# Show reverse dependencies (what requires this service)[root@host ~]# systemctl list-dependencies httpd --reverse
# Show dependencies of a target[root@host ~]# systemctl list-dependencies multi-user.target
# View unit file to see dependency configuration[root@host ~]# systemctl cat httpd
[Unit]
Description=Apache HTTP Server
After=network.target remote-fs.target nss-lookup.target
Wants=httpd-init.service
...
Unit Files
Unit files define how systemd manages a service. They contain configuration for startup, dependencies, and behavior.
# Unit file locations (in priority order)/etc/systemd/system/ - Local administrator configuration (highest priority)
/run/systemd/system/ - Runtime generated units
/usr/lib/systemd/system/ - Package-installed units (vendor)# View a unit file[root@host ~]# systemctl cat sshd.service
[Unit]
Description=OpenSSH server daemon
After=network.target sshd-keygen.target
Wants=sshd-keygen.target
[Service]
Type=notify
ExecStart=/usr/sbin/sshd -D $OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target
Never edit files in /usr/lib/systemd/system/ - they are overwritten on updates. Copy to /etc/systemd/system/ first.
Modifying Unit Files
# Edit unit file (creates drop-in override automatically)[root@host ~]# systemctl edit httpd
# Opens editor for /etc/systemd/system/httpd.service.d/override.conf# Example: Add environment variable[Service]
Environment="OPTIONS=-DFOREGROUND"# Edit full unit file (copy original to /etc/systemd/system/)[root@host ~]# systemctl edit --full httpd
# After editing, reload systemd configuration[root@host ~]# systemctl daemon-reload
# Then restart the service for changes to take effect[root@host ~]# systemctl restart httpd
# View all files affecting a unit[root@host ~]# systemctl cat httpd
# Shows base file plus any drop-in overrides# Revert to vendor unit file (remove local changes)[root@host ~]# systemctl revert httpd
Always run:systemctl daemon-reload after modifying unit files, then restart the service.
Best Practices
Do
Check status before and after changes
Use enable --now to enable and start
Use systemctl edit for customizations
Run daemon-reload after unit changes
Check logs when services fail
Test configuration before restart
Document service customizations
Understand dependencies before stopping
Do Not
Edit files in /usr/lib/systemd/system/
Forget daemon-reload after changes
Ignore failed services
Stop critical services without planning
Assume enable starts the service
Skip status checks in scripts
Mask services without documentation
Restart when reload would work
Troubleshooting workflow: status → logs (journalctl) → dependencies → unit file → fix → daemon-reload → restart → status
Key Takeaways
1
systemd: Init system managing services, mounts, targets. PID 1 on modern Linux. Controls the entire system lifecycle.