RED HAT ENTERPRISE LINUX

Image-Based Management

Building, Deploying, and Upgrading with bootc

College-Level Course Module | RHEL System Administration

Welcome to this forward-looking module on image-based installation management in Red Hat Enterprise Linux. This represents the modern approach to system deployment and management. Traditional package-based systems install and update individual RPM packages. Image-based systems deploy complete, immutable system images that can be atomically upgraded and rolled back. RHEL 9 introduces bootable container images (bootc) - you build your operating system as a container image, then deploy it to physical or virtual machines. This brings container workflow benefits to full operating systems: reproducible builds, atomic upgrades, easy rollbacks, and infrastructure-as-code practices. You will learn to create bootable container images using Containerfiles, deploy them to servers, perform atomic upgrades, and roll back when needed. These skills represent the future of Linux system management.

Learning Objectives

Understand image-based deployment concepts

Bootable containers, OSTree, atomic updates, and immutable infrastructure

Create bootable container images

Build custom OS images using Containerfiles and podman

Deploy image-based systems

Install servers from bootable container images

Manage upgrades and rollbacks

Perform atomic system updates with easy rollback capability

Traditional vs Image-Based

Traditional (Package-Based)

Install base OS
Add packages with dnf
Configure individually
Update packages incrementally
Each system potentially unique
Rollback is complex

Image-Based (bootc)

Build complete OS image
Deploy identical copies
Configuration in image
Update entire image atomically
All systems identical
Instant rollback available

Build Image

→

Test Image

→

Push to Registry

→

Deploy to Servers

Key insight: Image-based treats the OS like a container - build once, deploy many, update atomically, roll back instantly.

Let me contrast traditional and image-based approaches. Traditional package-based management installs a base OS, then adds and configures packages individually. Each system can drift over time as different updates are applied or configurations change. Rollback means keeping track of every change and reversing them - often impractical. Image-based management builds a complete OS image containing everything - base system, packages, configuration. You test this image, then deploy identical copies to all servers. Every server running the same image is identical by definition. Updates mean building a new image and deploying it atomically - the entire system switches to the new version at once. If problems occur, you roll back to the previous image instantly. This is the same workflow containers use for applications, now applied to entire operating systems. Benefits include reproducibility, consistency, easier testing, and reliable rollbacks.

What is bootc?

bootc (bootable containers) is technology that enables deploying and updating Linux systems using OCI container images. Build your OS as a container, deploy it as a server.

# bootc manages bootable container images on a running system
[root@server ~]# bootc status
Current booted image: registry.redhat.io/rhel9/rhel-bootc:9.3
    Image version: 9.3.0 (2024-01-15)
    Image digest: sha256:a1b2c3d4...
    
Staged for next boot: none
Rollback available: registry.redhat.io/rhel9/rhel-bootc:9.2
    Image version: 9.2.0 (2023-10-01)

# Key components:
# - OCI container images (standard format)
# - OSTree for atomic filesystem updates
# - bootc CLI for system management
# - Standard container tools (podman, buildah)

Benefits: Reproducible deployments, atomic updates, instant rollbacks, familiar container workflow, infrastructure as code.

bootc - bootable containers - is the technology that makes image-based Linux systems work. It combines several technologies: OCI container images provide a standard, portable image format. The same images work with podman, docker, and container registries. OSTree provides atomic filesystem updates. Instead of updating files in place, OSTree deploys complete filesystem trees and switches between them atomically. The bootc CLI manages these images on running systems - checking status, applying updates, and managing rollbacks. You use standard container tools - podman, buildah, Containerfiles - to build bootc images. Then bootc deploys them as running systems. The bootc status command shows what image is currently running, what is staged for the next boot, and what rollback options are available. This gives you visibility into exactly what version each system is running.

OSTree Foundation

OSTree is a system for managing bootable filesystem trees. It enables atomic updates by deploying complete filesystem snapshots rather than updating files in place.

Current Boot: rhel-bootc:9.3

Staged: rhel-bootc:9.4 (pending reboot)

Rollback: rhel-bootc:9.2

OSTree Repository (stores all versions)

# OSTree stores filesystem trees efficiently
# - Content-addressed storage (deduplication)
# - Atomic switching between versions
# - Boot loader integration for rollback

# View OSTree deployments
[root@server ~]# rpm-ostree status
State: idle
Deployments:
● ostree-image-signed:docker://registry.redhat.io/rhel9/rhel-bootc:9.3
                   Digest: sha256:a1b2c3d4...
  ostree-image-signed:docker://registry.redhat.io/rhel9/rhel-bootc:9.2
                   Digest: sha256:e5f6g7h8...

OSTree is the underlying technology that makes atomic updates possible. Traditional package managers update files in place - if an update fails halfway, you have a partially updated, potentially broken system. OSTree works differently. It stores complete filesystem trees in a content-addressed repository. Each version is a complete snapshot. When you update, OSTree downloads the new tree (only transferring changed files due to deduplication), then atomically switches the boot configuration to point to the new tree. The switch is atomic - it either completely succeeds or completely fails. There is no partial update state. The boot loader knows about multiple deployed trees, so you can boot into the previous version if the new one has problems. This is rollback - just boot the old tree. rpm-ostree status shows all deployed versions. The bullet point indicates the currently booted deployment. You typically have two or three deployments available: current, staged (if an update is pending), and previous (for rollback).

RHEL Bootc Images

Red Hat provides base bootc images that serve as foundations for custom images. These include the kernel, systemd, and core OS components needed to boot.

Image	Description	Use Case
rhel-bootc	Full RHEL bootable base image	General purpose servers
rhel-bootc-minimal	Minimal bootable image	Appliances, single-purpose systems

# Pull RHEL bootc base image
[user@workstation ~]$ podman pull registry.redhat.io/rhel9/rhel-bootc:9.3

# Inspect base image
[user@workstation ~]$ podman inspect registry.redhat.io/rhel9/rhel-bootc:9.3

# Base images include:
# - Linux kernel and firmware
# - systemd and core services
# - SELinux policies
# - Basic system utilities
# - bootc and rpm-ostree tools

Starting point: Always start FROM a bootc base image. Regular container images (like UBI) lack the kernel and boot components needed for a bootable system.

Red Hat provides base bootc images to build upon. These are different from regular container images like UBI. Bootc images include everything needed to boot as a standalone system: the Linux kernel, boot loader components, firmware, systemd, SELinux policies, and the bootc/rpm-ostree tools for managing updates. rhel-bootc is the full base image with a comprehensive set of packages. Use this for general-purpose servers where you will add your applications on top. rhel-bootc-minimal is a smaller base for appliances or single-purpose systems where you want tight control over what is included. You cannot use regular container images like UBI as a bootc base - they lack the kernel and boot infrastructure. Always start your bootable images FROM a bootc base image. Pull these images from registry.redhat.io, which requires Red Hat credentials. These images are updated regularly with security fixes and should be used as the foundation for your custom images.

Creating Custom Images

Build custom bootc images using standard Containerfiles. Add your packages, configurations, and customizations on top of the base bootc image.

# Containerfile for custom web server bootc image
FROM registry.redhat.io/rhel9/rhel-bootc:9.3

# Install packages
RUN dnf install -y httpd mod_ssl php php-mysqlnd && \
    dnf clean all && \
    systemctl enable httpd

# Add configuration files
COPY httpd.conf /etc/httpd/conf/httpd.conf
COPY ssl.conf /etc/httpd/conf.d/ssl.conf

# Add application files
COPY webapp/ /var/www/html/

# Set SELinux file contexts (important for bootc)
RUN setfiles -F /etc/selinux/targeted/contexts/files/file_contexts /var/www/html

# Labels for identification
LABEL description="Custom RHEL web server"
LABEL version="1.0"

Creating custom bootc images uses the same Containerfile syntax you know from application containers. The key difference is starting FROM a bootc base image rather than a regular container image. The structure is familiar: FROM specifies the base image. RUN executes commands - here we install packages and enable services. COPY adds configuration files and application code. Note the systemctl enable in the RUN command - this sets services to start at boot. Unlike application containers that run a single process, bootc systems run full systemd with multiple services. SELinux context is important. Files added during image build need correct SELinux labels. The setfiles command applies correct contexts based on the system policy. Without this, SELinux may block access to your files at runtime. LABEL adds metadata for identification. Include version information to track what is deployed. This Containerfile creates a complete, bootable web server image ready for deployment.

Building Images

# Build the bootc image
[user@workstation ~]$ podman build -t mywebserver:1.0 -f Containerfile .
STEP 1/8: FROM registry.redhat.io/rhel9/rhel-bootc:9.3
STEP 2/8: RUN dnf install -y httpd mod_ssl php...
...
STEP 8/8: LABEL version="1.0"
Successfully tagged localhost/mywebserver:1.0

# Verify the image
[user@workstation ~]$ podman images
REPOSITORY                               TAG    IMAGE ID      SIZE
localhost/mywebserver                    1.0    a1b2c3d4e5f6  2.1 GB
registry.redhat.io/rhel9/rhel-bootc      9.3    f6e5d4c3b2a1  1.8 GB

# Test the image (limited - cannot fully boot in container)
[user@workstation ~]$ podman run --rm -it mywebserver:1.0 rpm -q httpd
httpd-2.4.57-5.el9.x86_64

# Tag for registry
[user@workstation ~]$ podman tag mywebserver:1.0 registry.example.com/mywebserver:1.0

# Push to registry
[user@workstation ~]$ podman push registry.example.com/mywebserver:1.0

Building bootc images uses standard podman build. The process is identical to building application containers - podman reads the Containerfile, executes each instruction, and creates a layered image. The resulting image is larger than typical application containers because it includes the full operating system - kernel, libraries, utilities, plus your customizations. Several gigabytes is normal. You can test the image by running it as a container, but testing is limited. A bootc image in a container does not fully boot - there is no kernel boot, no systemd init. You can verify packages are installed and files are in place, but full testing requires deploying to actual hardware or a VM. After building, tag the image for your registry and push it. Use semantic versioning tags (1.0, 1.1, 2.0) rather than just "latest" for production deployments. This way you know exactly what version each system is running.

Registry Workflow

Build Image

→

Test

→

Push to Registry

→

Deploy/Update

# Login to registry
[user@workstation ~]$ podman login registry.example.com
Username: admin
Password: 
Login Succeeded!

# Push image to registry
[user@workstation ~]$ podman push registry.example.com/rhel-webserver:1.0

# Systems pull updates from registry
[root@server ~]# bootc upgrade
Pulling manifest for registry.example.com/rhel-webserver:latest
Importing ostree commit...
Staging deployment...
Queued for next boot: registry.example.com/rhel-webserver:1.1

# Registries can be:
# - Red Hat Quay (quay.io or self-hosted)
# - Docker Hub (docker.io)
# - Private registries (Harbor, Artifactory)
# - registry.redhat.io (Red Hat images)

The registry is central to image-based workflow. It stores your built images and serves them to systems that need updates. The workflow is: build image on your workstation or CI system, test it, push to registry, then systems pull and deploy. Standard container registries work with bootc images - Quay, Docker Hub, Harbor, Artifactory, or any OCI-compatible registry. For enterprise use, consider running a private registry for control over image distribution and to avoid external dependencies. Systems pull images from the registry using bootc upgrade. This downloads the new image, imports it into the OSTree repository, and stages it for the next boot. The registry URL is stored in the system configuration, so running bootc upgrade checks for newer versions of the current image. You can also switch to entirely different images if needed - perhaps switching a server from one role to another.

Installing from Images

Deploy bootc images to new systems using bootc-image-builder to create installable media, or use bootc install on existing systems.

# Method 1: Create installation ISO with bootc-image-builder
[user@workstation ~]$ sudo podman run --rm -it --privileged \
    --pull=newer \
    -v ./output:/output \
    registry.redhat.io/rhel9/bootc-image-builder:latest \
    --type iso \
    registry.example.com/mywebserver:1.0

Building ISO image...
Output: /output/bootiso/install.iso

# Boot the ISO on target hardware/VM and install

# Method 2: Convert existing system to bootc
[root@existing ~]# podman run --rm --privileged \
    -v /:/target \
    -v /var/lib/containers:/var/lib/containers \
    registry.example.com/mywebserver:1.0 \
    bootc install to-existing-root /target

# Reboot into new image-based system
[root@existing ~]# systemctl reboot

Deploying bootc images to systems can be done several ways. bootc-image-builder creates installable media from your container image. It can produce ISOs for traditional installation, disk images for cloud deployment, or other formats. Run it as a privileged container, pointing to your bootc image, and it outputs an installable ISO. Boot target hardware from this ISO and it installs your complete image-based system. For converting existing traditional RHEL systems to image-based, use bootc install to-existing-root. This transforms the system in place - run it from a container with access to the root filesystem, and it converts to an OSTree-based deployment of your image. After reboot, the system runs your bootc image. This is useful for migrating existing infrastructure to image-based management without reinstalling from scratch. Both methods result in a system managed by bootc, ready for atomic updates and rollbacks.

bootc Status

# Check current system status
[root@server ~]# bootc status
Current booted image: registry.example.com/rhel-webserver:1.0
    Image version: 1.0.0
    Image digest: sha256:a1b2c3d4e5f6...
    Timestamp: 2024-01-15T10:30:00Z
    
Staged for next boot: none

Rollback available: registry.example.com/rhel-webserver:0.9
    Image version: 0.9.0
    Image digest: sha256:e5f6g7h8i9j0...
    Timestamp: 2024-01-01T08:00:00Z

# Show detailed image information
[root@server ~]# bootc status --format=json | jq .

# Verify system is running expected image
[root@server ~]# bootc status --booted
registry.example.com/rhel-webserver:1.0

# Check if updates are available
[root@server ~]# bootc upgrade --check
Update available: registry.example.com/rhel-webserver:1.1

The bootc status command is your primary tool for understanding system state. It shows what image is currently booted, including the full image reference, version, digest (content hash), and build timestamp. If an update is staged for the next boot, it shows that too. Crucially, it shows available rollback options - the previous image you can revert to if needed. The --format=json option outputs JSON for scripting and automation. Pipe to jq for processing. The --booted flag shows just the currently booted image reference - useful in scripts that need to verify systems are running expected versions. bootc upgrade --check queries the registry for updates without applying them. This is useful for monitoring - you can check if updates are available before deciding to apply them.

Upgrading Systems

Atomic updates: bootc downloads the new image, stages it, and activates on reboot. The running system is unchanged until reboot - no partial updates.

# Check for and apply updates
[root@server ~]# bootc upgrade
Pulling manifest for registry.example.com/rhel-webserver:latest
Layers already present: 45/52
Pulling new layers: 7
Importing to ostree...
Staging deployment...
Upgrade staged. Reboot to apply.

# Reboot to activate new image
[root@server ~]# systemctl reboot

# After reboot, verify new version
[root@server ~]# bootc status
Current booted image: registry.example.com/rhel-webserver:1.1
Rollback available: registry.example.com/rhel-webserver:1.0

# Switch to specific image (not just latest)
[root@server ~]# bootc switch registry.example.com/rhel-webserver:2.0

Safe updates: The running system continues operating normally while the update is staged. Only reboot activates the change.

Upgrading bootc systems is straightforward and safe. bootc upgrade pulls the latest version of the current image from the registry. It downloads only changed layers - if 45 of 52 layers are unchanged, only 7 are downloaded. This is efficient for incremental updates. The new image is imported into OSTree and staged as a new deployment. Critically, the running system is unaffected. Services continue running on the current version. There is no risk of a partial update breaking the system. Only when you reboot does the system switch to the new image. After reboot, bootc status shows the new version as current and the old version as the rollback option. If you need a specific version rather than latest, use bootc switch with the full image reference. This is useful for controlled rollouts or when you need to pin systems to specific versions.

Rolling Back

Instant rollback: If an upgrade causes problems, roll back to the previous working image immediately. No complex recovery procedures needed.

# Scenario: Upgrade caused problems
[root@server ~]# bootc status
Current booted image: registry.example.com/rhel-webserver:1.1
Rollback available: registry.example.com/rhel-webserver:1.0

# Rollback to previous image
[root@server ~]# bootc rollback
Rollback staged. Reboot to apply.

# Reboot to activate rollback
[root@server ~]# systemctl reboot

# Alternative: Select previous deployment at boot menu
# GRUB shows multiple deployments - select older one

# After rollback
[root@server ~]# bootc status
Current booted image: registry.example.com/rhel-webserver:1.0
Rollback available: registry.example.com/rhel-webserver:1.1

Recovery time: Rollback is just a reboot - no reinstallation, no package downgrading, no complex recovery. System returns to known-good state instantly.

Rollback is one of the most powerful features of image-based systems. In traditional systems, rolling back an update means identifying every changed package, potentially resolving dependency issues, and hoping the data is compatible. With bootc, rollback is trivial. bootc rollback stages the previous deployment to become active on next boot. Reboot, and you are back to the previous image. Total downtime is just one reboot. You can also select previous deployments from the GRUB boot menu - useful if the new image fails to boot at all. Each deployment is independent, so rolling back does not affect the newer image. After rollback, the newer (problematic) version is still available as the rollback option. You could roll forward again if needed. This safety net fundamentally changes how you approach updates. You can update confidently knowing that recovery from a bad update is fast and reliable.

Managing Local Changes

Image-based systems are intended to be immutable, but some local state must persist: configuration, data, logs. These live outside the image in designated locations.

# Persistent directories (survive upgrades):
/etc       - Configuration (merged with image /etc)
/var       - Variable data, logs, databases
/home      - User home directories

# Immutable directories (from image, read-only concept):
/usr       - System binaries and libraries
/lib       - Libraries (symlink to /usr/lib)
/bin       - Binaries (symlink to /usr/bin)

# Configuration changes in /etc persist
[root@server ~]# vi /etc/myapp/config.yaml
# This change survives upgrades

# To add packages locally (temporary, testing only):
[root@server ~]# rpm-ostree install vim-enhanced
Staging deployment... Reboot to apply.

# Better approach: Add packages to the image
# Edit Containerfile, rebuild, upgrade

Image-based systems are designed to be immutable - the operating system comes from the image and should not be modified locally. But systems need some local state: configuration specific to this machine, application data, logs, user files. The solution is a split between immutable and persistent areas. /usr and related directories come from the image and are conceptually read-only. Do not modify these directly. /etc is for configuration. Changes here persist across upgrades. When you upgrade, the new image's /etc is merged with your local changes. /var holds variable data: logs, databases, application state. This persists and is not touched by upgrades. /home holds user data and persists. For configuration, edit files in /etc normally. For additional packages, the best practice is adding them to the image itself - edit the Containerfile, rebuild, and upgrade. This keeps systems consistent. For temporary testing, rpm-ostree install adds packages locally, but this creates drift from the image and complicates updates.

Containerfile Best Practices

FROM registry.redhat.io/rhel9/rhel-bootc:9.3

# Combine RUN commands to reduce layers
RUN dnf install -y \
        httpd \
        mod_ssl \
        php \
        php-mysqlnd \
    && dnf clean all \
    && systemctl enable httpd

# Copy configs before application files (better caching)
COPY etc/ /etc/

# Set correct SELinux contexts
RUN setfiles -F /etc/selinux/targeted/contexts/files/file_contexts /etc

# Application files last (most likely to change)
COPY webapp/ /var/www/html/
RUN setfiles -F /etc/selinux/targeted/contexts/files/file_contexts /var/www/html

# Always include metadata
LABEL org.opencontainers.image.version="1.0"
LABEL org.opencontainers.image.description="Production web server"
LABEL org.opencontainers.image.created="2024-01-15"

Caching tip: Put things that change frequently (app code) last in the Containerfile. Earlier layers are cached, making rebuilds faster.

Following best practices makes your bootc images efficient and maintainable. Combine related RUN commands with && to reduce layers. Each layer adds overhead, so combining package installation, cleanup, and service enablement into one RUN is efficient. Always run dnf clean all to remove cached package data and reduce image size. Order instructions for optimal caching. Docker/Podman caches layers and rebuilds from the first changed layer. Put stable things (package installation) early and frequently changing things (application code) late. When you update app code, only the final layers rebuild. Set SELinux contexts for all custom files. Use setfiles to apply correct labels based on system policy. Missing contexts cause runtime access denials. Include OCI standard labels for version, description, and creation date. This metadata helps track what is deployed and when it was built. Consider using build arguments (ARG) for version numbers so you can inject them from CI pipelines.

Multi-Stage Builds

# Stage 1: Build application
FROM registry.redhat.io/ubi9/go-toolset:latest AS builder
WORKDIR /app
COPY src/ .
RUN go build -o /app/myservice

# Stage 2: Create bootable image
FROM registry.redhat.io/rhel9/rhel-bootc:9.3

# Install runtime dependencies only (not build tools)
RUN dnf install -y minimal-runtime-deps && dnf clean all

# Copy built application from builder stage
COPY --from=builder /app/myservice /usr/local/bin/

# Add systemd service file
COPY myservice.service /etc/systemd/system/
RUN systemctl enable myservice

LABEL version="1.0"

Smaller images: Build tools are only in the builder stage, not the final image. The bootable image contains only what is needed to run.

Multi-stage builds create smaller, cleaner images by separating build and runtime environments. The first stage uses a toolchain image - here, go-toolset for compiling Go applications. It includes compilers, build tools, and development libraries. We build the application in this stage. The second stage starts fresh from the bootc base image. It only installs runtime dependencies, not build tools. The compiled application is copied from the builder stage using COPY --from=builder. The final image contains only what is needed to run - no compiler, no source code, no build artifacts. This dramatically reduces image size and attack surface. This pattern works for any compiled language: Go, Rust, C/C++. For interpreted languages like Python, you might use a build stage to install dependencies and copy only the installed packages to the final image. The key insight is that build-time dependencies do not belong in production images.

CI/CD Integration

Integrate bootc image building into CI/CD pipelines for automated, tested deployments. Build on commit, test automatically, deploy to staging, then production.

# Example CI pipeline (GitLab CI / GitHub Actions concept)
stages:
  - build
  - test
  - push
  - deploy-staging
  - deploy-production

build-image:
  script:
    - podman build -t myserver:${CI_COMMIT_SHA} .
    
test-image:
  script:
    - podman run myserver:${CI_COMMIT_SHA} /tests/run-tests.sh
    
push-image:
  script:
    - podman push registry.example.com/myserver:${CI_COMMIT_SHA}
    - podman push registry.example.com/myserver:latest
    
deploy-staging:
  script:
    - ssh staging "bootc upgrade && systemctl reboot"
    
deploy-production:
  script:
    - ssh prod-servers "bootc upgrade && systemctl reboot"
  when: manual

CI/CD pipelines automate the build, test, and deploy cycle for bootc images. A typical pipeline builds the image on every commit, using the commit SHA as a version tag for traceability. Tests run against the new image - you can run test suites, security scans, or compliance checks. After tests pass, push to the registry with both the commit SHA tag and latest. Deploy to staging first for integration testing. Staging servers run bootc upgrade to pull the new image, then reboot. Manual approval gates production deployment. Production servers similarly run bootc upgrade, but the trigger is manual approval after staging validation. This pipeline ensures every deployed image is built from version-controlled source, tested automatically, and traceable to a specific commit. Rollback is trivial - either push a previous version as latest and upgrade, or use bootc rollback on individual servers.

Troubleshooting

# Check bootc service status
[root@server ~]# systemctl status bootc-fetch-apply-updates.timer

# View bootc logs
[root@server ~]# journalctl -u bootc-fetch-apply-updates

# Check OSTree deployments
[root@server ~]# rpm-ostree status
[root@server ~]# ostree admin status

# Verify image signature (if using signed images)
[root@server ~]# bootc status --format=json | jq '.booted.image.signature'

# Debug upgrade issues
[root@server ~]# bootc upgrade --verbose 2>&1 | tee upgrade.log

# Common issues:
# - Registry authentication: podman login registry.example.com
# - Network connectivity: curl registry.example.com/v2/
# - SELinux denials: ausearch -m avc -ts recent
# - Disk space: df -h /var/lib/containers

# Emergency: Boot previous deployment from GRUB menu

When bootc operations fail, systematic troubleshooting identifies the cause. bootc-fetch-apply-updates is a systemd service that can automate updates. Check its status and logs for scheduled update issues. rpm-ostree status and ostree admin status show the OSTree layer that bootc uses. Useful for deep debugging of deployment issues. For upgrade failures, run bootc upgrade --verbose for detailed output. Common issues include registry authentication - ensure you are logged in with podman login. Network connectivity - verify the registry is reachable. SELinux denials - check audit logs with ausearch. Disk space - OSTree needs space for multiple deployments. If a system will not boot after upgrade, select the previous deployment from the GRUB menu. This is your emergency escape hatch. The previous deployment is always available as a boot option.

Security Considerations

Security Benefits

Immutable base reduces attack surface
Atomic updates prevent partial/broken states
Easy rollback from compromised images
Reproducible builds enable auditing
Image signing verifies authenticity
Consistent state across all systems

Security Practices

Use signed images in production
Scan images for vulnerabilities
Keep base images updated
Minimize installed packages
Apply SELinux contexts correctly
Secure registry access

# Sign images with cosign
[user@workstation ~]$ cosign sign registry.example.com/myserver:1.0

# Verify signature before deployment
[root@server ~]# cosign verify registry.example.com/myserver:1.0

# Scan for vulnerabilities
[user@workstation ~]$ podman image scan myserver:1.0

Image-based systems offer significant security benefits. Immutability means the base system is not modified at runtime - what you tested is what runs. Atomic updates eliminate partial update states where some packages are updated and others are not. If something is compromised, rollback to a known-good image is instant. Reproducible builds mean you can audit exactly what is in an image. Image signing cryptographically verifies images have not been tampered with. All systems running the same image are identical - no configuration drift. Best practices include signing all production images using tools like cosign. Scanning images for known vulnerabilities before deployment. Keeping base images updated - when Red Hat releases security fixes, rebuild your images on the new base. Minimizing installed packages to reduce attack surface. Applying correct SELinux contexts. Securing registry access with authentication and TLS.

Best Practices

Do

Start FROM official bootc base images
Use specific version tags, not :latest
Set SELinux contexts for added files
Enable services with systemctl enable
Clean dnf cache to reduce size
Test images before production deployment
Use CI/CD for automated building
Keep rollback images available

Do Not

Use non-bootc base images (UBI, etc.)
Modify /usr on running systems
Skip SELinux context configuration
Install packages locally with rpm-ostree
Deploy untested images to production
Ignore image signing in production
Let systems drift from the image
Delete all rollback deployments

Philosophy: Treat the OS like a container - build it once, test it, deploy identical copies everywhere, update atomically, roll back instantly.

Let me summarize best practices for image-based management. Always start from official bootc base images - they have the kernel and boot infrastructure needed. Use specific version tags for reproducibility. Set SELinux contexts on all custom files or they will be inaccessible. Enable services in the Containerfile so they start at boot. Clean up dnf cache to reduce image size. Test images before production - run them as containers for basic validation, deploy to staging for full testing. Use CI/CD pipelines for consistent, automated builds. Keep rollback options available - do not delete all previous deployments. Avoid using non-bootc bases that lack boot components. Do not modify /usr at runtime - it defeats the purpose of immutable systems. Do not install packages locally except for temporary testing. Do not deploy untested images. Do not skip signing in production. Do not let systems drift from the image. The fundamental philosophy: your OS is a container. Build, test, deploy, update atomically, roll back when needed.

Key Takeaways

Image-Based: bootc deploys complete OS images atomically. Build with Containerfiles, deploy with bootc, update and rollback instantly.

Building: Start FROM bootc base images. Add packages, configs, enable services. Set SELinux contexts. Push to registry.

Deploying: Use bootc-image-builder for install media, or bootc install to-existing-root for conversions.

Managing: bootc status/upgrade/rollback. Atomic updates with one reboot. Instant rollback to previous image.

LAB EXERCISES

Build a custom bootc image with web server and application
Push image to registry and deploy to VM
Update the image and perform atomic upgrade
Practice rollback after simulated problematic upgrade
Examine OSTree deployments with rpm-ostree status
Create CI pipeline for automated image building

Image-based management represents the future of Linux system administration

Let me summarize the essential image-based management skills. Image-based systems use bootc to deploy complete OS images atomically. Instead of managing individual packages, you build and deploy complete system images. This ensures consistency, enables instant rollback, and brings container workflow benefits to full operating systems. Building images uses familiar Containerfile syntax. Start FROM bootc base images which include kernel and boot components. Add your packages, configurations, and applications. Set SELinux contexts. Push to a registry for distribution. Deploying uses bootc-image-builder to create installable media for new systems, or bootc install to-existing-root to convert traditional systems in place. Both result in bootc-managed systems. Managing systems uses bootc status to see current state, bootc upgrade to pull and stage new images, and bootc rollback to revert to previous images. All updates are atomic - one reboot switches versions. Rollback is instant. Your lab exercises provide hands-on experience with this complete workflow. These skills represent the future direction of Linux system administration, combining the best of container practices with full operating system management.