## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Refraction Forger

Refraction Forger is a declarative image building tool that creates optimized OS images from simple specification files. It is designed for infrastructure engineers, DevOps teams, and OS distribution maintainers who need reproducible, cacheable image builds across multiple platforms.

## What It Does

Forger reads a `.kdl` specification file that declares what your image should contain — packages, files, users, boot configuration — and produces ready-to-deploy artifacts:

- QCOW2 virtual machine images with ZFS or ext4 filesystems
- OCI container images for container runtimes
- Tar archives for further processing or embedding

Built artifacts can be pushed directly to OCI registries like GHCR.

## Why Forger?

Traditional image building tools fall into two camps:

1. Shell script orchestrators (like the illumos `image-builder`) that require a matching host OS, root privileges, and careful manual sequencing.
2. ISO boot automators (like HashiCorp Packer) that spin up a full installer, type keystrokes into a virtual console, and wait — slowly.

Forger takes a different approach. It assembles images directly by calling package managers and filesystem tools, skipping the installer entirely. Builds that took 20+ minutes with Packer complete in a fraction of the time.

When your host doesn't match the target OS, Forger automatically spins up an ephemeral builder VM, transfers the spec, builds inside it, and retrieves the artifacts — no manual VM management needed.

## Primary Focus

Forger's primary focus is illumos distributions, particularly OmniOS. The illumos ecosystem uses IPS (Image Packaging System) and ZFS, both of which Forger understands natively.

Linux support (starting with Ubuntu) is included as a secondary target. The long-term goal is to bring IPS to Linux via a Rust implementation, making Forger's packaging model available across operating systems. In the meantime, popular Linux distributions are supported to provide a broad userbase familiar with tools like Packer.

## How This Book Is Organized

- Getting Started walks you through installation and building your first image.
- The KDL Spec Language is a complete guide to writing image specifications.
- Composability explains how to chain builds, share configuration, and create variants.
- Distro Guide covers distribution-specific details for illumos and Linux.
- Output Formats describes each artifact type and how to deploy them.
- Architecture explains the internal design for contributors and advanced users.
- Migration helps you move from `omnios-image-builder` or Packer.
- Reference provides CLI help, spec schema, and complete examples.

# Installation

## Prerequisites

Forger is written in Rust and builds as a single static binary. You need:

- Rust toolchain (1.85 or later) via rustup
- Git for fetching the source

For local builds (building images directly on your machine), you also need:

- The target OS's package manager available (e.g., `pkg` on OmniOS, `debootstrap` + `apt` on Ubuntu)
- Root/sudo access for filesystem operations
- `qemu-img` for QCOW2 conversion
- ZFS utilities if building ZFS-based images

For remote builds (the default when your host doesn't match the target), you only need:

- QEMU installed (for the builder VM)
- No root access required — Forger uses user-mode networking

## Building From Source

    git clone https://github.com/cloudnebulaproject/refraction-forger.git
    cd refraction-forger
    cargo build --release

The binary is at `target/release/forger`.

### Cross-Compilation for illumos

If you're building on Linux for deployment on illumos:

    # Install the cross-compilation tool
    cargo install cross

    # Build for illumos
    cross build --release --target x86_64-unknown-illumos

The project includes a `Cross.toml` with the illumos target preconfigured.

## Verify Installation

    forger --help

You should see the five subcommands: `build`, `validate`, `inspect`, `push`, and `targets`.

# Your First Image

Let's build a minimal OmniOS VM image. Create a file called `my-image.kdl`:

    metadata name="my-first-image" version="1.0.0" description="A minimal OmniOS image"

    repositories {
        publisher name="omnios" origin="https://pkg.omnios.org/bloody/core/"
        publisher name="extra.omnios" origin="https://pkg.omnios.org/bloody/extra/"
    }

    incorporation "entire"

    certificates {
        ca publisher="omnios" certfile="omniosce-ca.cert.pem"
    }

    variants {
        set name="opensolaris.zone" value="global"
    }

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
        package "/network/rsync"
    }

    target "vm" kind="qcow2" {
        disk-size "2G"
        bootloader "uefi"
        filesystem "zfs"
        pool {
            property name="ashift" value="12"
        }
    }

## Validate First

Before building, validate the spec to catch syntax errors:

    forger validate --spec my-image.kdl

## Inspect the Resolved Spec

See what the build will do after resolving includes and applying profiles:

    forger inspect --spec my-image.kdl

## List Targets

Check what targets are defined:

    forger targets --spec my-image.kdl

Output:

    vm (qcow2)

## Build

    forger build --spec my-image.kdl

This will:

1. Detect whether to build locally or spin up a builder VM
2. Phase 1: Create a rootfs by initializing IPS, setting publishers, and installing packages
3. Phase 2: Create a 2GB raw disk, set up a ZFS pool, populate it from the rootfs, install the UEFI bootloader, and convert to QCOW2

The output lands in `./output/` by default.

## Build a Specific Target

If your spec defines multiple targets, build just one:

    forger build --spec my-image.kdl --target vm

## Next Steps

- Add overlays for custom configuration files
- Set up base specs for caching
- Configure a builder VM for cross-platform builds

# Build Modes

Forger supports two build modes: local and remote. The mode is selected automatically based on your environment, or you can force it with CLI flags.

## Local Build

A local build runs directly on your host machine. This is the fastest option but requires:

- The target distro's package manager to be available
- Root/sudo access for filesystem operations
- Matching architecture (e.g., building OmniOS images on OmniOS)

    forger build --spec my-image.kdl --local

Use `--local` to skip builder VM detection and force a local build.

## Remote Build (Builder VM)

When your host doesn't match the target OS — for example, building OmniOS images from a Linux workstation — Forger spins up an ephemeral builder VM:

1. Downloads or uses a cached builder image (OCI reference, URL, or local file)
2. Creates a cloud-init configuration with an ephemeral SSH keypair
3. Starts a QEMU VM with user-mode networking (no root needed on host)
4. Transfers the `forger` binary, spec file, and overlay files via SSH
5. Runs the build inside the VM
6. Downloads the finished artifacts
7. Destroys the VM

    forger build --spec my-image.kdl --use-builder

Use `--use-builder` to force a remote build even when local build is possible.

### Default Builder Images

If no builder is specified in the spec or on the CLI, Forger uses sensible defaults:

Target Distro | Default Builder Image
--- | ---
OmniOS | `oci://ghcr.io/cloudnebulaproject/omnios-builder:latest`
Ubuntu | `oci://ghcr.io/cloudnebulaproject/ubuntu-builder:latest`

### Override the Builder Image

From the CLI:

    forger build --spec my-image.kdl --builder-image oci://my-registry/my-builder:v1

Or in the spec file (see Builder Configuration).

## How Forger Chooses

When neither `--local` nor `--use-builder` is specified, Forger checks whether the current host can satisfy the build requirements (package manager availability, OS match). If it can, it builds locally. Otherwise, it falls back to a builder VM.

# Spec Overview

Forger uses KDL (KDL Document Language) as its specification format. KDL is a node-based document language that's more expressive than TOML or JSON while being more readable than YAML.

## Why KDL?

- Node-based: natural fit for declaring image components (packages, files, targets)
- Arguments and properties: nodes can have positional arguments and named properties on the same line
- Children blocks: nested structure without deep indentation
- Type annotations: optional type safety
- Comments: `//` line comments and `/* */` block comments

## Spec File Structure

A complete spec file has these top-level sections:

    // Identity
    metadata name="my-image" version="1.0.0" description="What this image is"

    // Which OS (omit for OmniOS default)
    distro "ubuntu-22.04"

    // Composability
    base "path/to/parent.kdl"
    include "path/to/shared-steps.kdl"

    // Package sources
    repositories {
        // IPS publishers or APT mirrors
    }

    // IPS-specific
    incorporation "entire"
    variants { /* ... */ }
    certificates { /* ... */ }

    // What to install
    packages {
        package "pkg-name"
    }

    // Files and directories to place
    overlays {
        // file, ensure-dir, ensure-symlink, shadow, devfsadm, remove-files
    }

    // Users and system setup
    customization {
        user "username"
    }

    // Builder VM config (optional)
    builder {
        image "oci://..."
        vcpus 4
        memory 4096
    }

    // What to produce
    target "name" kind="qcow2" {
        // target-specific settings
    }

All sections are optional except that you need at least one `target` to build anything.

## File Resolution

Paths in `base`, `include`, and overlay `source` fields are resolved relative to the spec file's directory. This means you can organize specs and their supporting files together:

    images/
      omnios-base.kdl
      omnios-disk.kdl        # base "omnios-base.kdl"
      common.kdl
      devfs.kdl
      files/
        etc/
          hosts
          resolv.conf
          sshd_config
        omniosce-ca.cert.pem

## Validation

Always validate before building:

    forger validate --spec my-image.kdl

This checks:

- KDL syntax correctness
- Schema conformance (required fields, valid types)
- Include/base resolution (files exist, no circular references)

# Metadata & Distro

## Metadata

The `metadata` node identifies your image:

    metadata name="omnios-base" version="1.0.0" description="Base OmniOS bloody image"

Property | Required | Description
--- | --- | ---
`name` | Yes | Image name (used in output filenames)
`version` | Yes | Semantic version
`description` | No | Human-readable description

## Distro Selection

The `distro` node tells Forger which OS family to target. This determines the package manager, filesystem defaults, and build path.

    distro "ubuntu-22.04"

### Supported Values

Distro String | Family | Package Manager | Default Filesystem
--- | --- | --- | ---
(omitted) | OmniOS | IPS (`pkg`) | ZFS
`"ubuntu-22.04"` | Ubuntu | APT (`debootstrap` + `apt`) | ext4

### How Distro Affects the Build

The distro string maps to a `DistroFamily` internally:

- OmniOS: Uses `pkg image-create` to initialize IPS, sets publishers, handles incorporations, variants, and CA certificates. ZFS is the default filesystem for QCOW2 targets.
- Ubuntu: Uses `debootstrap` for initial rootfs, writes `sources.list`, runs `apt update` and `apt install`. ext4 is the default filesystem for QCOW2 targets.

If no `distro` is specified, OmniOS is assumed. This reflects Forger's primary focus on the illumos ecosystem.

### Distro and Builder Images

The distro also determines the default builder VM image when no explicit builder is configured:

Distro Family | Default Builder
--- | ---
OmniOS | `oci://ghcr.io/cloudnebulaproject/omnios-builder:latest`
Ubuntu | `oci://ghcr.io/cloudnebulaproject/ubuntu-builder:latest`

# Repositories & Publishers

The `repositories` block defines where packages are fetched from. The syntax differs between IPS (OmniOS) and APT (Ubuntu).

## IPS Publishers (OmniOS)

OmniOS uses IPS publishers — named package repositories with a URL origin:

    repositories {
        publisher name="omnios" origin="https://pkg.omnios.org/bloody/core/"
        publisher name="extra.omnios" origin="https://pkg.omnios.org/bloody/extra/"
    }

Property | Required | Description
--- | --- | ---
`name` | Yes | Publisher name (e.g., `"omnios"`, `"extra.omnios"`)
`origin` | Yes | Repository URL

### Common OmniOS Publishers

Name | Origin | Contents
--- | --- | ---
`omnios` | `https://pkg.omnios.org/bloody/core/` | Core OS packages (bloody)
`extra.omnios` | `https://pkg.omnios.org/bloody/extra/` | Additional packages (bloody)
`omnios` | `https://pkg.omnios.org/r151050/core/` | Core OS packages (stable)

### Publisher Verification

OmniOS publishers use signed packages. Configure CA certificates in the `certificates` block to enable verification:

    certificates {
        ca publisher="omnios" certfile="omniosce-ca.cert.pem"
    }

The `certfile` path is resolved relative to the spec file's directory.

## APT Mirrors (Ubuntu)

Ubuntu uses APT repositories with suite and component selection:

    repositories {
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy" components="main universe"
    }

Property | Required | Description
--- | --- | ---
(argument) | Yes | Mirror URL
`suite` | Yes | Distribution codename (e.g., `"jammy"`)
`components` | Yes | Space-separated component list (e.g., `"main universe"`)

### Common Ubuntu Mirrors

    repositories {
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy" components="main universe"
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy-updates" components="main universe"
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy-security" components="main universe"
    }

## Deduplication

When specs are composed via `base` or `include`, repositories are merged by name (IPS) or URL (APT). Duplicate entries are deduplicated automatically.

# Packages

The `packages` block declares which packages to install in the image. Package names follow the conventions of the target distro's package manager.

## Basic Usage

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
        package "/network/rsync"
        package "/service/network/ntpsec"
    }

Each `package` node takes a single argument: the package name.

## IPS Package Names (OmniOS)

IPS uses hierarchical FMRI (Fault Management Resource Identifier) paths:

    packages {
        package "/editor/vim"                    // Editors
        package "/network/openssh-server"        // Network services
        package "/ooce/developer/rust"           // OmniOS Community Extra (OOCE)
        package "/developer/build/gnu-make"      // Build tools
        package "/driver/virtio/vioif"           // Virtio network driver
    }

The leading `/` is conventional for IPS. You can find available packages with `pkg search` on an OmniOS system or browse pkg.omnios.org.

## APT Package Names (Ubuntu)

Ubuntu uses flat package names:

    packages {
        package "build-essential"
        package "curl"
        package "git"
        package "openssh-server"
        package "linux-image-generic"
    }

## Conditional Packages (Profiles)

Packages can be scoped to a profile using the `if` property. They're only installed when that profile is active:

    // Always installed
    packages {
        package "/editor/vim"
        package "/network/openssh-server"
    }

    // Only when building with --profile build
    packages if="build" {
        package "/developer/build-essential"
        package "/ooce/developer/omnios-build-tools"
        package "/developer/build/gnu-make"
    }

Build with a profile:

    forger build --spec my-image.kdl --profile build

Multiple `packages` blocks with different `if` conditions can coexist in a single spec.

## IPS Incorporations

For OmniOS, the `incorporation` node pins the entire OS to a consistent version set:

    incorporation "entire"

This ensures all installed packages are compatible. The `entire` incorporation is standard for OmniOS and should almost always be included.

## IPS Variants

Variants control which facets of packages are installed:

    variants {
        set name="opensolaris.zone" value="global"
    }

Setting `opensolaris.zone` to `"global"` ensures you get the full global-zone package set, including kernel modules and boot components. Use `"nonglobal"` for zone-only images.

# Overlays

Overlays let you place files, create directories, manage symlinks, and configure system state in the image. They run after package installation.

## File Overlay

Copy a local file into the image:

    overlays {
        file destination="/etc/ssh/sshd_config" source="files/etc/sshd_config"
    }

Property | Required | Description
--- | --- | ---
`destination` | Yes | Absolute path in the image
`source` | Yes | Local file path (relative to spec file)

## Ensure Directory

Create a directory with specific ownership and permissions:

    overlays {
        ensure-dir "/opt/myapp" owner="root" group="root" mode="755"
    }

Property | Required | Description
--- | --- | ---
(argument) | Yes | Directory path in the image
`owner` | No | Owner user
`group` | No | Owner group
`mode` | No | Octal permissions

## Ensure Symlink

Create a symbolic link:

    overlays {
        ensure-symlink "/etc/nsswitch.conf" target="/etc/nsswitch.dns"
    }

Property | Required | Description
--- | --- | ---
(argument) | Yes | Symlink path in the image
`target` | Yes | What the symlink points to

This is commonly used for SMF profiles on illumos:

    overlays {
        ensure-symlink "/etc/svc/profile/generic.xml" target="generic_limited_net.xml"
        ensure-symlink "/etc/svc/profile/inetd_services.xml" target="inetd_generic.xml"
        ensure-symlink "/etc/svc/profile/platform.xml" target="platform_none.xml"
        ensure-symlink "/etc/svc/profile/name_service.xml" target="ns_dns.xml"
    }

## Remove Files

Remove files or directories from the image:

    overlays {
        remove-files "/dev/dsk" "/dev/rdsk" "/dev/cfg" "/dev/usb"
    }

Takes one or more path arguments. Useful for cleaning up device nodes before recreating them with `devfsadm`.

## Shadow Password

Set a password hash for a user:

    overlays {
        shadow username="root" password="$5$rounds=10000$hashedvalue"
    }

Property | Required | Description
--- | --- | ---
`username` | Yes | User whose password to set
`password` | Yes | Password hash (crypt format)

## Device Filesystem (devfsadm)

Run `devfsadm` to populate `/dev` with device nodes. This is essential for bootable illumos images:

    overlays {
        devfsadm
    }

This is typically used in a dedicated `devfs.kdl` include file alongside directory creation and cleanup.

## Conditional Overlays

Like packages, overlay blocks can be scoped to profiles:

    overlays if="debug" {
        file destination="/etc/system" source="files/etc/system.debug"
    }

## Execution Order

Overlays execute in the order they appear in the spec. When multiple spec files are composed (base + includes), overlays from the base run first, followed by includes in order, then the current spec.

## Complete Example

    overlays {
        // Clean up device directories
        remove-files "/dev/dsk" "/dev/rdsk" "/dev/cfg" "/dev/usb"

        // Recreate them
        ensure-dir "/dev/cfg" owner="root" group="root" mode="755"
        ensure-dir "/dev/dsk" owner="root" group="root" mode="755"
        ensure-dir "/dev/rdsk" owner="root" group="root" mode="755"
        ensure-dir "/dev/usb" owner="root" group="root" mode="755"

        // Populate device nodes
        devfsadm

        // System configuration
        file destination="/etc/inet/hosts" source="files/etc/hosts"
        file destination="/etc/nodename" source="files/etc/nodename"
        ensure-symlink "/etc/nsswitch.conf" target="/etc/nsswitch.dns"

        // Root password
        shadow username="root" password="$5$rounds=10000$..."
    }

# Customizations

The `customization` block configures users and system-level settings.

## Users

Create a user in the image:

    customization {
        user "ci"
    }

Property | Required | Description
--- | --- | ---
(argument) | Yes | Username to create

The user is created with default settings appropriate for the target distro.

## Conditional Customizations

Like packages and overlays, customization blocks support profile filtering:

    customization if="ci" {
        user "ci"
    }

This user is only created when the `ci` profile is active:

    forger build --spec my-image.kdl --profile ci

## Combined with Overlays

For full user setup, combine customization with overlay operations:

    customization {
        user "deploy"
    }

    overlays {
        // Set user password
        shadow username="deploy" password="$5$..."

        // Ensure SSH directory
        ensure-dir "/home/deploy/.ssh" owner="deploy" group="staff" mode="700"

        // Deploy authorized keys
        file destination="/home/deploy/.ssh/authorized_keys" source="files/authorized_keys"
    }

# Targets

Targets define what Forger produces from the assembled rootfs. Each spec can have multiple targets, and you can build them selectively.

## Target Kinds

Kind | Description | Typical Use
--- | --- | ---
`qcow2` | Bootable VM disk image | Cloud VMs, hypervisors
`oci` | OCI container image | Container runtimes
`artifact` | Tar archive of the rootfs | Intermediate build stage, embedding

## QCOW2 Target

Produces a bootable virtual machine disk image:

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "uefi"
        filesystem "zfs"
        pool {
            property name="ashift" value="12"
        }
        push-to "ghcr.io/myorg/my-image:latest"
    }

Property | Required | Default | Description
--- | --- | --- | ---
`disk-size` | Yes | — | Disk size (e.g., `"2G"`, `"2000M"`)
`bootloader` | Yes | — | `"uefi"`, `"grub"`, or `"grub-efi-amd64-bin"`
`filesystem` | No | Distro default | `"zfs"` or `"ext4"`
`push-to` | No | — | OCI registry reference for auto-push
`pool` | No | — | ZFS pool properties (ZFS only)

### ZFS Pool Properties

    pool {
        property name="ashift" value="12"
    }

`ashift=12` sets the ZFS block alignment to 4K sectors, which is correct for modern storage.

### Bootloader Options

- `uefi`: UEFI bootloader — recommended for illumos and modern Linux
- `grub`: Legacy GRUB — for BIOS-boot OmniOS images
- `grub-efi-amd64-bin`: GRUB EFI for Ubuntu — use with ext4

## OCI Target

Produces an OCI container image:

    target "container" kind="oci" {
        entrypoint command="/bin/sh"
        environment {
            set "PATH" "/usr/bin:/bin:/usr/sbin:/sbin"
            set "TZ" "UTC"
        }
    }

Property | Required | Description
--- | --- | ---
`entrypoint` | No | Container entrypoint command
`environment` | No | Environment variables

The OCI target produces an OCI Image Layout directory containing:

- Compressed tar.gz layer(s) of the rootfs
- OCI manifest and config JSON
- SHA256-based blob digests

## Artifact Target

Produces a tar archive of the rootfs:

    target "archive" kind="artifact" {
    }

Artifact targets are the primary mechanism for multi-stage pipelines. A parent spec can produce an artifact that a child spec consumes as its base.

## Multiple Targets

A single spec can define multiple targets:

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "uefi"
        filesystem "zfs"
    }

    target "container" kind="oci" {
        entrypoint command="/usr/sbin/sshd"
    }

    target "archive" kind="artifact" {
    }

Build all targets:

    forger build --spec my-image.kdl

Build a specific target:

    forger build --spec my-image.kdl --target vm

## Auto-Push

When `push-to` is set on a target, Forger automatically pushes the artifact to the OCI registry after a successful build (unless `--skip-push` is used):

    # Build and push
    forger build --spec my-image.kdl

    # Build without pushing
    forger build --spec my-image.kdl --skip-push

## Listing Targets

View all targets in a spec:

    forger targets --spec my-image.kdl

# Builder Configuration

The `builder` block configures the ephemeral VM used for remote builds. This is optional — Forger selects sensible defaults when omitted.

## Syntax

    builder {
        image "oci://ghcr.io/cloudnebulaproject/omnios-builder:latest"
        vcpus 4
        memory 4096
        disk 20
    }

Property | Required | Default | Description
--- | --- | --- | ---
`image` | No | Distro-specific default | Builder VM image (OCI ref, URL, or path)
`vcpus` | No | Host-dependent | Number of virtual CPUs
`memory` | No | Host-dependent | Memory in MB
`disk` | No | 20 (GB) | Disk overlay size in GB

## Image Sources

The `image` property accepts three formats:

### OCI Registry Reference

    builder {
        image "oci://ghcr.io/myorg/my-builder:latest"
    }

Pulls a QCOW2 image from an OCI registry.

### URL

    builder {
        image "https://example.com/builders/omnios-builder.qcow2"
    }

Downloads the image directly.

### Local Path

    builder {
        image "/path/to/my-builder.qcow2"
    }

Uses a local QCOW2 file.

## Default Builder Images

When no builder image is specified (neither in the spec nor on the CLI), Forger uses:

Distro | Default Image
--- | ---
OmniOS | `oci://ghcr.io/cloudnebulaproject/omnios-builder:latest`
Ubuntu | `oci://ghcr.io/cloudnebulaproject/ubuntu-builder:latest`

## How the Builder VM Works

1. Image resolution: Downloads or locates the builder image
2. Cloud-init: Generates a user-data config with an ephemeral SSH keypair and a `builder` user with passwordless sudo
3. VM creation: Starts QEMU with user-mode networking (no host root needed)
4. SSH connection: Retries SSH connection for up to 120 seconds while the VM boots
5. Transfer: Uploads the `forger` binary, spec files, and overlay files
6. Build: Executes the build command inside the VM
7. Download: Retrieves the build artifacts
8. Teardown: Destroys the VM (guaranteed, even on build failure)

## CLI Override

The builder image can be overridden from the CLI, taking precedence over the spec:

    forger build --spec my-image.kdl --builder-image /path/to/custom-builder.qcow2

## Resource Sizing

For typical builds:

- OmniOS base image: 2 vCPUs, 2048 MB RAM is sufficient
- OmniOS with Rust/build tools: 4 vCPUs, 4096 MB RAM recommended
- Ubuntu with build-essential: 4 vCPUs, 4096 MB RAM recommended
- Disk: 20 GB is sufficient for most images; increase for large package sets

# Base Specs (Build Caching)

The `base` directive establishes a parent-child relationship between specs. This is Forger's primary caching mechanism: a parent spec produces an artifact, and a child spec consumes it.

## How It Works

    omnios-base.kdl          (parent)
      → builds artifact       (tar archive cached on disk or in registry)
        → omnios-disk.kdl     (child, consumes the artifact)
          → builds QCOW2

The parent handles the expensive, slow operations (OS installation, base package setup). The child adds customization and produces the final image. When the parent's output is cached, rebuilding the child skips the entire base installation.

## Syntax

In the child spec:

    base "omnios-base.kdl"

    // Child-specific additions
    packages {
        package "/system/cloud-init"
        package "/driver/virtio/vioif"
    }

    target "vm" kind="qcow2" {
        disk-size "2G"
        bootloader "uefi"
        filesystem "zfs"
    }

The parent spec (`omnios-base.kdl`) defines the foundation:

    metadata name="omnios-base" version="1.0.0" description="Base OmniOS configuration"

    repositories {
        publisher name="omnios" origin="https://pkg.omnios.org/bloody/core/"
        publisher name="extra.omnios" origin="https://pkg.omnios.org/bloody/extra/"
    }

    incorporation "entire"

    certificates {
        ca publisher="omnios" certfile="omniosce-ca.cert.pem"
    }

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
        package "/network/rsync"
    }

## What Gets Inherited

When a child references a parent via `base`:

- Repositories are merged (deduplicated by name/URL)
- Packages from both parent and child are installed
- Overlays from the parent execute first, then the child's
- Customizations from both are applied
- Metadata from the child overrides the parent
- Targets from the child are used (parent targets are not inherited)

## Caching Pipeline

The key insight is that `base` creates a build stage boundary. The parent's output (typically an artifact/tar) is the cache unit:

1. First build: parent runs Phase 1 (full package installation), produces artifact
2. Subsequent builds: if the parent spec hasn't changed, skip Phase 1 and start from the cached artifact
3. The child only needs to apply its additional packages, overlays, and produce its targets

This mirrors the strap→image→archive pipeline from the old `omnios-image-builder`, but expressed declaratively.

## Multi-Level Inheritance

Base specs can themselves have bases, creating a chain:

    distro-base.kdl
      → platform-base.kdl
        → application-image.kdl

Each level adds or refines what the previous level established. Circular references are detected and rejected.

## Base vs Include

| Base | Include
--- | --- | ---
Relationship | Parent → child (produces artifact for child to consume) | Sibling (shared steps imported)
Caching | Yes — parent output is the cache boundary | No — just DRY for config
Merging | Full merge with child overrides | Steps imported as-is
Targets | Child's targets used | No target interaction

# Includes (Shared Steps)

The `include` directive imports shared configuration steps from another spec file into the current spec. Unlike `base`, includes don't create a build-stage boundary — they simply pull in common definitions for reuse.

## Syntax

    include "common.kdl"
    include "devfs.kdl"

## What Includes Do

An include file is a regular `.kdl` spec containing overlays, packages, customizations, or other configuration. When included, its contents are merged into the including spec as if they were written inline.

### Example: `common.kdl`

    overlays {
        ensure-symlink "/etc/svc/profile/generic.xml" target="generic_limited_net.xml"
        ensure-symlink "/etc/svc/profile/inetd_services.xml" target="inetd_generic.xml"
        ensure-symlink "/etc/svc/profile/platform.xml" target="platform_none.xml"
        ensure-symlink "/etc/svc/profile/name_service.xml" target="ns_dns.xml"

        file destination="/etc/inet/hosts" source="files/etc/hosts"
        file destination="/etc/nodename" source="files/etc/nodename"
        file destination="/etc/resolv.conf" source="files/etc/resolv.conf"
        ensure-symlink "/etc/nsswitch.conf" target="/etc/nsswitch.dns"
    }

### Example: `devfs.kdl`

    overlays {
        devfsadm

        remove-files "/dev/dsk" "/dev/rdsk" "/dev/cfg" "/dev/usb"

        ensure-dir "/dev/cfg" owner="root" group="root" mode="755"
        ensure-dir "/dev/dsk" owner="root" group="root" mode="755"
        ensure-dir "/dev/rdsk" owner="root" group="root" mode="755"
        ensure-dir "/dev/usb" owner="root" group="root" mode="755"

        ensure-symlink "/dev/msglog" target="../devices/pseudo/log@0:msglog"
    }

## Using Includes

A disk image spec can import these shared steps:

    base "omnios-base.kdl"
    include "devfs.kdl"
    include "common.kdl"

    packages {
        package "/system/cloud-init"
    }

    overlays {
        file destination="/boot/conf.d/console" source="files/boot_console.115200"
        shadow username="root" password="$5$..."
    }

    target "vm" kind="qcow2" {
        disk-size "2G"
        bootloader "uefi"
        filesystem "zfs"
    }

## Execution Order

1. Base spec's packages, overlays, and customizations run first
2. Include files run in the order they appear
3. The current spec's content runs last

This gives you control over layering: device filesystem setup (`devfs.kdl`) before network configuration (`common.kdl`) before image-specific overlays.

## When to Use Include vs Base

- Include: Shared configuration snippets used across multiple specs (SMF profiles, network setup, device nodes)
- Base: A full image foundation that produces a cached artifact consumed by derivative images

Includes are lightweight — they don't trigger a separate build phase or produce intermediate artifacts.

# Profiles (Conditional Variants)

Profiles let you create multiple image variants from a single spec. Blocks tagged with `if="profile-name"` are only included when that profile is active.

## Syntax

Tag any `packages`, `overlays`, or `customization` block with an `if` property:

    // Always included (no condition)
    packages {
        package "/editor/vim"
        package "/network/openssh-server"
    }

    // Only when --profile build is active
    packages if="build" {
        package "/developer/build-essential"
        package "/ooce/developer/omnios-build-tools"
    }

    // Only when --profile ci is active
    customization if="ci" {
        user "ci"
    }

    overlays if="debug" {
        file destination="/etc/system" source="files/etc/system.debug"
    }

## Activating Profiles

Use the `--profile` flag (repeatable) when building or inspecting:

    # No profiles — just the base packages
    forger build --spec my-image.kdl

    # With build tools
    forger build --spec my-image.kdl --profile build

    # With build tools AND CI user
    forger build --spec my-image.kdl --profile build --profile ci

    # Inspect with profiles to see what would be included
    forger inspect --spec my-image.kdl --profile build --profile ci

## How Filtering Works

Profile filtering happens after spec resolution (base + include merging) but before the build starts:

1. Parse and resolve the full spec (with all blocks)
2. Apply profile filter:
   - Blocks with no `if` condition → always kept
   - Blocks where `if` value matches an active profile → kept
   - Blocks where `if` value doesn't match any active profile → removed
3. Build proceeds with the filtered spec

## Use Cases

### Development vs Production

    packages {
        package "/network/openssh-server"
    }

    packages if="dev" {
        package "/developer/build-essential"
        package "/diagnostic/top"
    }

    overlays if="dev" {
        shadow username="root" password="$5$..."
    }

### CI Variants

    packages if="rust-ci" {
        package "/ooce/developer/rust"
        package "/developer/build/gnu-make"
    }

    packages if="go-ci" {
        package "/ooce/developer/go"
    }

### Cloud Provider Specific

    overlays if="aws" {
        file destination="/boot/conf.d/console" source="files/boot_console.aws"
        file destination="/etc/dhcp/dhcpagent" source="files/dhcpagent.aws"
    }

    overlays if="digitalocean" {
        file destination="/boot/conf.d/console" source="files/boot_console.do"
    }

## Combining with Base and Includes

Profiles work across the full composition chain. Conditional blocks in base specs and includes are filtered together with the current spec's blocks:

    // omnios-base.kdl
    packages if="build" {
        package "/developer/build-essential"
    }

    // my-image.kdl
    base "omnios-base.kdl"

    packages if="build" {
        package "/ooce/developer/rust"
    }

Building with `--profile build` activates both blocks from both files.

# Multi-Stage Pipelines

Forger's composability model enables multi-stage build pipelines where each stage's output feeds as input to the next. This is the key to fast, cacheable image builds.

## The Pipeline Pattern

A typical pipeline for a bootable OmniOS VM image looks like this:

    Stage 1: Base (expensive, cached)
      omnios-base.kdl
        → Initializes IPS
        → Sets publishers
        → Installs core packages
        → Produces: artifact (tar archive)

    Stage 2: Image (fast, incremental)
      omnios-disk.kdl (base: omnios-base.kdl)
        → Consumes base artifact
        → Adds cloud-init, drivers
        → Applies overlays (config files, device nodes)
        → Produces: QCOW2 image

The first build runs both stages. Subsequent builds skip Stage 1 if the base hasn't changed, jumping straight to Stage 2.

## Designing Your Pipeline

### Identify the Cache Boundary

The most expensive operation in image building is package installation — downloading and extracting hundreds of packages from a repository. Put this in the base spec:

    // omnios-base.kdl — the slow part, cached
    metadata name="omnios-base" version="1.0.0"

    repositories {
        publisher name="omnios" origin="https://pkg.omnios.org/bloody/core/"
        publisher name="extra.omnios" origin="https://pkg.omnios.org/bloody/extra/"
    }

    incorporation "entire"

    certificates {
        ca publisher="omnios" certfile="omniosce-ca.cert.pem"
    }

    variants {
        set name="opensolaris.zone" value="global"
    }

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
        package "/network/rsync"
        package "/service/network/ntpsec"
        package "/web/curl"
        package "/web/wget"
    }

### Derive Specific Images

Then create derivative specs that add target-specific configuration:

    // omnios-disk.kdl — fast, builds on cached base
    base "omnios-base.kdl"
    include "devfs.kdl"
    include "common.kdl"

    packages {
        package "/system/cloud-init"
        package "/driver/virtio/vioif"
        package "/driver/virtio/vioblk"
    }

    overlays {
        file destination="/boot/conf.d/console" source="files/boot_console.115200"
        shadow username="root" password="$5$rounds=10000$..."
    }

    target "vm" kind="qcow2" {
        disk-size "2G"
        bootloader "uefi"
        filesystem "zfs"
        pool {
            property name="ashift" value="12"
        }
    }

### Multiple Derivatives from One Base

    omnios-base.kdl
      ├── omnios-disk.kdl          → QCOW2 VM image
      ├── omnios-rust-ci.kdl       → Rust CI image (adds rust, git, build tools)
      ├── omnios-container.kdl     → OCI container
      └── omnios-aws.kdl           → AWS-specific VM image

Each derivative shares the same cached base, so building all four images only runs the expensive Stage 1 once.

## Comparison with Predecessor Tools

### omnios-image-builder (Shell + JSON)

The old `omnios-image-builder` achieved the same pattern with ZFS snapshots:

    01-strap.json  → pkg install entire → ZFS snapshot "strap"
    02-image.json  → pkg install extras → ZFS snapshot "image"
    03-archive.json → pack_tar → omnios-bloody.tar.gz
    aws.json       → unpack_tar + make_bootable → raw disk

Each ZFS snapshot was a cache point. The `-f` flag forced a full rebuild.

Forger replaces this with the `base` directive. No ZFS on the build host required. No manual snapshot management. The caching is implicit in the spec relationship.

### Packer (HCL)

Packer has no native multi-stage caching. Each build starts from an ISO installation, runs provisioner scripts, and captures the result. There's no way to say "skip the OS install and start from here."

Forger's pipeline model is fundamentally more efficient for iterative development.

# illumos Overview

Forger's primary focus is the illumos ecosystem. This chapter provides essential background for image developers working with illumos distributions.

## What is illumos?

illumos is a Unix operating system kernel derived from OpenSolaris. It powers several distributions including OmniOS, OpenIndiana, and SmartOS. Key technologies that distinguish illumos from Linux:

### ZFS

ZFS is the default and recommended filesystem for illumos. It provides:

- Copy-on-write: Snapshots and clones are instant and space-efficient
- Boot Environments (BEs): Multiple bootable system states on the same pool
- Data integrity: End-to-end checksumming
- Built-in compression: LZ4 by default on modern pools

Forger creates ZFS pools natively for QCOW2 targets on illumos.

### IPS (Image Packaging System)

IPS is illumos's package manager. Key concepts:

- Publishers: Named package repositories (e.g., `omnios`, `extra.omnios`)
- Incorporations: Meta-packages that constrain version compatibility (e.g., `entire`)
- Variants: Facets that select package subsets (e.g., `opensolaris.zone=global`)
- FMRIs: Hierarchical package names (e.g., `/network/openssh-server`)
- Signed packages: CA-verified package integrity

Forger wraps IPS operations directly — no shell scripting needed.

### SMF (Service Management Facility)

SMF is illumos's service manager (similar to systemd on Linux, but predates it). Services are defined by XML manifests and managed via profiles:

- Profiles control which services are enabled at boot
- generic_limited_net.xml: Basic networking
- inetd_generic.xml: Internet daemon services
- platform_none.xml: Platform-specific (none for VM images)
- ns_dns.xml: DNS name service

Forger configures SMF profiles through symlink overlays:

    overlays {
        ensure-symlink "/etc/svc/profile/generic.xml" target="generic_limited_net.xml"
        ensure-symlink "/etc/svc/profile/name_service.xml" target="ns_dns.xml"
    }

### Zones

illumos zones are lightweight OS-level containers (similar to LXC/Docker, but predating both). The `opensolaris.zone` variant controls whether packages include global-zone-only components:

- `global`: Full system including kernel modules, boot components, and hardware drivers
- `nonglobal`: Zone-only packages (no kernel or hardware support)

For VM images, always use `global`:

    variants {
        set name="opensolaris.zone" value="global"
    }

### Device Filesystem (devfs)

illumos manages device nodes through `devfsadm`, which discovers hardware and creates entries in `/dev`. For image building, this means:

1. Clean up stale device nodes from the build environment
2. Run `devfsadm` to create correct entries for the target hardware
3. Ensure required directories exist (`/dev/dsk`, `/dev/rdsk`, `/dev/cfg`, `/dev/usb`)

Forger handles this through the `devfsadm` overlay and the conventional `devfs.kdl` include.

## illumos Distributions

Distribution | Focus | Publisher URL
--- | --- | ---
OmniOS | Server/cloud, minimal, stable | `pkg.omnios.org`
OpenIndiana | Desktop/general-purpose, broader package set | `pkg.openindiana.org`
SmartOS | Hypervisor/container host (Joyent) | Not IPS-based

Forger currently supports OmniOS. OpenIndiana support follows the same IPS path.

# OmniOS

OmniOS is Forger's primary target distribution. It's a server-focused illumos distribution maintained by the OmniOS Community Edition (OmniOSce) project.

## Release Branches

Branch | URL Path | Use Case
--- | --- | ---
bloody | `/bloody/core/` | Rolling development, latest packages
stable (e.g., r151050) | `/r151050/core/` | Production, LTS releases

### Choosing a Branch

- Use bloody for CI images, development, and testing the latest software
- Use stable for production deployments where predictability matters

## Minimal Spec

    metadata name="omnios-base" version="1.0.0"

    repositories {
        publisher name="omnios" origin="https://pkg.omnios.org/bloody/core/"
        publisher name="extra.omnios" origin="https://pkg.omnios.org/bloody/extra/"
    }

    incorporation "entire"

    certificates {
        ca publisher="omnios" certfile="omniosce-ca.cert.pem"
    }

    variants {
        set name="opensolaris.zone" value="global"
    }

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
    }

### Required Elements

- `incorporation "entire"`: Pins all packages to a consistent version set. Without this, you may get incompatible package versions.
- `certificates`: OmniOS packages are signed. The CA certificate file (`omniosce-ca.cert.pem`) must be available relative to the spec.
- `variants` with `opensolaris.zone=global`: Required for bootable VM images. Omitting this may exclude kernel modules.

## Common Packages

### Core System

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
        package "/network/rsync"
        package "/service/network/ntpsec"
        package "/web/curl"
        package "/web/wget"
    }

### Cloud & Virtualization

    packages {
        package "/system/cloud-init"
        package "/driver/virtio/vioif"      // Virtio network
        package "/driver/virtio/vioblk"     // Virtio block storage
        package "/driver/virtio/vio9p"      // 9p filesystem sharing
        package "/driver/virtio/vioscsi"    // Virtio SCSI
        package "/driver/virtio/viorand"    // Virtio RNG
    }

### Development

    packages if="build" {
        package "/developer/build-essential"
        package "/ooce/developer/omnios-build-tools"
        package "/developer/build/gnu-make"
    }

### Rust Toolchain

    packages if="rust" {
        package "/ooce/developer/rust"
        package "/developer/versioning/git"
    }

## Boot Configuration

OmniOS VM images need console and boot configuration through overlays:

### Serial Console (115200 baud)

    overlays {
        file destination="/boot/conf.d/console" source="files/boot_console.115200"
        file destination="/etc/ttydefs" source="files/ttydefs.115200"
        file destination="/etc/default/init" source="files/default_init.utc"
    }

The boot console file configures which console device is used:

- `ttya`: Serial console (standard for cloud VMs)
- `text`: Framebuffer (for interactive debugging)

### SMF Profiles

    overlays {
        ensure-symlink "/etc/svc/profile/generic.xml" target="generic_limited_net.xml"
        ensure-symlink "/etc/svc/profile/inetd_services.xml" target="inetd_generic.xml"
        ensure-symlink "/etc/svc/profile/platform.xml" target="platform_none.xml"
        ensure-symlink "/etc/svc/profile/name_service.xml" target="ns_dns.xml"
    }

## QCOW2 Target Settings

For OmniOS, always use ZFS with UEFI:

    target "vm" kind="qcow2" {
        disk-size "2G"
        bootloader "uefi"
        filesystem "zfs"
        pool {
            property name="ashift" value="12"
        }
    }

- `ashift=12`: 4K sector alignment, correct for modern disks and virtual storage
- UEFI: Standard boot method for OmniOS

## Complete Example

See `images/omnios-bloody-disk.kdl` in the repository for a full bootable OmniOS image spec, or the Example Specs chapter.

## OmniOS Extra Publisher

The `extra.omnios` publisher provides community-maintained packages not in the core repository, including:

- Rust (`/ooce/developer/rust`)
- Go (`/ooce/developer/go`)
- Python versions (`/ooce/runtime/python-*`)
- Node.js (`/ooce/runtime/node-*`)
- Build tools (`/ooce/developer/omnios-build-tools`)

Always add this publisher if you need development tools or language runtimes.

# Ubuntu

Ubuntu is Forger's secondary target, providing Linux support for teams that need both illumos and Linux images from the same tooling.

## How Ubuntu Builds Differ

Aspect | OmniOS | Ubuntu
--- | --- | ---
Bootstrap | `pkg image-create` | `debootstrap`
Package manager | IPS (`pkg`) | APT (`apt`)
Repository config | Publishers | `sources.list`
Default filesystem | ZFS | ext4
Bootloader | UEFI (illumos) | `grub-efi-amd64-bin`
Init system | SMF | systemd

## Minimal Spec

    metadata name="ubuntu-base" version="1.0.0" description="Ubuntu 22.04 base"

    distro "ubuntu-22.04"

    repositories {
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy" components="main universe"
    }

    packages {
        package "openssh-server"
        package "curl"
    }

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "grub-efi-amd64-bin"
        filesystem "ext4"
    }

### Key Differences from OmniOS

- `distro "ubuntu-22.04"` is required — without it, Forger assumes OmniOS
- No incorporation, variants, or certificates — these are IPS concepts
- Bootloader is `grub-efi-amd64-bin` — the Ubuntu GRUB EFI package
- Filesystem is `ext4` — ZFS is possible but not the Ubuntu default

## Repositories

Ubuntu uses APT mirrors with suite and component selection:

    repositories {
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy" components="main universe"
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy-updates" components="main universe"
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy-security" components="main universe"
    }

### Components

- main: Officially supported free software
- universe: Community-maintained free software
- restricted: Proprietary drivers
- multiverse: Non-free software

For most server images, `main universe` covers all needed packages.

## Common Packages

### Core System

    packages {
        package "openssh-server"
        package "curl"
        package "git"
        package "cloud-init"
        package "linux-image-generic"
    }

> Important: Include `linux-image-generic` for bootable VM images. Without a kernel, the image won't boot.

### Build Tools

    packages if="build" {
        package "build-essential"
        package "pkg-config"
        package "libssl-dev"
    }

### Rust Toolchain

    packages if="rust" {
        package "rustc"
        package "cargo"
        package "build-essential"
        package "libssl-dev"
        package "pkg-config"
    }

## Builder VM

Ubuntu builds need an Ubuntu builder VM. Specify it explicitly or let Forger use the default:

    builder {
        image "oci://ghcr.io/cloudnebulaproject/ubuntu-builder:latest"
        vcpus 4
        memory 4096
        disk 20
    }

## Complete Example

    metadata name="ubuntu-rust-ci" version="1.0.0" description="Ubuntu Rust CI image"

    distro "ubuntu-22.04"

    repositories {
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy" components="main universe"
    }

    packages {
        package "build-essential"
        package "rustc"
        package "cargo"
        package "git"
        package "curl"
        package "openssh-server"
        package "cloud-init"
        package "linux-image-generic"
        package "grub-efi-amd64-bin"
        package "libssl-dev"
        package "pkg-config"
    }

    customization {
        user "ci"
    }

    builder {
        image "oci://ghcr.io/cloudnebulaproject/ubuntu-builder:latest"
        vcpus 4
        memory 4096
        disk 20
    }

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "grub-efi-amd64-bin"
        filesystem "ext4"
        push-to "ghcr.io/cloudnebulaproject/ubuntu-rust:latest"
    }

## Future: IPS on Linux

A long-term goal of the Forger project is to bring IPS to Linux via a Rust implementation. This would allow Linux images to use the same publisher-based, signed-package, incorporation-constrained model that makes illumos packaging robust. When this is available, Ubuntu and other Linux distros will gain IPS as an alternative package manager option within Forger.

# Adding a New Distro

Forger's distro support is built around the `DistroFamily` abstraction. Adding a new distribution means extending this abstraction at the spec-parsing and build-engine levels.

## Architecture Overview

The distro system has three touch points:

1. `spec-parser`: Maps distro strings to `DistroFamily` enum
2. `forge-engine` Phase 1: Distro-specific rootfs assembly logic
3. `forge-builder`: Default builder image selection

## Step 1: Extend the DistroFamily Enum

In `crates/spec-parser/src/lib.rs`, add your distro to the enum:

    #![allow(unused)]
    fn main() {
    pub enum DistroFamily {
        OmniOS,
        Ubuntu,
        Fedora,   // New
    }
    }

Update the detection logic that maps the `distro` string to a family:

    #![allow(unused)]
    fn main() {
    fn detect_family(distro: &str) -> DistroFamily {
        if distro.starts_with("ubuntu") {
            DistroFamily::Ubuntu
        } else if distro.starts_with("fedora") {
            DistroFamily::Fedora
        } else {
            DistroFamily::OmniOS
        }
    }
    }

## Step 2: Add Repository Type

If your distro uses a different repository format, add it to the `repositories` parsing in the spec:

    repositories {
        // Existing
        publisher name="..." origin="..."    // IPS
        apt-mirror "..." suite="..." components="..."  // APT

        // New: DNF/YUM
        dnf-repo name="fedora" baseurl="https://..." gpgkey="..."
    }

## Step 3: Implement Phase 1 Logic

In `crates/forge-engine/src/phase1/mod.rs`, add the rootfs assembly path for your distro. This is the core work — each distro has its own bootstrap process:

- OmniOS: `pkg image-create` → set publishers → install
- Ubuntu: `debootstrap` → write sources.list → apt install
- Fedora: `dnf --installroot` → write repo files → dnf install

The key operations:

1. Initialize a package manager root in the staging directory
2. Configure repositories/mirrors
3. Install the base package set
4. Install user-specified packages

## Step 4: Default Builder Image

In `crates/forge-builder/src/lib.rs`, add a default builder image:

    #![allow(unused)]
    fn main() {
    match distro_family {
        DistroFamily::OmniOS => "oci://ghcr.io/cloudnebulaproject/omnios-builder:latest",
        DistroFamily::Ubuntu => "oci://ghcr.io/cloudnebulaproject/ubuntu-builder:latest",
        DistroFamily::Fedora => "oci://ghcr.io/cloudnebulaproject/fedora-builder:latest",
    }
    }

You'll also need to build and publish the builder image itself.

## Step 5: Add Example Specs

Create example specs in the `images/` directory showing common patterns for the new distro.

## Step 6: Document

Add a chapter to this book under Distro Guide covering:

- Required and recommended packages
- Repository configuration
- Filesystem and bootloader defaults
- Any distro-specific overlay patterns

## Design Principles

When adding a distro, keep these principles in mind:

- The `ToolRunner` trait wraps all external tool execution. Use it for any new package manager commands — this enables testing without root access.
- Phase 1 is distro-specific, Phase 2 is not. The QCOW2/OCI/artifact target production is shared across all distros. Only rootfs assembly changes.
- Filesystem defaults should match what the distro community expects (ZFS for illumos, ext4 for most Linux).
- Error messages should use miette diagnostics to tell the user exactly what's wrong and how to fix it.

# QCOW2 VM Images

QCOW2 (QEMU Copy-On-Write v2) is the primary output format for bootable virtual machine images.

## How QCOW2 Targets Work

Phase 2 for a QCOW2 target follows this sequence:

1. Create a raw disk of the specified size
2. Attach via loopback device (or equivalent)
3. Create filesystem:
   - ZFS: Create pool → create boot environment dataset → mount
   - ext4: Partition disk → format → mount
4. Populate from Phase 1 rootfs (copy files into mounted filesystem)
5. Install bootloader (UEFI or GRUB)
6. Finalize:
   - ZFS: Set bootfs property → unmount → export pool
   - ext4: Unmount
7. Detach loopback device
8. Convert raw disk to QCOW2 via `qemu-img convert`

## ZFS-Based Images (illumos)

ZFS is the default and recommended filesystem for OmniOS images:

    target "vm" kind="qcow2" {
        disk-size "2G"
        bootloader "uefi"
        filesystem "zfs"
        pool {
            property name="ashift" value="12"
        }
    }

### ZFS Pool Details

During build, Forger creates a uniquely-named ZFS pool (e.g., `forgebuild_12345`) to avoid conflicts with existing pools on the build host. After export, the pool is named `rpool` in the final image.

### Pool Properties

    pool {
        property name="ashift" value="12"
    }

- `ashift=12`: 4KB sector alignment. Use this for modern storage and virtual disks.
- Additional ZFS pool properties can be set using the same syntax.

### Boot Environments

ZFS images use boot environments (BEs), a core illumos concept. The image contains a single BE that becomes the default boot target. On first boot, the system can create new BEs for upgrades, allowing rollback to previous states.

## ext4-Based Images (Linux)

ext4 is the default filesystem for Ubuntu images:

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "grub-efi-amd64-bin"
        filesystem "ext4"
    }

The disk is partitioned with an EFI System Partition and a root partition formatted as ext4.

## Disk Sizing

Specify the disk size as a string with a unit suffix:

    disk-size "2G"       // 2 gigabytes
    disk-size "2000M"    // 2000 megabytes
    disk-size "8G"       // 8 gigabytes

Choose a size that accommodates your installed packages plus reasonable free space. Typical sizes:

- Minimal OmniOS: 2G
- OmniOS with development tools: 4-8G
- Ubuntu with build tools: 8G

## Bootloader Options

Value | Platform | Description
--- | --- | ---
`uefi` | illumos | Native UEFI boot (recommended for OmniOS)
`grub` | illumos | Legacy GRUB (BIOS boot)
`grub-efi-amd64-bin` | Ubuntu | GRUB EFI for x86_64 Linux

## Auto-Push

QCOW2 images can be automatically pushed to an OCI registry as artifacts:

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "uefi"
        filesystem "zfs"
        push-to "ghcr.io/myorg/omnios-image:latest"
    }

The QCOW2 file is wrapped as an OCI artifact with custom media types (`application/vnd.cloudnebula.qcow2.*`) and pushed to the registry. This allows distributing VM images through container registries.

## Deploying QCOW2 Images

QCOW2 images work with:

- QEMU/KVM: Direct use (`qemu-system-x86_64 -drive file=image.qcow2,format=qcow2`)
- libvirt/virt-manager: Import as existing disk
- Proxmox: Upload to storage, create VM from disk
- Cloud platforms: Convert to platform-specific format if needed

To convert to raw (for AWS, DigitalOcean, etc.):

    qemu-img convert -f qcow2 -O raw image.qcow2 image.raw

# OCI Container Images

OCI targets produce container images compatible with Docker, Podman, and any OCI-compliant runtime.

## Syntax

    target "container" kind="oci" {
        entrypoint command="/bin/sh"
        environment {
            set "PATH" "/usr/bin:/bin:/usr/sbin:/sbin"
            set "TZ" "UTC"
        }
    }

## How OCI Targets Work

Phase 2 for an OCI target:

1. Compress the Phase 1 rootfs into a gzip tar layer
2. Compute SHA256 digests for all blobs
3. Build OCI config JSON (entrypoint, environment, layer diff IDs)
4. Build OCI manifest JSON (media types, blob references, sizes)
5. Write an OCI Image Layout directory:

    output/container/
    ├── oci-layout          # {"imageLayoutVersion": "1.0.0"}
    ├── index.json          # Points to manifest
    └── blobs/
        └── sha256/
            ├── <manifest>  # OCI manifest JSON
            ├── <config>    # OCI config JSON
            └── <layer>     # Compressed rootfs layer

## Configuration

### Entrypoint

The command to run when the container starts:

    entrypoint command="/usr/sbin/sshd"

If omitted, no entrypoint is set and the container runtime's default applies.

### Environment Variables

    environment {
        set "PATH" "/usr/bin:/bin:/usr/sbin:/sbin"
        set "LANG" "C.UTF-8"
        set "TZ" "UTC"
    }

## Using the Output

### Load into Docker

    # From OCI Image Layout directory
    docker load < output/container/

    # Or use skopeo
    skopeo copy oci:output/container docker-daemon:myimage:latest

### Load into Podman

    podman load < output/container/

### Push to Registry

Use the `push` command or `push-to` in the target:

    forger push --image output/container/ --reference ghcr.io/myorg/myimage:latest

Or configure auto-push in the spec (see OCI Registry Push).

## OmniOS Containers

OmniOS in a container is useful for CI builds and lightweight services:

    metadata name="omnios-container" version="1.0.0"

    repositories {
        publisher name="omnios" origin="https://pkg.omnios.org/bloody/core/"
    }

    incorporation "entire"

    variants {
        set name="opensolaris.zone" value="nonglobal"
    }

    packages {
        package "/editor/vim"
        package "/web/curl"
    }

    target "container" kind="oci" {
        entrypoint command="/bin/bash"
        environment {
            set "PATH" "/usr/bin:/bin:/usr/sbin:/sbin"
        }
    }

Note: For containers, use `opensolaris.zone=nonglobal` to exclude kernel modules and hardware drivers that aren't needed in a container context.

# Tar Artifacts

Artifact targets produce a tar archive of the assembled rootfs. This is the simplest output format and serves as the building block for multi-stage pipelines.

## Syntax

    target "archive" kind="artifact" {
    }

## Use Cases

### Build Stage Caching

The primary use of artifacts is as intermediate outputs in a multi-stage pipeline. A parent spec produces an artifact that a child spec consumes:

    // base.kdl — produces artifact
    target "base-archive" kind="artifact" {
    }

    // disk.kdl — consumes artifact from base
    base "base.kdl"

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "uefi"
    }

### External Processing

Artifacts can be consumed by external tools for further processing:

- Import into a zone or container manually
- Feed into another build system
- Archive for distribution

### Rootfs Inspection

Build an artifact to inspect what the rootfs looks like without committing to a disk image:

    forger build --spec my-image.kdl --target archive
    tar -tzf output/archive.tar.gz | head -50

## Output

The artifact is written to the output directory as a tar archive (typically gzip-compressed). The archive contains the full rootfs tree starting from `/`.

# OCI Registry Push

Forger can push built images directly to OCI-compliant registries, including GitHub Container Registry (GHCR), Docker Hub, and self-hosted registries.

## Auto-Push (Target Configuration)

Set `push-to` on a target to automatically push after a successful build:

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "uefi"
        push-to "ghcr.io/myorg/omnios-image:latest"
    }

    target "container" kind="oci" {
        push-to "ghcr.io/myorg/omnios-container:latest"
    }

Skip the push with:

    forger build --spec my-image.kdl --skip-push

## Manual Push

Push a previously built image:

    # Push OCI Image Layout
    forger push --image output/container/ --reference ghcr.io/myorg/myimage:latest

    # Push QCOW2 as OCI artifact
    forger push --image output/vm.qcow2 --reference ghcr.io/myorg/myvm:latest --artifact

### Options

Flag | Description
--- | ---
`--image <PATH>` | Path to OCI Image Layout directory or QCOW2 file
`--reference <REF>` | Registry reference (e.g., `ghcr.io/org/image:tag`)
`--artifact` | Push QCOW2 as OCI artifact (custom media types)
`--auth-file <PATH>` | JSON auth file for registry authentication

## Authentication

### GitHub Container Registry (GHCR)

Forger automatically uses the `GITHUB_TOKEN` environment variable when pushing to `ghcr.io`:

    export GITHUB_TOKEN=ghp_...
    forger build --spec my-image.kdl

In GitHub Actions, the token is available automatically:

    - name: Build and push
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: forger build --spec images/omnios-rust-ci.kdl

### Auth File

For other registries, provide a JSON auth file:

    {
        "username": "myuser",
        "password": "mypassword"
    }

Or with a token:

    {
        "token": "my-registry-token"
    }

    forger push --image output/container/ \
      --reference registry.example.com/myimage:latest \
      --auth-file auth.json

### Anonymous Push

Local registries (localhost, 127.0.0.1) are accessed without authentication over HTTP (insecure mode).

## QCOW2 as OCI Artifact

When pushing QCOW2 images with `--artifact`, Forger uses custom OCI media types:

- Config: `application/vnd.cloudnebula.qcow2.config.v1+json`
- Layer: `application/vnd.cloudnebula.qcow2.layer.v1`

This allows distributing VM disk images through container registries alongside container images, using a unified registry infrastructure.

## Pulling QCOW2 Artifacts

QCOW2 artifacts pushed to a registry can be pulled back as builder images or for deployment:

    builder {
        image "oci://ghcr.io/myorg/omnios-builder:latest"
    }

Forger resolves `oci://` references by pulling from the registry.

# Design Overview

Refraction Forger is designed around three principles: declarative specs, two-phase execution, and distro abstraction.

## High-Level Architecture

                        ┌─────────────┐
                        │  KDL Spec   │
                        └──────┬──────┘
                               │
                        ┌──────▼──────┐
                        │ spec-parser │  Parse → Resolve → Filter
                        └──────┬──────┘
                               │
                        ┌──────▼──────┐
                        │   forger    │  CLI routing
                        └──────┬──────┘
                               │
                  ┌────────────┼────────────┐
                  │                         │
           ┌──────▼──────┐          ┌──────▼──────┐
           │ Local Build  │          │forge-builder│
           └──────┬──────┘          └──────┬──────┘
                  │                         │
                  │                  Ephemeral VM
                  │                  ┌─────────────┐
                  │                  │  SSH + SCP   │
                  │                  └──────┬──────┘
                  │                         │
           ┌──────▼─────────────────────────▼──────┐
           │            forge-engine                │
           │  ┌──────────┐     ┌─────────────────┐ │
           │  │ Phase 1   │────▶│    Phase 2      │ │
           │  │ Rootfs    │     │ QCOW2/OCI/Tar  │ │
           │  └──────────┘     └────────┬────────┘ │
           └────────────────────────────┼──────────┘
                                        │
                                 ┌──────▼──────┐
                                 │  forge-oci  │  Registry push
                                 └─────────────┘

## Key Design Decisions

### Declarative Over Imperative

The old tools used shell scripts to orchestrate builds — ordering mattered, error handling was manual, and reuse required copy-paste. Forger uses a declarative spec where you describe what the image should contain, and the engine handles how.

### Direct Assembly Over Installation

Packer boots an ISO, types keystrokes into a virtual console, and waits for an installer to finish. Forger calls the package manager directly to assemble a rootfs, skipping the installer entirely. This is faster and more reliable.

### Host Independence

The old `omnios-image-builder` required an illumos host with ZFS and pfexec. Forger can build from any platform by spinning up an ephemeral builder VM. The build environment is part of the spec, not a prerequisite.

### OCI as Distribution Channel

Instead of custom upload scripts for each cloud provider, Forger uses OCI registries as a universal distribution mechanism. VM images, container images, and tar artifacts all flow through the same registry infrastructure.

## Error Handling

Forger uses miette for rich error diagnostics. When something fails, you get:

- The error message
- Context about what was being attempted
- Suggestions for how to fix it
- Source location where the error originated

This is deliberate — image building involves many external tools and system operations, and clear error messages are essential for debugging.

# Two-Phase Build Pipeline

Every Forger build follows a two-phase architecture. Understanding this split is key to building efficient images and debugging build failures.

## Phase 1: Rootfs Assembly

Phase 1 creates a populated filesystem tree in a staging directory. The work is entirely distro-specific:

### OmniOS Path

1. `pkg image-create` — Initialize an IPS package image
2. `pkg set-publisher` — Configure each publisher (name + origin URL)
3. `pkg change-variant` — Set zone variant (`global` or `nonglobal`)
4. `pkg approve-ca-cert` — Trust CA certificates for signed packages
5. `pkg install` — Install all specified packages
6. Apply customizations (create users)
7. Apply overlays (files, directories, symlinks, shadow passwords, devfsadm)

### Ubuntu Path

1. `debootstrap` — Bootstrap a minimal Debian/Ubuntu rootfs
2. Write `/etc/apt/sources.list` from repository configuration
3. `apt update` — Refresh package lists
4. `apt install` — Install all specified packages
5. Apply customizations and overlays (same as OmniOS)

### Output

Phase 1 produces a staging directory containing a complete rootfs. This directory is consumed by Phase 2.

## Phase 2: Target Production

Phase 2 takes the rootfs from Phase 1 and packages it into the requested output format. This logic is shared across all distros.

### QCOW2 Path

    Create raw disk file (specified size)
      → Attach as loopback device
        → ZFS: create pool → create BE dataset → mount
        → ext4: partition → format → mount
          → Copy rootfs into mounted filesystem
            → Install bootloader (UEFI/GRUB)
              → ZFS: set bootfs → unmount → export pool
              → ext4: unmount
                → Detach loopback
                  → qemu-img convert raw → qcow2

The ZFS path creates a unique pool name during build (e.g., `forgebuild_12345`) and renames to `rpool` after export. This prevents conflicts with existing pools on the build host.

### OCI Path

    Compress rootfs → tar.gz layer
      → Compute SHA256 digest
        → Build OCI config JSON (entrypoint, env)
          → Build OCI manifest JSON
            → Write OCI Image Layout directory

### Artifact Path

    Create tar archive from rootfs

## Why Two Phases?

The split serves several purposes:

1. Separation of concerns: Distro-specific logic (Phase 1) doesn't leak into format-specific logic (Phase 2)
2. Multiple targets: One Phase 1 rootfs can produce QCOW2, OCI, and artifact targets without rebuilding
3. Caching boundary: The base/child relationship creates a cache point between Phase 1 and Phase 2
4. Testability: Each phase can be tested independently

## Error Recovery

If Phase 2 fails (e.g., disk too small, bootloader installation error), Forger cleans up:

- Unmounts filesystems
- Exports ZFS pools
- Detaches loopback devices
- Removes temporary files

Cleanup runs even on failure, preventing resource leaks on the build host.

# Crate Structure

Forger is a Cargo workspace with five crates, each with a clear responsibility.

## Workspace Layout

    crates/
    ├── forger/          CLI entry point
    ├── spec-parser/     KDL parsing and resolution
    ├── forge-engine/    Build execution (Phase 1 + Phase 2)
    ├── forge-builder/   Remote VM builds
    └── forge-oci/       OCI image and registry operations

## forger (CLI)

The binary crate. Defines five subcommands via clap:

Command | Description
--- | ---
`build` | Build image from spec
`validate` | Parse and check a spec
`inspect` | Show resolved, profile-filtered spec
`push` | Push artifact to OCI registry
`targets` | List targets in a spec

This crate is thin — it parses CLI arguments and delegates to the library crates.

## spec-parser

Handles everything related to KDL spec files:

- Parsing: Uses the `knuffel` crate to deserialize KDL into Rust structs (`ImageSpec`, `Target`, `Package`, etc.)
- Resolution: Recursively resolves `base` and `include` references, merging specs while detecting circular dependencies
- Profile filtering: Removes conditional blocks that don't match active profiles

Key types:

- `ImageSpec` — Root spec structure
- `DistroFamily` — Enum discriminating OmniOS vs Ubuntu
- `Target` / `TargetKind` — Output target definitions
- `Overlays` — File operations (file, ensure-dir, ensure-symlink, shadow, devfsadm, remove-files)

## forge-engine

The build engine implementing both phases:

- Phase 1 (`phase1/mod.rs`): Distro-specific rootfs assembly
  - IPS operations for OmniOS
  - debootstrap + APT for Ubuntu
  - Overlay application (shared across distros)
- Phase 2 (`phase2/mod.rs`): Target production
  - `qcow2_zfs.rs`: ZFS pool creation, BE management
  - `qcow2_ext4.rs`: Partitioning, ext4 formatting
  - OCI image layout creation
  - Tar artifact packaging

Key abstraction: `ToolRunner` trait — wraps all external tool execution (`pkg`, `apt`, `qemu-img`, `zfs`, `parted`, etc.) through a single interface. The real implementation (`SystemToolRunner`) uses `tokio::process::Command`. This trait enables testing without root access.

## forge-builder

Manages remote builds via ephemeral VMs:

1. Resolve builder image (OCI ref, URL, or path)
2. Generate ephemeral SSH keypair
3. Create cloud-init config
4. Start QEMU via `vm-manager` crate (user-mode networking)
5. SSH connect with retry (up to 120s)
6. Transfer forger binary + spec + files via SCP
7. Execute remote build via SSH
8. Download artifacts
9. Destroy VM (guaranteed cleanup)

Uses the external `vm-manager` crate for hypervisor abstraction (`RouterHypervisor` auto-detects QEMU/bhyve).

## forge-oci

OCI-specific operations:

- tar_layer: Compress rootfs into gzip layer
- manifest: Build OCI config and manifest JSON
- layout: Write OCI Image Layout directory structure
- artifact: Package QCOW2 as OCI artifact with custom media types
- registry: Push to OCI registries (token auth, basic auth, anonymous)
- AuthConfig: GHCR token auto-detection, auth file parsing

## Dependency Flow

    forger
      ├── spec-parser
      ├── forge-engine
      │     └── spec-parser
      ├── forge-builder
      │     ├── spec-parser
      │     └── vm-manager (external)
      └── forge-oci

All crates are async (tokio) and use miette for error diagnostics.

## Key External Dependencies

Category | Crate | Purpose
--- | --- | ---
KDL parsing | `knuffel` 3.2 | Spec deserialization
CLI | `clap` 4.5 | Argument parsing
Async | `tokio` 1 | Runtime, process, fs
OCI | `oci-spec`, `oci-client` | Image spec, registry client
SSH | `ssh2` | Remote builder communication
Errors | `miette`, `thiserror` | Rich diagnostics
VM | `vm-manager` (path dep) | Hypervisor abstraction

# Remote Builder VMs

When the build host doesn't match the target OS, Forger delegates to an ephemeral builder VM. This chapter explains the internals.

## Lifecycle

    1. Image Resolution
       ├── OCI reference → pull from registry
       ├── URL → download
       └── Local path → use directly

    2. Cloud-Init Generation
       ├── Generate ephemeral Ed25519 SSH keypair
       ├── Create user-data: builder user, SSH key, passwordless sudo
       └── Create cloud-init ISO

    3. VM Creation
       ├── Create VmSpec (CPU, memory, disk, network)
       ├── Network: user-mode (SLIRP) — no root required
       ├── Disk: overlay on builder image (20GB default)
       └── Hypervisor: auto-detect via vm-manager

    4. Boot & Connect
       ├── Start VM via hypervisor
       └── SSH retry loop (up to 120 seconds)

    5. Transfer
       ├── Upload forger binary via SCP
       ├── Upload spec files via SCP
       └── Upload overlay files via SCP

    6. Build
       └── Execute forger build command via SSH

    7. Download
       └── Retrieve artifacts via SCP

    8. Teardown (guaranteed, even on failure)
       └── Destroy VM via hypervisor

## User-Mode Networking

Builder VMs use QEMU's user-mode networking (SLIRP). This means:

- No root access needed on the host
- No bridge interfaces to configure
- No firewall rules to manage
- Guest can access the internet through NAT
- Host communicates with guest via port forwarding

DNS is automatically configured if needed.

## Security Model

- SSH keys are ephemeral — generated per build, discarded after
- The builder user has passwordless sudo but only exists for the build duration
- The VM is destroyed after every build, even on failure
- No persistent state from previous builds leaks into new ones

## Custom Builder Images

Builder images are QCOW2 files with cloud-init support. To create your own:

1. Build a base image with Forger (or any tool)
2. Ensure `cloud-init` is installed and enabled
3. Ensure SSH server is running
4. Include all build dependencies (`pkg`, `qemu-img`, `zfs`, etc.)
5. Push to an OCI registry or host as a downloadable file

### Minimum Requirements for a Builder Image

- Cloud-init (for SSH key injection and user setup)
- SSH server
- The target distro's package manager
- `qemu-img` (for QCOW2 conversion)
- Filesystem tools (`zfs`/`zpool` for ZFS, `parted`/`mkfs.ext4` for ext4)
- Bootloader tools (GRUB, UEFI support)

## Disk Overlay

The builder VM uses a disk overlay (copy-on-write layer) on top of the builder image. This means:

- The original builder image is never modified
- The overlay provides additional working space (20GB by default)
- Multiple builds can run concurrently from the same builder image

## Troubleshooting

### VM Fails to Boot

- Check that the builder image has cloud-init enabled
- Verify QEMU is installed and accessible
- Check system resources (enough RAM and disk for the VM)

### SSH Connection Times Out

- The 120-second timeout may be insufficient for slow storage
- Check that the builder image's SSH server starts on boot
- Verify user-mode networking isn't blocked by firewall

### Build Fails Inside VM

- The error output from the remote build is captured and displayed
- Check that the builder image has all required tools installed
- For disk space issues, increase the disk overlay size in the builder config

# From omnios-image-builder

This guide helps you migrate from the shell-based `omnios-image-builder` to Forger.

## Architecture Comparison

omnios-image-builder | Forger
--- | ---
Shell scripts (`strap.sh`, `aws.sh`, etc.) | Single `forger` binary
JSON templates with step arrays | KDL specs with declarative blocks
ZFS snapshots for caching | `base` directive for caching
Requires illumos host with ZFS + pfexec | Any host (remote builder VM)
Manual pipeline (01-strap → 02-image → 03-archive → final) | Implicit pipeline via `base`/`include`

## Mapping Concepts

### JSON Steps → KDL Blocks

The old JSON templates used a `steps` array with typed objects:

    {
      "steps": [
        { "t": "pkg_image_create", "publisher": "omnios", "uri": "https://..." },
        { "t": "pkg_install", "pkgs": ["entire"] },
        { "t": "pkg_change_variant", "variant": "opensolaris.zone", "value": "global" }
      ]
    }

In Forger, these become declarative blocks:

    repositories {
        publisher name="omnios" origin="https://pkg.omnios.org/bloody/core/"
    }

    incorporation "entire"

    variants {
        set name="opensolaris.zone" value="global"
    }

    packages {
        package "/editor/vim"
    }

### Step Type Mapping

JSON Step | KDL Equivalent
--- | ---
`pkg_image_create` | `repositories { publisher ... }` (implicit)
`pkg_set_publisher` | `repositories { publisher ... }`
`pkg_install` | `packages { package "..." }`
`pkg_change_variant` | `variants { set ... }`
`pkg_approve_ca_cert` | `certificates { ca ... }`
`pkg_uninstall` | Not needed (don't install unwanted packages)
`pkg_purge_history` | Handled automatically
`pkg_set_property` | Handled automatically
`ensure_file` | `overlays { file ... }`
`ensure_dir` | `overlays { ensure-dir ... }`
`ensure_symlink` | `overlays { ensure-symlink ... }`
`remove_files` | `overlays { remove-files ... }`
`shadow` | `overlays { shadow ... }`
`devfsadm` | `overlays { devfsadm }`
`seed_smf` | Handled automatically
`include` | `include "file.kdl"`
`create_be` | Handled automatically (QCOW2 target)
`unpack_tar` | Handled automatically (`base` directive)
`make_bootable` | `target ... { bootloader "uefi" }`
`pack_tar` | `target "..." kind="artifact"`

### Three-Stage Pipeline → Base + Child

Old approach (three JSON files + shell orchestration):

    01-strap.json  → pkg install entire → snapshot "strap"
    02-image.json  → pkg install extras → snapshot "image"
    03-archive.json → pack_tar → omnios-bloody.tar.gz
    aws.json       → unpack_tar + make_bootable → raw disk

New approach (two KDL files):

    // omnios-base.kdl
    repositories { publisher name="omnios" ... }
    incorporation "entire"
    packages { package "/editor/vim" ... }

    // omnios-disk.kdl
    base "omnios-base.kdl"
    include "devfs.kdl"
    include "common.kdl"

    packages { package "/system/cloud-init" }
    overlays { file destination="/boot/conf.d/console" ... }

    target "vm" kind="qcow2" { ... }

The `base` directive replaces the ZFS snapshot pipeline. The parent's output is cached, and the child builds on top.

### Pool Configuration

Old (`pool` in JSON):

    {
      "pool": { "name": "rpool", "ashift": 12, "uefi": true, "size": 2000 }
    }

New (in target block):

    target "vm" kind="qcow2" {
        disk-size "2000M"
        bootloader "uefi"
        filesystem "zfs"
        pool {
            property name="ashift" value="12"
        }
    }

### Output Formats

Old: Raw disk images only (convert to QCOW2/AMI externally)

New: QCOW2 natively, plus OCI container images and tar artifacts

### Cloud Provider Scripts

The old `aws.sh`, `digitalocean.sh`, and `smartosbhyve.sh` scripts are replaced by profiles or separate specs:

    // Cloud-provider-specific overlays via profiles
    overlays if="aws" {
        file destination="/boot/conf.d/console" source="files/boot_console.aws"
    }

    overlays if="digitalocean" {
        file destination="/boot/conf.d/console" source="files/boot_console.do"
    }

## Migration Checklist

1. Identify your JSON templates and their step sequences
2. Map each step to the equivalent KDL block (see table above)
3. Split the three-stage pipeline into base + child specs
4. Move overlay files to a `files/` directory relative to the specs
5. Replace `setup.sh` dependency management with `forger` binary installation
6. Test with `forger validate` and `forger inspect` before building

# From Packer (oi-packer)

This guide helps you migrate from HashiCorp Packer-based illumos image building to Forger.

## Why Migrate?

Packer | Forger
--- | ---
Boots ISO, types keystrokes, waits | Direct package manager assembly
20+ minute builds | Minutes
No native caching | Multi-stage caching via `base`
Shell provisioners (order-sensitive) | Declarative KDL spec
Vagrant boxes, raw disks | QCOW2, OCI, tar artifacts
External upload scripts | Built-in OCI registry push

## Mapping Concepts

### Packer Source → Not Needed

Packer sources define the VM that runs the installer:

    source "qemu" "oi-hipster" {
      iso_url          = "https://dlc.openindiana.org/isos/..."
      iso_checksum     = "sha256:..."
      disk_size        = "51200M"
      memory           = 4096
      accelerator      = "kvm"
    }

Forger doesn't boot an ISO. It calls the package manager directly, so there's no installer VM to configure.

### Boot Command → Not Needed

Packer's boot command types keystrokes into the installer:

    boot_command = [
      "<wait10><wait10><wait10>",
      "47<enter><wait>",    // Select installation
      "7<enter><wait>",     // Select bootloader
      // ... dozens more keystrokes
    ]

Forger skips the installer entirely. Package installation happens through direct `pkg` or `apt` calls.

### Shell Provisioners → KDL Blocks

Packer (shell scripts that SSH into the VM):

    provisioner "shell" {
      scripts = [
        "scripts/update.sh",      // pkg update
        "scripts/vagrant.sh",     // Create vagrant user, install SSH keys
        "scripts/cleanup.sh",     // Remove SSH host keys, clear logs
      ]
    }

Forger (declarative):

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
    }

    customization {
        user "deploy"
    }

    overlays {
        file destination="/home/deploy/.ssh/authorized_keys" source="files/authorized_keys"
        shadow username="deploy" password="$5$..."
    }

### Post-Processors → Targets

Packer (Vagrant box post-processor):

    post-processor "vagrant" {
      compression_level = 9
      output            = "OI-hipster-{{.Provider}}.box"
    }

Forger (target block):

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "uefi"
        push-to "ghcr.io/myorg/my-image:latest"
    }

Forger doesn't produce Vagrant boxes directly. Instead, it produces QCOW2 images that work with QEMU/KVM, Proxmox, and other hypervisors. OCI container images are also available.

### Variables → Profiles

Packer (HCL variables):

    variable "build_version" { default = "20240426" }

Forger (profiles for variants):

    packages if="dev" {
        package "/diagnostic/top"
    }

## Full Migration Example

### Packer Template (before)

    source "qemu" "oi-hipster" {
      iso_url     = "https://dlc.openindiana.org/isos/hipster/20240426/OI-hipster-text-20240426.iso"
      disk_size   = "51200M"
      memory      = 4096
      format      = "qcow2"
      boot_command = ["<wait30>", "47<enter>", /* ... 30 more lines ... */]
    }

    build {
      sources = ["source.qemu.oi-hipster"]

      provisioner "shell" { scripts = ["scripts/update.sh"] }
      provisioner "shell" { scripts = ["scripts/vagrant.sh"] }
      provisioner "shell" { scripts = ["scripts/cleanup.sh"] }

      post-processor "vagrant" {
        output = "OI-hipster-{{.Provider}}.box"
      }
    }

### Forger Spec (after)

    metadata name="oi-hipster" version="1.0.0" description="OpenIndiana Hipster image"

    repositories {
        publisher name="openindiana.org" origin="http://pkg.openindiana.org/hipster/"
    }

    incorporation "entire"

    variants {
        set name="opensolaris.zone" value="global"
    }

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
        package "/network/rsync"
        package "/driver/network/vioif"
        package "/driver/storage/vioblk"
    }

    customization {
        user "deploy"
    }

    overlays {
        file destination="/home/deploy/.ssh/authorized_keys" source="files/authorized_keys"
    }

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "uefi"
        filesystem "zfs"
        push-to "ghcr.io/myorg/oi-hipster:latest"
    }

Build time drops from 20+ minutes (ISO boot + install + provisioning) to a few minutes (direct package installation).

## Migration Checklist

1. Identify packages installed by your shell provisioners
2. List files copied or modified by provisioners
3. Translate to KDL `packages`, `overlays`, and `customization` blocks
4. Replace ISO-based installation with `repositories` + `incorporation`
5. Replace Vagrant post-processor with QCOW2 target + OCI push
6. Test with `forger validate` and `forger inspect`
7. Build and verify the image boots correctly

# CLI Reference

## forger build

Build an image from a spec file.

    forger build [OPTIONS] --spec <PATH>

Option | Description
--- | ---
`--spec <PATH>` | Path to the KDL spec file (required)
`--target <NAME>` | Build only this target (default: all)
`--profile <PROFILE>` | Activate a profile (repeatable)
`--output-dir <PATH>` | Output directory (default: `./output`)
`--local` | Force local build (skip builder VM)
`--use-builder` | Force build inside a builder VM
`--skip-push` | Don't push to registry after build
`--builder-image <SPEC>` | Override builder VM image

### Examples

    # Build all targets
    forger build --spec images/omnios-bloody-disk.kdl

    # Build specific target with profile
    forger build --spec images/omnios-bloody-disk.kdl --target vm --profile build

    # Force local build, custom output directory
    forger build --spec images/omnios-bloody-disk.kdl --local --output-dir /tmp/images

    # Build with custom builder image
    forger build --spec images/ubuntu-rust-ci.kdl --builder-image /path/to/builder.qcow2

## forger validate

Parse and check a spec file for errors.

    forger validate --spec <PATH>

Option | Description
--- | ---
`--spec <PATH>` | Path to the KDL spec file (required)

Checks:

- KDL syntax
- Schema conformance
- Include/base resolution
- Circular dependency detection

### Example

    forger validate --spec images/omnios-bloody-disk.kdl

## forger inspect

Parse, resolve, and apply profiles, then display the resolved spec.

    forger inspect [OPTIONS] --spec <PATH>

Option | Description
--- | ---
`--spec <PATH>` | Path to the KDL spec file (required)
`--profile <PROFILE>` | Activate a profile (repeatable)

### Examples

    # Inspect without profiles
    forger inspect --spec images/omnios-bloody-disk.kdl

    # Inspect with build profile to see what's included
    forger inspect --spec images/omnios-bloody-disk.kdl --profile build

## forger push

Push a built artifact to an OCI registry.

    forger push [OPTIONS] --image <PATH> --reference <REF>

Option | Description
--- | ---
`--image <PATH>` | Path to OCI Image Layout directory or QCOW2 file
`--reference <REF>` | Registry reference (e.g., `ghcr.io/org/image:tag`)
`--auth-file <PATH>` | JSON auth file (optional)
`--artifact` | Push QCOW2 as OCI artifact

### Examples

    # Push OCI image
    forger push --image output/container/ --reference ghcr.io/myorg/myimage:latest

    # Push QCOW2 as artifact
    forger push --image output/vm.qcow2 --reference ghcr.io/myorg/myvm:latest --artifact

    # Push with auth file
    forger push --image output/container/ \
      --reference registry.example.com/myimage:latest \
      --auth-file auth.json

## forger targets

List targets defined in a spec file.

    forger targets --spec <PATH>

Option | Description
--- | ---
`--spec <PATH>` | Path to the KDL spec file (required)

### Example

    forger targets --spec images/omnios-bloody-disk.kdl

Output:

    vm (qcow2)

## Environment Variables

Variable | Description
--- | ---
`GITHUB_TOKEN` | Used for GHCR authentication when pushing
`RUST_LOG` | Logging level (e.g., `info`, `debug`, `trace`)

# KDL Spec Reference

Complete reference for all KDL spec nodes and properties.

## Top-Level Nodes

### metadata

    metadata name="<string>" version="<string>" description="<string>"

Property | Type | Required | Description
--- | --- | --- | ---
`name` | string | Yes | Image name
`version` | string | Yes | Semantic version
`description` | string | No | Human-readable description

### distro

    distro "<string>"

Distro identifier. Determines the build path (IPS vs APT). Omit for OmniOS (default).

Value | Family
--- | ---
(omitted) | OmniOS
`"ubuntu-22.04"` | Ubuntu

### base

    base "<path>"

Parent spec file. Creates a build-stage boundary for caching. Path is relative to the current spec file.

### include

    include "<path>"

Sibling spec file to merge. Imports shared steps. Path is relative to the current spec file.

### incorporation

    incorporation "<string>"

IPS incorporation constraint (OmniOS only). Typically `"entire"`.

## Block Nodes

### repositories

    repositories {
        publisher name="<string>" origin="<url>"
        apt-mirror "<url>" suite="<string>" components="<string>"
    }

publisher (IPS):

Property | Type | Required
--- | --- | ---
`name` | string | Yes
`origin` | string (URL) | Yes

apt-mirror (APT):

Argument | Type | Required
--- | --- | ---
(positional) | string (URL) | Yes
`suite` | string | Yes
`components` | string (space-separated) | Yes

### certificates

    certificates {
        ca publisher="<string>" certfile="<path>"
    }

Property | Type | Required
--- | --- | ---
`publisher` | string | Yes
`certfile` | string (path) | Yes

### variants

    variants {
        set name="<string>" value="<string>"
    }

IPS variant settings. Common: `name="opensolaris.zone" value="global"`.

### packages

    packages if="<profile>" {
        package "<name>"
    }

Property | Type | Required
--- | --- | ---
`if` | string | No

The `if` property enables profile filtering. Omit for unconditional packages.

### overlays

    overlays if="<profile>" {
        file destination="<path>" source="<path>"
        ensure-dir "<path>" owner="<string>" group="<string>" mode="<string>"
        ensure-symlink "<path>" target="<path>"
        remove-files "<path>" "<path>" ...
        shadow username="<string>" password="<string>"
        devfsadm
    }

file:

Property | Type | Required
--- | --- | ---
`destination` | string (absolute path) | Yes
`source` | string (relative path) | Yes

ensure-dir:

Argument/Property | Type | Required
--- | --- | ---
(positional) | string (path) | Yes
`owner` | string | No
`group` | string | No
`mode` | string (octal) | No

ensure-symlink:

Argument/Property | Type | Required
--- | --- | ---
(positional) | string (link path) | Yes
`target` | string (target path) | Yes

remove-files: One or more path arguments.

shadow:

Property | Type | Required
--- | --- | ---
`username` | string | Yes
`password` | string (crypt hash) | Yes

devfsadm: No arguments.

### customization

    customization if="<profile>" {
        user "<username>"
    }

### builder

    builder {
        image "<string>"
        vcpus <integer>
        memory <integer>
        disk <integer>
    }

Property | Type | Required | Default
--- | --- | --- | ---
`image` | string | No | Distro default
`vcpus` | integer | No | Host-dependent
`memory` | integer (MB) | No | Host-dependent
`disk` | integer (GB) | No | 20

### target

    target "<name>" kind="<kind>" {
        // kind-specific properties
    }

Property | Type | Required
--- | --- | ---
(positional) | string (name) | Yes
`kind` | string | Yes

kind="qcow2":

Property | Type | Required
--- | --- | ---
`disk-size` | string (e.g., "8G") | Yes
`bootloader` | string | Yes
`filesystem` | string | No
`push-to` | string (registry ref) | No
`pool` children | property nodes | No

kind="oci":

Property | Type | Required
--- | --- | ---
`entrypoint` | node with `command` | No
`environment` children | `set` nodes | No
`push-to` | string | No

kind="artifact": No additional properties.

# Example Specs

Complete, production-ready examples from the `images/` directory.

## OmniOS Bloody Base

A reusable base spec for OmniOS bloody images. This is typically used as a `base` for other specs.

    metadata name="omnios-bloody-base" version="1.0.0" description="Base OmniOS bloody image"

    repositories {
        publisher name="omnios" origin="https://pkg.omnios.org/bloody/core/"
        publisher name="extra.omnios" origin="https://pkg.omnios.org/bloody/extra/"
    }

    incorporation "entire"

    certificates {
        ca publisher="omnios" certfile="omniosce-ca.cert.pem"
    }

    variants {
        set name="opensolaris.zone" value="global"
    }

    packages {
        package "/editor/vim"
        package "/network/openssh-server"
        package "/network/rsync"
        package "/service/network/ntpsec"
        package "/web/curl"
        package "/web/wget"
        package "/system/cloud-init"
    }

    packages if="build" {
        package "/developer/build-essential"
        package "/ooce/developer/omnios-build-tools"
        package "/developer/build/gnu-make"
    }

## OmniOS Bloody Bootable Disk

A bootable QCOW2 VM image built on top of the base. Includes device filesystem setup, network configuration, and boot console settings.

    metadata name="omnios-bloody-disk" version="1.0.0" description="OmniOS bloody bootable disk"

    base "omnios-bloody-base.kdl"
    include "devfs.kdl"
    include "common.kdl"

    packages {
        package "/system/cloud-init"
        package "/driver/virtio/viorand"
        package "/driver/virtio/vioif"
        package "/driver/virtio/vioblk"
        package "/driver/virtio/vio9p"
        package "/driver/virtio/vioscsi"
    }

    overlays {
        file destination="/boot/conf.d/console" source="files/boot_console.115200"
        file destination="/etc/ttydefs" source="files/ttydefs.115200"
        file destination="/etc/default/init" source="files/default_init.utc"
        shadow username="root" password="$5$rounds=10000$..."
    }

    target "vm" kind="qcow2" {
        disk-size "2000M"
        bootloader "uefi"
        filesystem "zfs"
        pool {
            property name="ashift" value="12"
        }
    }

## OmniOS Rust CI Image

An OmniOS image with the Rust toolchain for continuous integration.

    metadata name="omnios-rust-ci" version="1.0.0" description="OmniOS Rust CI image"

    base "omnios-bloody-base.kdl"
    include "devfs.kdl"
    include "common.kdl"

    packages {
        package "/ooce/developer/rust"
        package "/developer/versioning/git"
        package "/system/cloud-init"
        package "/driver/virtio/viorand"
        package "/driver/virtio/vioif"
        package "/driver/virtio/vioblk"
        package "/driver/virtio/vio9p"
        package "/driver/virtio/vioscsi"
    }

    builder {
        image "oci://ghcr.io/cloudnebulaproject/omnios-builder:latest"
        vcpus 4
        memory 4096
        disk 20
    }

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "uefi"
        filesystem "zfs"
        pool {
            property name="ashift" value="12"
        }
        push-to "ghcr.io/cloudnebulaproject/omnios-rust:latest"
    }

## Ubuntu Rust CI Image

An Ubuntu 22.04 image with the Rust toolchain.

    metadata name="ubuntu-rust-ci" version="1.0.0" description="Ubuntu 22.04 Rust CI image"

    distro "ubuntu-22.04"

    repositories {
        apt-mirror "http://archive.ubuntu.com/ubuntu" suite="jammy" components="main universe"
    }

    packages {
        package "build-essential"
        package "rustc"
        package "cargo"
        package "git"
        package "curl"
        package "openssh-server"
        package "cloud-init"
        package "linux-image-generic"
        package "grub-efi-amd64-bin"
        package "libssl-dev"
        package "pkg-config"
    }

    customization {
        user "ci"
    }

    builder {
        image "oci://ghcr.io/cloudnebulaproject/ubuntu-builder:latest"
        vcpus 4
        memory 4096
        disk 20
    }

    target "vm" kind="qcow2" {
        disk-size "8G"
        bootloader "grub-efi-amd64-bin"
        filesystem "ext4"
        push-to "ghcr.io/cloudnebulaproject/ubuntu-rust:latest"
    }

## Device Filesystem Include

Standard device node setup for bootable illumos images.

    overlays {
        devfsadm

        remove-files "/dev/dsk" "/dev/rdsk" "/dev/cfg" "/dev/usb"

        ensure-dir "/dev/cfg" owner="root" group="root" mode="755"
        ensure-dir "/dev/dsk" owner="root" group="root" mode="755"
        ensure-dir "/dev/rdsk" owner="root" group="root" mode="755"
        ensure-dir "/dev/usb" owner="root" group="root" mode="755"

        ensure-symlink "/dev/msglog" target="../devices/pseudo/log@0:msglog"
    }

## Common System Configuration Include

Standard network and SMF profile setup for illumos images.

    overlays {
        ensure-symlink "/etc/svc/profile/generic.xml" target="generic_limited_net.xml"
        ensure-symlink "/etc/svc/profile/inetd_services.xml" target="inetd_generic.xml"
        ensure-symlink "/etc/svc/profile/platform.xml" target="platform_none.xml"
        ensure-symlink "/etc/svc/profile/name_service.xml" target="ns_dns.xml"

        file destination="/etc/inet/hosts" source="files/etc/hosts"
        file destination="/etc/nodename" source="files/etc/nodename"
        file destination="/etc/resolv.conf" source="files/etc/resolv.conf"
        ensure-symlink "/etc/nsswitch.conf" target="/etc/nsswitch.dns"
    }