Solstice CI

Build your code on multiple operating systems easily — with first‑class support for Illumos and Linux, and a simple, observable pipeline.

Solstice CI is a Rust workspace that provides:

• Orchestration to provision per‑job virtual machines (libvirt/KVM) (bHyve)
• A simple workflow format (KDL) and runner
• Integration layers for forges (Forgejo/Gitea, GitHub) based on messaging (RabbitMQ)
• Shared telemetry and utilities

This README follows best practices inspired by awesome-readme. It focuses on how to get productive locally, what you need to run it, and how to contribute.

Table of Contents

• Overview
• Features
• OS and Tooling Requirements
• Quickstart: Local Development
• Running the Services
• Configuration (Environment Variables)
• Testing
• Troubleshooting
• Contributing (Codeberg Pull Requests)
• License

Overview

Solstice CI is a modular CI system:

• crates/orchestrator — schedules jobs and provisions VMs per job
• crates/forge-integration — receives webhooks and enqueues job requests
• crates/workflow-runner — executes a KDL workflow inside a VM/agent
• crates/common — shared types, telemetry, and messaging helpers
• crates/ciadm and crates/cidev — small CLIs to aid development and admin tasks

Features

• Per-job VMs via libvirt/KVM with image prefetch and capacity controls
• RabbitMQ for job request queueing (durable, backpressure via prefetch)
• gRPC/HTTP components and structured telemetry (tracing + OTLP)
• Simple development flows with cargo and docker-compose

OS and Tooling Requirements

Minimum supported host OS for end-to-end local runs:

• Linux x86_64 (recommended): required for libvirt/KVM hypervisor path
• macOS / Windows: supported for building, running non-hypervisor services, and exercising CLIs; full VM orchestration is Linux-only currently

Required tools:

• Rust (stable, edition 2024). Install via rustup: https://rustup.rs
• Cargo (comes with rustup)
• Docker or Podman for running RabbitMQ locally (docker-compose file provided)
• Git

Optional but recommended:

• Postgres (DATABASE_URL). Some binaries accept a database URL; current snapshots may not require a live DB for basic dev loops
• An OpenTelemetry collector (OTLP) for tracing (or rely on stderr logs)

Hardware hints for Linux/local VM testing:

• CPU virtualization enabled (VT-x/AMD-V)
• 8 GB RAM+ recommended for running VMs alongside tooling

Quickstart: Local Development

1. Clone the repo

• git clone https://codeberg.org/your-namespace/solstice-ci.git
• cd solstice-ci

2. Start RabbitMQ locally

• docker compose up -d rabbitmq
• Management UI: http://localhost:15672 (guest/guest)

3. Build everything

• cargo build --workspace

4. (Linux) Prepare an example orchestrator config

• The orchestrator defaults to examples/orchestrator-image-map.yaml. Edit it to point local_path to a writable location; the orchestrator will download images as needed.

5. Run services in separate terminals

• Forge integration (webhooks) or enqueue sample jobs
• Orchestrator (VM scheduler/hypervisor)

6. Inspect logs

• Logs are structured via tracing. Set RUST_LOG=info (or debug/trace) as needed.

Running the Services

Environment defaults are sensible for local dev; override via flags or env vars.

Forge Integration (HTTP webhook receiver) — also supports a manual enqueue mode:

• cargo run -p forge-integration -- --help
• cargo run -p forge-integration -- enqueue --repo-url https://codeberg.org/example/repo.git --commit-sha deadbeef --runs-on illumos-latest
• cargo run -p forge-integration -- --http-addr 0.0.0.0:8080 --webhook-path /webhooks/forgejo

Orchestrator (Linux + libvirt/KVM):

• cargo run -p orchestrator -- --config examples/orchestrator-image-map.yaml --grpc-addr 0.0.0.0:50051
• You can cap concurrency and per-label capacity with --max-concurrency and --capacity-map, e.g. --capacity-map illumos-latest=1,ubuntu-22.04=2

Workflow runner (agent, useful for inspection of KDL files):

• cargo run -p workflow-runner -- --workflow path/to/workflow.kdl

Developer helper (validate/list/show workflow KDL):

• cargo run -p cidev -- validate --path path/to/workflow.kdl
• cargo run -p cidev -- list --path path/to/workflow.kdl
• cargo run -p cidev -- show --path path/to/workflow.kdl --job build

Configuration (Environment Variables)

Common env (see crates and guidelines for full list):

• RUST_LOG=info
• OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 (optional)
• AMQP_URL=amqp://127.0.0.1:5672/%2f
• AMQP_EXCHANGE=solstice.jobs
• AMQP_QUEUE=solstice.jobs.v1
• AMQP_ROUTING_KEY=jobrequest.v1
• AMQP_PREFETCH=64

Orchestrator specifics:

• ORCH_CONFIG=examples/orchestrator-image-map.yaml
• MAX_CONCURRENCY=2
• CAPACITY_MAP=illumos-latest=1,ubuntu-22.04=2
• GRPC_ADDR=0.0.0.0:50051
• DATABASE_URL=postgres://user:pass@localhost:5432/solstice (if used)
• LIBVIRT_URI=qemu:///system (Linux)
• LIBVIRT_NETWORK=default

Forge integration specifics:

• HTTP_ADDR=0.0.0.0:8080
• WEBHOOK_PATH=/webhooks/forgejo
• WEBHOOK_SECRET=... (recommended in real setups)

Testing

• Run all tests across the workspace:
  • cargo test --workspace
• Run a specific crate’s tests:
  • cargo test -p orchestrator
• Run an integration test by name filter:
  • cargo test smoke

Troubleshooting

• No logs? Ensure tracing is initialized once and set RUST_LOG appropriately.
• RabbitMQ not reachable? Verify docker compose is running and ports 5672/15672 are open.
• VM provisioning errors on non-Linux hosts: hypervisor features require Linux. Use a Linux machine or dev container/VM.
• Image downloads fail: check examples/orchestrator-image-map.yaml entries and local_path permissions.

Contributing (Codeberg Pull Requests)

We welcome contributions via the standard fork-and-pull-request workflow on Codeberg.

1. Fork the repository on Codeberg

• https://codeberg.org/your-namespace/solstice-ci (replace with the actual path)

2. Create a feature branch

• git checkout -b feat/short-description

3. Make changes and commit

• Follow conventional, descriptive commit messages when practical
• cargo fmt; cargo clippy --workspace --all-targets --all-features
• cargo test --workspace

4. Push your branch to your fork

• git push origin feat/short-description

5. Open a Pull Request on Codeberg

• Describe the change, motivations, and testing steps
• Link to any related issues

6. Review process

• Automated checks will run
• A maintainer will review and may request changes

7. Getting your PR merged

• Ensure feedback is addressed
• Squash/rebase as requested by maintainers

Security and credentials:

• Do not commit secrets. Use env vars locally and a secret store in deployments.

License

This project is licensed under the Mozilla Public License 2.0 (MPL-2.0). See LICENSE for details.