Commit graph

17 commits

Author SHA256 Message Date
Till Wegmueller
0d944c2075 Add genisoimage to orchestrator container image 2026-04-07 21:01:57 +02:00
Till Wegmueller
b5c7078adc Switch vm-manager to git dep + multi-stage Containerfile
- Use HTTPS git dep for vm-manager (works in CI and container builds)
- Add .cargo/ to .gitignore (local dev patch override)
- Restore multi-stage Containerfile: Rust build stage fetches vm-manager
  from GitHub, Ubuntu 24.04 runtime with QEMU
- Host orchestrator stopped and disabled (container-only from now on)
2026-04-07 17:24:17 +02:00
Till Wegmueller
21f4977839 Fix orchestrator container: pre-built binary + Ubuntu 24.04 base
Switch to pre-built binary Containerfile to avoid workspace dependency
issues with vm-manager path dep. Use Ubuntu 24.04 to match host glibc.
2026-04-07 16:50:55 +02:00
Till Wegmueller
39509039e3 Update orchestrator Containerfile: drop libvirt, add QEMU
- Remove libvirt-dev from build stage, libvirt-clients/libvirt0 from runtime
- Remove genisoimage (vm-manager has pure-Rust ISO builder)
- Add qemu-system-x86 to runtime for direct VM execution
- Keep qemu-utils for qemu-img overlay creation
- Remove --features libvirt from cargo build
- Remove libvirt socket/config volume mounts from compose.yml
- Remove LIBVIRT_URI/LIBVIRT_NETWORK env vars
- Add optional NETWORK_BRIDGE env var for TAP mode
- Container now only needs /dev/kvm device access
2026-04-07 15:57:10 +02:00
Till Wegmueller
d8ef6ef236 Add log delivery and step state reporting to Forgejo runner
Fetches logs from logs-service per category, uploads them to Forgejo
via UpdateLog, and reports per-step StepState entries so the Forgejo
UI shows individual step results and log output.
2026-04-06 23:59:26 +02:00
Till Wegmueller
70605a3c3a Add Forgejo Runner integration service
New crate that registers as a Forgejo Actions Runner, polls for tasks
via connect-rpc, translates them into Solstice JobRequests (with 3-tier
fallback: KDL workflow → Actions YAML run steps → unsupported error),
and reports results back to Forgejo.

Includes Containerfile and compose.yml service definition.
2026-04-06 23:34:53 +02:00
Till Wegmueller
4c5a8567a4
Add webhook crate for extensible signature validation and integration
- Introduce a new `webhook` crate to centralize signature validation for GitHub, Hookdeck, and Forgejo webhooks.
- Enable `github-integration` to perform unified webhook signature verification using the `webhook` crate.
- Refactor `github-integration`: replace legacy HMAC verification with the reusable `webhook` structure.
- Extend Podman configuration for Hookdeck webhook signature handling and improve documentation.
- Clean up unused dependencies by migrating to the new implementation.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2026-01-25 22:16:11 +01:00
Till Wegmueller
e33ddf2892
Update container build cache mounts, enhance Traefik and GitHub integration support
- Use `sharing=locked` for cargo build cache mounts across multiple Containerfiles to improve caching efficiency.
- Upgrade Traefik to version 3.6 and add support for `DOCKER_API_VERSION` for Podman compatibility.
- Extend `.env.sample` with GitHub integration variables and update `.gitignore` with new secrets.
- Document GitHub App configuration and webhook integration in Podman README.
- Update `github-integration` compose service with environment variables for webhook secret, app ID, key, and API base.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2026-01-25 18:38:28 +01:00
Till Wegmueller
7fc4e8edb7
Introduce logs-service for structured job logs management; bump version to 0.1.13
- Add `logs-service` crate as a separate microservice to handle job log storage, retrieval, and categorization.
- Update orchestrator to redirect log endpoints to the new service with optional permanent redirects using `LOGS_BASE_URL`.
- Enhance log persistence by introducing structured fields such as category, level, and error flags.
- Implement migration to add new columns and indexes for job logs.
- Add ANSI escape sequence stripping and structured logging for cleaner log storage.
- Improve SSH log handling with interleaved stdout/stderr processing and pty request support.
- Revise Docker files and compose setup to include logs-service, with support for PostgreSQL and secure connections.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-11-18 11:48:09 +01:00
Till Wegmueller
fad8e60ec1
Add Debian packaging support and network configuration enhancements
- Introduce Debian package build script using `cargo-deb` for orchestrator releases.
- Add systemd unit file and post-installation script for automatic service setup.
- Update `compose.yml` with host-only port bindings for Postgres and RabbitMQ.
- Introduce NGINX-based log proxy for orchestrator logs with Traefik support.
- Bump orchestrator version to 0.1.1 and update related Cargo metadata for packaging.
- Add example environment file for orchestrator configuration.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-11-17 19:57:19 +01:00
Till Wegmueller
c2fefb5167
Add per-job SSH key support, refactor scheduler for SSH-based job execution, and remove unused runner endpoint
- Introduce fields in `JobContext` for per-job SSH configuration, including user, key paths, and PEM contents.
- Update the scheduler to support SSH-based execution of jobs, including VM lifecycle management and SSH session handling.
- Add utility functions for SSH execution, guest IP discovery, and runner deployment.
- Remove the unused `/runners/{name}` HTTP endpoint and its associated logic.
- Simplify router creation by refactoring out disabled runner directory handling.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-11-15 18:37:30 +01:00
Till Wegmueller
930efe547f
Add public runner URL configuration and enhance log streaming support
- Introduce options for specifying public runner base URLs (`SOLSTICE_RUNNER_BASE_URL`) and orchestrator contact addresses (`ORCH_CONTACT_ADDR`).
- Update `.env.sample` and `compose.yml` with new configuration fields for external log streaming and runner binary serving.
- Refactor runner URL handling and generation logic for improved flexibility.
- Enhance `cloud-init` templates with updated runner URL environment variables (`RUNNER_SINGLE` and `RUNNER_URLS`).
- Add unit tests for runner URL generation to verify various input cases.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-11-11 20:24:20 +01:00
Till Wegmueller
248885bdf8
Add runner binary serving via orchestrator, update configurations and documentation
- Extend `.env.sample` with `RUNNER_DIR_HOST` for serving workflow runner binaries.
- Update `compose.yml` with `RUNNER_DIR` and corresponding volume mount.
- Add instructions for runner binary setup and serving in `README.md`.
- Enhance `mise.toml` with new tooling dependencies for building runners.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-11-09 19:02:42 +01:00
Till Wegmueller
888aa26388
Add libvirt/KVM integration and Forgejo webhook support to Podman stack
- Extend `.env.sample` with libvirt configuration, Forgejo secrets, and image mapping defaults.
- Update `compose.yml` to enable libvirt integration, including required mounts, devices, and environment variables.
- Add Forgejo webhook configuration and commit status reporting with optional HMAC validation.
- Enhance the orchestrator container with libvirt dependencies and optional features for VM management.
- Document host preparation for libvirt/KVM and image directories in the README.
- Set default fallback values for Traefik ACME CA server.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-11-09 17:58:36 +01:00
Till Wegmueller
fe7b4b9ce0
Update Podman deployment for rootless support and DNS fixes
- Document rootless Podman port binding limitations and workarounds in README.
- Update `.env.sample` with notes and default high ports for rootless runs.
- Adjust `compose.yml` for network configuration and privileged port handling.
- Introduce fixes for Traefik DNS timeouts using explicit public resolvers and network tweaks.
- Switch MinIO and MinIO setup to use the latest images for better compatibility.
2025-11-08 21:55:27 +00:00
Till Wegmueller
4228c7ae6c
Add .env to Podman deployment .gitignore 2025-11-08 20:26:19 +00:00
Till Wegmueller
1c5dc338f5
Add Podman Compose deployment stack with Traefik and services integration
This commit introduces:
- A production-ready Podman Compose stack using Traefik as a reverse proxy with Let's Encrypt integration.
- Per-environment logical separation for Postgres, RabbitMQ, and MinIO services.
- New deployment utilities, including a `.env.sample` template, `compose.yml`, and setup scripts for MinIO and Postgres.
- Updates to `github-integration` HTTP server with basic webhook handling using `axum` and configurable paths.
- Adjustments to packaging tasks for better tarball generation via `git archive`.
- Expanded dependencies for `PKGBUILD` to support SQLite and PostgreSQL libraries.
- Containerfiles for orchestrator and integration services to enable Rust multi-stage builds without sccache.

This enables simplified and secure CI deployments with automatic routing, TLS, and volume persistence.
2025-11-08 20:21:57 +00:00