- Introduce `slugify_step_name` to generate URL-friendly step name slugs.
- Attach per-step categories to NDJSON logs for better traceability.
- Update stdout and stderr logging with step-specific categories.
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
- Add detection and usage of GNU `tar` for platforms where BSD `tar` is incompatible with required options.
- Refactor `job.sh` to delegate all environment setup to newly introduced per-OS setup scripts.
- Add initial support for workflow setups via `workflow.kdl`, running pre-defined setup scripts before executing workflow steps.
- Integrate step-wise execution and logging for workflows, with structured NDJSON output for detailed traceability.
- Increment orchestrator version to 0.1.16.
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
- Add parsing and execution support for `.solstice/workflow.kdl` with job-specific configurations, including `runs_on`, `script path`, and `workflow_job_id`.
- Enable job grouping via `group_id` for cohesive workflow processing.
- Update orchestrator to pass workflow-specific parameters to `cloud-init` for finer control over execution.
- Refactor enqueue logic to handle multiple jobs per workflow with fallback to single job when no workflow is defined.
- Enhance dependencies for workflow parsing by integrating `base64`, `regex`, and `uuid`.
- Increment orchestrator version to 0.1.14 for release.
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
- Enhance runner logging with status-specific messages and structured JSON fields for better traceability.
- Add SHA256 object format detection and initialization for Git repos when applicable.
- Improve shell script execution by adding verbose mode and safe commands handling.
- Extend package installation scripts to support Clang and related tooling across multiple environments.
- Increment orchestrator version for release.
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
- Enhance runner tool check diagnostics with more descriptive output and JSON fields for better observability.
- Replace raw SQL queries in `logs-service` with ORM-based logic for portable and backend-agnostic log categorization.
- Add error category aggregation and structured summary reporting in logs-service.
- Improve environment variable fallback mechanics for runner workdir selection.
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
- Add `logs-service` crate as a separate microservice to handle job log storage, retrieval, and categorization.
- Update orchestrator to redirect log endpoints to the new service with optional permanent redirects using `LOGS_BASE_URL`.
- Enhance log persistence by introducing structured fields such as category, level, and error flags.
- Implement migration to add new columns and indexes for job logs.
- Add ANSI escape sequence stripping and structured logging for cleaner log storage.
- Improve SSH log handling with interleaved stdout/stderr processing and pty request support.
- Revise Docker files and compose setup to include logs-service, with support for PostgreSQL and secure connections.
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
- Introduce fields in `JobContext` for per-job SSH configuration, including user, key paths, and PEM contents.
- Update the scheduler to support SSH-based execution of jobs, including VM lifecycle management and SSH session handling.
- Add utility functions for SSH execution, guest IP discovery, and runner deployment.
- Remove the unused `/runners/{name}` HTTP endpoint and its associated logic.
- Simplify router creation by refactoring out disabled runner directory handling.
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
This commit introduces the following updates:
- Adds an environment variable (`SOLSTICE_ALLOW_INSECURE`) to enable insecure TLS as a fallback for curl.
- Improves CA certificate handling and automatic installation on SunOS using IPS or pkgin.
- Extends fallback logic for repository fetching to cover scenarios with missing CA bundles.
- Updates Solstice job script dependencies to include `cmake`.
This commit improves the runner's functionality by adding:
- Detailed log streaming with request ID, stdout, and stderr line counts.
- Fallback mechanisms for repository fetch using HTTP archive when git commands fail.
- Enhanced error reporting for missing job scripts and reading errors.
- Updates to ensure compatibility with SunOS environments and non-interactive shells.
This commit introduces gRPC-based log streaming between the VM runner (`solstice-runner`) and orchestrator. Key updates include:
- Implemented gRPC server in the orchestrator for receiving and processing runner logs.
- Added log streaming and job result reporting in the `solstice-runner` client.
- Defined `runner.proto` with messages (`LogItem`, `JobEnd`) and the `Runner` service.
- Updated orchestrator to accept gRPC settings and start the server.
- Modified cloud-init user data to include gRPC endpoint and request ID for runners.
- Enhanced message queue logic to handle job results via `publish_job_result`.
- Configured `Cross.toml` for cross-compilation of the runner.
