solstice-ci/docs/ai/2025-10-26-workflow-runner-and-cross.md

### Solstice CI — Workflow Runner bootstrap and cross builds (MVP)

Summary
- Implemented a minimal VM workflow runner binary (solstice-runner) that the orchestrator’s cloud-init bootstraps and executes inside the guest.
- The runner is cross-compilable with cross for convenient deployment to local dev VMs.

What the runner does (today)
- Reads job context from env or a small YAML file:
  - Env: SOLSTICE_REPO_URL, SOLSTICE_COMMIT_SHA
  - File: SOLSTICE_JOB_FILE (defaults to /etc/solstice/job.yaml) with keys repo_url and commit_sha
- Prepares a workspace (default /root/work; overridable with SOLSTICE_WORKDIR).
- Uses system git to fetch the repository at the exact commit (avoids libgit2 to make cross builds simpler).
- Executes .solstice/job.sh when present, streaming stdout/stderr. Exits with the script’s exit code.
- Logs via tracing (stderr), compatible with the serial console setup added to libvirt.

Why this design
- Keeps the guest-side binary very small, with minimal dependencies, easing cross builds.
- Shelling out to git leverages whatever the base image provides and avoids cross-compiling libgit2.
- Aligns with the orchestrator’s cloud-init that writes /etc/solstice/job.yaml and exports the same env var names.

Build and usage
- Build all: cargo build --workspace
- Build only the runner: cargo build -p workflow-runner
- Binary name in target directory: solstice-runner

Cross compiling
- A Cross.toml is provided at the workspace root. Example targets:
  - x86_64-unknown-linux-gnu
  - x86_64-unknown-illumos (requires a recent toolchain with illumos std)

Examples:
- cross build -p workflow-runner --target x86_64-unknown-linux-gnu --release
- cross build -p workflow-runner --target x86_64-unknown-illumos --release

Notes:
- Ensure the base VM image has /bin/sh and git installed (runner relies on both).
- On OpenIndiana/illumos images, prefer installing git via pkg or image packaging beforehand; the runner will not attempt to install packages.
- The orchestrator cloud-init already bootstraps /usr/local/bin/solstice-runner and calls it; set SOLSTICE_RUNNER_URL to point to an HTTP(S) URL hosting the cross-built artifact for quick iteration.

Next steps
- Add optional KDL workflow execution when .solstice/job.sh is absent (parse .solstice/workflow.kdl and run steps).
- Stream logs back to the orchestrator over gRPC and report final status to the Integration layer.
- Secrets injection and masking in logs.
-												Add gRPC support for VM runner log streaming and orchestrator integration

This commit introduces gRPC-based log streaming between the VM runner (`solstice-runner`) and orchestrator. Key updates include:
- Implemented gRPC server in the orchestrator for receiving and processing runner logs.
- Added log streaming and job result reporting in the `solstice-runner` client.
- Defined `runner.proto` with messages (`LogItem`, `JobEnd`) and the `Runner` service.
- Updated orchestrator to accept gRPC settings and start the server.
- Modified cloud-init user data to include gRPC endpoint and request ID for runners.
- Enhanced message queue logic to handle job results via `publish_job_result`.
- Configured `Cross.toml` for cross-compilation of the runner.

											
										
										
											2025-11-01 12:14:50 +01:00
+								### Solstice CI — Workflow Runner bootstrap and cross builds (MVP)
 								Summary
 								- Implemented a minimal VM workflow runner binary (solstice-runner) that the orchestrator’s cloud-init bootstraps and executes inside the guest.
 								- The runner is cross-compilable with cross for convenient deployment to local dev VMs.
 								What the runner does (today)
 								- Reads job context from env or a small YAML file:
 								  - Env: SOLSTICE_REPO_URL, SOLSTICE_COMMIT_SHA
 								  - File: SOLSTICE_JOB_FILE (defaults to /etc/solstice/job.yaml) with keys repo_url and commit_sha
 								- Prepares a workspace (default /root/work; overridable with SOLSTICE_WORKDIR).
 								- Uses system git to fetch the repository at the exact commit (avoids libgit2 to make cross builds simpler).
 								- Executes .solstice/job.sh when present, streaming stdout/stderr. Exits with the script’s exit code.
 								- Logs via tracing (stderr), compatible with the serial console setup added to libvirt.
 								Why this design
 								- Keeps the guest-side binary very small, with minimal dependencies, easing cross builds.
 								- Shelling out to git leverages whatever the base image provides and avoids cross-compiling libgit2.
 								- Aligns with the orchestrator’s cloud-init that writes /etc/solstice/job.yaml and exports the same env var names.
 								Build and usage
 								- Build all: cargo build --workspace
 								- Build only the runner: cargo build -p workflow-runner
 								- Binary name in target directory: solstice-runner
 								Cross compiling
 								- A Cross.toml is provided at the workspace root. Example targets:
 								  - x86_64-unknown-linux-gnu
 								  - x86_64-unknown-illumos (requires a recent toolchain with illumos std)
 								Examples:
 								- cross build -p workflow-runner --target x86_64-unknown-linux-gnu --release
 								- cross build -p workflow-runner --target x86_64-unknown-illumos --release
 								Notes:
 								- Ensure the base VM image has /bin/sh and git installed (runner relies on both).
 								- On OpenIndiana/illumos images, prefer installing git via pkg or image packaging beforehand; the runner will not attempt to install packages.
 								- The orchestrator cloud-init already bootstraps /usr/local/bin/solstice-runner and calls it; set SOLSTICE_RUNNER_URL to point to an HTTP(S) URL hosting the cross-built artifact for quick iteration.
 								Next steps
 								- Add optional KDL workflow execution when .solstice/job.sh is absent (parse .solstice/workflow.kdl and run steps).
 								- Stream logs back to the orchestrator over gRPC and report final status to the Integration layer.
 								- Secrets injection and masking in logs.