CloudNebulaProject/ips
1
0
Fork 0
mirror of https://codeberg.org/Toasterson/ips.git synced 2026-04-10 13:20:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Add api module for forge/pkgdev high-level integration

Browse source 
- Introduced `api.rs` to provide a stable, struct-first API surface for building, linting, resolving, and publishing IPS packages.
- Encapsulated existing functionality for better abstraction and usability in external integrations.
- Enhanced `libips` with a high-level repository and transaction interface.
- Added support for dependency generation, manifest transformations, and linting via configurable rulesets.
- Updated documentation with integration flow examples and usage guidelines.
This commit is contained in:
Till Wegmueller 2025-08-30 22:25:45 +02:00
parent a156a3f863
commit 29ef35f350
No known key found for this signature in database
3 changed files with 1063 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

251
doc/forge_docs/ips_integration.md Normal file
View file

@ -0,0 +1,251 @@
# IPS Library Integration Plan for pkgdev (struct-first)
This document updates the earlier integration plan to align with moving away from text representations (filelist.fmt, .p5m strings, pkgfmt) and toward constructing typed manifests and packages directly via the Rust IPS library (libips). Publishing and repository operations should be performed using the PublisherClient provided by libips.
Repository: https://github.com/OpenFlowLabs/ips.git
Key goals:
- No intermediate text filelists/manifests during the in-memory pipeline.
- Build typed Manifest/Action structures from prototype directories and recipe data.
- Apply transform rules programmatically (instead of pkgmogrify text transforms).
- Generate and resolve dependencies via library analyzers/resolvers.
- Use a typed Repository and PublisherClient to publish transactions atomically.
## Targeted Replacements and Proposed (Typed) APIs
Note: Names are indicative. The libips repository should expose these or equivalent types. Error handling should follow our project guidelines (thiserror + miette::Diagnostic) with codes like `ips::category_error`.
### Core Types (in libips)
- enum Action
- File { path: PathBuf, mode: Option<u32>, owner: Option<String>, group: Option<String>, hash: Option<String>, ... }
- Dir { path: PathBuf, ... }
- Link { path: PathBuf, target: PathBuf, ... }
- Hardlink { path: PathBuf, target: PathBuf, ... }
- License { path: PathBuf, license: String }
- Set { name: String, value: String }
- Depend { type_: String, fmri: String }
- … (other IPS actions as needed)
- struct Manifest { actions: Vec<Action> }
- Methods: validate(), to_string() (for debug/export only), from_string() (optional for interop)
- struct ManifestBuilder
- from_prototype_dir(proto: &Path) -> Result<Manifest, IpsError>
- with_base_metadata(meta: BaseMeta) -> &mut Self
- apply_rules(rules: &[TransformRule]) -> Result<&mut Self, IpsError>
- build() -> Manifest
- struct TransformRule
- Selectors and operations analogous to pkgmogrify, but typed and programmatic.
- struct DependencyGenerator
- generate(proto: &Path, manifest: &Manifest) -> Result<Manifest, IpsError>
- Adds Depend actions into the provided manifest (returns a new manifest or mutates in place).
- struct Resolver
- resolve(manifests: &mut [Manifest]) -> Result<(), IpsError>
- Converts provisional dependency notations into resolved FMRIs across the set.
- struct Repository
- open(path: &Path) -> Result<Repository, IpsError>
- create(path: &Path) -> Result<Repository, IpsError>
- has_publisher(name: &str) -> Result<bool, IpsError>
- add_publisher(name: &str) -> Result<(), IpsError>
- struct PublisherClient
- new(repo: Repository, publisher: String) -> Result<PublisherClient, IpsError>
- begin() -> Result<Txn, IpsError>
- struct Txn
- add_payload_dir(dir: &Path) -> Result<(), IpsError>
- add_manifest(manifest: &Manifest) -> Result<(), IpsError>
- commit() -> Result<(), IpsError>
### 1) Generate Manifest actions from prototype (replaces `pkgsend generate` + `pkgfmt`)
- API: ManifestBuilder::from_prototype_dir(proto)
- In pkgdev: call into libips to produce a Manifest (no filelist.fmt). Save only when exporting/debugging.
### 2) Apply transforms (replaces `pkgmogrify` + `pkgfmt`)
- API: ManifestBuilder::apply_rules(rules)
- In pkgdev: translate recipe/gate transforms into TransformRule structs and apply to the manifest.
### 3) Generate dependencies (replaces `pkgdepend generate` + `pkgfmt`)
- API: DependencyGenerator::generate(proto, &manifest) -> Manifest
- In pkgdev: obtain a Manifest with Depend actions injected.
### 4) Resolve dependencies against repository (replaces `pkgdepend resolve`)
- API: Resolver::resolve_with_repo(repo, publisher, &mut manifests)
- In pkgdev: call resolver with a repository handle so dependencies resolve to already-published packages.
### 5) Lint manifests (replaces `pkglint`)
- API: lint::lint_manifest(&manifest)
- In pkgdev: pass Manifest (typed), receive diagnostics.
### 6) Repository and publisher management (replaces `pkgrepo create/add-publisher`)
- APIs on Repository: create/open/has_publisher/add_publisher
- In pkgdev: ensure repo exists and publisher is present via libips types.
### 7) Publish (replaces `pkgsend publish`)
- APIs: Repository::open/create; PublisherClient::new; Txn::begin -> add_payload_dir -> add_manifest -> commit
- In pkgdev: feed prototype/unpack/pkg dirs via add_payload_dir; add each Manifest; commit.
## Integration points in pkgdev (cfg(feature = "libips"))
The following functions in crates/pkgdev/src/build/ips.rs will switch to libips when the feature is enabled. For now:
- pkgdev constructs a pre-filled typed Manifest struct for base metadata (fmri, summary, classification, upstream/source URLs, license) and writes a debug export as <name>-typed.manifest.json in the manifest directory. This is a temporary internal representation until libips types are available.
- Other libips paths return descriptive TODO errors pointing here.
When implementing with real libips types:
- run_generate_filelist: Build a Manifest via ManifestBuilder::from_prototype_dir (no filelist.fmt). Optionally serialize for debug output only.
- generate_manifest_files: Build base metadata into Manifest; compute TransformRule set from gate and recipe; apply via ManifestBuilder::apply_rules.
- run_generate_pkgdepend: Use DependencyGenerator::generate to add Depend actions to the Manifest; avoid text .dep intermediates.
- run_resolve_dependencies: Use Resolver::resolve_with_repo(repo, publisher, &mut manifests) so dependencies are resolved against a repository of published packages.
- run_lint: Call lint::lint_manifest(&Manifest) and convert diagnostics to miette.
- ensure_repo_with_publisher_exists: Use Repository::{create,has_publisher,add_publisher}.
- publish: Use PublisherClient with a Txn to add payload dirs and the Manifest, then commit.
## Error Handling Guidance
- Define specific error enums with thiserror + miette::Diagnostic.
- Codes: `ips::validation_error`, `ips::repo_error`, `ips::publish_error`, with specific variants (e.g., `ips::repo_error::publisher_missing`).
- Provide helpful diagnostics, including labels for problematic manifest actions when applicable.
## Next Steps in OpenFlowLabs/ips
1. Establish the crate module layout and expose the typed APIs outlined above.
2. Implement modules incrementally:
- manifest::{Action, Manifest, ManifestBuilder, TransformRule}
- dep::{DependencyGenerator, Resolver}
- lint::lint_manifest
- repo::{Repository, PublisherClient, Txn}
3. Mirror CLI behaviors with tests; provide conversion utilities to import/export .p5m where needed for interop only.
4. Once available, update pkgdev’s libips feature to depend on the crate and replace TODO stubs with real calls.
## Enabling Integration in pkgdev
- The Cargo feature `libips` is defined but does not yet pull the dependency. When libips stabilizes, add it as an optional dependency in crates/pkgdev/Cargo.toml, and wire cfg(feature = "libips") paths to call the typed APIs.
- Avoid introducing new text-based intermediates in the libips path; persist manifests only for debugging/exports.
## Gate KDL Transform Configuration (AST)
The gate crate now supports defining transform rules in KDL and exposes them as a typed AST rather than plain text lines. This enables constructing typed libips TransformRule objects in the future without round-tripping through pkgmogrify text.
KDL schema excerpt within a transform node:
- transform
- [legacy] positional arguments: textual mogrify lines (kept for compatibility)
- [legacy] include: optional include file path (string)
- rule (0..N)
- select (0..N): properties
- action: optional string (e.g., "file", "link", "hardlink", "dir")
- attr: optional string (e.g., "path", "mode", "owner")
- pattern: optional string (glob/regex or exact string)
- op (0..N): first argument is operation name; properties:
- key: optional string (e.g., attribute name for set/delete/default)
- value: optional string (e.g., value for set/default)
Example:
transform {
rule {
select action="file" attr="path" pattern=".*"
op "set" key="keep" value="true"
}
}
In Rust (gate crate):
- Transform has a field `rules: Vec<TransformRuleAst>`; use `ast_rules()` to access.
- Legacy fields `actions: Vec<String>` and `include: Option<String>` remain available and are used by existing pkgdev text-based paths.
When libips exposes typed TransformRule, pkgdev can map TransformRuleAst to libips structures directly.
## Additional integration points to audit in this repo
This section lists other areas in the forge codebase that should be considered when replacing CLI tools with libips and moving to typed, in-memory manifests.
1) pkgdev build orchestrator
- File: crates/pkgdev/src/build/mod.rs
- Function: run_ips_actions(...)
- Role: Orchestrates the full IPS pipeline: generate filelist -> build manifests -> deps generate -> deps resolve -> lint -> repo/publisher ensure -> publish.
- Libips path notes:
- Transition from file-based intermediates to in-memory Manifest values passed between steps.
- Consider adjusting function signatures to return and accept typed Manifests and dependency results rather than file paths.
- Preserve step ordering; add structured logging around each lib call.
2) pkgdev development dependency helper (pkg(1) CLI usage)
- File: crates/pkgdev/src/build/dependencies.rs
- Current behavior: Shells out to pkg list and pfexec pkg install to ensure dev dependencies are present on the build host.
- Potential libips integration:
- If libips exposes image management APIs, consider:
- struct Image { open(root: &Path) -> Result<Image, IpsError>; list_installed() -> Result<Vec<Fmri>, IpsError>; }
- impl Image { begin() -> Result<ImageTxn, IpsError> } and ImageTxn::install(pkgs: &[Fmri]) -> Result<(), IpsError>
- Otherwise, keep CLI for now and document this as out-of-scope for publisher/repo features.
- Documentation gap for libips: clarify whether host image management is in scope.
3) Typed base metadata/template construction
- Location: crates/pkgdev/src/build/ips.rs::generate_manifest_files (libips feature path writes <name>-typed.manifest.json for now).
- Libips API expectations:
- struct BaseMeta { fmri: Fmri, summary: String, classification: String, upstream_url: Url, source_url: Url, license: LicenseSpec }
- struct Fmri { pkg: String, version: Semver-ish; build, branch, revision components } with a Display/Parse.
- ManifestBuilder::with_base_metadata(BaseMeta).
4) Transform include semantics from CLI flags
- Location: pkgdev BuildArgs includes an optional include dir (-I) forwarded to pkgmogrify in the CLI path.
- Libips mapping:
- Gate KDL AST (see Gate TransformRuleAst) should be primary.
- For legacy include files, libips could expose a loader/parser for transform files, or pkgdev should translate those into typed TransformRule values.
5) Dependency resolution across multi-package builds
- Location: Orchestrator collects multiple ManifestCollection items and resolves dependencies together.
- Libips mapping:
- Resolver::resolve(&mut [Manifest]) should support a set of manifests at once so inter-package deps are satisfied without temporary .dep/.dep.res files.
- Provide diagnostics that identify unresolved FMRIs and which manifest/action caused them.
6) Lint configuration and reference catalogs
- Current CLI path uses pkglint possibly with cache/reference repos (see sample_data/make-rules for patterns).
- Libips mapping:
- LintConfig { reference_repos: Vec<PathBuf>, rulesets: Vec<String> } and lint::lint_manifest(manifest, &LintConfig).
- Ensure ability to suppress/waive or categorize warnings vs errors for CI.
7) Repository and publisher management lifecycles
- Current path: ensure_repo_with_publisher_exists uses pkgrepo create/add-publisher and path checks.
- Libips mapping:
- Repository::create/open, Repository::has_publisher/add_publisher, and helpers for repository initialization (e.g., metadata files, permissions).
- Consider explicit error variants: ips::repo_error::{not_found, invalid_format, publisher_missing}.
8) Publishing multiple manifests and payload dirs
- Location: pkgdev publish step publishes each manifest with multiple -d payload directories.
- Libips mapping:
- PublisherClient::new(repo, publisher).begin() -> Txn.
- Txn::add_payload_dir for prototype, unpack, and pkg directories.
- Txn::add_manifest for each typed Manifest; support multiple manifests per txn or separate txns.
- Txn::commit with atomic behavior and clear diagnostics.
9) Sample make-rules parity (reference only)
- Files: sample_data/make-rules/*.mk and sample_data/components/**.p5m
- Purpose: Serve as behavioral reference for transform chains, dependency generation, and publishing patterns.
- Action: Do not wire into pkgdev; use to define tests in libips and to validate parity.
10) Workspace layout and persistence decisions
- Files: crates/workspace (not IPS-specific but used heavily by pkgdev)
- Notes:
- Under libips, avoid persisting intermediate files except for debug/export.
- Ensure functions that currently assume files exist (e.g., reading .dep.res) are refactored to accept in-memory Manifests.
11) Future optional: Conversion utilities for interop
- Provide import/export for .p5m and .dep/.dep.res to ease migration and to compare outputs in tests.
- This is already noted, but emphasize the need for round-trip tests in libips.
12) Logging and diagnostics
- Adopt tracing across pkgdev paths interacting with libips.
- Ensure libips returns miette Diagnostics with codes per guidelines to integrate well in pkgdev error reporting.

811
libips/src/api.rs Normal file
View file

@ -0,0 +1,811 @@
// This Source Code Form is subject to the terms of
// the Mozilla Public License, v. 2.0. If a copy of the
// MPL was not distributed with this file, You can
// obtain one at https://mozilla.org/MPL/2.0/.
//! High-level, struct-first APIs for forge/pkgdev integration.
//!
//! These facades wrap existing libips modules to provide a stable API surface
//! for building, transforming, linting, resolving, and publishing IPS packages
//! entirely in memory using typed structures.
//!
//! See doc/forge_docs/ips_integration.md for an overview of the end-to-end flow.
//!
//! Quickstart (ignore): Build, lint, resolve, and publish
//! ```ignore
//! use libips::api as ips;
//! use std::path::Path;
//!
//! // 1) Build a Manifest from a prototype directory and base metadata
//! let proto = Path::new("/path/to/proto");
//! let mut manifest = ips::ManifestBuilder::from_prototype_dir(proto)?
//! .with_base_metadata(ips::BaseMeta {
//! fmri: Some(ips::Fmri::parse("pkg://pub/example@1.0")?),
//! summary: Some("Example package".into()),
//! classification: Some("org.opensolaris.category.2008:Applications/Other".into()),
//! upstream_url: Some("https://example.com".into()),
//! source_url: Some("https://example.com/src.tar.gz".into()),
//! license: Some("MIT".into()),
//! })
//! .build();
//!
//! // 2) Generate dependencies with a repository (for FMRI mapping)
//! let mut backend = libips::repository::file_backend::FileBackend::open(Path::new("/repo"))?;
//! manifest = ips::DependencyGenerator::generate_with_repo(&mut backend, Some("pub"), proto, &manifest, ips::DependGenerateOptions::default())?;
//!
//! // 3) Lint and optionally filter rules
//! let mut lint_cfg = ips::LintConfig::default();
//! lint_cfg.disabled_rules = vec!["manifest.summary".into()];
//! let diags = ips::lint::lint_manifest(&manifest, &lint_cfg)?;
//! assert!(diags.is_empty(), "Diagnostics: {:?}", diags);
//!
//! // 4) Publish
//! let repo = ips::Repository::open(Path::new("/repo"))?;
//! if !repo.has_publisher("pub")? { repo.add_publisher("pub")?; }
//! let client = ips::PublisherClient::new(repo, "pub");
//! let mut txn = client.begin()?;
//! txn.add_payload_dir(proto)?;
//! txn.add_manifest(&manifest);
//! txn.commit()?;
//! # Ok::<(), libips::api::IpsError>(())
//! ```
use std::path::{Path, PathBuf};
use miette::Diagnostic;
use thiserror::Error;
use walkdir::WalkDir;
pub use crate::actions::Manifest;
// Core typed manifest
use crate::actions::{Attr, File as FileAction};
pub use crate::fmri::Fmri;
pub use crate::depend::{FileDep, GenerateOptions as DependGenerateOptions};
// For BaseMeta
use crate::repository::file_backend::{FileBackend, Transaction};
use crate::repository::{ReadableRepository, RepositoryError, RepositoryVersion, WritableRepository};
use crate::transformer;
pub use crate::transformer::TransformRule;
/// Unified error type for API-level operations
#[derive(Debug, Error, Diagnostic)]
pub enum IpsError {
#[error(transparent)]
#[diagnostic(transparent)]
Repository(#[from] RepositoryError),
#[error(transparent)]
#[diagnostic(transparent)]
Transform(#[from] transformer::TransformError),
#[error(transparent)]
#[diagnostic(transparent)]
Depend(#[from] crate::depend::DependError),
#[error("I/O error: {0}")]
#[diagnostic(code(ips::api_error::io), help("Check file paths and permissions"))]
Io(String),
#[error("Unimplemented feature: {feature}")]
#[diagnostic(code(ips::api_error::unimplemented), help("See doc/forge_docs/ips_integration.md for roadmap."))]
Unimplemented { feature: &'static str },
}
/// Base package metadata used by ManifestBuilder.
///
/// Fields are optional to support incremental construction. At minimum,
/// providing `fmri` and `summary` is recommended.
///
/// Example:
/// ```
/// use libips::api::{BaseMeta, Fmri};
/// let meta = BaseMeta {
/// fmri: Some(Fmri::parse("pkg://pub/example@1.0").unwrap()),
/// summary: Some("Example".into()),
/// classification: Some("org.opensolaris.category.2008:Applications/Other".into()),
/// upstream_url: Some("https://example.com".into()),
/// source_url: Some("https://example.com/src.tar.gz".into()),
/// license: Some("MIT".into()),
/// };
/// ```
#[derive(Debug, Clone, Default)]
pub struct BaseMeta {
pub fmri: Option<Fmri>,
pub summary: Option<String>,
pub classification: Option<String>,
pub upstream_url: Option<String>,
pub source_url: Option<String>,
pub license: Option<String>,
}
/// Build or enrich typed manifests using a fluent builder.
///
/// Example (no_run):
/// ```no_run
/// use libips::api as ips;
/// use std::path::Path;
/// let proto = Path::new("/proto");
/// let mut manifest = ips::ManifestBuilder::new()
/// .with_base_metadata(ips::BaseMeta {
/// fmri: Some(ips::Fmri::parse("pkg://pub/name@1.0").unwrap()),
/// summary: Some("Summary".into()),
/// classification: None,
/// upstream_url: None,
/// source_url: None,
/// license: None,
/// })
/// .build();
/// # Ok::<(), ips::IpsError>(())
/// ```
pub struct ManifestBuilder {
manifest: Manifest,
}
impl ManifestBuilder {
/// Start a new empty builder
pub fn new() -> Self {
Self { manifest: Manifest::new() }
}
/// Convenience: construct a Manifest directly by scanning a prototype directory.
/// Paths in the manifest are stored relative to `proto`.
pub fn from_prototype_dir(proto: &Path) -> Result<Manifest, IpsError> {
if !proto.exists() {
return Err(IpsError::Io(format!(
"prototype directory does not exist: {}",
proto.display()
)));
}
let root = proto
.canonicalize()
.map_err(|e| IpsError::Io(format!("failed to canonicalize {}: {}", proto.display(), e)))?;
let mut m = Manifest::new();
for entry in WalkDir::new(&root).into_iter().filter_map(|e| e.ok()) {
let p = entry.path();
if p.is_file() {
// Build File action from absolute path
let mut f = FileAction::read_from_path(p).map_err(RepositoryError::from)?;
// Store path relative to root
let rel = p
.strip_prefix(&root)
.map_err(RepositoryError::from)?
.to_string_lossy()
.to_string();
f.path = rel;
m.add_file(f);
}
}
Ok(m)
}
/// Add base metadata to the manifest using typed fields.
pub fn with_base_metadata(mut self, meta: BaseMeta) -> Self {
// Helper to push an attribute set action
let mut push_attr = |key: &str, val: String| {
self.manifest.attributes.push(Attr {
key: key.to_string(),
values: vec![val],
properties: Default::default(),
});
};
if let Some(fmri) = meta.fmri {
push_attr("pkg.fmri", fmri.to_string());
}
if let Some(s) = meta.summary { push_attr("pkg.summary", s); }
if let Some(c) = meta.classification { push_attr("info.classification", c); }
if let Some(u) = meta.upstream_url { push_attr("info.upstream-url", u); }
if let Some(su) = meta.source_url { push_attr("info.source-url", su); }
if let Some(l) = meta.license {
// Represent base license via an attribute named 'license'; callers may add dedicated license actions separately
self.manifest.attributes.push(Attr {
key: "license".to_string(),
values: vec![l],
properties: Default::default(),
});
}
self
}
/// Apply typed transform rules to the manifest (in place)
pub fn apply_rules(mut self, rules: &[TransformRule]) -> Result<Self, IpsError> {
let rules: Vec<crate::actions::Transform> = rules.iter().cloned().map(Into::into).collect();
transformer::apply(&mut self.manifest, &rules)?;
Ok(self)
}
/// Finalize and return the Manifest
pub fn build(self) -> Manifest {
self.manifest
}
}
/// Minimal repository facade backed by an on-disk file repository.
///
/// Example (no_run):
/// ```no_run
/// use libips::api::Repository;
/// use std::path::Path;
/// let repo_path = Path::new("/repo");
/// // Create if needed
/// let _ = Repository::create(repo_path);
/// // Open and ensure publisher
/// let repo = Repository::open(repo_path)?;
/// if !repo.has_publisher("pub")? { repo.add_publisher("pub")?; }
/// # Ok::<(), libips::api::IpsError>(())
/// ```
pub struct Repository {
path: PathBuf,
}
impl Repository {
pub fn open(path: &Path) -> Result<Self, IpsError> {
// Validate by opening backend
let _ = FileBackend::open(path)?;
Ok(Self { path: path.to_path_buf() })
}
pub fn create(path: &Path) -> Result<Self, IpsError> {
let _ = FileBackend::create(path, RepositoryVersion::default())?;
Ok(Self { path: path.to_path_buf() })
}
pub fn has_publisher(&self, name: &str) -> Result<bool, IpsError> {
let backend = FileBackend::open(&self.path)?;
let info = backend.get_info()?;
Ok(info.publishers.iter().any(|p| p.name == name))
}
pub fn add_publisher(&self, name: &str) -> Result<(), IpsError> {
let mut backend = FileBackend::open(&self.path)?;
backend.add_publisher(name)?;
Ok(())
}
pub fn path(&self) -> &Path { &self.path }
}
/// High-level publishing client for starting repository transactions.
///
/// Example (no_run):
/// ```no_run
/// use libips::api as ips;
/// use std::path::Path;
/// let repo = ips::Repository::open(Path::new("/repo"))?;
/// let client = ips::PublisherClient::new(repo, "pub");
/// let mut tx = client.begin()?;
/// // Add payloads and manifests, then commit
/// # Ok::<(), ips::IpsError>(())
/// ```
pub struct PublisherClient {
repo: Repository,
publisher: String,
}
impl PublisherClient {
pub fn new(repo: Repository, publisher: impl Into<String>) -> Self {
Self { repo, publisher: publisher.into() }
}
/// Begin a new transaction
pub fn begin(&self) -> Result<Txn, IpsError> {
let backend = FileBackend::open(self.repo.path())?;
let tx = backend.begin_transaction()?; // returns Transaction bound to repo path
Ok(Txn { backend_path: self.repo.path().to_path_buf(), tx, publisher: self.publisher.clone() })
}
}
/// Transaction wrapper exposing add_payload_dir/add_manifest/commit.
///
/// Start a transaction via PublisherClient::begin, add payload directories and manifests,
/// then commit to publish.
///
/// Example (no_run):
/// ```no_run
/// use libips::api as ips;
/// use std::path::Path;
/// let repo = ips::Repository::open(Path::new("/repo"))?;
/// let client = ips::PublisherClient::new(repo, "pub");
/// let mut tx = client.begin()?;
/// tx.add_payload_dir(Path::new("/proto"))?;
/// tx.add_manifest(&ips::Manifest::new());
/// tx.commit()?;
/// # Ok::<(), ips::IpsError>(())
/// ```
pub struct Txn {
backend_path: PathBuf,
tx: Transaction,
publisher: String,
}
impl Txn {
/// Add all files from the given payload/prototype directory
pub fn add_payload_dir(&mut self, dir: &Path) -> Result<(), IpsError> {
let root = dir
.canonicalize()
.map_err(|e| IpsError::Io(format!("failed to canonicalize {}: {}", dir.display(), e)))?;
for entry in WalkDir::new(&root).into_iter().filter_map(|e| e.ok()) {
let p = entry.path();
if p.is_file() {
let mut f = FileAction::read_from_path(p).map_err(RepositoryError::from)?;
let rel = p
.strip_prefix(&root)
.map_err(RepositoryError::from)?
.to_string_lossy()
.to_string();
f.path = rel;
self.tx.add_file(f, p)?;
}
}
Ok(())
}
/// Merge the provided manifest into the transaction's manifest
pub fn add_manifest(&mut self, manifest: &Manifest) {
self.tx.update_manifest(manifest.clone());
}
/// Commit the transaction to the repository for the preselected publisher
pub fn commit(mut self) -> Result<(), IpsError> {
self.tx.set_publisher(&self.publisher);
self.tx.commit()?;
// Rebuild metadata (catalog and index)
let backend = FileBackend::open(&self.backend_path)?;
backend.rebuild(Some(&self.publisher), false, false)?;
Ok(())
}
}
/// Dependency generation facade.
///
/// Use this to compute file-level dependencies and resolve them to package
/// FMRIs using a repository.
///
/// Example: generate dependencies with a repository (no_run)
/// ```no_run
/// use libips::api as ips;
/// use std::path::Path;
/// use libips::repository::{FileBackend, ReadableRepository};
/// let proto = Path::new("/proto");
/// let mut backend = FileBackend::open(Path::new("/repo"))?;
/// let manifest = ips::Manifest::new();
/// let manifest = ips::DependencyGenerator::generate_with_repo(
/// &mut backend,
/// Some("pub"),
/// proto,
/// &manifest,
/// ips::DependGenerateOptions::default(),
/// )?;
/// # Ok::<(), ips::IpsError>(())
/// ```
pub struct DependencyGenerator;
impl DependencyGenerator {
/// Compute file-level dependencies for the given manifest, using `proto` as base for local file resolution.
/// This is a helper for callers that want to inspect raw file deps before mapping them to package FMRIs.
pub fn file_deps(proto: &Path, manifest: &Manifest, mut opts: DependGenerateOptions) -> Result<Vec<FileDep>, IpsError> {
if opts.proto_dir.is_none() {
opts.proto_dir = Some(proto.to_path_buf());
}
let deps = crate::depend::generate_file_dependencies_from_manifest(manifest, &opts)?;
Ok(deps)
}
/// Generate dependencies and return a new manifest with Depend actions injected.
/// Intentionally not implemented in this facade: mapping raw file dependencies to package FMRIs
/// requires repository/catalog context. Call `generate_with_repo` instead.
pub fn generate(_proto: &Path, _manifest: &Manifest) -> Result<Manifest, IpsError> {
Err(IpsError::Unimplemented { feature: "DependencyGenerator::generate (use generate_with_repo)" })
}
/// Generate dependencies using a repository to resolve file-level deps into package FMRIs.
pub fn generate_with_repo<R: ReadableRepository>(
repo: &mut R,
publisher: Option<&str>,
proto: &Path,
manifest: &Manifest,
mut opts: DependGenerateOptions,
) -> Result<Manifest, IpsError> {
if opts.proto_dir.is_none() {
opts.proto_dir = Some(proto.to_path_buf());
}
let file_deps = crate::depend::generate_file_dependencies_from_manifest(manifest, &opts)?;
let deps = crate::depend::resolve_dependencies(repo, publisher, &file_deps)?;
let mut out = manifest.clone();
out.dependencies.extend(deps);
Ok(out)
}
}
/// Cross-manifest dependency resolver.
///
/// This helper fills missing publisher/version on dependency FMRIs either by
/// inspecting peer manifests in-memory or by querying a repository.
///
/// Examples (no_run):
/// ```no_run
/// use libips::api as ips;
/// // Peer-manifest resolve
/// let mut manifests: Vec<ips::Manifest> = vec![]; // populate with manifests that depend on each other
/// ips::Resolver::resolve(&mut manifests)?;
/// # Ok::<(), ips::IpsError>(())
/// ```
/// ```no_run
/// use libips::api as ips;
/// use std::path::Path;
/// // Repository-backed resolve
/// use libips::repository::{FileBackend, ReadableRepository};
/// let backend = FileBackend::open(Path::new("/repo"))?;
/// let mut manifests: Vec<ips::Manifest> = vec![]; // populate
/// ips::Resolver::resolve_with_repo(&backend, Some("pub"), &mut manifests)?;
/// # Ok::<(), ips::IpsError>(())
/// ```
pub struct Resolver;
impl Resolver {
/// Best-effort peer-manifest resolver.
/// Note: For production resolution against published packages, prefer resolve_with_repo().
pub fn resolve(manifests: &mut [Manifest]) -> Result<(), IpsError> {
// Build a map from package name (stem) to full FMRI from the provided manifests
use std::collections::HashMap;
let mut providers: HashMap<String, Fmri> = HashMap::new();
for m in manifests.iter() {
if let Some(f) = manifest_fmri(m) {
providers.insert(f.stem().to_string(), f);
}
}
// For each manifest dependency that has an FMRI with missing publisher/version,
// fill in from providers if there is a matching manifest by name.
for m in manifests.iter_mut() {
for dep in &mut m.dependencies {
if let Some(ref mut f) = dep.fmri {
// Only attempt if version is missing
if f.version.is_none() {
if let Some(p) = providers.get(f.stem()) {
// Fill publisher if missing and version from provider
if f.publisher.is_none() {
f.publisher = p.publisher.clone();
}
if f.version.is_none() {
f.version = p.version.clone();
}
}
}
}
}
}
Ok(())
}
/// Resolve dependency FMRIs using a repository of already-published packages.
/// For each dependency with a name-only FMRI (missing version), if exactly one
/// package with that name exists in the given publisher, fill in publisher and version.
/// If multiple or zero matches are found, the dependency is left unchanged.
pub fn resolve_with_repo<R: ReadableRepository>(
repo: &R,
publisher: Option<&str>,
manifests: &mut [Manifest],
) -> Result<(), IpsError> {
for m in manifests.iter_mut() {
for dep in &mut m.dependencies {
if let Some(ref mut f) = dep.fmri {
if f.version.is_none() {
// Query repository for this package name
let pkgs = repo.list_packages(publisher, Some(&f.name))?;
let matches: Vec<&crate::repository::PackageInfo> = pkgs
.iter()
.filter(|pi| pi.fmri.name == f.name)
.collect();
if matches.len() == 1 {
let fmri = &matches[0].fmri;
if f.publisher.is_none() {
f.publisher = fmri.publisher.clone();
}
if f.version.is_none() {
f.version = fmri.version.clone();
}
}
}
}
}
}
Ok(())
}
}
// Helper: extract the package FMRI from a manifest's attributes
fn manifest_fmri(manifest: &Manifest) -> Option<Fmri> {
for attr in &manifest.attributes {
if attr.key == "pkg.fmri" {
if let Some(val) = attr.values.get(0) {
if let Ok(f) = Fmri::parse(val) {
return Some(f);
}
}
}
}
None
}
/// Lint facade providing a typed, extensible rule engine with enable/disable controls.
///
/// Configure which rules to run, override severities, and pass rule-specific parameters.
///
/// Example: disable a rule and run lint (no_run)
/// ```no_run
/// use libips::api as ips;
/// let mut cfg = ips::LintConfig::default();
/// cfg.disabled_rules = vec!["manifest.summary".into()];
/// let mut m = ips::Manifest::new();
/// m.attributes.push(libips::actions::Attr{ key: "pkg.fmri".into(), values: vec!["pkg://pub/name@1.0".into()], properties: Default::default() });
/// let diags = ips::lint::lint_manifest(&m, &cfg)?;
/// assert!(diags.is_empty());
/// # Ok::<(), ips::IpsError>(())
/// ```
#[derive(Debug, Clone, Default)]
pub struct LintConfig {
pub reference_repos: Vec<PathBuf>,
pub rulesets: Vec<String>,
// Rule configurability
pub disabled_rules: Vec<String>, // rule IDs to disable
pub enabled_only: Option<Vec<String>>, // if Some, only these rule IDs run
pub severity_overrides: std::collections::HashMap<String, lint::LintSeverity>,
pub rule_params: std::collections::HashMap<String, std::collections::HashMap<String, String>>, // rule_id -> (key->val)
}
pub mod lint {
use super::*;
use miette::Diagnostic;
use std::collections::HashSet;
use thiserror::Error;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum LintSeverity {
Error,
Warning,
Info,
}
#[derive(Debug, Error, Diagnostic)]
pub enum LintIssue {
#[error("Manifest is missing pkg.fmri or it is invalid")]
#[diagnostic(code(ips::lint_error::missing_fmri), help("Add a valid set name=pkg.fmri value=... attribute"))]
MissingOrInvalidFmri,
#[error("Manifest has multiple pkg.fmri attributes")]
#[diagnostic(code(ips::lint_error::duplicate_fmri), help("Ensure only one pkg.fmri set action is present"))]
DuplicateFmri,
#[error("Manifest is missing pkg.summary")]
#[diagnostic(code(ips::lint_error::missing_summary), help("Add a set name=pkg.summary value=... attribute"))]
MissingSummary,
#[error("Dependency is missing FMRI or name")]
#[diagnostic(code(ips::lint_error::dependency_missing_fmri), help("Each depend action should include a valid fmri (name or full fmri)"))]
DependencyMissingFmri,
#[error("Dependency type is missing")]
#[diagnostic(code(ips::lint_error::dependency_missing_type), help("Set depend type (e.g., require, incorporate, optional)"))]
DependencyMissingType,
}
pub trait LintRule {
fn id(&self) -> &'static str;
fn description(&self) -> &'static str;
fn default_severity(&self) -> LintSeverity { LintSeverity::Error }
/// Run this rule against the manifest. Implementors may ignore `config` (prefix with `_`) if not needed.
/// The config carries enable/disable lists, severity overrides and rule-specific parameters for extensibility.
fn check(&self, manifest: &Manifest, config: &LintConfig) -> Vec<miette::Report>;
}
struct RuleManifestFmri;
impl LintRule for RuleManifestFmri {
fn id(&self) -> &'static str { "manifest.fmri" }
fn description(&self) -> &'static str { "Validate pkg.fmri presence/uniqueness/parse" }
fn check(&self, manifest: &Manifest, _config: &LintConfig) -> Vec<miette::Report> {
let mut diags = Vec::new();
let mut fmri_attr_count = 0usize;
let mut fmri_text: Option<String> = None;
for attr in &manifest.attributes {
if attr.key == "pkg.fmri" {
fmri_attr_count += 1;
if let Some(v) = attr.values.get(0) { fmri_text = Some(v.clone()); }
}
}
if fmri_attr_count > 1 { diags.push(miette::Report::new(LintIssue::DuplicateFmri)); }
match (fmri_attr_count, fmri_text) {
(0, _) => diags.push(miette::Report::new(LintIssue::MissingOrInvalidFmri)),
(_, Some(txt)) => { if Fmri::parse(&txt).is_err() { diags.push(miette::Report::new(LintIssue::MissingOrInvalidFmri)); } },
(_, None) => diags.push(miette::Report::new(LintIssue::MissingOrInvalidFmri)),
}
diags
}
}
struct RuleManifestSummary;
impl LintRule for RuleManifestSummary {
fn id(&self) -> &'static str { "manifest.summary" }
fn description(&self) -> &'static str { "Validate pkg.summary presence" }
fn check(&self, manifest: &Manifest, _config: &LintConfig) -> Vec<miette::Report> {
let mut diags = Vec::new();
let has_summary = manifest
.attributes
.iter()
.any(|a| a.key == "pkg.summary" && a.values.iter().any(|v| !v.trim().is_empty()));
if !has_summary { diags.push(miette::Report::new(LintIssue::MissingSummary)); }
diags
}
}
struct RuleDependencyFields;
impl LintRule for RuleDependencyFields {
fn id(&self) -> &'static str { "depend.fields" }
fn description(&self) -> &'static str { "Validate basic dependency fields" }
fn check(&self, manifest: &Manifest, _config: &LintConfig) -> Vec<miette::Report> {
let mut diags = Vec::new();
for dep in &manifest.dependencies {
let fmri_ok = dep.fmri.as_ref().map(|f| !f.name.trim().is_empty()).unwrap_or(false);
if !fmri_ok { diags.push(miette::Report::new(LintIssue::DependencyMissingFmri)); }
if dep.dependency_type.trim().is_empty() { diags.push(miette::Report::new(LintIssue::DependencyMissingType)); }
}
diags
}
}
fn default_rules() -> Vec<Box<dyn LintRule>> {
vec![
Box::new(RuleManifestFmri),
Box::new(RuleManifestSummary),
Box::new(RuleDependencyFields),
]
}
fn rule_enabled(rule_id: &str, cfg: &LintConfig) -> bool {
if let Some(only) = &cfg.enabled_only {
let set: HashSet<&str> = only.iter().map(|s| s.as_str()).collect();
return set.contains(rule_id);
}
let disabled: HashSet<&str> = cfg.disabled_rules.iter().map(|s| s.as_str()).collect();
!disabled.contains(rule_id)
}
/// Lint a manifest and return diagnostics. Does not fail the call; diagnostics are returned as reports.
///
/// Example (no_run):
/// ```no_run
/// use libips::api as ips;
/// let mut m = ips::Manifest::new();
/// m.attributes.push(libips::actions::Attr{ key: "pkg.fmri".into(), values: vec!["pkg://pub/name@1.0".into()], properties: Default::default() });
/// let cfg = ips::LintConfig::default();
/// let diags = ips::lint::lint_manifest(&m, &cfg)?;
/// assert!(diags.is_empty());
/// # Ok::<(), ips::IpsError>(())
/// ```
pub fn lint_manifest(manifest: &Manifest, config: &LintConfig) -> Result<Vec<miette::Report>, IpsError> {
let mut diags: Vec<miette::Report> = Vec::new();
for rule in default_rules().into_iter() {
if rule_enabled(rule.id(), config) {
diags.extend(rule.check(manifest, config).into_iter());
}
}
Ok(diags)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::actions::{Attr, Dependency as ManifestDependency};
fn make_manifest_with_fmri(fmri_str: &str) -> Manifest {
let mut m = Manifest::new();
m.attributes.push(Attr { key: "pkg.fmri".into(), values: vec![fmri_str.to_string()], properties: Default::default() });
m
}
#[test]
fn resolver_fills_version_and_publisher_from_peer_manifest() {
// Provider manifest: pkgA with publisher and version
let provider = make_manifest_with_fmri("pkg://pub/pkgA@1.0");
// Consumer manifest with dependency on pkgA without version/publisher
let mut consumer = make_manifest_with_fmri("pkg://pub/consumer@0.1");
let dep_fmri = Fmri::parse("pkgA").unwrap();
consumer.dependencies.push(ManifestDependency {
fmri: Some(dep_fmri),
dependency_type: "require".to_string(),
predicate: None,
root_image: String::new(),
optional: Vec::new(),
facets: Default::default(),
});
let mut manifests = vec![provider, consumer];
Resolver::resolve(&mut manifests).unwrap();
// After resolve, the consumer's first dependency should have version and publisher set
let consumer_after = &manifests[1];
let dep = &consumer_after.dependencies[0];
let fmri = dep.fmri.as_ref().unwrap();
assert_eq!(fmri.name, "pkgA");
assert_eq!(fmri.publisher.as_deref(), Some("pub"));
assert!(fmri.version.is_some(), "expected version to be filled from provider");
assert_eq!(fmri.version.as_ref().unwrap().to_string(), "1.0");
}
#[test]
fn resolver_uses_repository_for_provider() {
use crate::repository::file_backend::FileBackend;
use crate::repository::RepositoryVersion;
// Create a temporary repository and add a publisher
let tmp = tempfile::tempdir().unwrap();
let repo_path = tmp.path().join("repo");
let mut backend = FileBackend::create(&repo_path, RepositoryVersion::default()).unwrap();
backend.add_publisher("pub").unwrap();
// Publish provider package pkgA@1.0
let mut provider = Manifest::new();
provider.attributes.push(Attr { key: "pkg.fmri".into(), values: vec!["pkg://pub/pkgA@1.0".to_string()], properties: Default::default() });
let mut tx = backend.begin_transaction().unwrap();
tx.update_manifest(provider);
tx.set_publisher("pub");
tx.commit().unwrap();
backend.rebuild(Some("pub"), false, false).unwrap();
// Build consumer with name-only dependency
let mut consumer = make_manifest_with_fmri("pkg://pub/consumer@0.1");
let dep_fmri = Fmri::parse("pkgA").unwrap();
consumer.dependencies.push(ManifestDependency {
fmri: Some(dep_fmri),
dependency_type: "require".to_string(),
predicate: None,
root_image: String::new(),
optional: Vec::new(),
facets: Default::default(),
});
let mut manifests = vec![consumer];
Resolver::resolve_with_repo(&backend, Some("pub"), &mut manifests).unwrap();
let dep = &manifests[0].dependencies[0];
let fmri = dep.fmri.as_ref().unwrap();
assert_eq!(fmri.publisher.as_deref(), Some("pub"));
assert_eq!(fmri.version.as_ref().unwrap().to_string(), "1.0");
}
#[test]
fn lint_reports_missing_fmri_and_summary() {
let m = Manifest::new();
let cfg = LintConfig::default();
let diags = lint::lint_manifest(&m, &cfg).unwrap();
assert!(!diags.is_empty());
}
#[test]
fn lint_accepts_valid_manifest() {
let mut m = Manifest::new();
m.attributes.push(Attr { key: "pkg.fmri".into(), values: vec!["pkg://pub/name@1.0".to_string()], properties: Default::default() });
m.attributes.push(Attr { key: "pkg.summary".into(), values: vec!["A package".to_string()], properties: Default::default() });
let cfg = LintConfig::default();
let diags = lint::lint_manifest(&m, &cfg).unwrap();
assert!(diags.is_empty(), "unexpected diags: {:?}", diags);
}
#[test]
fn lint_disable_summary_rule() {
// Manifest with valid fmri but missing summary
let mut m = Manifest::new();
m.attributes.push(Attr { key: "pkg.fmri".into(), values: vec!["pkg://pub/name@1.0".to_string()], properties: Default::default() });
// Disable the summary rule; expect no diagnostics
let mut cfg = LintConfig::default();
cfg.disabled_rules = vec!["manifest.summary".to_string()];
let diags = lint::lint_manifest(&m, &cfg).unwrap();
// fmri is valid, dependencies empty, summary rule disabled => no diags
assert!(diags.is_empty(), "expected no diagnostics when summary rule disabled, got: {:?}", diags);
}
}

1
libips/src/lib.rs
View file

@ -14,6 +14,7 @@ pub mod publisher;
pub mod transformer; pub mod transformer;
pub mod solver; pub mod solver;
pub mod depend; pub mod depend;
pub mod api;
mod test_json_manifest; mod test_json_manifest;
#[cfg(test)] #[cfg(test)]