Commit graph

235 commits

Author SHA1 Message Date
Till Wegmueller
d763bf1c34
Simplify PkgTreeError struct and consolidate error handling logic across main.rs. Update OmniOS workflows and Vagrantfile to include additional libraries and improve configuration consistency. 2026-01-18 17:05:59 +01:00
Till Wegmueller
0e873ee114
Update OmniOS workflows to use ooce packages
- Replaced `extra.omnios` publisher setup and outdated package names with `ooce` packages in `rust.yml` and `release.yml`.
2026-01-18 16:45:43 +01:00
Till Wegmueller
d7c7079374
Add OmniOS extra publisher setup to workflows
- Updated both `rust.yml` and `release.yml` to configure the extra OmniOS publisher for additional package installations.
2026-01-18 16:32:27 +01:00
Till Wegmueller
4d96cf1d2b
Update Illumos build workflows to use OmniOS VM
- Replaced OpenIndiana VM with OmniOS VM in Illumos build steps across `rust.yml` and `release.yml` workflows.
- Updated related workflow labels and comments for consistency.
2026-01-18 16:01:24 +01:00
Till Wegmueller
7fb3793c54
Remove Jenkinsfile, add Illumos builds to GitHub Actions, and document release process
- Deleted the Jenkinsfile, transitioning all CI workflows to GitHub Actions.
- Added Illumos (OpenIndiana) build and release steps to GitHub workflows.
- Created a new release pipeline triggered by version tags starting with `v`.
- Updated README with release instructions using `cargo-release`.
2026-01-18 14:59:24 +01:00
Till Wegmueller
898ec20ad8
Update binaries and workflows for pkg6 rename
- Renamed `pkg6dev` to `pkg6` across build scripts, workflows, and documentation.
- Added support for additional binaries (`pkg6repo` and `pkg6depotd`) in release builds.
- Disabled `RUSTFLAGS` warnings-as-errors policy in workflows for improved flexibility.
- Simplified error handling in `manifest_fmri` with streamlined conditionals.
- Introduced `#[allow(clippy::result_large_err)]` to suppress clippy warnings for large error types.
2026-01-18 14:30:05 +01:00
Till Wegmueller
b8c625a11e
Refactor attribute value access and error handling
- Replaced `values.get(0)` with `values.first()` for improved clarity and consistency across multiple files.
- Simplified conditional checks with `.is_empty()` and `format_args!` to enhance readability.
- Implemented `Default` trait for `VariableMode` directly within the enum.
- Updated error handling by boxing errors in `RepositoryError` and `CatalogError`.
- Replaced redundant `else` blocks with streamlined logic.
2026-01-18 13:00:21 +01:00
Till Wegmueller
9512eb3b12
chore: cargo fmt 2026-01-18 12:51:55 +01:00
Till Wegmueller
8f089656ba
Add search functionality to repository and route handlers (currently disabled)
- Introduced a searchable index with structured `IndexEntry` support for packages, files, directories, and dependencies.
- Added `search` method in `DepotRepo` with wildcard and case-sensitive query handling.
- Created `/search/0` and `/search/1` routes for search API, supporting publishers and token-based queries.
- Updated `SearchIndex` handling to map tokens to detailed `IndexEntry` structures.
- Improved index building to include attributes for files, directories, and dependencies.
2026-01-18 12:29:44 +01:00
Till Wegmueller
b080288114
Add legacy manifest handling with JSON fallback for transaction commits
- Introduced `legacy_manifest_content` field in `Transaction` to allow preserving byte-identical legacy manifests.
- Updated `commit` method to save both JSON and legacy manifests, falling back to JSON if no legacy content is provided.
- Enhanced manifest copy logic to handle both JSON and legacy formats during transaction finalization.
- Improved catalog rebuild to skip `.json` files when a corresponding legacy manifest exists.
2025-12-23 14:09:14 +01:00
Till Wegmueller
7dc475ed2d
Fix FMRI and catalog processing inconsistencies
- Corrected FMRI formatting to `pkg:/name` for entries without a publisher.
- Excluded `pkg.fmri` from summary action extraction in file backend.
- Removed unused signature handling in catalog package processing.
- Reordered `PackageVersionEntry` struct fields for logical consistency.
2025-12-23 13:19:51 +01:00
Till Wegmueller
e0de265d17
Add rebuild_test_repo.sh script for rebuilding test repositories with PostgreSQL packages 2025-12-23 12:33:31 +01:00
Till Wegmueller
b32ace705f
Refactor whitespace handling in JSON serialization and improve ISO-8601 timestamp formatting
- Removed unnecessary spaces in JSON key-value and array formatting for Python-style compatibility.
- Enhanced `format_iso8601_basic` to ensure precise microsecond-level time formatting.
- Updated `write_update_log` to return and store SHA-1 signatures for update log verification.
2025-12-23 12:33:12 +01:00
Till Wegmueller
ff0b9f4319
Add support for file URL without algorithm and refactor JSON serialization for Python-style compatibility
- Added `/file/1/{digest}` route to support file retrieval without specifying the algorithm.
- Implemented a Python-compatible JSON serializer to ensure consistent formatting for catalog artifacts.
- Replaced `HashMap` with `BTreeMap` for deterministic ordering in catalog serialization and updates.
- Updated integration tests to validate the new route functionality and ensure response correctness.
- Refactored `format_iso8601_basic` to improve timestamp formatting consistency.
2025-12-22 22:42:56 +01:00
Till Wegmueller
a921c99eb6
Reorder struct fields for consistency and align with Rust formatting standards. 2025-12-22 20:11:08 +01:00
Till Wegmueller
d2d1c297cc
Refactor to align with Rust formatting guidelines and enhance code readability.
- Adjusted indentation and line breaks for structs, functions, and method calls to comply with Rust formatting standards.
- Improved error message formatting and consistency across `PkgTreeError` instances.
- Restructured long function arguments and chained calls for clarity and maintainability.
- Simplified conditional statements and loops for better readability.
- No functional changes introduced.
2025-12-22 20:10:17 +01:00
Till Wegmueller
d0fcdbec20
Update Vagrantfile with memory configuration, synced folder, and provisioning enhancements
- Added provider-specific memory allocation (8192 MB) for VirtualBox and Libvirt.
- Configured synced folder using rsync with `.git/` and `target/` exclusions.
- Streamlined provisioning by replacing GCC dependencies with Rust and updating package installation list.
- Removed redundant provisioning script for Rust setup.
2025-12-22 19:39:15 +01:00
Till Wegmueller
81f00a0624
Add caching headers, admin routes, and configurable cache max-age support
- Introduced caching headers (`Cache-Control`, `ETag`, `Last-Modified`) for file responses to improve client-side caching.
- Added HTTP admin routes for health check and authentication validation.
- Made `cache_max_age` configurable via server configuration with a default of 3600 seconds.
- Enhanced file handler with metadata-based `Last-Modified` computation.
- Updated integration tests to cover new functionality and ensure correctness.
2025-12-09 20:23:00 +01:00
Till Wegmueller
3457b4acba
Add version parsing, FMRI formatting, and license content resolution
- Enhanced `get_info` handler to parse version components into `Build Release`, `Branch`, and `Packaging Date`.
- Added proper FMRI formatting in `pkg://<publisher>/<name>@<version>` format.
- Implemented license content resolution with file-based lookup, gzip decompression, and content truncation.
- Introduced compressed and uncompressed package size calculations for manifest actions.
- Updated dependencies to include `chrono` and `flate2` for date parsing and gzip decompression.
2025-12-09 16:42:21 +01:00
Till Wegmueller
cff3d5d960
Add content-type handling for JSON catalog artifacts and weak ETag for manifests
- Ensured correct `Content-Type` header for catalog artifacts (`catalog.attrs` and `catalog.*`) in HTTP responses.
- Added SHA-1 based weak ETag generation for manifest responses to improve caching and legacy compatibility.
- Updated `integration_tests` to validate content-type and ETag correctness.
- Added new dependency `sha1` for hashing support.
2025-12-09 16:02:02 +01:00
Till Wegmueller
e87d1a3166
Add batched catalog rebuild support and INI-only repo test
- Introduced `BatchOptions` for configurable batched catalog rebuilds.
- Added `rebuild_catalog_batched` in `file_backend` for batched processing with deterministic package order and progress tracking.
- Updated `rebuild_catalog` to leverage the new batched implementation with default options.
- Added integration test `test_ini_only_repo_serving_catalog` to validate functionality with an INI-only repository configuration.
- Enhanced test coverage for legacy catalog handling via `pkg5.repository`.
2025-12-09 14:23:55 +01:00
Till Wegmueller
bd67e06012
Add catalog_writer module for atomic catalog updates with SHA-1 signatures
- Introduced `write_catalog_attrs`, `write_catalog_part`, and `write_update_log` functions for streamlined and secure file writing.
- Refactored `file_backend` to use `catalog_writer` for managing catalog updates, improving readability and maintainability.
- Updated `save_catalog_attrs`, `save_catalog_part`, and `append_update` to leverage atomic writes and ensure signature computation.
- Replaced manual serialization logic with centralized writing utilities for consistency and error resilience.
- Updated dependencies for JSON handling and signature computation.
2025-12-09 12:49:25 +01:00
Till Wegmueller
a948f87e6f
Add legacy repository support and SHA-1 signature handling
- Introduced fallback for legacy `pkg5.repository` configuration in INI format alongside the existing `pkg6.repository` JSON format.
- Enabled SHA-1 signature computation for compatibility with legacy catalog signatures.
- Added methods to save update logs in legacy format and enhance catalog compatibility.
- Updated dependencies to include `sha1` for hashing.
2025-12-09 12:12:57 +01:00
Till Wegmueller
c4910bb434
Refactor error handling in repository module to include structured error variants with detailed context.
- Replaced string-based `FileReadError`, `FileWriteError`, and `DirectoryCreateError` with structured error types using `PathBuf` and `source`.
- Updated all affected methods in `rest_backend`, `file_backend`, and tests to use the new structured error variants.
- Improved error messages for clarity and debugging by embedding file paths and sources directly into error data.
2025-12-08 23:13:27 +01:00
Till Wegmueller
ee02bf3cf0
Simplify config loading by replacing match with unwrap_or_else for improved readability. 2025-12-08 22:49:35 +01:00
Till Wegmueller
81bc7b8574
Refactor catalog handling by replacing get_legacy_catalog with get_catalog_file_path for improved clarity and consistency.
- Removed now-redundant catalog path resolution logic in `repo.rs`.
- Updated `file_backend` with a unified method for fetching catalog file paths, addressing invalid path prefixes.
- Adjusted HTTP handler to use the updated method.
2025-12-08 22:45:39 +01:00
Till Wegmueller
55decc16ff
Simplify CatalogPart::load by removing manual JSON parsing fallbacks and introducing test coverage for sample catalogs. 2025-12-08 22:39:28 +01:00
Till Wegmueller
84b2c50ed6
Remove deprecated Catalog v0 support and refactor legacy catalog handling.
- Removed `get_catalog` handler and associated Catalog v0 route.
- Updated `get_legacy_catalog` to accept `filename` for enhanced flexibility.
- Adjusted integration tests to validate changes and focus on Catalog v1 support.
- Refactored `SupportedOperation` definitions to exclude Catalog v0.
2025-12-08 22:10:11 +01:00
Till Wegmueller
048dddf487
Refactor get_versions response with structured VersionsResponse model.
- Added `Operation` enum and `SupportedOperation` struct to encapsulate supported operations and versions.
- Updated response generation to dynamically include the `pkg6depotd` version and supported operations.
- Enhanced integration tests to validate version string format changes.
2025-12-08 21:41:02 +01:00
Till Wegmueller
0b3a974ca6
Add REST API v1 endpoints and legacy catalog handling for pkg6depotd
- Expanded repository structure by introducing methods for fetching legacy catalogs, catalog file paths, and repository info.
- Added new REST API v1 endpoints for catalog, manifest, file, and publisher handling.
- Implemented `publisher` handler module with `get_publisher_v0` and `get_publisher_v1` methods to retrieve publisher details in pkg5 format.
- Updated `integration_tests` to validate new endpoints and ensure compatibility with legacy and modern catalog/manifest handling.
- Removed unused dependency `walkdir` and refactored test cases for clarity and efficiency.
2025-12-08 21:36:37 +01:00
Till Wegmueller
cd15e21420
Add repository handling and foundational HTTP routes for pkg6depotd
- Implemented `DepotRepo` for repository access, including methods for catalog path, file path, and manifest retrieval.
- Introduced foundational HTTP routes for catalog, manifest, file, and package info retrieval.
- Added integration tests to validate repository setup and basic server functionality.
- Modularized HTTP handlers for better maintainability and extended them with new implementations like `info` and `manifest` handling.
- Refactored `main` function to simplify initialization and leverage reusable `run` logic in a new `lib.rs`.
- Updated `Cargo.toml` and `Cargo.lock` to include new dependencies: `walkdir` and updated testing utilities.
2025-12-08 20:50:20 +01:00
Till Wegmueller
f2a3bc4d7c
Introduce foundational structure for pkg6depotd
- Added initial implementation of the `pkg6depotd` server with modular components for CLI parsing, configuration management, HTTP handling, repository access, and daemonization.
- Implemented basic server startup logic with a default router and placeholder handlers.
- Integrated telemetry initialization and configuration fallback mechanism for ease of development.
- Updated `Cargo.toml` and `Cargo.lock` to include dependencies necessary for server functionality.
2025-12-08 20:11:05 +01:00
Till Wegmueller
340e58ca09
Add comprehensive documentation for IPS search index design and implementation
- Documented indexer workflow, consistency mechanisms, and file layout for client-side and server-side indexing.
- Detailed fast incremental updates and full rebuild criteria with corresponding file modifications.
- Specified encoding formats, invariants, and validation rules for on-disk index files.
- Enhanced developer documentation for maintaining and troubleshooting the search index.
- Updated VCS configuration to remove unused submodule mapping.
2025-12-08 20:10:30 +01:00
Till Wegmueller
6be608164d
Add detailed documentation for Redb-based IPS Search Index schema and encoding plans
- Introduced a series of planning documents detailing the Redb-based IPS index design, schema specification, and encoding strategies.
- Added a high-level overview of the core search index schema and the use of Redb MVCC transactions for consistency and performance improvements.
- Documented simplified schema definitions avoiding optional elements, focusing on compact encodings.
- Defined transitions to postcard-encoded binary formats, aligning with Rust’s serde for standardized serialization.
- Outlined migration strategies, invariants, error handling, and testing plans for index adoption.
- Enhanced documentation with structured explanations for developers to implement, extend, and migrate seamlessly to the new index model.
2025-12-08 20:10:04 +01:00
Till Wegmueller
33dd228df7
Update VCS mappings to include oi-userland 2025-09-02 16:05:53 +02:00
Till Wegmueller
1198a6a934
Remove Submodule
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-08-31 10:44:35 +02:00
Till Wegmueller
23815a2aab
Expand ManifestBuilder with new helper methods for setting attributes, licenses, links, and dependencies.
- Added `add_set`, `add_license`, `add_link`, and `add_depend` methods for streamlined manifest construction.
- Updated documentation with new examples showcasing the extended `ManifestBuilder` API.
- Enhanced test coverage to validate the new helper methods.
2025-08-31 00:17:15 +02:00
Till Wegmueller
d78cd9f659
Refactor depend module
- Simplified `split_dir_base` signature by removing unnecessary lifetime annotation.
- Refactored `compute_python_runpaths` to eliminate mutable variable and streamline `insert_default_runpath`.
2025-08-30 23:26:17 +02:00
Till Wegmueller
29ef35f350
Add api module for forge/pkgdev high-level integration
- Introduced `api.rs` to provide a stable, struct-first API surface for building, linting, resolving, and publishing IPS packages.
- Encapsulated existing functionality for better abstraction and usability in external integrations.
- Enhanced `libips` with a high-level repository and transaction interface.
- Added support for dependency generation, manifest transformations, and linting via configurable rulesets.
- Updated documentation with integration flow examples and usage guidelines.
2025-08-30 22:25:45 +02:00
Till Wegmueller
a156a3f863
Remove pkg6dev module and related code
- Deleted `pkg6dev` crate, including all related files (`src`, `Cargo.toml`, `Cargo.lock`, and `README.md`).
- Removed error handling, CLI logic, and publishing functionality from the project.
- Migrated relevant high-level APIs to `libips`, ensuring consistency with existing modules.
- Updated `libips` with a new `api` module to encapsulate `forge/pkgdev` integration logic.
- Simplified repository structure to exclude now-unused components and dependencies.
2025-08-30 21:59:52 +02:00
Till Wegmueller
77f02fdfbd
Add depend module for file-level dependency generation
- Introduced `depend.rs` to handle dependency generation for ELF files, scripts, Python modules, and SMF manifests.
- Implemented file classification and analysis logic with configurable bypass rules and runpath handling.
- Added utility functions to resolve file dependencies into manifest actions using a provided repository.
- Updated `Cargo.toml` with `goblin` dependency for ELF processing.
- Enhanced codebase with default runpath insertion, dynamic token expansion, and Python module import detection.
- Included `pkgdepend` documentation for dependency resolution overview.
2025-08-30 18:35:41 +02:00
Till Wegmueller
7cffa6c4e6
Add transformer module for manifest transformation logic
- Introduced `transformer.rs` with a structured approach for parsing and applying transformation rules.
- Added support for operations like `add`, `default`, `delete`, `drop`, `edit`, `emit`, and `set` on attributes, files, directories, and other targets.
- Implemented regex-based matching for patterns and backreference handling in transformations.
- Enhanced manifest modification functionality, including attribute/facet operations and deferred action emission.
- Added comprehensive unit tests to validate transformation rules and their applications.
2025-08-28 23:50:59 +02:00
Till Wegmueller
88e06b488d
Refactor catalog processing to merge parts by version
- Introduced `process_publisher_merged` to consolidate catalog parts per `stem@version` before processing.
- Replaced part-wise processing with merged version logic for efficiency and improved determinism.
- Added `process_catalog_part` dead code annotation and cleaned up outdated logic.
- Enhanced handling of merged actions and signature deduplication across catalog parts.
- Improved logging for import and obsolete package metrics.
2025-08-27 00:05:36 +02:00
Till Wegmueller
6ed8bb5b5b
Improve dependency constraint extraction and branch locking logic
- Updated `extract_constraint` to derive constraints from both optional properties and dependency FMRI version strings.
- Ensured child dependencies inherit parent's branch when not explicitly constrained, mirroring solver behavior.
2025-08-26 21:51:05 +02:00
Till Wegmueller
d77e61f90f
Add solver advice system for user-focused installation guidance
- Introduced a new `advice` module in the solver to provide user-oriented guidance for resolving dependency issues.
- Added structured failure details to `SolverError` for improved diagnosis and advice generation.
- Updated `pkg6` to display actionable advice when dependency resolution fails, including catalog refresh, publisher verification, and incorporation constraints.
- Enhanced CLI tools with a dangling dependency scan to identify and output unresolved stems.
- Improved modularity and caching mechanisms for catalog and manifest handling to support the advice system.
- Refined error messages and logging for better user feedback during installation errors.
2025-08-26 21:09:06 +02:00
Till Wegmueller
8bdc6d6641
Enhance pkgtree with advisor and error analysis modes
- Added advisor mode to analyze package installability with recursive dependency checks.
- Introduced solver error file analysis to diagnose and suggest fixes for specific install issues.
- Updated CLI options to support advisor and solver error analysis functionalities.
- Improved dependency processing logic to filter by applicable dependency types.
- Added support for configurable recursion depth and dependency limits in advisor mode.
- Enhanced graph mode logic to prioritize error analysis and provide better logging for failure cases.
- Included new helper functions for version parsing, dependency constraint extraction, and caching.
- Added tests for version parsing and dependency handling.
2025-08-26 14:53:31 +02:00
Till Wegmueller
c8a6c8c781
Introduce pkgtree: CLI tool for analyzing IPS package dependency trees
- Added `pkgtree` with functionalities to analyze dependency trees and detect cycles in IPS package images.
- Implemented CLI options for filtering by publisher, package substring, and output format (Tree/JSON).
- Integrated cycle detection and suggestions for resolving dependency cycles.
- Added testing support for cycle detection.
- Updated workspace dependencies and added `pkgtree` to the workspace configuration.
2025-08-26 12:38:36 +02:00
Till Wegmueller
9ac8f98b38
Refactor solver and manifest handling
- Replaced `CatalogProvider` with database-backed solution, improving manifest retrieval logic.
- Added fallback and LZ4 decoding support for catalog-stored manifests.
- Enhanced incorporation lock handling with direct database queries.
- Updated sample install script to use `debug` logging for better traceability.
2025-08-21 23:52:11 +02:00
Till Wegmueller
e4bd9a748a
Refactor ImageCatalog and apply manifest logic
- Added support for obsoleted package database with separate handling in `ImageCatalog`.
- Enhanced apply manifest functionality with progress callback support and processing statistics.
- Introduced LZ4 compression for manifest storage in `ImageCatalog`.
- Removed debugging eprintln statements, replaced with structured logging.
- Updated `pkg6` image creation and installation logic to improve user feedback and error handling.
- Updated database initialization and build processes to handle new obsoleted logic.
2025-08-19 22:43:50 +02:00
Till Wegmueller
77147999b3
Add incorporation lock support and update dependencies across the project
- Implemented incorporation lock functionality for packages, including pre-checks, addition, and retrieval methods.
- Enhanced package installation logic to handle incorporation constraints.
- Updated dependency versions in `Cargo.toml` and `Cargo.lock`, aligning with latest compatible releases.
- Introduced new tests to verify incorporation lock behavior, including scenarios with and without locks.
- Refactored code for improved readability and added error handling improvements.
- Fixed minor typos and inconsistencies in comments and log messages.
2025-08-19 14:30:55 +02:00