Commit graph

34 commits

Author SHA1 Message Date
Till Wegmueller
e236f30f6e
Convert trait methods to use &self instead of &mut self, introduce Mutex for interior mutability, optimize HTTP client creation, and implement parallel payload processing using Rayon. 2026-02-05 15:57:56 +01:00
Till Wegmueller
def11a1dfb
Refactor: Replace redb with rusqlite for catalog handling
- Transitioned the catalog backend from `redb` to `rusqlite` for better compatibility and concurrency.
- Updated `IpsProvider` to use SQLite for package querying, dependency resolution, and obsolescence handling.
- Removed `decode_manifest_bytes_local` and unused `manifest` cache logic.
- Simplified catalog-related functions and integrated `sqlite_catalog` module.
- Enhanced test functions and added schemas for managing SQLite databases.
2026-02-04 22:39:42 +01:00
Till Wegmueller
4ab529f4c7
chore(format): format code 2026-01-25 23:17:49 +01:00
Till Wegmueller
1c0619ca55
Add pkg6recv package for receiving IPS repositories.
- Introduced `pkg6recv` for downloading packages from IPS repositories with support for recursive dependency fetching.
- Implemented CLI commands with `clap` for source/destination setup, package selection, and publisher defaults.
- Added `ConsoleProgressReporter` for detailed receive progress visibility.
- Enhanced the `Manifest` parser to support JSON format alongside IPS text format.
- Updated `FileBackend` and `RestBackend` repositories to fetch manifests and payloads in new operations.
- Extended `Digest` utilities for payload integrity checks.
- Added tests to verify basic functionality and manifest format preservation.
2026-01-20 20:16:58 +01:00
Till Wegmueller
22178cffd7
Add manual testing setup for pkg6depotd
- Introduced scripts and configurations for manual testing using `anyvm` with OpenIndiana and OmniOS.
- Implemented repository fetching (`fetch_repo.sh`) and server startup (`run_depotd.sh`) scripts.
- Enhanced `pkg6depotd` to support default publisher routes and trailing slashes.
- Updated integration tests to verify new publisher route behavior.
2026-01-20 17:44:36 +01:00
Till Wegmueller
898ec20ad8
Update binaries and workflows for pkg6 rename
- Renamed `pkg6dev` to `pkg6` across build scripts, workflows, and documentation.
- Added support for additional binaries (`pkg6repo` and `pkg6depotd`) in release builds.
- Disabled `RUSTFLAGS` warnings-as-errors policy in workflows for improved flexibility.
- Simplified error handling in `manifest_fmri` with streamlined conditionals.
- Introduced `#[allow(clippy::result_large_err)]` to suppress clippy warnings for large error types.
2026-01-18 14:30:05 +01:00
Till Wegmueller
b8c625a11e
Refactor attribute value access and error handling
- Replaced `values.get(0)` with `values.first()` for improved clarity and consistency across multiple files.
- Simplified conditional checks with `.is_empty()` and `format_args!` to enhance readability.
- Implemented `Default` trait for `VariableMode` directly within the enum.
- Updated error handling by boxing errors in `RepositoryError` and `CatalogError`.
- Replaced redundant `else` blocks with streamlined logic.
2026-01-18 13:00:21 +01:00
Till Wegmueller
8f089656ba
Add search functionality to repository and route handlers (currently disabled)
- Introduced a searchable index with structured `IndexEntry` support for packages, files, directories, and dependencies.
- Added `search` method in `DepotRepo` with wildcard and case-sensitive query handling.
- Created `/search/0` and `/search/1` routes for search API, supporting publishers and token-based queries.
- Updated `SearchIndex` handling to map tokens to detailed `IndexEntry` structures.
- Improved index building to include attributes for files, directories, and dependencies.
2026-01-18 12:29:44 +01:00
Till Wegmueller
d2d1c297cc
Refactor to align with Rust formatting guidelines and enhance code readability.
- Adjusted indentation and line breaks for structs, functions, and method calls to comply with Rust formatting standards.
- Improved error message formatting and consistency across `PkgTreeError` instances.
- Restructured long function arguments and chained calls for clarity and maintainability.
- Simplified conditional statements and loops for better readability.
- No functional changes introduced.
2025-12-22 20:10:17 +01:00
Till Wegmueller
e87d1a3166
Add batched catalog rebuild support and INI-only repo test
- Introduced `BatchOptions` for configurable batched catalog rebuilds.
- Added `rebuild_catalog_batched` in `file_backend` for batched processing with deterministic package order and progress tracking.
- Updated `rebuild_catalog` to leverage the new batched implementation with default options.
- Added integration test `test_ini_only_repo_serving_catalog` to validate functionality with an INI-only repository configuration.
- Enhanced test coverage for legacy catalog handling via `pkg5.repository`.
2025-12-09 14:23:55 +01:00
Till Wegmueller
bd67e06012
Add catalog_writer module for atomic catalog updates with SHA-1 signatures
- Introduced `write_catalog_attrs`, `write_catalog_part`, and `write_update_log` functions for streamlined and secure file writing.
- Refactored `file_backend` to use `catalog_writer` for managing catalog updates, improving readability and maintainability.
- Updated `save_catalog_attrs`, `save_catalog_part`, and `append_update` to leverage atomic writes and ensure signature computation.
- Replaced manual serialization logic with centralized writing utilities for consistency and error resilience.
- Updated dependencies for JSON handling and signature computation.
2025-12-09 12:49:25 +01:00
Till Wegmueller
c4910bb434
Refactor error handling in repository module to include structured error variants with detailed context.
- Replaced string-based `FileReadError`, `FileWriteError`, and `DirectoryCreateError` with structured error types using `PathBuf` and `source`.
- Updated all affected methods in `rest_backend`, `file_backend`, and tests to use the new structured error variants.
- Improved error messages for clarity and debugging by embedding file paths and sources directly into error data.
2025-12-08 23:13:27 +01:00
Till Wegmueller
7cffa6c4e6
Add transformer module for manifest transformation logic
- Introduced `transformer.rs` with a structured approach for parsing and applying transformation rules.
- Added support for operations like `add`, `default`, `delete`, `drop`, `edit`, `emit`, and `set` on attributes, files, directories, and other targets.
- Implemented regex-based matching for patterns and backreference handling in transformations.
- Enhanced manifest modification functionality, including attribute/facet operations and deferred action emission.
- Added comprehensive unit tests to validate transformation rules and their applications.
2025-08-28 23:50:59 +02:00
Till Wegmueller
77147999b3
Add incorporation lock support and update dependencies across the project
- Implemented incorporation lock functionality for packages, including pre-checks, addition, and retrieval methods.
- Enhanced package installation logic to handle incorporation constraints.
- Updated dependency versions in `Cargo.toml` and `Cargo.lock`, aligning with latest compatible releases.
- Introduced new tests to verify incorporation lock behavior, including scenarios with and without locks.
- Refactored code for improved readability and added error handling improvements.
- Fixed minor typos and inconsistencies in comments and log messages.
2025-08-19 14:30:55 +02:00
Till Wegmueller
d483e2a995
implement manifest fetching and fix CI errors 2025-08-13 23:23:45 +02:00
Till Wegmueller
e27cd35d8d
Add ImageCatalog and InstalledPackages modules for package management
- Introduced `ImageCatalog` for handling catalog-related operations, including initialization, building from publishers, querying, and manifest management.
- Added `InstalledPackages` for managing installed package operations, such as adding, removing, querying, and retrieving manifests.
- Defined database schemas and error types to support both modules.
- Refactored table handling, ensuring consistent transaction scoping with `redb`.
2025-08-04 22:01:38 +02:00
Till Wegmueller
2777f153c9
Introduce progress reporting infrastructure and error management refinements
- Add `ProgressReporter` trait with `start`, `update`, and `finish` methods for customizable progress tracking during operations.
- Implement `ProgressInfo` struct for detailed progress state reporting, including operation, progress, and context information.
- Create a no-op implementation `NoopProgressReporter` for cases where progress reporting is not needed.
- Enhance `pkg6` with structured error handling using `Pkg6Error` and new diagnostic variants for I/O, JSON, FMRI, and logging errors.
- Update `Image` functionality to leverage progress reporting during catalog downloads and include robust publisher management.
- Introduce `TestProgressReporter` for unit tests to validate progress reporting integration.
2025-08-03 14:28:36 +02:00
Till Wegmueller
9b2f74c5c1
Introduce obsoleted package management system in IPS
- Add `obsoleted.rs` module to handle storing, metadata management, and operations for obsoleted packages.
- Implement commands for marking, listing, searching, restoring, exporting, and importing obsoleted packages (`pkg6repo`).
- Enhance `RepositoryError` with `From` implementations for various error types to manage database and serialization-related errors.
- Introduce reusable data structures for obsoleted package metadata and export representation.
- Update `Cargo.toml` and `Cargo.lock` to include new dependencies (`redb`, `bincode`, etc.).
- Document obsoleted package workflow and integration details in `doc/obsoleted_packages.md` for contributors.
- Refactor repository internals to integrate obsoleted package support without disrupting existing workflow.
- Add robust error handling, logging, and pagination for enhanced usability and scalability.
2025-07-29 16:16:12 +02:00
Till Wegmueller
21de26ae82
Refactor codebase to improve formatting, logging clarity, and error handling
- Standardize formatting by aligning and cleaning up indentation, spacing, and line breaks across multiple modules.
- Enhance logging with improved message formatting for better context and readability.
- Simplify error handling in `pkg6dev`, `pkg6repo`, and core libraries by consolidating redundant patterns and improving clarity.
- Update test cases to reflect formatting and logging changes while extending coverage of edge cases.
- Perform general cleanup, including removing unused imports, refining comments, and ensuring consistent style.
2025-07-27 15:22:49 +02:00
Till Wegmueller
f4d1ce73b6
Refactor repository operations to use immutable references
- Modify `rebuild` and `refresh` methods to take `&self` instead of `&mut self`.
- Replace mutable repository instances in `pkg6dev`, `pkg6repo`, and core library with immutable ones.
- Introduce `debug` logging for enhanced traceability during metadata rebuild and refresh.
- Extract and simplify catalog rebuild logic into a dedicated `rebuild_catalog` method.
- General cleanup: remove redundant methods, reorganize logic, and refine comments across affected files.
2025-07-27 13:43:56 +02:00
Till Wegmueller
7f4ecf6346
Organize repository files into hierarchical directory structure for improved scalability
- Update `FileBackend` to store files in a multi-level directory structure (`xx/yy/hash`) based on hash prefixes.
- Adjust method signatures and logic across `FileBackend`, `pkg6dev`, and `pkg6repo` to support mutable repository operations.
- Add test cases and scripts (`test_file_structure.rs`, `test_repo_operations.sh`) to validate the new file structure.
- Refactor catalog manager to use `RefCell` for interior mutability to enable lazy initialization.
- Clean up redundant and dead code, enhance comments, and replace outdated logic for file handling and metadata operations.
2025-07-27 13:15:52 +02:00
Till Wegmueller
4649608408
Replace anyhow with custom RepositoryError for improved error specificity and consistency. Remove anyhow dependency. 2025-07-26 16:50:10 +02:00
Till Wegmueller
a33a3246b6
chore(fmt): format with cargo fmt 2025-07-26 12:54:01 +02:00
Till Wegmueller
a5cdb7e673
Add JSON and TSV output to repository commands, integrate serde for struct serialization, and introduce search subcommand with flexible filtering options. 2025-07-26 12:35:44 +02:00
Till Wegmueller
4f0e2fe066
Add unit and end-to-end tests for repository functionality, including setup scripts, package management operations, and TESTING.md documentation. 2025-07-26 10:52:25 +02:00
Till Wegmueller
ead9ad12af
Refactor repository traits into ReadableRepository and WritableRepository, introduce Repository to combine traits for backward compatibility, and update relevant modules and tests to use the new structure. 2025-07-26 10:34:45 +02:00
Till Wegmueller
a0fcc13033
Introduce Catalog module with structs and functionality for managing package metadata, enhance file_backend and CLI to handle catalog operations, and update dependencies (chrono and others) to support new features. 2025-07-24 00:28:33 +02:00
Till Wegmueller
63f2d1da62
Add search method to repository module for querying packages, update dependencies to support search functionality. 2025-07-23 23:22:24 +02:00
Till Wegmueller
2cb63fbef0
Introduce PackageContents struct for structured representation of package data, refactor show_contents for multiple content types, and enhance handling of manifests with improved filtering and output formatting. 2025-07-23 22:39:49 +02:00
Till Wegmueller
5d987ca0cb
Refactor package FMRI handling by introducing Fmri struct, update dependencies, and adjust repository and action modules for structured FMRI data processing
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-22 14:10:37 +02:00
Till Wegmueller
e3662eaf23
Introduce PackageInfo struct for structured package data, refactor list_packages to use it, and enhance publisher directory handling
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-22 11:57:24 +02:00
Till Wegmueller
7332e0f7b5
Refactor repository info retrieval to return structured data, add detailed publisher information, and update tests accordingly
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-22 10:21:16 +02:00
Till Wegmueller
a9584fa6d2
Add package publishing functionality and default publisher management
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-21 23:20:19 +02:00
Till Wegmueller
1a5e0e053d
Add repository module and update dependencies
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-21 22:02:05 +02:00