Commit graph

167 commits

Author SHA1 Message Date
Till Wegmueller
99dd0fe87c
Refactor repository structure to use publisher-specific directories
- Introduce publisher-specific hierarchy (`publisher/<publisher_name>`) for catalog, package, and file storage.
- Replace hardcoded paths with helper methods (`construct_catalog_path`, `construct_package_dir`, `construct_file_path_with_publisher`) for consistent path generation.
- Remove outdated tests and simplify redundant assertions in repository tests.
- Update `CatalogManager` and file handling logic to leverage publisher-specific paths.
- Enhance debug logs to include publisher context where applicable.
2025-07-31 00:18:21 +02:00
Till Wegmueller
845ffec13b
Switch to CBOR for metadata serialization and simplify key management
- Replace `bincode` serialization with `CBOR` for obsoleted package metadata to improve interoperability and error messaging.
- Simplify key handling by directly using FMRI strings as keys, removing the need for intermediate serialization steps.
- Update deserialization, error handling, and logging to align with `CBOR` serialization.
- Add `serde_cbor` as a dependency, updating `Cargo.toml` and `Cargo.lock` accordingly.
- Refactor `ObsoletedPackageKey` to include conversion between FMRI strings and data components.
2025-07-29 23:14:47 +02:00
Till Wegmueller
a5f1b7d815
Refactor obsoleted.rs to optimize database schema and enhance error handling
- Replace `FMRI_TO_HASH_TABLE` and `HASH_TO_METADATA_TABLE` with a unified `FMRI_TO_METADATA_TABLE` for direct FMRI-to-metadata mapping, reducing complexity and eliminating intermediate hash lookups.
- Implement robust error handling and logging in database transactions, serialization, and deserialization procedures.
- Introduce on-the-fly manifest generation for entries with `NULL_HASH`, streamlining workflows for minimal manifest creation.
- Update indexing, querying, and removal logic to align with the new table and error handling improvements.
- Streamline `list_obsoleted_packages` by introducing index-based fetching with structured fallbacks to filesystem methods.
2025-07-29 21:24:32 +02:00
Till Wegmueller
5b4b719b42
Introduce cleanup functionality for obsoleted packages with TTL support
- Add methods to find and clean up obsoleted packages older than a specified TTL (`find_obsoleted_packages_older_than_ttl` and `cleanup_obsoleted_packages_older_than_ttl`) in `libips`.
- Implement a new `CleanupObsoleted` command in `pkg6repo` to handle cleanup operations.
- Update workspace `Cargo.toml` files with unified attributes for better consistency.
- Adjust dependencies (`libips` version patterns, `thiserror` updates) and enhance metadata management for obsoleted packages.
- Enhance repository operations by adding batch processing and robust logging during cleanup.
2025-07-29 19:08:18 +02:00
Till Wegmueller
7633feb36f
Add support for null hash in obsoleted package metadata
- Introduce `NULL_HASH` constant to indicate the absence of original manifest storage.
- Add `new_with_null_hash` method to create metadata for obsoleted packages without storing full manifests.
- Implement on-the-fly generation of minimal manifests with obsoletion attributes for packages with null hash.
- Update `pkg5_import` to use null hash for obsoleted packages, reducing storage requirements.
- Document null hash usage and its benefits in `obsoleted_packages.md`.
2025-07-29 18:40:34 +02:00
Till Wegmueller
9b2f74c5c1
Introduce obsoleted package management system in IPS
- Add `obsoleted.rs` module to handle storing, metadata management, and operations for obsoleted packages.
- Implement commands for marking, listing, searching, restoring, exporting, and importing obsoleted packages (`pkg6repo`).
- Enhance `RepositoryError` with `From` implementations for various error types to manage database and serialization-related errors.
- Introduce reusable data structures for obsoleted package metadata and export representation.
- Update `Cargo.toml` and `Cargo.lock` to include new dependencies (`redb`, `bincode`, etc.).
- Document obsoleted package workflow and integration details in `doc/obsoleted_packages.md` for contributors.
- Refactor repository internals to integrate obsoleted package support without disrupting existing workflow.
- Add robust error handling, logging, and pagination for enhanced usability and scalability.
2025-07-29 16:16:12 +02:00
Till Wegmueller
88b55c4a70
Add serde attribute defaults and improve JSON deserialization handling in manifests
- Introduce `#[serde(skip_serializing_if = "is_empty", default)]` for various optional fields to streamline serialization and ensure defaults are applied during deserialization.
- Add `tracing::debug` logging for enhanced error context in JSON deserialization fallback logic.
- Update tests to reflect changes in manifest parsing, including new cases for the updated JSON format.
2025-07-29 11:38:36 +02:00
Till Wegmueller
81eb4a7447
Use walkdir for efficient recursive traversal in FileBackend::build_search_index
- Replace manual recursive directory scanning with `walkdir` for simplified and efficient file traversal.
- Update dependencies in `Cargo.toml` and `Cargo.lock` to include `walkdir`.
- Enhance code maintainability by replacing redundant logic with streamlined traversal and processing.
2025-07-27 19:48:52 +02:00
Till Wegmueller
1286db23fd
Implement two-level file path hierarchy in pkg5_import.rs and remove redundant e2e tests
- Introduce a two-level directory structure for file paths (`xx/yy/hash`) in `pkg5_import.rs` for improved scalability, with fallback to the old structure for compatibility.
- Remove outdated end-to-end tests for `pkg5` imports, reducing redundancy and streamlining test coverage.
2025-07-27 16:27:22 +02:00
Till Wegmueller
87eea546aa
Remove obsolete test_manifest_parsing.rs and refactor FileBackend for robust manifest handling
- Eliminate redundant `test_manifest_parsing.rs` as its functionality is covered during `FileBackend` implementation.
- Define `PackageContentVectors` for streamlined data organization.
- Enhance manifest handling in `FileBackend` with recursive search, improved content extraction, and robust error handling.
- Simplify and centralize manifest parsing logic for maintainability and code clarity.
2025-07-27 16:12:59 +02:00
Till Wegmueller
21de26ae82
Refactor codebase to improve formatting, logging clarity, and error handling
- Standardize formatting by aligning and cleaning up indentation, spacing, and line breaks across multiple modules.
- Enhance logging with improved message formatting for better context and readability.
- Simplify error handling in `pkg6dev`, `pkg6repo`, and core libraries by consolidating redundant patterns and improving clarity.
- Update test cases to reflect formatting and logging changes while extending coverage of edge cases.
- Perform general cleanup, including removing unused imports, refining comments, and ensuring consistent style.
2025-07-27 15:22:49 +02:00
Till Wegmueller
abb997df43
Refactor FileBackend for improved manifest handling and directory structure
- Extract manifest path construction and recursive manifest search into helper methods (`construct_manifest_path` and `find_manifests_recursive`).
- Enhance package information extraction by separating stem and version details from FMRI.
- Adjust directory creation logic to include per-package subdirectories for better organization.
- Simplify and centralize manifest parsing logic, improving error handling and maintainability.
2025-07-27 15:20:45 +02:00
Till Wegmueller
f4d1ce73b6
Refactor repository operations to use immutable references
- Modify `rebuild` and `refresh` methods to take `&self` instead of `&mut self`.
- Replace mutable repository instances in `pkg6dev`, `pkg6repo`, and core library with immutable ones.
- Introduce `debug` logging for enhanced traceability during metadata rebuild and refresh.
- Extract and simplify catalog rebuild logic into a dedicated `rebuild_catalog` method.
- General cleanup: remove redundant methods, reorganize logic, and refine comments across affected files.
2025-07-27 13:43:56 +02:00
Till Wegmueller
7f4ecf6346
Organize repository files into hierarchical directory structure for improved scalability
- Update `FileBackend` to store files in a multi-level directory structure (`xx/yy/hash`) based on hash prefixes.
- Adjust method signatures and logic across `FileBackend`, `pkg6dev`, and `pkg6repo` to support mutable repository operations.
- Add test cases and scripts (`test_file_structure.rs`, `test_repo_operations.sh`) to validate the new file structure.
- Refactor catalog manager to use `RefCell` for interior mutability to enable lazy initialization.
- Clean up redundant and dead code, enhance comments, and replace outdated logic for file handling and metadata operations.
2025-07-27 13:15:52 +02:00
Till Wegmueller
633e742528
Remove backward compatibility logic for root pkg directory manifest copy in file_backend.rs 2025-07-27 12:26:57 +02:00
Till Wegmueller
d00de23d52
Simplify pkg5_import.rs by removing redundant directory creation logic; delegate to Transaction.commit() for handling. 2025-07-27 12:19:38 +02:00
Till Wegmueller
ea5eddbc6a
Simplify pkg5_import.rs by removing redundant directory creation logic; delegate to Transaction.commit() for handling. 2025-07-27 12:05:22 +02:00
Till Wegmueller
9b271d81b5
Add support for User, Group, Driver, Legacy, and Transform actions in Manifest parsing
- Implement parsing and mapping of new action types: `User`, `Group`, `Driver`, `Legacy`, and `Transform` within `Manifest`.
- Introduce comprehensive test coverage for the newly supported actions, including edge cases for `ftpuser` property in manifests.
- Refactor to leverage `HashSet` for user `services` to avoid duplicates.
- Improve comments and error handling during action parsing for enhanced clarity and robustness.
2025-07-27 11:30:24 +02:00
Till Wegmueller
7889dffdea
Update workspace structure, unify metadata, enhance logging, and expand error handling
- Centralize package metadata in workspace for consistency.
- Upgrade `pkg6dev` and `pkg6repo` to version `0.5.1`, aligning with new workspace configuration.
- Introduce `EnvFilter` for dynamic logging configuration, defaulting to debug level.
- Add `LoggingEnvError` variant to `Pkg6DevError` with structured diagnostics for improved error handling.
2025-07-27 11:08:39 +02:00
Till Wegmueller
bf5d60c77c
Update Cargo.lock dependencies to latest versions to ensure compatibility and improve functionality. 2025-07-27 11:02:37 +02:00
Till Wegmueller
8760bf0c4d
Remove unused imports and the synchronous flag in pkg6repo, adjust comments for clarity and grammar, and refine logging statements. 2025-07-27 10:42:03 +02:00
Till Wegmueller
a3b2686b0a
Refine imports and comments in pkg5_import.rs for clarity, remove unused imports, adjust variable naming, and resolve grammar issues in comments. 2025-07-26 23:54:34 +02:00
Till Wegmueller
54a5844d14
Update GitHub Actions workflows: upgrade actions/checkout and cache to v4, replace actions-rs/toolchain with dtolnay/rust-toolchain, and simplify toolchain setup. 2025-07-26 23:49:57 +02:00
Till Wegmueller
e222937201
Fix tripplet typo in function and variable names, replace unwrap with proper error handling, refine comments for clarity and grammar, and enhance file existence check by returning an error instead of a warning. 2025-07-26 23:49:26 +02:00
Till Wegmueller
ad153a65ab
Replace println! statements with structured logging using the tracing crate, refactor error handling to use error! macros for logging, improve comments for clarity, and simplify type usage by utilizing HashMap and HashSet from the prelude. 2025-07-26 23:41:09 +02:00
Till Wegmueller
ce535c830d
Add support for importing pkg5 repositories (directory and p5p format) into pkg6, update README with usage examples, and enhance e2e tests for validation. 2025-07-26 23:14:34 +02:00
Till Wegmueller
c3ff6ac28e
Introduce Pkg5Importer for importing pkg5 repositories (directory or p5p format) into pkg6, extend error handling with ActionError, update dependencies (tempfile, flate2, and thiserror), and enhance CI workflows. 2025-07-26 23:02:56 +02:00
Till Wegmueller
cbd3dd987d
Replace all println! statements with structured logging using tracing crate for enhanced diagnostics and consistency across pkg6repo and pkg6dev. 2025-07-26 21:48:07 +02:00
Till Wegmueller
245e67f5f8
Expand Pkg6DevError with new error variants for component validation, manifest handling, Makefile parsing, and publisher configuration. Enhance error handling and diagnostics across pkg6dev operations. 2025-07-26 21:30:48 +02:00
Till Wegmueller
cb75c045e5
Introduce Pkg6DevError for improved error handling across pkg6dev, replace anyhow with custom error types, enable structured diagnostics with miette, update dependencies, and add tracing for logging. 2025-07-26 21:20:50 +02:00
Till Wegmueller
b451099b54
Refactor tests and stubs: update parsed_manifest assertions with more descriptive values and simplify unused variable patterns in repository rest backend methods. 2025-07-26 18:04:58 +02:00
Till Wegmueller
3ea955fc5f
Integrate GitHub Actions workflows for CI, replacing legacy scripts with xtask automation for formatting, linting, builds, tests, and documentation. Update Jenkinsfile to clarify its restricted usage for illumos builds. 2025-07-26 17:58:11 +02:00
Till Wegmueller
8c2ec7e965
Disable RsMainFunctionNotFound inspection in project default settings. 2025-07-26 17:50:08 +02:00
Till Wegmueller
098c62b645
Replace legacy setup_test_env.sh script with xtask for environment setup, refactor e2e tests to use pre-built binaries, and update xtask commands and documentation for improved automation and reliability. 2025-07-26 17:49:56 +02:00
Till Wegmueller
38fba9b9db
Introduce custom Pkg6RepoError for improved error handling, replace anyhow, and update dependencies and test cases for consistency. 2025-07-26 17:07:59 +02:00
Till Wegmueller
4649608408
Replace anyhow with custom RepositoryError for improved error specificity and consistency. Remove anyhow dependency. 2025-07-26 16:50:10 +02:00
Till Wegmueller
40560e5960
Add xtask for build and test automation, replacing legacy scripts, and integrate commands for environment setup, build, test, and formatting. Update documentation for cargo-xtask usage. 2025-07-26 14:18:30 +02:00
Till Wegmueller
56b50c755a
Add comprehensive project guidelines for Junie, detailing structure, dependencies, error handling, testing, build process, code style, and workflow. 2025-07-26 13:33:56 +02:00
Till Wegmueller
a33a3246b6
chore(fmt): format with cargo fmt 2025-07-26 12:54:01 +02:00
Till Wegmueller
f5b80a7d12
Refactor repository property and publisher filtering in show_contents, improve clarity and efficiency by introducing section_filtered_properties and adding publisher-based property filtering logic. Update get_info to support publisher filtering. 2025-07-26 12:52:55 +02:00
Till Wegmueller
362042268a
Add publisher filtering to show_contents, enhance property filtering for repository configurations, and update synchronous operation handling across backends. 2025-07-26 12:44:39 +02:00
Till Wegmueller
a5cdb7e673
Add JSON and TSV output to repository commands, integrate serde for struct serialization, and introduce search subcommand with flexible filtering options. 2025-07-26 12:35:44 +02:00
Till Wegmueller
307dfddc7c
Remove TESTING.md, fix CLI argument naming (--repo-version), enhance transaction publisher handling, and improve manifest storage logic. 2025-07-26 11:52:42 +02:00
Till Wegmueller
4f0e2fe066
Add unit and end-to-end tests for repository functionality, including setup scripts, package management operations, and TESTING.md documentation. 2025-07-26 10:52:25 +02:00
Till Wegmueller
ead9ad12af
Refactor repository traits into ReadableRepository and WritableRepository, introduce Repository to combine traits for backward compatibility, and update relevant modules and tests to use the new structure. 2025-07-26 10:34:45 +02:00
Till Wegmueller
a0fcc13033
Introduce Catalog module with structs and functionality for managing package metadata, enhance file_backend and CLI to handle catalog operations, and update dependencies (chrono and others) to support new features. 2025-07-24 00:28:33 +02:00
Till Wegmueller
63f2d1da62
Add search method to repository module for querying packages, update dependencies to support search functionality. 2025-07-23 23:22:24 +02:00
Till Wegmueller
2cb63fbef0
Introduce PackageContents struct for structured representation of package data, refactor show_contents for multiple content types, and enhance handling of manifests with improved filtering and output formatting. 2025-07-23 22:39:49 +02:00
Till Wegmueller
01b059bf5d
Expand pkg5 documentation with additional details on repository operations, including administrative tasks, enhanced metadata handling, new operation versions, and improved search functionality. 2025-07-23 22:09:09 +02:00
Till Wegmueller
ca7ce75008
Enhance show_contents to parse manifests directly, add filtering by publishers, patterns, and action types, and handle multiple content types (files, directories, links, dependencies, licenses). 2025-07-22 19:39:46 +02:00