Commit graph

65 commits

Author SHA1 Message Date
Till Wegmueller
dfe8ac1305
Add support for backward compatibility with legacy repository files
- Introduced creation of `pub.p5i` files for publishers to maintain compatibility with older repository formats.
- Implemented saving repository configuration in legacy INI format (`pkg5.repository`).
- Updated tests to validate the generation and structure of legacy files.
- Added new dependencies (`rust-ini`) and updated `Cargo.toml` and `Cargo.lock` accordingly.
2025-08-02 13:17:49 +02:00
Till Wegmueller
fc00304038
Remove redundant file checks and update test assertions for publisher-specific file paths
- Eliminate obsolete assertions for the root `file` directory as it is no longer created.
- Update tests to validate file paths in `publisher`-specific directories, aligning with the updated repository structure.
- Simplify `create_directories` by removing unnecessary `file` directory creation.
2025-07-31 00:29:53 +02:00
Till Wegmueller
99dd0fe87c
Refactor repository structure to use publisher-specific directories
- Introduce publisher-specific hierarchy (`publisher/<publisher_name>`) for catalog, package, and file storage.
- Replace hardcoded paths with helper methods (`construct_catalog_path`, `construct_package_dir`, `construct_file_path_with_publisher`) for consistent path generation.
- Remove outdated tests and simplify redundant assertions in repository tests.
- Update `CatalogManager` and file handling logic to leverage publisher-specific paths.
- Enhance debug logs to include publisher context where applicable.
2025-07-31 00:18:21 +02:00
Till Wegmueller
845ffec13b
Switch to CBOR for metadata serialization and simplify key management
- Replace `bincode` serialization with `CBOR` for obsoleted package metadata to improve interoperability and error messaging.
- Simplify key handling by directly using FMRI strings as keys, removing the need for intermediate serialization steps.
- Update deserialization, error handling, and logging to align with `CBOR` serialization.
- Add `serde_cbor` as a dependency, updating `Cargo.toml` and `Cargo.lock` accordingly.
- Refactor `ObsoletedPackageKey` to include conversion between FMRI strings and data components.
2025-07-29 23:14:47 +02:00
Till Wegmueller
a5f1b7d815
Refactor obsoleted.rs to optimize database schema and enhance error handling
- Replace `FMRI_TO_HASH_TABLE` and `HASH_TO_METADATA_TABLE` with a unified `FMRI_TO_METADATA_TABLE` for direct FMRI-to-metadata mapping, reducing complexity and eliminating intermediate hash lookups.
- Implement robust error handling and logging in database transactions, serialization, and deserialization procedures.
- Introduce on-the-fly manifest generation for entries with `NULL_HASH`, streamlining workflows for minimal manifest creation.
- Update indexing, querying, and removal logic to align with the new table and error handling improvements.
- Streamline `list_obsoleted_packages` by introducing index-based fetching with structured fallbacks to filesystem methods.
2025-07-29 21:24:32 +02:00
Till Wegmueller
5b4b719b42
Introduce cleanup functionality for obsoleted packages with TTL support
- Add methods to find and clean up obsoleted packages older than a specified TTL (`find_obsoleted_packages_older_than_ttl` and `cleanup_obsoleted_packages_older_than_ttl`) in `libips`.
- Implement a new `CleanupObsoleted` command in `pkg6repo` to handle cleanup operations.
- Update workspace `Cargo.toml` files with unified attributes for better consistency.
- Adjust dependencies (`libips` version patterns, `thiserror` updates) and enhance metadata management for obsoleted packages.
- Enhance repository operations by adding batch processing and robust logging during cleanup.
2025-07-29 19:08:18 +02:00
Till Wegmueller
7633feb36f
Add support for null hash in obsoleted package metadata
- Introduce `NULL_HASH` constant to indicate the absence of original manifest storage.
- Add `new_with_null_hash` method to create metadata for obsoleted packages without storing full manifests.
- Implement on-the-fly generation of minimal manifests with obsoletion attributes for packages with null hash.
- Update `pkg5_import` to use null hash for obsoleted packages, reducing storage requirements.
- Document null hash usage and its benefits in `obsoleted_packages.md`.
2025-07-29 18:40:34 +02:00
Till Wegmueller
9b2f74c5c1
Introduce obsoleted package management system in IPS
- Add `obsoleted.rs` module to handle storing, metadata management, and operations for obsoleted packages.
- Implement commands for marking, listing, searching, restoring, exporting, and importing obsoleted packages (`pkg6repo`).
- Enhance `RepositoryError` with `From` implementations for various error types to manage database and serialization-related errors.
- Introduce reusable data structures for obsoleted package metadata and export representation.
- Update `Cargo.toml` and `Cargo.lock` to include new dependencies (`redb`, `bincode`, etc.).
- Document obsoleted package workflow and integration details in `doc/obsoleted_packages.md` for contributors.
- Refactor repository internals to integrate obsoleted package support without disrupting existing workflow.
- Add robust error handling, logging, and pagination for enhanced usability and scalability.
2025-07-29 16:16:12 +02:00
Till Wegmueller
88b55c4a70
Add serde attribute defaults and improve JSON deserialization handling in manifests
- Introduce `#[serde(skip_serializing_if = "is_empty", default)]` for various optional fields to streamline serialization and ensure defaults are applied during deserialization.
- Add `tracing::debug` logging for enhanced error context in JSON deserialization fallback logic.
- Update tests to reflect changes in manifest parsing, including new cases for the updated JSON format.
2025-07-29 11:38:36 +02:00
Till Wegmueller
81eb4a7447
Use walkdir for efficient recursive traversal in FileBackend::build_search_index
- Replace manual recursive directory scanning with `walkdir` for simplified and efficient file traversal.
- Update dependencies in `Cargo.toml` and `Cargo.lock` to include `walkdir`.
- Enhance code maintainability by replacing redundant logic with streamlined traversal and processing.
2025-07-27 19:48:52 +02:00
Till Wegmueller
87eea546aa
Remove obsolete test_manifest_parsing.rs and refactor FileBackend for robust manifest handling
- Eliminate redundant `test_manifest_parsing.rs` as its functionality is covered during `FileBackend` implementation.
- Define `PackageContentVectors` for streamlined data organization.
- Enhance manifest handling in `FileBackend` with recursive search, improved content extraction, and robust error handling.
- Simplify and centralize manifest parsing logic for maintainability and code clarity.
2025-07-27 16:12:59 +02:00
Till Wegmueller
21de26ae82
Refactor codebase to improve formatting, logging clarity, and error handling
- Standardize formatting by aligning and cleaning up indentation, spacing, and line breaks across multiple modules.
- Enhance logging with improved message formatting for better context and readability.
- Simplify error handling in `pkg6dev`, `pkg6repo`, and core libraries by consolidating redundant patterns and improving clarity.
- Update test cases to reflect formatting and logging changes while extending coverage of edge cases.
- Perform general cleanup, including removing unused imports, refining comments, and ensuring consistent style.
2025-07-27 15:22:49 +02:00
Till Wegmueller
abb997df43
Refactor FileBackend for improved manifest handling and directory structure
- Extract manifest path construction and recursive manifest search into helper methods (`construct_manifest_path` and `find_manifests_recursive`).
- Enhance package information extraction by separating stem and version details from FMRI.
- Adjust directory creation logic to include per-package subdirectories for better organization.
- Simplify and centralize manifest parsing logic, improving error handling and maintainability.
2025-07-27 15:20:45 +02:00
Till Wegmueller
f4d1ce73b6
Refactor repository operations to use immutable references
- Modify `rebuild` and `refresh` methods to take `&self` instead of `&mut self`.
- Replace mutable repository instances in `pkg6dev`, `pkg6repo`, and core library with immutable ones.
- Introduce `debug` logging for enhanced traceability during metadata rebuild and refresh.
- Extract and simplify catalog rebuild logic into a dedicated `rebuild_catalog` method.
- General cleanup: remove redundant methods, reorganize logic, and refine comments across affected files.
2025-07-27 13:43:56 +02:00
Till Wegmueller
7f4ecf6346
Organize repository files into hierarchical directory structure for improved scalability
- Update `FileBackend` to store files in a multi-level directory structure (`xx/yy/hash`) based on hash prefixes.
- Adjust method signatures and logic across `FileBackend`, `pkg6dev`, and `pkg6repo` to support mutable repository operations.
- Add test cases and scripts (`test_file_structure.rs`, `test_repo_operations.sh`) to validate the new file structure.
- Refactor catalog manager to use `RefCell` for interior mutability to enable lazy initialization.
- Clean up redundant and dead code, enhance comments, and replace outdated logic for file handling and metadata operations.
2025-07-27 13:15:52 +02:00
Till Wegmueller
633e742528
Remove backward compatibility logic for root pkg directory manifest copy in file_backend.rs 2025-07-27 12:26:57 +02:00
Till Wegmueller
d00de23d52
Simplify pkg5_import.rs by removing redundant directory creation logic; delegate to Transaction.commit() for handling. 2025-07-27 12:19:38 +02:00
Till Wegmueller
9b271d81b5
Add support for User, Group, Driver, Legacy, and Transform actions in Manifest parsing
- Implement parsing and mapping of new action types: `User`, `Group`, `Driver`, `Legacy`, and `Transform` within `Manifest`.
- Introduce comprehensive test coverage for the newly supported actions, including edge cases for `ftpuser` property in manifests.
- Refactor to leverage `HashSet` for user `services` to avoid duplicates.
- Improve comments and error handling during action parsing for enhanced clarity and robustness.
2025-07-27 11:30:24 +02:00
Till Wegmueller
ad153a65ab
Replace println! statements with structured logging using the tracing crate, refactor error handling to use error! macros for logging, improve comments for clarity, and simplify type usage by utilizing HashMap and HashSet from the prelude. 2025-07-26 23:41:09 +02:00
Till Wegmueller
cb75c045e5
Introduce Pkg6DevError for improved error handling across pkg6dev, replace anyhow with custom error types, enable structured diagnostics with miette, update dependencies, and add tracing for logging. 2025-07-26 21:20:50 +02:00
Till Wegmueller
b451099b54
Refactor tests and stubs: update parsed_manifest assertions with more descriptive values and simplify unused variable patterns in repository rest backend methods. 2025-07-26 18:04:58 +02:00
Till Wegmueller
4649608408
Replace anyhow with custom RepositoryError for improved error specificity and consistency. Remove anyhow dependency. 2025-07-26 16:50:10 +02:00
Till Wegmueller
a33a3246b6
chore(fmt): format with cargo fmt 2025-07-26 12:54:01 +02:00
Till Wegmueller
a5cdb7e673
Add JSON and TSV output to repository commands, integrate serde for struct serialization, and introduce search subcommand with flexible filtering options. 2025-07-26 12:35:44 +02:00
Till Wegmueller
307dfddc7c
Remove TESTING.md, fix CLI argument naming (--repo-version), enhance transaction publisher handling, and improve manifest storage logic. 2025-07-26 11:52:42 +02:00
Till Wegmueller
4f0e2fe066
Add unit and end-to-end tests for repository functionality, including setup scripts, package management operations, and TESTING.md documentation. 2025-07-26 10:52:25 +02:00
Till Wegmueller
ead9ad12af
Refactor repository traits into ReadableRepository and WritableRepository, introduce Repository to combine traits for backward compatibility, and update relevant modules and tests to use the new structure. 2025-07-26 10:34:45 +02:00
Till Wegmueller
a0fcc13033
Introduce Catalog module with structs and functionality for managing package metadata, enhance file_backend and CLI to handle catalog operations, and update dependencies (chrono and others) to support new features. 2025-07-24 00:28:33 +02:00
Till Wegmueller
63f2d1da62
Add search method to repository module for querying packages, update dependencies to support search functionality. 2025-07-23 23:22:24 +02:00
Till Wegmueller
2cb63fbef0
Introduce PackageContents struct for structured representation of package data, refactor show_contents for multiple content types, and enhance handling of manifests with improved filtering and output formatting. 2025-07-23 22:39:49 +02:00
Till Wegmueller
ca7ce75008
Enhance show_contents to parse manifests directly, add filtering by publishers, patterns, and action types, and handle multiple content types (files, directories, links, dependencies, licenses). 2025-07-22 19:39:46 +02:00
Till Wegmueller
5d987ca0cb
Refactor package FMRI handling by introducing Fmri struct, update dependencies, and adjust repository and action modules for structured FMRI data processing
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-22 14:10:37 +02:00
Till Wegmueller
e3662eaf23
Introduce PackageInfo struct for structured package data, refactor list_packages to use it, and enhance publisher directory handling
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-22 11:57:24 +02:00
Till Wegmueller
7332e0f7b5
Refactor repository info retrieval to return structured data, add detailed publisher information, and update tests accordingly
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-22 10:21:16 +02:00
Till Wegmueller
a9584fa6d2
Add package publishing functionality and default publisher management
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-21 23:20:19 +02:00
Till Wegmueller
8a32bf3176
Add compression support (Gzip/LZ4) and update dependencies
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-21 22:37:08 +02:00
Till Wegmueller
1a5e0e053d
Add repository module and update dependencies
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2025-07-21 22:02:05 +02:00
Till Wegmueller
bef5442afd
Adding diff-struct
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2024-08-15 21:27:00 +02:00
Till Wegmueller
5cea5186d7
Add serde and image starter module
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2024-08-14 20:02:29 +02:00
Till Wegmueller
a8f84ef3f6
fix derivable impls
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2023-03-25 16:57:51 +01:00
Till Wegmueller
2b4d01df8a
test problems
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2023-03-25 13:13:41 +01:00
Till Wegmueller
640fc4f611
fixing clippy issues
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2023-03-25 13:08:40 +01:00
Till Wegmueller
59ae8ea4cc
fixing clippy issues
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2023-03-25 13:01:05 +01:00
Till Wegmueller
8b9e69a787
fixing clippy issues
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2023-03-25 12:48:09 +01:00
Till Wegmueller
5291bb304f
fixing clippy issues
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2023-03-25 12:47:54 +01:00
Till Wegmueller
4ebecbd153 Adding ports and specfile crates. Switch to thiserror for libraries 2022-09-02 23:48:26 +02:00
Till Wegmueller
bf5c8253a8 Remove unused import 2022-07-31 16:51:03 -03:00
dependabot[bot]
b4458da5f3
Bump regex from 1.4.5 to 1.5.5
Bumps [regex](https://github.com/rust-lang/regex) from 1.4.5 to 1.5.5.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.4.5...1.5.5)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-06 20:14:34 +00:00
Till Wegmueller
3481814af2 Switch to anyhow and thiserror
Update clap to version 3
Make golang file parseable by removing line breaks
2022-03-24 19:48:41 -03:00
Till Wegmueller
d129518d33 Add Unicode parsing
Add Linebreak parsing
2021-09-04 23:58:43 -03:00