The previous digest fallback used Display format (source:algorithm:hash)
which caused REST servers to look up files by SHA256 compressed hash
instead of the expected SHA1 primary hash. Now tries raw hash first
(compatible with pkg5 REST servers), then formatted variants with
algorithm info (compatible with local FileBackend storage).
Multiple methods held a read or write lock on self.index while calling
fallback methods that also needed to acquire locks, causing deadlocks:
- get_obsoleted_package_metadata: read lock held during filesystem
fallback that calls update_index_entry (write lock)
- get_obsoleted_package_manifest: same pattern
- remove_obsoleted_package: write lock held while calling build_index
(also needs write lock)
- search_obsoleted_packages: read lock held during fallback
Fix: scope all lock acquisitions in blocks so locks are dropped before
calling any method that may also acquire a lock. For remove, extract a
needs_rebuild flag set inside the lock scope, then call build_index
after release.
All 95 libips tests now pass (previously 3 hung indefinitely).
- Use tempdir_in(repo_path) instead of system /tmp to keep temp files on
the same filesystem, preventing cross-device rename failures
- Replace fs::copy with fs::rename for atomic file moves in transaction
commit and signature payload storage
- Fix Digest construction to use correct SHA256 algorithm instead of
defaulting to SHA1 for bare hex hashes
- Try compressed hash (additional_identifiers) as fallback when primary
hash lookup fails during payload fetch
- Remove duplicate publisher resolution block in Transaction::commit()
- Add integration test for multi-file parallel package receive
Closes: https://codeberg.org/Toasterson/ips/issues/21
- Made `receive_one` public to allow external use.
- Improved signature payload fetching, storing, and fallback logic for `FileBackend` repositories.
- Added test case `test_receive_with_signature` to verify signature handling and repository storage.
- Extended manifest fields in transaction logic to support additional attributes (e.g., `users`, `groups`, `drivers`).
- Updated `RestBackend` to handle extended file path lookups for signature payloads.
- Added `resolve_packages` to centralize wildcard pattern processing and match the latest package versions consistently.
- Updated both `FileBackend` and `RestBackend` to support wildcard patterns via `glob_to_regex` logic.
- Made `glob_to_regex` public to facilitate regex conversion for glob patterns.
- Improved `catalog_manager` handling in `RestBackend` by explicitly loading catalog parts to ensure accurate package matching.
- Replaced redundant FMRI parsing logic with `resolve_packages` for cleaner and more maintainable code.
- Migrated `_SIGNATURE` handling in `CatalogPart`, `UpdateLog`, and `PackageUpdateEntry` to use metadata fields.
- Introduced `set_signature` and `signature` methods for cleaner access and manipulation of signature data.
- Updated deserialization logic to accommodate flattened metadata storage.
- Improved structure by centralizing signature and metadata interactions.
- Introduced specific error variants (e.g., `FileOpenError`, `FileWriteError`, `DirectoryCreateError`) in `RepositoryError` for better error diagnostics.
- Applied `.map_err()` handling for operations like file creation, copying, renaming, and directory manipulation.
- Replaced direct `HashMap` usage with `Mutex<HashMap>` for thread-safe access to catalog managers in `RestBackend`.
- Refactored `list_packages` to use `list_packages_from_catalog`, removing reliance on the search API.
- Added temporary directory management for non-local-cache scenarios in `RestBackend`.
- Implemented support for fetching, importing, and storing signature payloads during transaction creation.
- Added the `Signature` struct to represent signature-related action data.
- Updated `Manifest` to include and process `signatures` as part of its fields.
- Enabled signature file imports with proper path resolution in `pkg6repo`.
- Updated `FileBackend::open` calls to use immutable `repo` variables across multiple modules.
- Improved dependency action string construction by integrating optional properties, predicates, and facets for better clarity and flexibility.
- Eliminated LZ4-based compression and related decoding utilities for manifest handling.
- Removed unused private helper and legacy methods in `catalog.rs`.
- Standardized database path handling, replacing `catalog_db_path` and `obsoleted_db_path` with `active_db_path` and `obsolete_db_path`.
- Added `#[allow(dead_code)]` annotations for unused methods in `file_backend.rs` to reduce warnings.
- Introduced `CLAUDE.md` to provide detailed project documentation, including architecture, configuration, and implementation details.
- Added build and testing instructions, focusing on `cargo` commands and `cargo-nextest` for improved testing workflow.
- Described application modules (`settings`, `storage`, `jwks`, `web`) and key features like PKCE, WebAuthn, and 2FA.
- Covered OpenID Connect, OAuth 2.0 endpoints, and claims handling (AMR/ACR).
- Documented admin API, WebAuthn client setup, OIDC conformance, and migration steps for existing deployments.
- Transitioned the catalog backend from `redb` to `rusqlite` for better compatibility and concurrency.
- Updated `IpsProvider` to use SQLite for package querying, dependency resolution, and obsolescence handling.
- Removed `decode_manifest_bytes_local` and unused `manifest` cache logic.
- Simplified catalog-related functions and integrated `sqlite_catalog` module.
- Enhanced test functions and added schemas for managing SQLite databases.
- Introduced `.solstice/workflow.kdl` for defining Solstice CI workflows, including Linux and illumos builds.
- Added `setup-linux.sh` and `setup-illumos.sh` scripts for per-OS environment preparation.
- Implemented `job.sh` as a legacy script hook for additional build steps.
- Introduced `pkg6recv` for downloading packages from IPS repositories with support for recursive dependency fetching.
- Implemented CLI commands with `clap` for source/destination setup, package selection, and publisher defaults.
- Added `ConsoleProgressReporter` for detailed receive progress visibility.
- Enhanced the `Manifest` parser to support JSON format alongside IPS text format.
- Updated `FileBackend` and `RestBackend` repositories to fetch manifests and payloads in new operations.
- Extended `Digest` utilities for payload integrity checks.
- Added tests to verify basic functionality and manifest format preservation.
- Introduced scripts and configurations for manual testing using `anyvm` with OpenIndiana and OmniOS.
- Implemented repository fetching (`fetch_repo.sh`) and server startup (`run_depotd.sh`) scripts.
- Enhanced `pkg6depotd` to support default publisher routes and trailing slashes.
- Updated integration tests to verify new publisher route behavior.
- Replaced OpenIndiana VM with OmniOS VM in Illumos build steps across `rust.yml` and `release.yml` workflows.
- Updated related workflow labels and comments for consistency.
- Deleted the Jenkinsfile, transitioning all CI workflows to GitHub Actions.
- Added Illumos (OpenIndiana) build and release steps to GitHub workflows.
- Created a new release pipeline triggered by version tags starting with `v`.
- Updated README with release instructions using `cargo-release`.
- Renamed `pkg6dev` to `pkg6` across build scripts, workflows, and documentation.
- Added support for additional binaries (`pkg6repo` and `pkg6depotd`) in release builds.
- Disabled `RUSTFLAGS` warnings-as-errors policy in workflows for improved flexibility.
- Simplified error handling in `manifest_fmri` with streamlined conditionals.
- Introduced `#[allow(clippy::result_large_err)]` to suppress clippy warnings for large error types.
- Replaced `values.get(0)` with `values.first()` for improved clarity and consistency across multiple files.
- Simplified conditional checks with `.is_empty()` and `format_args!` to enhance readability.
- Implemented `Default` trait for `VariableMode` directly within the enum.
- Updated error handling by boxing errors in `RepositoryError` and `CatalogError`.
- Replaced redundant `else` blocks with streamlined logic.
- Introduced a searchable index with structured `IndexEntry` support for packages, files, directories, and dependencies.
- Added `search` method in `DepotRepo` with wildcard and case-sensitive query handling.
- Created `/search/0` and `/search/1` routes for search API, supporting publishers and token-based queries.
- Updated `SearchIndex` handling to map tokens to detailed `IndexEntry` structures.
- Improved index building to include attributes for files, directories, and dependencies.
- Introduced `legacy_manifest_content` field in `Transaction` to allow preserving byte-identical legacy manifests.
- Updated `commit` method to save both JSON and legacy manifests, falling back to JSON if no legacy content is provided.
- Enhanced manifest copy logic to handle both JSON and legacy formats during transaction finalization.
- Improved catalog rebuild to skip `.json` files when a corresponding legacy manifest exists.
- Corrected FMRI formatting to `pkg:/name` for entries without a publisher.
- Excluded `pkg.fmri` from summary action extraction in file backend.
- Removed unused signature handling in catalog package processing.
- Reordered `PackageVersionEntry` struct fields for logical consistency.
- Removed unnecessary spaces in JSON key-value and array formatting for Python-style compatibility.
- Enhanced `format_iso8601_basic` to ensure precise microsecond-level time formatting.
- Updated `write_update_log` to return and store SHA-1 signatures for update log verification.
- Added `/file/1/{digest}` route to support file retrieval without specifying the algorithm.
- Implemented a Python-compatible JSON serializer to ensure consistent formatting for catalog artifacts.
- Replaced `HashMap` with `BTreeMap` for deterministic ordering in catalog serialization and updates.
- Updated integration tests to validate the new route functionality and ensure response correctness.
- Refactored `format_iso8601_basic` to improve timestamp formatting consistency.
- Adjusted indentation and line breaks for structs, functions, and method calls to comply with Rust formatting standards.
- Improved error message formatting and consistency across `PkgTreeError` instances.
- Restructured long function arguments and chained calls for clarity and maintainability.
- Simplified conditional statements and loops for better readability.
- No functional changes introduced.
- Added provider-specific memory allocation (8192 MB) for VirtualBox and Libvirt.
- Configured synced folder using rsync with `.git/` and `target/` exclusions.
- Streamlined provisioning by replacing GCC dependencies with Rust and updating package installation list.
- Removed redundant provisioning script for Rust setup.
- Introduced caching headers (`Cache-Control`, `ETag`, `Last-Modified`) for file responses to improve client-side caching.
- Added HTTP admin routes for health check and authentication validation.
- Made `cache_max_age` configurable via server configuration with a default of 3600 seconds.
- Enhanced file handler with metadata-based `Last-Modified` computation.
- Updated integration tests to cover new functionality and ensure correctness.
- Enhanced `get_info` handler to parse version components into `Build Release`, `Branch`, and `Packaging Date`.
- Added proper FMRI formatting in `pkg://<publisher>/<name>@<version>` format.
- Implemented license content resolution with file-based lookup, gzip decompression, and content truncation.
- Introduced compressed and uncompressed package size calculations for manifest actions.
- Updated dependencies to include `chrono` and `flate2` for date parsing and gzip decompression.
