Commit graph

278 commits

Author SHA1 Message Date
Till Wegmueller
7d5ddb626f
feat: Add copy buttons for FMRI and install command on detail page
Replace the cramped FMRI badge and install-cmd div with full-width
copyable blocks that have labeled headers, single-line horizontal
scrolling (no wrapping), and a copy-to-clipboard button. The button
shows a checkmark on success. The install command copy strips the
leading $ prompt.
2026-03-15 23:22:32 +01:00
Till Wegmueller
195863f6d8
fix: Show actual repo last-modified time and widen publisher cards
Read last-modified from catalog.attrs instead of using get_info() which
falls back to current time when the package directory scan finds nothing.
Widened publisher card grid min-width from 320px to 420px so the
formatted date doesn't get cut off.
2026-03-15 23:06:09 +01:00
Till Wegmueller
0d737168e0
fix: Resolve dependency links to latest package version
Dependency FMRIs are version constraints (e.g. SUNWcs@0.5.11-151056.0)
not exact versions, so they don't match any specific manifest. Now
dependency links use the package name only, and the detail handler
resolves name-only or partial-version FMRIs by finding the latest
version from list_packages and redirecting to it.
2026-03-15 23:00:20 +01:00
Till Wegmueller
503efb6aed
fix: Inline CSS in HTML template to guarantee styling loads
The external CSS file wasn't loading on the deployed server despite
being embedded in the binary — likely a route matching or reverse
proxy issue. Moved all CSS into an inline <style> tag in the base
template. Since Askama compiles templates into the binary, this
guarantees the styles render with the HTML on first paint with zero
external dependencies. HTMX JS remains as an embedded route handler.
2026-03-15 22:57:37 +01:00
Till Wegmueller
4591767a0a
fix: Embed static assets in binary and fix font loading
Static files (CSS, JS) were served via ServeDir using CARGO_MANIFEST_DIR
which only exists on the build machine, not on deployed servers. This
caused: white unstyled page, always-visible loading indicators, and
broken manifest loading.

- Embed style.css and htmx.min.js into the binary via include_str!
- Serve them from dedicated handlers with proper content-type and caching
- Move Google Fonts from CSS @import to <link> tags with preconnect
  for non-blocking font loading
- Remove ServeDir dependency from routes
2026-03-15 22:49:33 +01:00
Till Wegmueller
5d13eaa936
fix: FMRI URL routing and drop Pico.css for custom dark theme
Fix package detail 404: FMRI in URL paths now uses just name@version
instead of the full pkg://publisher/name@version, since the publisher
is already a separate path segment. This affected package detail,
manifest loading, dependency links, and search result links.

Drop Pico.css CDN dependency — it was fighting the custom dark theme
causing visual issues. Replaced with a complete standalone stylesheet
with proper reset, giving full control over the design.
2026-03-15 22:42:51 +01:00
Till Wegmueller
2b6613dbfe
feat: Redesign web UI with dark theme and fix display bugs
Design overhaul:
- Dark "Solaris Engineering" theme with amber accent colors
- JetBrains Mono + Source Sans 3 typography via Google Fonts
- Publisher cards with accent borders and stats layout
- Styled package tables with hover states
- Breadcrumb navigation and active nav indicators
- Colored dependency type badges (require/optional/incorporate)
- Terminal-style install command display
- Floating pill-shaped P5I cart with enter animation
- Custom scrollbars for manifest viewer

Bug fixes:
- P5I cart now hidden by default (was visible with 0 items)
- "Updated" timestamp now formatted via format_packaging_date
- Package count falls back to list_packages when get_info reports 0
2026-03-15 22:25:29 +01:00
Till Wegmueller
fad83901f7
feat: Redirect / to /ui/ so the web UI is the default landing page 2026-03-15 22:05:13 +01:00
Till Wegmueller
d49bb3c306
feat: Add web UI for browsing packages in pkg6depotd
Add a human-facing web interface at /ui/ for browsing IPS package
repositories. Uses Askama templates, HTMX for interactivity, and
Pico.css for styling.

Routes:
- /ui/ - Publisher list with package counts
- /ui/packages/:publisher - Paginated package list
- /ui/search - Search with HTMX search-as-you-type
- /ui/package/:publisher/*fmri - Package detail with lazy manifest
- /ui/p5i - P5I file generation for installing package sets
2026-03-15 21:55:10 +01:00
Till Wegmueller
6f5040978b fix: Deduplicate payload downloads to prevent parallel rename races
Multiple file actions in a manifest can reference the same payload hash.
When downloaded in parallel via rayon, multiple threads would write to
the same temp file path simultaneously, causing rename failures.

Now deduplicates payloads by digest before parallel download, then maps
results back to all file actions that reference each payload.
2026-03-15 21:14:35 +01:00
Till Wegmueller
bcea795848 fix: Store payloads under primary hash and fix digest detection
- Store compressed payloads under the primary (uncompressed) hash as
  the main key, matching IPS protocol where clients request files by
  the manifest's primary hash via /file/0/<hash>
- Create hardlink under compressed hash for internal lookups
- Detect SHA256 (64 chars) vs SHA1 (40 chars) from bare hex hash
  length in Digest::from_str instead of always defaulting to SHA1
- Remove digest verification for local file copies in FileBackend
  (files are compressed, hash is of uncompressed content)
- Simplify recv.rs payload fetch to use primary hash directly
  instead of trying multiple digest formats against the source
2026-03-15 20:58:44 +01:00
Till Wegmueller
a0fe229ba4 fix: Store payload files under both compressed and primary hash
Files were only stored under their compressed SHA256 hash, but pkg5
clients and the IPS protocol look up files by the manifest's primary
hash (typically SHA1/SHA256 uncompressed). Create a hardlink from the
primary hash path to the compressed hash file so both old and new
clients can find payloads via the server's /file/0/<hash> endpoint.
2026-03-15 20:44:43 +01:00
Till Wegmueller
0a28909e9e fix: Try raw hash before formatted digest in payload fetch fallback
The previous digest fallback used Display format (source:algorithm:hash)
which caused REST servers to look up files by SHA256 compressed hash
instead of the expected SHA1 primary hash. Now tries raw hash first
(compatible with pkg5 REST servers), then formatted variants with
algorithm info (compatible with local FileBackend storage).
2026-03-15 20:39:50 +01:00
Till Wegmueller
d5ed328ab2 fix: Resolve RwLock deadlocks in ObsoletedPackageManager causing test hangs
Multiple methods held a read or write lock on self.index while calling
fallback methods that also needed to acquire locks, causing deadlocks:

- get_obsoleted_package_metadata: read lock held during filesystem
  fallback that calls update_index_entry (write lock)
- get_obsoleted_package_manifest: same pattern
- remove_obsoleted_package: write lock held while calling build_index
  (also needs write lock)
- search_obsoleted_packages: read lock held during fallback

Fix: scope all lock acquisitions in blocks so locks are dropped before
calling any method that may also acquire a lock. For remove, extract a
needs_rebuild flag set inside the lock scope, then call build_index
after release.

All 95 libips tests now pass (previously 3 hung indefinitely).
2026-03-15 20:27:57 +01:00
Till Wegmueller
750df8dcc7 fix: Resolve pkg6recv multithreaded download race conditions and file loss
- Use tempdir_in(repo_path) instead of system /tmp to keep temp files on
  the same filesystem, preventing cross-device rename failures
- Replace fs::copy with fs::rename for atomic file moves in transaction
  commit and signature payload storage
- Fix Digest construction to use correct SHA256 algorithm instead of
  defaulting to SHA1 for bare hex hashes
- Try compressed hash (additional_identifiers) as fallback when primary
  hash lookup fails during payload fetch
- Remove duplicate publisher resolution block in Transaction::commit()
- Add integration test for multi-file parallel package receive

Closes: https://codeberg.org/Toasterson/ips/issues/21
2026-03-15 19:32:24 +01:00
Till Wegmueller
ec9c55daf3
Update docker-compose for test deployment
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2026-03-14 23:08:44 +01:00
Till Wegmueller
6865996190
feat: Add Docker deployment for pkg.solstice-ci.org
Multi-stage Dockerfile (rust builder + debian-slim runtime),
docker-compose, and container-specific pkg6depotd config.

Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2026-03-14 22:56:32 +01:00
Till Wegmueller
e83f2b7284
feat: Add full-text search via FTS5 and enable search endpoints
Add FTS5 search functions to sqlite_catalog (sanitize_fts_query,
search_fts, resolve_latest_fmris), enable search in versions response,
add integration tests, and remove legacy search code from file_backend.
2026-03-14 22:01:48 +01:00
Till Wegmueller
6e60e9cdd1
Update readme and remove old info
Signed-off-by: Till Wegmueller <toasterson@gmail.com>
2026-02-07 14:51:09 +01:00
Till Wegmueller
ac9ab8c447
feat: Enhance signature payload handling and add test coverage
- Made `receive_one` public to allow external use.
- Improved signature payload fetching, storing, and fallback logic for `FileBackend` repositories.
- Added test case `test_receive_with_signature` to verify signature handling and repository storage.
- Extended manifest fields in transaction logic to support additional attributes (e.g., `users`, `groups`, `drivers`).
- Updated `RestBackend` to handle extended file path lookups for signature payloads.
2026-02-06 00:40:59 +01:00
Till Wegmueller
e4f49cd7c8
Refactor: Introduce resolve_packages for wildcard handling and enhance repository package resolution
- Added `resolve_packages` to centralize wildcard pattern processing and match the latest package versions consistently.
- Updated both `FileBackend` and `RestBackend` to support wildcard patterns via `glob_to_regex` logic.
- Made `glob_to_regex` public to facilitate regex conversion for glob patterns.
- Improved `catalog_manager` handling in `RestBackend` by explicitly loading catalog parts to ensure accurate package matching.
- Replaced redundant FMRI parsing logic with `resolve_packages` for cleaner and more maintainable code.
2026-02-06 00:04:23 +01:00
Till Wegmueller
f8068364bc
Refactor: Replace direct signature fields with metadata-based storage
- Migrated `_SIGNATURE` handling in `CatalogPart`, `UpdateLog`, and `PackageUpdateEntry` to use metadata fields.
- Introduced `set_signature` and `signature` methods for cleaner access and manipulation of signature data.
- Updated deserialization logic to accommodate flattened metadata storage.
- Improved structure by centralizing signature and metadata interactions.
2026-02-05 23:34:15 +01:00
Till Wegmueller
7a3373a17d
Add detailed error handling for file and directory operations
- Introduced specific error variants (e.g., `FileOpenError`, `FileWriteError`, `DirectoryCreateError`) in `RepositoryError` for better error diagnostics.
- Applied `.map_err()` handling for operations like file creation, copying, renaming, and directory manipulation.
- Replaced direct `HashMap` usage with `Mutex<HashMap>` for thread-safe access to catalog managers in `RestBackend`.
- Refactored `list_packages` to use `list_packages_from_catalog`, removing reliance on the search API.
- Added temporary directory management for non-local-cache scenarios in `RestBackend`.
2026-02-05 23:16:02 +01:00
Till Wegmueller
dfc24725b8
Add signature handling for manifests
- Implemented support for fetching, importing, and storing signature payloads during transaction creation.
- Added the `Signature` struct to represent signature-related action data.
- Updated `Manifest` to include and process `signatures` as part of its fields.
- Enabled signature file imports with proper path resolution in `pkg6repo`.
2026-02-05 22:26:53 +01:00
Till Wegmueller
38baf16b6f
Refactor: Replace mutable repo assignments with immutable and enhance dependency action formatting
- Updated `FileBackend::open` calls to use immutable `repo` variables across multiple modules.
- Improved dependency action string construction by integrating optional properties, predicates, and facets for better clarity and flexibility.
2026-02-05 21:56:53 +01:00
Till Wegmueller
e236f30f6e
Convert trait methods to use &self instead of &mut self, introduce Mutex for interior mutability, optimize HTTP client creation, and implement parallel payload processing using Rayon. 2026-02-05 15:57:56 +01:00
Till Wegmueller
0de84b80c8
Refactor: Remove LZ4 compression utilities and clean up obsolete catalog functions
- Eliminated LZ4-based compression and related decoding utilities for manifest handling.
- Removed unused private helper and legacy methods in `catalog.rs`.
- Standardized database path handling, replacing `catalog_db_path` and `obsoleted_db_path` with `active_db_path` and `obsolete_db_path`.
- Added `#[allow(dead_code)]` annotations for unused methods in `file_backend.rs` to reduce warnings.
2026-02-04 23:03:20 +01:00
Till Wegmueller
7b9391f36e
Refactor: Remove unused imports and clean up redundant code
- Eliminated unused imports across multiple modules, including `info`, `trace`, `warn`, `base64`, `PathBuf`, and `fs`.
- Replaced mutable variable assignment with immutable in `populate_obsolete_db`.
- Simplified loop variable handling in `pkg6depotd` shard handler.
2026-02-04 22:47:44 +01:00
Till Wegmueller
ba82317da5
docs: Add comprehensive CLAUDE.md with project overview and guidelines
- Introduced `CLAUDE.md` to provide detailed project documentation, including architecture, configuration, and implementation details.
- Added build and testing instructions, focusing on `cargo` commands and `cargo-nextest` for improved testing workflow.
- Described application modules (`settings`, `storage`, `jwks`, `web`) and key features like PKCE, WebAuthn, and 2FA.
- Covered OpenID Connect, OAuth 2.0 endpoints, and claims handling (AMR/ACR).
- Documented admin API, WebAuthn client setup, OIDC conformance, and migration steps for existing deployments.
2026-02-04 22:42:46 +01:00
Till Wegmueller
240df1f5b9
chore(format): apply consistent code formatting across modules 2026-02-04 22:40:51 +01:00
Till Wegmueller
def11a1dfb
Refactor: Replace redb with rusqlite for catalog handling
- Transitioned the catalog backend from `redb` to `rusqlite` for better compatibility and concurrency.
- Updated `IpsProvider` to use SQLite for package querying, dependency resolution, and obsolescence handling.
- Removed `decode_manifest_bytes_local` and unused `manifest` cache logic.
- Simplified catalog-related functions and integrated `sqlite_catalog` module.
- Enhanced test functions and added schemas for managing SQLite databases.
2026-02-04 22:39:42 +01:00
Till Wegmueller
4ab529f4c7
chore(format): format code 2026-01-25 23:17:49 +01:00
Till Wegmueller
e7dc9d5a4c
Add comment for testing in Solstice CI workflow configuration 2026-01-25 22:02:03 +01:00
Till Wegmueller
04f37476f0
Add Solstice CI workflow with per-OS setup scripts
- Introduced `.solstice/workflow.kdl` for defining Solstice CI workflows, including Linux and illumos builds.
- Added `setup-linux.sh` and `setup-illumos.sh` scripts for per-OS environment preparation.
- Implemented `job.sh` as a legacy script hook for additional build steps.
2026-01-25 20:05:07 +01:00
Till Wegmueller
e1ce390abd
Refactor code to simplify handling and remove unused functions
- Removed `add_package` function from `FileBackend`, consolidating package indexing logic.
- Allowed unused assignments using `#![allow(unused_assignments)]` where necessary.
- Addressed unused imports in integration tests.
- Simplified `Digest` error handling by removing unreachable branches for unknown algorithms.
2026-01-20 22:19:25 +01:00
Till Wegmueller
1c0619ca55
Add pkg6recv package for receiving IPS repositories.
- Introduced `pkg6recv` for downloading packages from IPS repositories with support for recursive dependency fetching.
- Implemented CLI commands with `clap` for source/destination setup, package selection, and publisher defaults.
- Added `ConsoleProgressReporter` for detailed receive progress visibility.
- Enhanced the `Manifest` parser to support JSON format alongside IPS text format.
- Updated `FileBackend` and `RestBackend` repositories to fetch manifests and payloads in new operations.
- Extended `Digest` utilities for payload integrity checks.
- Added tests to verify basic functionality and manifest format preservation.
2026-01-20 20:16:58 +01:00
Till Wegmueller
22178cffd7
Add manual testing setup for pkg6depotd
- Introduced scripts and configurations for manual testing using `anyvm` with OpenIndiana and OmniOS.
- Implemented repository fetching (`fetch_repo.sh`) and server startup (`run_depotd.sh`) scripts.
- Enhanced `pkg6depotd` to support default publisher routes and trailing slashes.
- Updated integration tests to verify new publisher route behavior.
2026-01-20 17:44:36 +01:00
Till Wegmueller
d16e5339ce
chore: Release 2026-01-19 00:28:58 +01:00
Till Wegmueller
d0722bdacd
Enable push and tag in workspace release configuration. 2026-01-19 00:28:41 +01:00
Till Wegmueller
074dad33bf
chore: Release {{version}} 2026-01-19 00:27:43 +01:00
Till Wegmueller
a8174e8da2
Mark package as non-publishable in workspace configuration. 2026-01-19 00:27:17 +01:00
Till Wegmueller
6ee6392b01
Standardize description, version, authors, and edition fields across packages using workspace inheritance. 2026-01-19 00:25:47 +01:00
Till Wegmueller
86e1a2eb53
Update OmniOS workflows and Vagrantfile to replace library/lz4 with compress/lz4 2026-01-18 17:27:54 +01:00
Till Wegmueller
d763bf1c34
Simplify PkgTreeError struct and consolidate error handling logic across main.rs. Update OmniOS workflows and Vagrantfile to include additional libraries and improve configuration consistency. 2026-01-18 17:05:59 +01:00
Till Wegmueller
0e873ee114
Update OmniOS workflows to use ooce packages
- Replaced `extra.omnios` publisher setup and outdated package names with `ooce` packages in `rust.yml` and `release.yml`.
2026-01-18 16:45:43 +01:00
Till Wegmueller
d7c7079374
Add OmniOS extra publisher setup to workflows
- Updated both `rust.yml` and `release.yml` to configure the extra OmniOS publisher for additional package installations.
2026-01-18 16:32:27 +01:00
Till Wegmueller
4d96cf1d2b
Update Illumos build workflows to use OmniOS VM
- Replaced OpenIndiana VM with OmniOS VM in Illumos build steps across `rust.yml` and `release.yml` workflows.
- Updated related workflow labels and comments for consistency.
2026-01-18 16:01:24 +01:00
Till Wegmueller
7fb3793c54
Remove Jenkinsfile, add Illumos builds to GitHub Actions, and document release process
- Deleted the Jenkinsfile, transitioning all CI workflows to GitHub Actions.
- Added Illumos (OpenIndiana) build and release steps to GitHub workflows.
- Created a new release pipeline triggered by version tags starting with `v`.
- Updated README with release instructions using `cargo-release`.
2026-01-18 14:59:24 +01:00
Till Wegmueller
898ec20ad8
Update binaries and workflows for pkg6 rename
- Renamed `pkg6dev` to `pkg6` across build scripts, workflows, and documentation.
- Added support for additional binaries (`pkg6repo` and `pkg6depotd`) in release builds.
- Disabled `RUSTFLAGS` warnings-as-errors policy in workflows for improved flexibility.
- Simplified error handling in `manifest_fmri` with streamlined conditionals.
- Introduced `#[allow(clippy::result_large_err)]` to suppress clippy warnings for large error types.
2026-01-18 14:30:05 +01:00
Till Wegmueller
b8c625a11e
Refactor attribute value access and error handling
- Replaced `values.get(0)` with `values.first()` for improved clarity and consistency across multiple files.
- Simplified conditional checks with `.is_empty()` and `format_args!` to enhance readability.
- Implemented `Default` trait for `VariableMode` directly within the enum.
- Updated error handling by boxing errors in `RepositoryError` and `CatalogError`.
- Replaced redundant `else` blocks with streamlined logic.
2026-01-18 13:00:21 +01:00