mirror of
https://github.com/CloudNebulaProject/wayray.git
synced 2026-04-10 13:10:41 +00:00
167c6c17c6
Comprehensive documentation for WayRay, a SunRay-like thin client Wayland compositor targeting illumos and Linux: - CLAUDE.md: project context and conventions - docs/ai/plans: 6-phase implementation roadmap - docs/ai/adr: 9 architecture decision records (Smithay, QUIC, frame encoding, session management, rendering, audio, project structure, illumos support, pluggable window management) - docs/architecture: system architecture overview with diagrams - docs/protocols: WayRay wire protocol specification - book/: mdbook user guide (introduction, concepts, server/client guides, admin, development) - RESEARCH.md: deep research on remote display protocols
200 lines
8 KiB
Markdown
200 lines
8 KiB
Markdown
# Implementation Roadmap
|
|
|
|
## Phase 0: Foundation (Weeks 1-2)
|
|
|
|
### 0.1 Project Structure
|
|
- Set up Cargo workspace with four crates: `wayray-server`, `wayray-client`, `wayray-protocol`, `wayray-ctl`
|
|
- Configure shared dependencies, feature flags, CI (Linux + illumos)
|
|
- Set up tracing/logging infrastructure with miette error handling
|
|
- Smithay with `default-features = false` + portable features only in core
|
|
- Platform-specific backends behind `cfg(target_os)` + feature flags
|
|
|
|
### 0.2 Minimal Compositor (Server)
|
|
- Implement a minimal Smithay compositor based on Smallvil patterns
|
|
- Support: wl_compositor, xdg_shell, wl_shm, wl_seat, wl_output
|
|
- Use Winit backend for development/testing (works on both Linux and illumos)
|
|
- Use `shm_open` with `memfd_create` fallback for shared memory portability
|
|
- Verify basic Wayland clients (foot terminal, weston-info) can connect and render
|
|
|
|
### 0.3 Framebuffer Capture
|
|
- Implement ExportMem-based framebuffer capture after each render pass
|
|
- Integrate OutputDamageTracker for efficient dirty region tracking
|
|
- Benchmark: capture latency, memory bandwidth
|
|
|
|
## Phase 1: Network Protocol (Weeks 3-5)
|
|
|
|
### 1.1 Protocol Definition
|
|
- Define WayRay wire protocol in wayray-protocol crate
|
|
- Message types: FrameUpdate, InputEvent, SessionControl, AudioChunk, USBData
|
|
- Serialization: serde + bincode or postcard for low overhead
|
|
- Version negotiation and capability exchange
|
|
|
|
### 1.2 QUIC Transport Layer
|
|
- Implement QUIC server (quinn) in wayray-server
|
|
- Implement QUIC client (quinn) in wayray-client
|
|
- Stream mapping:
|
|
- Stream 0: Control channel (session mgmt, capabilities)
|
|
- Stream 1: Display channel (frame updates, damage regions)
|
|
- Stream 2: Input channel (keyboard, mouse, touch)
|
|
- Stream 3: Audio channel (Opus frames)
|
|
- Stream 4+: USB device channels (one per device)
|
|
- Connection handling: TLS certificates, authentication
|
|
|
|
### 1.3 Frame Encoding Pipeline
|
|
- Implement frame differencing (XOR diff against previous frame)
|
|
- Implement region-based compression (zstd for lossless regions)
|
|
- Implement content-adaptive encoding:
|
|
- Static regions: lossless zstd diff
|
|
- Video regions: H.264 via ffmpeg-next or VAAPI
|
|
- Text regions: lossless PNG-style encoding
|
|
- Damage rectangle merging and optimization
|
|
|
|
## Phase 2: Client Viewer (Weeks 5-7)
|
|
|
|
### 2.1 Display Client
|
|
- Implement wayray-client as a standalone application
|
|
- Use winit + wgpu for cross-platform display
|
|
- Frame decoding pipeline: receive -> decompress -> decode -> upload to GPU -> display
|
|
- Double-buffered rendering with VSync
|
|
|
|
### 2.2 Input Capture & Forwarding
|
|
- Capture keyboard events (with proper keymap forwarding via xkb)
|
|
- Capture mouse events (motion, buttons, scroll)
|
|
- Capture touch events
|
|
- Serialize and send over QUIC input stream
|
|
- Handle keyboard grab/release for compositor key passthrough
|
|
|
|
### 2.3 Cursor Handling
|
|
- Server-side cursor rendering (simplest)
|
|
- Client-side cursor rendering with cursor image forwarding (lower latency)
|
|
- Cursor shape protocol support
|
|
|
|
## Phase 2.5: Pluggable Window Management (Weeks 7-8)
|
|
|
|
### 2.5.1 WM Protocol Definition
|
|
- Define `wayray_wm_manager_v1` Wayland protocol XML
|
|
- Define `wayray_wm_window_v1`, `wayray_wm_seat_v1`, `wayray_wm_workspace_v1`
|
|
- Implement two-phase transaction model (manage + render sequences)
|
|
- Generate Rust bindings via wayland-scanner
|
|
|
|
### 2.5.2 WM Protocol Server (in compositor)
|
|
- Implement WM global in wayray-server
|
|
- Window lifecycle events (new, closed, properties)
|
|
- Manage phase: receive policy decisions, send configures
|
|
- Render phase: apply positions/z-order atomically
|
|
- Keybinding registration and dispatch via seat interface
|
|
|
|
### 2.5.3 Built-in Floating WM
|
|
- Default WM active when no external WM is connected
|
|
- Basic floating behavior: centered new windows, focus-follows-click
|
|
- Keyboard shortcuts: Alt+F4 close, Alt+Tab cycle, Super+Arrow snap
|
|
- Yields to external WM on connect
|
|
|
|
### 2.5.4 Example Tiling WM
|
|
- Ship a reference tiling WM as a separate binary (`wayray-wm-tiling`)
|
|
- Demonstrates the protocol for third-party WM developers
|
|
- Basic BSP tiling with keyboard-driven focus
|
|
|
|
## Phase 3: Session Management (Weeks 8-11)
|
|
|
|
### 3.1 Session Persistence
|
|
- Session state machine: Created -> Active -> Suspended -> Resumed -> Destroyed
|
|
- Session storage: in-memory with optional persistence (SeaORM + SQLite)
|
|
- Session timeout and cleanup policies
|
|
|
|
### 3.2 Authentication
|
|
- Token-based session identification (smart card ID, badge, or software token)
|
|
- PAM integration for user authentication
|
|
- Session-token binding in session store
|
|
|
|
### 3.3 Hot-Desking (Session Mobility)
|
|
- Token insertion triggers session lookup across server pool
|
|
- Session reconnection: rebind existing session to new client endpoint
|
|
- Session disconnect: unbind from client, keep session running
|
|
- Sub-second reconnection target (< 500ms)
|
|
|
|
### 3.4 Multi-Server Support
|
|
- Server discovery protocol (mDNS or custom)
|
|
- Session registry: which sessions live on which servers
|
|
- Cross-server session redirect
|
|
- Load balancing for new session placement
|
|
|
|
## Phase 4: Audio & Peripherals (Weeks 10-13)
|
|
|
|
### 4.1 Audio Forwarding
|
|
- Trait-based audio backend: PipeWire (Linux), PulseAudio (illumos/Linux fallback)
|
|
- PipeWire integration on server side for audio capture (Linux)
|
|
- Opus encoding for low-latency audio streaming
|
|
- Audio stream over dedicated QUIC stream
|
|
- Playback synchronization with display frames
|
|
- Microphone input forwarding (bidirectional audio)
|
|
|
|
### 4.2 USB Device Forwarding
|
|
- Userspace USB forwarding protocol over QUIC (not kernel USB/IP, for illumos portability)
|
|
- Consider usbredir as wire format or design custom
|
|
- Device hotplug detection on client (udev on Linux, sysevent on illumos)
|
|
- Device attach/detach over QUIC channels
|
|
- Security: device class filtering (allow/deny policies)
|
|
|
|
### 4.3 Clipboard Synchronization
|
|
- Intercept wl_data_device on server
|
|
- Forward clipboard content types and data over control channel
|
|
- Handle large clipboard entries (images) efficiently
|
|
- Security: optional clipboard direction restrictions
|
|
|
|
## Phase 5: Production Hardening (Weeks 14-17)
|
|
|
|
### 5.1 Platform-Specific Backends
|
|
- Linux: DRM/KMS backend for running wayray-server on hardware (optional, feature-gated)
|
|
- Linux: Multi-GPU support via MultiRenderer
|
|
- Linux: Session management via logind/libseat
|
|
- illumos: Custom input backend for `/dev/kbd` + `/dev/mouse` (local console use)
|
|
- illumos: Zones integration for session isolation
|
|
- CI: Test matrix for both Linux and illumos
|
|
|
|
### 5.2 XWayland Support
|
|
- Integrate Smithay's XWayland module for X11 application compatibility
|
|
- Handle X11 clipboard integration
|
|
|
|
### 5.3 Performance Optimization
|
|
- Adaptive bitrate based on network conditions
|
|
- Hardware encoding path (VAAPI, NVENC)
|
|
- Zero-copy frame capture via DMA-BUF export to encoder
|
|
- Client-side frame interpolation for network jitter compensation
|
|
|
|
### 5.4 Security
|
|
- TLS 1.3 for all QUIC connections (mandatory)
|
|
- Certificate-based mutual authentication
|
|
- Session encryption at rest
|
|
- Audit logging for session lifecycle events
|
|
- AppArmor/seccomp profiles for server process
|
|
|
|
## Phase 6: Management & Operations (Weeks 16-20)
|
|
|
|
### 6.1 Administration
|
|
- CLI tool: `wayray-ctl` for server/session management
|
|
- REST API for external integration
|
|
- Session monitoring: active sessions, resource usage, network stats
|
|
|
|
### 6.2 Multi-Tenancy
|
|
- User session isolation (namespaces, cgroups)
|
|
- Resource quotas (CPU, memory, GPU per session)
|
|
- Fair scheduling across sessions
|
|
|
|
### 6.3 High Availability
|
|
- Server failover groups
|
|
- Session state replication for seamless failover
|
|
- Health checking and automatic server removal
|
|
|
|
## Milestones
|
|
|
|
| Milestone | Phase | Description |
|
|
|-----------|-------|-------------|
|
|
| M0 | 0 | Wayland clients render in local compositor |
|
|
| M1 | 1 | Remote viewer sees compositor output over network |
|
|
| M2 | 2 | Interactive remote session (display + input) |
|
|
| M2.5 | 2.5 | External WM can control window layout via protocol |
|
|
| M3 | 3 | Session persists across client disconnects, hot-desking works |
|
|
| M4 | 4 | Audio and USB forwarding functional |
|
|
| M5 | 5 | Production-ready with platform backends, XWayland, illumos CI |
|
|
| M6 | 6 | Multi-server deployment with HA and management tools |
|