wayray/docs/ai/specs/2026-04-07-phase1-remote-display-design.md

# Phase 1: Remote Display Pipeline — Design Spec

## Goal

Build a working remote display pipeline: `wrsrvd` runs headless on a
Linux server, composites Wayland clients with PixmanRenderer, encodes
frame diffs with zstd, and sends them over QUIC to `wrclient` running
on macOS, which displays them in a native Cocoa window. Input flows
back from client to server.

This combines the roadmap's Phase 1 (protocol + transport + encoding)
and Phase 2 (client viewer) into a single deliverable, producing an
end-to-end working system.

## Context

Phase 0 built a working Smithay compositor with Winit backend, but
visual testing requires a display server — which is broken under UTM's
virtualized GPU. The solution: make the server headless (no display
server needed) and move the visual output to the client over the
network. This is the actual WayRay production architecture.

## Architecture

```
wrsrvd (Linux, headless)              wrclient (macOS)
─────────────────────                 ──────────────────
Wayland clients render
  → PixmanRenderer composites
  → OutputDamageTracker finds dirty rects
  → XOR diff against previous frame
  → zstd compress per region
  → QUIC Stream 1 (display)    →→→    Receive FrameUpdate
                                      → zstd decompress
                                      → XOR apply to local framebuffer
                                      → Upload to GPU texture (wgpu)
                                      → Display in winit/Cocoa window

  ← QUIC Stream 2 (input)     ←←←    Capture keyboard/mouse (winit)
  → Inject into Smithay seat          → Serialize as protocol messages
```

## Deliverables

### 1. Wire Protocol (`wayray-protocol` crate)

Message types serialized with `postcard` (compact binary, serde-based,
no-std compatible).

**Control messages (QUIC Stream 0 — bidirectional):**

| Message | Fields | Direction |
|---------|--------|-----------|
| `ClientHello` | `version: u32`, `capabilities: Vec<String>` | client→server |
| `ServerHello` | `version: u32`, `session_id: u64`, `output_width: u32`, `output_height: u32` | server→client |
| `Ping` | `timestamp: u64` | either |
| `Pong` | `timestamp: u64` | either |

**Display messages (QUIC Stream 1 — server→client, unidirectional):**

| Message | Fields |
|---------|--------|
| `FrameUpdate` | `sequence: u64`, `regions: Vec<DamageRegion>` |
| `DamageRegion` | `x: u32`, `y: u32`, `width: u32`, `height: u32`, `data: Vec<u8>` (zstd-compressed XOR diff) |

**Input messages (QUIC Stream 2 — client→server, unidirectional):**

| Message | Fields |
|---------|--------|
| `KeyboardEvent` | `keycode: u32`, `state: KeyState`, `time: u32` |
| `PointerMotion` | `x: f64`, `y: f64`, `time: u32` |
| `PointerButton` | `button: u32`, `state: ButtonState`, `time: u32` |
| `PointerAxis` | `axis: Axis`, `value: f64`, `time: u32` |

**Flow control:** Client sends `FrameAck { sequence: u64 }` on Stream 0
after processing each frame. Server limits in-flight frames to prevent
buffer bloat.

**Serialization:** Each message is length-prefixed (4-byte little-endian
length, then postcard-encoded payload) for framing on QUIC streams.

### 2. QUIC Transport

**Server (wrsrvd):** quinn-based QUIC listener. Accepts one client at a
time (multi-client comes with session management in Phase 3).

**Client (wrclient):** quinn-based QUIC connection to the server.

**TLS:** Self-signed certificates generated at first run, cached to
`~/.config/wayray/`. Client uses `danger_accept_any_cert` for now —
trust-on-first-use and proper PKI deferred to Phase 3.

**Stream model (logical channels, not literal QUIC stream IDs):**
- Control channel: Bidirectional stream — hello, ping/pong, frame ack
- Display channel: Unidirectional server→client stream — frame updates
- Input channel: Unidirectional client→server stream — input events

Each channel opens its own QUIC stream at connection time. QUIC assigns
the actual stream IDs based on directionality and open order.

### 3. Headless Backend (wrsrvd)

Replace the Winit backend as the default with a headless backend using
PixmanRenderer.

**PixmanRenderer path:**
- Create `PixmanRenderer` (no GPU, no EGL, no display server)
- Allocate an in-memory framebuffer as the render target
- Render with `OutputDamageTracker` for damage tracking
- Read pixels directly from RAM (no `ExportMem` needed)
- Drive the render loop on a calloop timer (render-on-damage, cap 60fps)

**Backend selection via CLI:**
- `wrsrvd` — headless (default)
- `wrsrvd --backend winit` — Winit window (dev/debug, requires display)

**Winit backend stays** as-is in a separate module, feature-gated or
behind the CLI flag.

### 4. Frame Encoding (Tier 1)

Simplest viable encoding — optimize later.

**Encoding (server):**
1. After render, get the new framebuffer (ARGB8888 pixels in RAM)
2. XOR against the previous frame to produce a diff
3. For each damage rectangle from OutputDamageTracker:
   - Extract the XOR diff region
   - Compress with zstd (level 1 — fast)
   - Package as a `DamageRegion` in the `FrameUpdate` message
4. Store current frame as the new "previous frame"

**Decoding (client):**
1. Receive `FrameUpdate`
2. For each `DamageRegion`:
   - Decompress with zstd
   - XOR-apply onto the local framebuffer copy at the given position
3. Upload the updated framebuffer to a GPU texture
4. Display

**Why XOR + zstd:** XOR diff makes unchanged pixels zero. zstd compresses
runs of zeros extremely well. For typical desktop content (mostly static
with small changes), this gives 10-100x compression with minimal CPU.

### 5. Client Viewer (wrclient)

Native application using winit + wgpu.

**Window:** winit creates a platform-native window (Cocoa on macOS).
Sized to match the server's output dimensions from `ServerHello`.

**Rendering:** wgpu renders a fullscreen textured quad. Each frame,
the updated pixel buffer is uploaded to a GPU texture and drawn.
Double-buffered with vsync.

**Input capture:** winit captures keyboard and mouse events from the
native window. Events are serialized as protocol messages and sent
over QUIC Stream 2.

**CLI:**
```
wrclient <host>:<port>
```

## Dependencies

**New workspace dependencies (`Cargo.toml`):**

| Crate | Purpose |
|-------|---------|
| `quinn` | QUIC transport |
| `rustls` | TLS for QUIC |
| `rcgen` | Self-signed certificate generation |
| `postcard` + `serde` | Wire protocol serialization |
| `zstd` | Frame compression |
| `winit` | Window creation (wrclient) |
| `wgpu` | GPU rendering (wrclient) |

**Smithay feature changes (wrsrvd):**
- Add: `renderer_pixman` (headless rendering)
- Keep: `wayland_frontend`, `desktop`, `renderer_gl`, `backend_winit`
  (behind feature flag for dev mode)

## Module Structure

```
crates/
├── wayray-protocol/src/
│   ├── lib.rs          # Re-exports
│   ├── messages.rs     # All message types + serde derives
│   ├── codec.rs        # Length-prefixed framing (encode/decode)
│   └── version.rs      # Protocol version constant
│
├── wrsrvd/src/
│   ├── main.rs         # CLI arg parsing, backend selection
│   ├── state.rs        # WayRay compositor state (unchanged)
│   ├── handlers/       # Wayland protocol handlers (unchanged)
│   ├── backend/
│   │   ├── mod.rs      # Backend trait/enum
│   │   ├── headless.rs # PixmanRenderer + in-memory framebuffer
│   │   └── winit.rs    # Existing Winit backend (moved from main.rs)
│   ├── render.rs       # Render logic (adapted for both backends)
│   ├── encoder.rs      # XOR diff + zstd compression
│   └── network/
│       ├── mod.rs      # Re-exports
│       └── server.rs   # QUIC server, frame sending, input receiving
│
├── wrclient/src/
│   ├── main.rs         # CLI args, connect, main loop
│   ├── decoder.rs      # zstd decompress + XOR apply
│   ├── display.rs      # winit window + wgpu rendering
│   ├── input.rs        # Keyboard/mouse capture + serialization
│   └── network/
│       ├── mod.rs      # Re-exports
│       └── client.rs   # QUIC client, frame receiving, input sending
│
└── wradm/src/
    └── main.rs         # Unchanged (placeholder)
```

## Success Criteria

1. `wrsrvd` starts headless on the Linux host, no display server needed
2. `wrclient` connects from macOS over the local network
3. Launch a Wayland client (foot, weston-terminal) into wrsrvd
4. The client window appears on the Mac in the wrclient window
5. Typing and clicking in the wrclient window reaches the Wayland client
6. Frame updates are visible in real-time

## Design Notes

**Cursor:** Rendered server-side into the framebuffer for Phase 1
(simplest). No separate cursor channel or client-side cursor.

**Input injection:** Network input events are injected into the Smithay
seat by calling the seat's keyboard/pointer methods directly (same
pattern as the existing Winit input handler in `state.rs`). No custom
`InputBackend` needed.

**Protocol evolution:** The message types defined here are Phase 1
subsets. They will evolve toward the full wire protocol spec
(`docs/protocols/wayray-wire-protocol.md`) in later phases.

**PixmanRenderer pixel access:** Verify early that PixmanRenderer's
framebuffer can be read directly from RAM. If Smithay requires
`ExportMem` even for Pixman, adapt accordingly — Pixman's `ExportMem`
is a simple memcpy (no GPU readback).

## Out of Scope

- Audio forwarding (Phase 4)
- USB forwarding (Phase 4)
- Session management / hot-desking (Phase 3)
- Content-adaptive encoding (Tier 2/3 — future optimization)
- Hardware video encoding (H.264/AV1 — future optimization)
- Multi-client support (Phase 3)
- Proper TLS certificate validation (Phase 3)
- Window management protocol (Phase 2.5)