wayray/docs/protocols/wayray-wire-protocol.md

WayRay Wire Protocol Specification

Overview

The WayRay wire protocol defines communication between WayRay server and client over QUIC. All messages are serialized with postcard (compact binary, no-std compatible) and transmitted over typed QUIC streams.

Protocol Version

Protocol version is exchanged during the initial handshake on the control stream. Both sides must agree on a compatible version before proceeding.

ProtocolVersion {
    major: u16,  // Breaking changes
    minor: u16,  // Additive features
    patch: u16,  // Bug fixes
}

QUIC Stream Layout

Stream 0: Control (Bidirectional, Reliable)

Session management, capability negotiation, configuration.

Stream 1: Display (Server -> Client, Semi-Reliable)

Frame updates. Old undelivered frames may be skipped in favor of newer ones.

Stream 2: Input (Client -> Server, Reliable)

Keyboard, mouse, touch, tablet input events.

Stream 3: Audio (Bidirectional, Semi-Reliable)

Opus-encoded audio frames. Tolerant of packet loss.

Stream 4+: USB (Bidirectional, Reliable)

One stream per USB device. USB/IP or usbredir encapsulated data.

Message Format

Every message on the wire is framed as:

┌──────────┬──────────┬──────────────────┐
│ type: u8 │ len: u32 │ payload: [u8]    │
└──────────┴──────────┴──────────────────┘

• type: Message type discriminant
• len: Payload length in bytes (little-endian)
• payload: Postcard-serialized message body

Control Messages (Stream 0)

ClientHello

Client -> Server. Sent immediately after QUIC connection established.

ClientHello {
    protocol_version: ProtocolVersion,
    token: SessionToken,            // Session identity
    client_capabilities: ClientCaps,
    display_info: Vec<DisplayInfo>, // Client's physical displays
}

ClientCaps {
    max_width: u32,
    max_height: u32,
    supports_h264: bool,
    supports_av1: bool,
    supports_opus: bool,
    supports_usb: bool,
    preferred_refresh_rate: u32,
}

DisplayInfo {
    width: u32,
    height: u32,
    scale_factor: f64,
    refresh_rate: u32,
}

ServerHello

Server -> Client. Response to ClientHello.

ServerHello {
    protocol_version: ProtocolVersion,
    session_id: Uuid,
    session_state: SessionState,  // New, Resumed
    server_capabilities: ServerCaps,
    output_config: OutputConfig,  // Assigned resolution/refresh
    keymap: Vec<u8>,              // XKB keymap for input
}

SessionSuspend / SessionResume

Bidirectional. Lifecycle transitions.

Ping / Pong

Bidirectional. Latency measurement and keepalive.

Display Messages (Stream 1)

FrameUpdate

Server -> Client. Contains one or more encoded regions.

FrameUpdate {
    sequence: u64,
    timestamp_us: u64,
    cursor: Option<CursorUpdate>,
    regions: Vec<EncodedRegion>,
}

EncodedRegion {
    x: u32,
    y: u32,
    width: u32,
    height: u32,
    encoding: RegionEncoding,
    data: Vec<u8>,
}

enum RegionEncoding {
    Raw,           // Uncompressed RGBA
    Zstd,          // Zstd-compressed diff against previous frame
    Jpeg,          // JPEG for photographic content
    H264,          // H.264 NAL units
    Av1,           // AV1 OBU
}

CursorUpdate {
    x: i32,
    y: i32,
    shape: Option<CursorShape>,  // Only sent when shape changes
}

CursorShape {
    width: u32,
    height: u32,
    hotspot_x: u32,
    hotspot_y: u32,
    pixels: Vec<u8>,  // RGBA
}

FrameAck

Client -> Server. Acknowledges frame receipt for flow control.

FrameAck {
    sequence: u64,
    render_time_us: u64,  // How long client took to decode + render
}

Input Messages (Stream 2)

KeyboardEvent

KeyboardEvent {
    timestamp_us: u64,
    keycode: u32,
    state: KeyState,       // Pressed, Released
    modifiers: Modifiers,  // Current modifier state
}

PointerMotion

PointerMotion {
    timestamp_us: u64,
    x: f64,  // Absolute position in output coordinates
    y: f64,
}

PointerButton

PointerButton {
    timestamp_us: u64,
    button: u32,
    state: ButtonState,
}

PointerAxis

PointerAxis {
    timestamp_us: u64,
    axis: Axis,         // Horizontal, Vertical
    value: f64,
    discrete: Option<i32>,
}

TouchEvent

TouchEvent {
    timestamp_us: u64,
    touch_type: TouchType,  // Down, Up, Motion, Frame, Cancel
    id: i32,
    x: f64,
    y: f64,
}

Audio Messages (Stream 3)

AudioFrame

AudioFrame {
    timestamp_us: u64,
    channels: u8,
    sample_rate: u32,
    codec: AudioCodec,     // Opus
    data: Vec<u8>,         // Encoded audio data
}

AudioConfig

Sent during session setup to negotiate audio parameters.

AudioConfig {
    sample_rate: u32,      // 48000
    channels: u8,          // 2 (stereo)
    frame_duration_ms: u8, // 20
    codec: AudioCodec,
}

USB Messages (Stream 4+)

USBDeviceAttach

USBDeviceAttach {
    device_id: u32,
    vendor_id: u16,
    product_id: u16,
    device_class: u8,
    device_subclass: u8,
    device_protocol: u8,
    description: String,
}

USBDeviceDetach

USBDeviceDetach {
    device_id: u32,
}

USBData

Encapsulated USB/IP or usbredir protocol data.

USBData {
    device_id: u32,
    data: Vec<u8>,
}