fix(runner): fail undeclared sessions at reattach, loudly #337

Merged

toasterson merged 1 commit from claude/archive-releases-slot into main

2026-07-20 21:06:23 +00:00

toasterson commented

2026-07-20 21:06:15 +00:00

Owner

Kills the recurring zombie class (three today, one per fleet restart): a runner's re-Hello resumable_sessions list is complete — built from its process-lifetime session map — so a bound live session missing from it cannot come back. The reconcile still gave those the full 30-minute rebind window, leaving the session running and its thread routing prompts into the void.

Now an undeclared session older than a short grace (ANIMA_RUNNER_ATTACH_ORPHAN_GRACE_SECS, default 60s — an Assign may still be racing the Hello) is failed at attach, and its thread gets a system note ("Session was lost in a runner restart — send another message to start fresh") instead of a typing indicator over a corpse. The staleness gate moves from updated_at (which heartbeat vouching kept fresh right up to the kill) to created_at. Server restarts are unaffected — the runner process survives those and declares its sessions.

Server+db only; no proto change, no migrations.

🤖 Generated with Claude Code

Kills the recurring zombie class (three today, one per fleet restart): a runner's re-Hello `resumable_sessions` list is complete — built from its process-lifetime session map — so a bound live session missing from it cannot come back. The reconcile still gave those the full 30-minute rebind window, leaving the session `running` and its thread routing prompts into the void. Now an undeclared session older than a short grace (`ANIMA_RUNNER_ATTACH_ORPHAN_GRACE_SECS`, default 60s — an Assign may still be racing the Hello) is failed at attach, and its thread gets a system note ("Session was lost in a runner restart — send another message to start fresh") instead of a typing indicator over a corpse. The staleness gate moves from `updated_at` (which heartbeat vouching kept fresh right up to the kill) to `created_at`. Server restarts are unaffected — the runner process survives those and declares its sessions. Server+db only; no proto change, no migrations. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

toasterson added 1 commit

2026-07-20 21:06:15 +00:00

fix(runner): fail undeclared sessions at reattach, loudly

CI / conflict-check (push) Successful in 26s

Details

CI / docker (push) Successful in 10m27s

Details

4379d16526

The runner's re-Hello resumable_sessions list is COMPLETE (built from
its process-lifetime session map), so a bound live session missing from
it cannot come back — its warm state died with the old process. The
reconcile still gave those the full 30-minute rebind window, leaving
the session 'running' and its thread routing prompts into the void
after every fleet restart (three live zombies on 2026-07-20, each
manually failed in the DB).

Now an undeclared session older than a short grace
(ANIMA_RUNNER_ATTACH_ORPHAN_GRACE_SECS, default 60s — an Assign may
still be racing the Hello) is failed at attach, and the thread gets a
system note ('Session was lost in a runner restart — send another
message to start fresh') instead of a typing indicator over a corpse.
Server restarts are unaffected: the runner process survives those and
declares its sessions. The staleness gate moves from updated_at (which
heartbeat vouching kept fresh right up to the kill) to created_at.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

toasterson merged commit 334d47a79f into main

2026-07-20 21:06:23 +00:00

toasterson referenced this pull request from a commit

2026-07-20 21:06:24 +00:00

Merge pull request 'fix(runner): fail undeclared sessions at reattach, loudly' (#337) from claude/archive-releases-slot into main

toasterson referenced this pull request

2026-07-25 13:03:03 +00:00

A failed turn is indistinguishable from an empty one: server ignores stop_reason, WI re-dispatches forever in working #366