toasterson/Anima

Watch

Fork 0

A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks #523

Merged

toasterson merged 10 commits from claude/wi-019fdd2a-a-fully-cleared-exit-bar-does-not-advanc into main

2026-08-09 18:44:12 +00:00

toasterson commented

2026-08-08 12:43:55 +00:00

Owner

Anima work item 019fdd2a-205d-7451-9de9-fcd7e706dd87.

Symptom

A work item whose exit bar is fully satisfied for its current phase is never moved to the next phase. Instead the scheduler keeps dispatching fresh implementation sessions against it until the retry budget runs out and it parks as retry-budget. The item is finished; the pipeline just never notices.

Evidence (WI-404, PR #485)

work_item_exit_bar_state for WI-404 phase Implementation: machine_cleared=t, ai_gate_cleared=t, updated 2026-08-06 23:48.
workflow_phases for Implementation: exit_bar_type='ai', ai_gate_enabled=t, machine_cleared and human_review_accepted NOT required. So the bar was fully satisfied at 23:48.
The PR head did not move again after ed873c90 (23:16).
Despite that, session.started (executor claude-code) fired at 00:17, 00:55, 01:42, 02:16, 02:53, 03:29 — each an implementation session that re-read the task, produced no diff, and ended stop_reason: end_turn.
At 04:00 the item parked retry-budget.
The operator had to hand-carry it: update_work_item(current_phase=testing) then approve_and_merge, which merged first try.

Roughly 8 wasted Opus sessions on an item that was already done, overnight, on one item.

Not the same as #500

#500 gave implementation sessions workitem.write so a session that knows it finished can advance itself. This defect is the case where the session does not know: it re-reads a task that is already implemented, correctly concludes there is nothing to do, and ends the turn. Nothing on the server side consults the exit bar and promotes the item, so "nothing to do" loops forever.

Blast radius

Five items are sitting in this state right now, all with machine_cleared AND ai_gate_cleared on Implementation and all parked rather than in testing: WI-384, WI-455, WI-453, WI-454, WI-402. Any item that clears its bar between reconciler passes lands here.

Expected

The phase-advance decision must be driven by the exit bar, server-side, not by a session volunteering to advance itself:

When a gate write satisfies the phase's exit bar (submit_review_verdict approve, CI green, human accept), advance the item to the next phase in the same transaction.
A periodic reconciler sweeps items whose bar is satisfied but whose phase has not advanced — this is the safety net for gates written by any path, including operator/manual writes and anything that crashed between the two steps.
The scheduler must refuse to dispatch an implementation session for an item whose current-phase exit bar is already satisfied. That is the loud failure: dispatching implementation work at a cleared bar is always a bug, and today it is silent.

Acceptance criteria

A test that sets the bar to satisfied and asserts the item advances without any session running.
A test that asserts assemble_and_send refuses (and logs loudly) when the current phase's exit bar is already cleared.
The reconciler pass is covered by a test with a bar satisfied "in the past" and a phase that never moved.
Backfill or a one-shot sweep for the five items above.

Anima work item `019fdd2a-205d-7451-9de9-fcd7e706dd87`. ## Symptom A work item whose exit bar is fully satisfied for its current phase is never moved to the next phase. Instead the scheduler keeps dispatching fresh *implementation* sessions against it until the retry budget runs out and it parks as `retry-budget`. The item is finished; the pipeline just never notices. ## Evidence (WI-404, PR #485) - `work_item_exit_bar_state` for WI-404 phase `Implementation`: `machine_cleared=t`, `ai_gate_cleared=t`, updated `2026-08-06 23:48`. - `workflow_phases` for `Implementation`: `exit_bar_type='ai'`, `ai_gate_enabled=t`, `machine_cleared` and `human_review_accepted` NOT required. So the bar was fully satisfied at 23:48. - The PR head did not move again after `ed873c90` (23:16). - Despite that, `session.started` (executor `claude-code`) fired at 00:17, 00:55, 01:42, 02:16, 02:53, 03:29 — each an implementation session that re-read the task, produced no diff, and ended `stop_reason: end_turn`. - At 04:00 the item parked `retry-budget`. - The operator had to hand-carry it: `update_work_item(current_phase=testing)` then `approve_and_merge`, which merged first try. Roughly 8 wasted Opus sessions on an item that was already done, overnight, on one item. ## Not the same as #500 #500 gave implementation sessions `workitem.write` so a session that *knows* it finished can advance itself. This defect is the case where the session does not know: it re-reads a task that is already implemented, correctly concludes there is nothing to do, and ends the turn. Nothing on the server side consults the exit bar and promotes the item, so "nothing to do" loops forever. ## Blast radius Five items are sitting in this state right now, all with `machine_cleared AND ai_gate_cleared` on `Implementation` and all parked rather than in testing: WI-384, WI-455, WI-453, WI-454, WI-402. Any item that clears its bar between reconciler passes lands here. ## Expected The phase-advance decision must be driven by the exit bar, server-side, not by a session volunteering to advance itself: 1. When a gate write satisfies the phase's exit bar (`submit_review_verdict` approve, CI green, human accept), advance the item to the next phase in the same transaction. 2. A periodic reconciler sweeps items whose bar is satisfied but whose phase has not advanced — this is the safety net for gates written by any path, including operator/manual writes and anything that crashed between the two steps. 3. The scheduler must refuse to dispatch an implementation session for an item whose current-phase exit bar is already satisfied. That is the loud failure: dispatching implementation work at a cleared bar is always a bug, and today it is silent. ## Acceptance criteria - A test that sets the bar to satisfied and asserts the item advances without any session running. - A test that asserts `assemble_and_send` refuses (and logs loudly) when the current phase's exit bar is already cleared. - The reconciler pass is covered by a test with a bar satisfied "in the past" and a phase that never moved. - Backfill or a one-shot sweep for the five items above.

toasterson added 1 commit

2026-08-08 12:45:50 +00:00

A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks c9e334f65f

Anima implementation session.

toasterson added 1 commit

2026-08-08 13:38:13 +00:00

A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks 64ae527ed8

Anima implementation session.

toasterson added 1 commit

2026-08-08 14:38:01 +00:00

A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks ca4d1e3ec8

Anima implementation session.

toasterson added 1 commit

2026-08-08 14:58:25 +00:00

A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks 1568f2966a

Anima implementation session.

toasterson added 1 commit

2026-08-08 15:49:58 +00:00

Fix exit bar advancement tests to match actual workflow configuration 90c90b32da

The tests were assuming the working phase had a machine exit bar and the
testing phase had a human exit bar, but the actual workflow configuration
(after migration 049) has:
- Working phase (order 1): AI exit bar (ai_gate_enabled=true)
- Testing phase (order 2): empty exit bar (always satisfied, no next phase)

Updated tests to:
1. Use AI gate clearing for working phase tests instead of machine gate
2. Renamed human_gate_write_advances_when_bar_satisfied to
   human_gate_write_handles_no_next_phase to reflect that testing is the
   last phase and correctly assert no advancement
3. Removed assertions about initial bar satisfaction state that were
   dependent on workflow configuration details
4. Kept the core acceptance test coverage: transactional advancement on
   gate writes, reconciler sweep, and scheduler guard

All 6 tests now pass, demonstrating the implementation correctly handles:
- AI gate transactional advancement (working → testing)
- Reconciler sweep for satisfied but un-advanced WIs
- Scheduler guard refusing dispatch when bar already satisfied
- Edge case of no next phase to advance to

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

toasterson added 1 commit

2026-08-08 16:42:13 +00:00

A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks 570492d2d5

Anima implementation session.

toasterson added 1 commit

2026-08-08 20:22:16 +00:00

ci: re-trigger — prior run lost in orchestrator restart 17:30Z (run 3b0f0ff2 zombie-running / 817fac10 stranded-queued) 4a6fa9d819

toasterson added 1 commit

2026-08-09 10:31:20 +00:00

ci: re-trigger solstice run after lost dispatch (babysit 2026-08-09) 3818394dcb

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

toasterson added 1 commit

2026-08-09 14:56:38 +00:00

ci: re-trigger — prior job request vanished in degraded broker (babysit 2026-08-09) 6478d012b5

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

toasterson force-pushed claude/wi-019fdd2a-a-fully-cleared-exit-bar-does-not-advanc from 6478d012b5 to 26bcbbd1b6

2026-08-09 15:27:49 +00:00

Compare

toasterson changed title from ~~WIP: A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks~~ to A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks

2026-08-09 18:43:56 +00:00

toasterson merged commit 8000bf3344 into main

2026-08-09 18:44:12 +00:00

toasterson referenced this pull request from a commit

2026-08-09 18:44:15 +00:00

Merge pull request 'A fully-cleared exit bar does not advance the work item: the scheduler keeps re-dispatching implementation sessions until the item parks' (#523) from claude/wi-019fdd2a-a-fully-cleared-exit-bar-does-not-advanc into main