toasterson/akh-medu
Watch
1
0
Fork
You've already forked akh-medu
 0
Code Issues 10 Pull requests 22 Projects Releases 2 Packages Wiki Activity Actions 7

WIP: Phase 44n — context-rot curve bench (per-model MECW + positional sensitivity) #292

Draft
toasterson wants to merge 1 commit from claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo into main
pull from: claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
merge into: toasterson:main
Conversation 0 Commits 1 Files changed 2 +1009
toasterson commented 2026-08-06 12:06:48 +00:00
Owner
Copy link

Anima work item 019fbf62-03e5-74e1-a1a4-e178708b3410.

Measure per-backing-model context-rot curves (needle-depth × context-length grid against live Ollama/Candle backends) and emit two regression-trackable numbers per model: positional_sensitivity(L) (middle-vs-edge accuracy drop) and effective_length (practical MECW at threshold 0.8). Follows the bench_thinking template: logic in src/bench_phase44/rot.rs, runnable examples/bench_context_rot.rs, CSVs to target/bench-results/context-rot/, dated findings report in docs/ai/.

Sub-items: 44n.1 core bench (RotProbe trait + Ollama num_ctx-aware / Candle probes, seeded nonce needles, distractor-vs-neutral filler modes, truncation detector, metrics + CSV); 44n.2 assembly-aware rescue-rate mode (grid-ified salient_present through each ContextManager strategy vs PassThrough, LLM-free assembly-survival leg + live end-to-end leg); 44n.3 budget wiring (akh.toml effective_context override, run_agentic_turn budgets min(advertised, measured)).

Plan: docs/ai/plans/2026-08-02-phase44n-context-rot-bench.md (Seshat slug phase44n-context-rot-bench). Lineage: ADR 065, ADR 070, 44j report deferred section (docs/ai/2026-06-21-phase44j-strategy-ab.md).

Anima work item `019fbf62-03e5-74e1-a1a4-e178708b3410`. Measure per-backing-model context-rot curves (needle-depth × context-length grid against live Ollama/Candle backends) and emit two regression-trackable numbers per model: positional_sensitivity(L) (middle-vs-edge accuracy drop) and effective_length (practical MECW at threshold 0.8). Follows the bench_thinking template: logic in src/bench_phase44/rot.rs, runnable examples/bench_context_rot.rs, CSVs to target/bench-results/context-rot/, dated findings report in docs/ai/. Sub-items: 44n.1 core bench (RotProbe trait + Ollama num_ctx-aware / Candle probes, seeded nonce needles, distractor-vs-neutral filler modes, truncation detector, metrics + CSV); 44n.2 assembly-aware rescue-rate mode (grid-ified salient_present through each ContextManager strategy vs PassThrough, LLM-free assembly-survival leg + live end-to-end leg); 44n.3 budget wiring (akh.toml effective_context override, run_agentic_turn budgets min(advertised, measured)). Plan: docs/ai/plans/2026-08-02-phase44n-context-rot-bench.md (Seshat slug phase44n-context-rot-bench). Lineage: ADR 065, ADR 070, 44j report deferred section (docs/ai/2026-06-21-phase44j-strategy-ab.md).
Phase 44n — context-rot curve bench (per-model MECW + positional sensitivity)
Some checks failed
CI / check-seshat (push) Failing after 11m16s
Details
CI / publish-chart (pull_request) Successful in 11m18s
Details
CI / publish-chart (push) Successful in 11m20s
Details
CI / check-seshat (pull_request) Successful in 13m31s
Details
CI / docker-seshd (pull_request) Successful in 18m17s
Details
CI / docker-seshd (push) Successful in 18m32s
Details
11e9a0e9a7 
Anima implementation session.
Some checks failed
CI / check-seshat (push) Failing after 11m16s
Details
CI / publish-chart (pull_request) Successful in 11m18s
Details
CI / publish-chart (push) Successful in 11m20s
Details
CI / check-seshat (pull_request) Successful in 13m31s
Details
CI / docker-seshd (pull_request) Successful in 18m17s
Details
CI / docker-seshd (push) Successful in 18m32s
Details
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo:claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git switch claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git switch claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git rebase main
git switch main
git merge --ff-only claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git switch claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git rebase main
git switch main
git merge --no-ff claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git switch main
git merge --squash claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git switch main
git merge --ff-only claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git switch main
git merge claude/wi-019fbf62-phase-44n-context-rot-curve-bench-per-mo
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
toasterson/akh-medu!292
No description provided.