toasterson/akh-medu
Watch
1
0
Fork
You've already forked akh-medu
 0
Code Issues 12 Pull requests 37 Projects Releases 3 Packages 8 Wiki Activity Actions 1

Draft: Electra training provenance, native qualification and mentor evaluation harness #561

Open
toasterson wants to merge 6 commits from codex/electra-harness into main
pull from: codex/electra-harness
merge into: toasterson:main
Conversation 0 Commits 6 Files changed 21 +1885
toasterson commented 2026-10-06 13:26:34 +00:00
Owner
Copy link

Add Electra experiment drivers for the existing Akh-Medu harness: baseline preparation, repository/library/native outcome collection, small prediction components, Lilith-initialized backbone continuation and merge provenance, pinned Candle qualification, frozen reasoning pilots, and evidence-citing mentor judges. Keep outcome prediction and disposition artifacts separate from the language-model serving baseline.

The evaluation workflow preserves failed attempts and provider responses, requires event citations, uses null for unobservable axes, and separates held-out assessment from supportive practice. Qualification verifies clean main ancestry and pinned model/tokenizer/binaries; fixtures are installed in the allocated sandbox before model input.

Validation: 16 Python tests pass. Real Electra Candle Vulkan trials passed exact readiness and brokered file read after runtime PR #558; Electra and hosted GLM each passed the two frozen BBH pilot items. GLM-5.3, GPT-OSS:120B and Astra independently reviewed sanitized packets. These are small component observations, not full-agent or leaderboard results.

Draft limitation: full-catalog prefill on shared Hephaistos hit an experiment memory limit and then a GPU reset. Lilith inference was restored using unchanged service configuration. Further shared-device GPU trials and full cluster activation are held pending an appropriate compute window/machine. Terminal-Bench and DeepSWE are pinned but unrun. No learning-gain claim or new evaluation-derived weight training is made.

Add Electra experiment drivers for the existing Akh-Medu harness: baseline preparation, repository/library/native outcome collection, small prediction components, Lilith-initialized backbone continuation and merge provenance, pinned Candle qualification, frozen reasoning pilots, and evidence-citing mentor judges. Keep outcome prediction and disposition artifacts separate from the language-model serving baseline. The evaluation workflow preserves failed attempts and provider responses, requires event citations, uses null for unobservable axes, and separates held-out assessment from supportive practice. Qualification verifies clean main ancestry and pinned model/tokenizer/binaries; fixtures are installed in the allocated sandbox before model input. Validation: 16 Python tests pass. Real Electra Candle Vulkan trials passed exact readiness and brokered file read after runtime PR #558; Electra and hosted GLM each passed the two frozen BBH pilot items. GLM-5.3, GPT-OSS:120B and Astra independently reviewed sanitized packets. These are small component observations, not full-agent or leaderboard results. Draft limitation: full-catalog prefill on shared Hephaistos hit an experiment memory limit and then a GPU reset. Lilith inference was restored using unchanged service configuration. Further shared-device GPU trials and full cluster activation are held pending an appropriate compute window/machine. Terminal-Bench and DeepSWE are pinned but unrun. No learning-gain claim or new evaluation-derived weight training is made.
Place qualification fixtures inside the allocated sandbox
Some checks failed
CI / test-session-platform (pull_request) Has been cancelled
Details
CI / test-sokar (pull_request) Has been cancelled
Details
CI / check-seshat (pull_request) Has been cancelled
Details
CI / check-akh-medu (pull_request) Has been cancelled
Details
CI / test-akh-medu (pull_request) Has been cancelled
Details
CI / docker-seshd (pull_request) Has been cancelled
Details
CI / docker-sokard (pull_request) Has been cancelled
Details
CI / docker-akh-transcripts (pull_request) Has been cancelled
Details
CI / publish-chart (pull_request) Has been cancelled
Details
CI / platform-images-home (pull_request) Has been cancelled
Details
CI / platform-images-core (pull_request) Has been cancelled
Details
CI / platform-images-daemons (pull_request) Has been cancelled
Details
CI / platform-images-fleet (pull_request) Has been cancelled
Details
CI / platform-images-collect (pull_request) Has been cancelled
Details
4df8166d89
Retain judge transport status and retry-after without credentials
All checks were successful
CI / check-seshat (pull_request) Successful in 14m42s
Details
CI / docker-seshd (pull_request) Successful in 16m2s
Details
CI / test-session-platform (pull_request) Successful in 32m12s
Details
CI / docker-sokard (pull_request) Successful in 22m57s
Details
CI / test-sokar (pull_request) Successful in 34m31s
Details
CI / test-akh-medu (pull_request) Successful in 33m44s
Details
CI / publish-chart (pull_request) Successful in 24m3s
Details
CI / docker-akh-transcripts (pull_request) Successful in 33m18s
Details
CI / check-akh-medu (pull_request) Successful in 1h13m38s
Details
CI / platform-images-daemons (pull_request) Successful in 3m49s
Details
CI / platform-images-core (pull_request) Successful in 8m11s
Details
CI / platform-images-home (pull_request) Successful in 15m28s
Details
CI / platform-images-fleet (pull_request) Successful in 12m26s
Details
CI / platform-images-collect (pull_request) Successful in 2h56m26s
Details
df102e6517
All checks were successful
CI / check-seshat (pull_request) Successful in 14m42s
Details
CI / docker-seshd (pull_request) Successful in 16m2s
Details
CI / test-session-platform (pull_request) Successful in 32m12s
Details
CI / docker-sokard (pull_request) Successful in 22m57s
Details
CI / test-sokar (pull_request) Successful in 34m31s
Details
CI / test-akh-medu (pull_request) Successful in 33m44s
Details
CI / publish-chart (pull_request) Successful in 24m3s
Details
CI / docker-akh-transcripts (pull_request) Successful in 33m18s
Details
CI / check-akh-medu (pull_request) Successful in 1h13m38s
Details
CI / platform-images-daemons (pull_request) Successful in 3m49s
Details
CI / platform-images-core (pull_request) Successful in 8m11s
Details
CI / platform-images-home (pull_request) Successful in 15m28s
Details
CI / platform-images-fleet (pull_request) Successful in 12m26s
Details
CI / platform-images-collect (pull_request) Successful in 2h56m26s
Details
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin codex/electra-harness:codex/electra-harness
git switch codex/electra-harness

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff codex/electra-harness
git switch codex/electra-harness
git rebase main
git switch main
git merge --ff-only codex/electra-harness
git switch codex/electra-harness
git rebase main
git switch main
git merge --no-ff codex/electra-harness
git switch main
git merge --squash codex/electra-harness
git switch main
git merge --ff-only codex/electra-harness
git switch main
git merge codex/electra-harness
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
follow-up
Documented operational follow-up

No labels
follow-up
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
toasterson/akh-medu!561
No description provided.