toasterson/akh-medu
Watch
1
0
Fork
You've already forked akh-medu
 0
Code Issues 10 Pull requests 22 Projects Releases 2 Packages Wiki Activity Actions 7

WIP: Anima integration — workspace daemon that pulls session transcripts from Seshat via Anima queue API #272

Draft
toasterson wants to merge 2 commits from claude/wi-019db1bd-anima-integration-workspace-daemon-that into main
pull from: claude/wi-019db1bd-anima-integration-workspace-daemon-that
merge into: toasterson:main
Conversation 0 Commits 2 Files changed 3 +424
toasterson commented 2026-07-27 21:52:01 +00:00
Owner
Copy link

Anima work item 019db1bd-0f30-72b1-a481-b58319de0447.

Cross-project work item: Akh-Medu side of docs/design/2026-04-21-document-system.md (in the Anima repo). Corresponds to Phase PA5.

Anima writes LLM session transcripts directly to Seshat (not to akh-medu) and exposes a queue API for akh-medu workspaces to pull them on their own cadence:

GET  https://anima.wegmueller.it/api/v1/workspaces/{workspace}/ungraphed_documents?limit=N
     → [{document_id, seshat_document_id, doc_type, created_at}]
POST https://anima.wegmueller.it/api/v1/documents/{document_id}/mark_graphed
     body: {akhmedu_symbol_id, facts_added, facts_updated}

Auth: Bearer PAT scoped akhmedu.read, akhmedu.write, single-workspace. Anima provisions + rotates this PAT; helm values inject it.

Implement in akh-medu:

  1. New daemon / cron inside akh-medu's existing daemon subsystem (see daemon_start / daemon_status MCP tools). Per-workspace schedule: poll every 10 min (configurable per workspace via a new ADR / plan).
  2. For each ungraphed document:
    • Fetch text from Seshat via the existing Seshat client inside akh-medu (seshat_search or the archive fetch when available)
    • Run the existing /library ingest pipeline against the workspace (parser → chunker → NLU → VSA → triples)
    • On success, POST mark_graphed back to Anima with the doc-root symbol id + counts
  3. Backoff + retry on failure. Bounded in-flight (max 4 concurrent ingestions per workspace).
  4. Cold-start catch-up: on first boot against a newly-attached workspace, pull in batches of 50 with 1-second pauses to avoid swamping the engine.
  5. Configurable via workspace config: 
    [anima-integration]
api_url = "https://anima.wegmueller.it"
bearer_token_env = "ANIMA_DAEMON_PAT"
poll_interval_sec = 600
batch_size = 50

Acceptance (as verifiable from the Anima side):

  • After Anima's A3 ships and one session transcript exists, within 15 min the Anima documents row has akhmedu_graphed_at set and akhmedu_symbol_id populated.
  • provenance_of(symbol_id) in the relevant workspace returns the document's extraction chain.
  • Poll interval is honoured (verified via Anima request logs).

References: docs/design/2026-04-21-document-system.md §7 (session-transcript pull-loop), §14 (PA5). Blocks the closing of Anima A4's loop.

2026-06-04 21:20 UTC

Summary

Create a background daemon in Akh-Medu that periodically pulls ungraphed session transcripts from Anima's API, retrieves their content from Seshat, orchestrates them through the existing ingestion pipeline, and updates Anima with the extracted fact metrics.

Approach

Extend Akh-Medu's workspace TOML parser to read the new [anima-integration] config block. Implement an Anima HTTP client using the configured environment variable token to poll the GET .../ungraphed_documents endpoint on the specified interval. Integrate this polling loop into Akh-Medu's existing daemon subsystem, adding state to handle cold starts (chunking into batches of 50 with a 1-second pause). For each retrieved document, download the text via the local Seshat client, process it through the /library ingest pipeline (parser → chunker → NLU → VSA → triples) utilizing bounded concurrency (max 4 in-flight), and implement exponential backoff for transient failures. Upon successful pipeline execution, fire a callback using the POST .../mark_graphed Anima endpoint with the resulting akhmedu_symbol_id, facts_added, and facts_updated.

Files likely to change

  • src/config/workspace.rs (guess)
  • src/clients/anima.rs (guess)
  • src/daemon/anima_pull.rs (guess)
  • src/daemon/mod.rs (guess)

Open questions

  • What is the exact backoff strategy (e.g., max retries, max delay ceiling) for network or ingestion failures?
  • How should we handle "poison pill" documents that consistently fail the ingestion pipeline—should they be marked as failed in Anima, or just dropped locally after N retries?
  • Does the existing /library ingest pipeline already return distinct facts_added versus facts_updated counts, or will the pipeline need to be modified to surface these metrics?
  • Does the 1-second cold-start delay apply between fetching individual documents from Seshat, or between overall batches?

Complexity

M: Modifying the config schema, building a new external API client with auth, coordinating bounded async ingestion with rate limits, and hooking into the daemon lifecycle involves moderate complexity spanning 1-3 days.

Anima work item `019db1bd-0f30-72b1-a481-b58319de0447`. Cross-project work item: **Akh-Medu side** of docs/design/2026-04-21-document-system.md (in the Anima repo). Corresponds to Phase PA5. Anima writes LLM session transcripts directly to Seshat (not to akh-medu) and exposes a queue API for akh-medu workspaces to pull them on their own cadence: ``` GET https://anima.wegmueller.it/api/v1/workspaces/{workspace}/ungraphed_documents?limit=N → [{document_id, seshat_document_id, doc_type, created_at}] POST https://anima.wegmueller.it/api/v1/documents/{document_id}/mark_graphed body: {akhmedu_symbol_id, facts_added, facts_updated} ``` Auth: Bearer PAT scoped `akhmedu.read`, `akhmedu.write`, single-workspace. Anima provisions + rotates this PAT; helm values inject it. **Implement in akh-medu:** 1. New daemon / cron inside akh-medu's existing daemon subsystem (see `daemon_start` / `daemon_status` MCP tools). Per-workspace schedule: poll every 10 min (configurable per workspace via a new ADR / plan). 2. For each ungraphed document: - Fetch text from Seshat via the existing Seshat client inside akh-medu (`seshat_search` or the archive fetch when available) - Run the existing `/library` ingest pipeline against the workspace (parser → chunker → NLU → VSA → triples) - On success, POST `mark_graphed` back to Anima with the doc-root symbol id + counts 3. Backoff + retry on failure. Bounded in-flight (max 4 concurrent ingestions per workspace). 4. Cold-start catch-up: on first boot against a newly-attached workspace, pull in batches of 50 with 1-second pauses to avoid swamping the engine. 5. Configurable via workspace config: ```toml [anima-integration] api_url = "https://anima.wegmueller.it" bearer_token_env = "ANIMA_DAEMON_PAT" poll_interval_sec = 600 batch_size = 50 ``` **Acceptance (as verifiable from the Anima side):** - After Anima's A3 ships and one session transcript exists, within 15 min the Anima `documents` row has `akhmedu_graphed_at` set and `akhmedu_symbol_id` populated. - `provenance_of(symbol_id)` in the relevant workspace returns the document's extraction chain. - Poll interval is honoured (verified via Anima request logs). **References:** docs/design/2026-04-21-document-system.md §7 (session-transcript pull-loop), §14 (PA5). Blocks the closing of Anima A4's loop. <!-- ANIMA TRIAGE PLAN --> 2026-06-04 21:20 UTC ## Summary Create a background daemon in Akh-Medu that periodically pulls ungraphed session transcripts from Anima's API, retrieves their content from Seshat, orchestrates them through the existing ingestion pipeline, and updates Anima with the extracted fact metrics. ## Approach Extend Akh-Medu's workspace TOML parser to read the new `[anima-integration]` config block. Implement an Anima HTTP client using the configured environment variable token to poll the `GET .../ungraphed_documents` endpoint on the specified interval. Integrate this polling loop into Akh-Medu's existing daemon subsystem, adding state to handle cold starts (chunking into batches of 50 with a 1-second pause). For each retrieved document, download the text via the local Seshat client, process it through the `/library` ingest pipeline (parser → chunker → NLU → VSA → triples) utilizing bounded concurrency (max 4 in-flight), and implement exponential backoff for transient failures. Upon successful pipeline execution, fire a callback using the `POST .../mark_graphed` Anima endpoint with the resulting `akhmedu_symbol_id`, `facts_added`, and `facts_updated`. ## Files likely to change * `src/config/workspace.rs` (guess) * `src/clients/anima.rs` (guess) * `src/daemon/anima_pull.rs` (guess) * `src/daemon/mod.rs` (guess) ## Open questions * What is the exact backoff strategy (e.g., max retries, max delay ceiling) for network or ingestion failures? * How should we handle "poison pill" documents that consistently fail the ingestion pipeline—should they be marked as failed in Anima, or just dropped locally after N retries? * Does the existing `/library` ingest pipeline already return distinct `facts_added` versus `facts_updated` counts, or will the pipeline need to be modified to surface these metrics? * Does the 1-second cold-start delay apply between fetching individual documents from Seshat, or between overall batches? ## Complexity M: Modifying the config schema, building a new external API client with auth, coordinating bounded async ingestion with rate limits, and hooking into the daemon lifecycle involves moderate complexity spanning 1-3 days.
Anima integration — workspace daemon that pulls session transcripts from Seshat via Anima queue API
All checks were successful
CI / check-seshat (pull_request) Successful in 23m9s
Details
CI / publish-chart (pull_request) Successful in 20m0s
Details
CI / publish-chart (push) Successful in 14m38s
Details
CI / docker-seshd (pull_request) Successful in 30m28s
Details
CI / check-seshat (push) Successful in 24m2s
Details
CI / docker-seshd (push) Successful in 29m54s
Details
60a3787b94 
Anima implementation session.
All checks were successful
CI / check-seshat (pull_request) Successful in 23m9s
Details
CI / publish-chart (pull_request) Successful in 20m0s
Details
CI / publish-chart (push) Successful in 14m38s
Details
CI / docker-seshd (pull_request) Successful in 30m28s
Details
CI / check-seshat (push) Successful in 24m2s
Details
CI / docker-seshd (push) Successful in 29m54s
Details
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin claude/wi-019db1bd-anima-integration-workspace-daemon-that:claude/wi-019db1bd-anima-integration-workspace-daemon-that
git switch claude/wi-019db1bd-anima-integration-workspace-daemon-that

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff claude/wi-019db1bd-anima-integration-workspace-daemon-that
git switch claude/wi-019db1bd-anima-integration-workspace-daemon-that
git rebase main
git switch main
git merge --ff-only claude/wi-019db1bd-anima-integration-workspace-daemon-that
git switch claude/wi-019db1bd-anima-integration-workspace-daemon-that
git rebase main
git switch main
git merge --no-ff claude/wi-019db1bd-anima-integration-workspace-daemon-that
git switch main
git merge --squash claude/wi-019db1bd-anima-integration-workspace-daemon-that
git switch main
git merge --ff-only claude/wi-019db1bd-anima-integration-workspace-daemon-that
git switch main
git merge claude/wi-019db1bd-anima-integration-workspace-daemon-that
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
toasterson/akh-medu!272
No description provided.