Run an agent on nodes matching a query:
poc-memory agent run linker --query 'key ~ "bcachefs" | limit 10'
Resolves the query to node keys, then passes all as seeds to the agent.
For large batches, should be queued to daemon (future work).
experience_mine and journal_enrich are replaced by the observation
agent. enrich.rs reduced from 465 to 40 lines — only extract_conversation
and split_on_compaction remain (used by observation fragment selection).
-455 lines.
Remove unused StoreView imports, unused store imports, dead
install_default_file, dead make_report_slug, dead fact-mine/
experience-mine spawning loops in daemon. Fix mut warnings.
Zero compiler warnings now.
Adds run_one_agent_with_keys() which bypasses the agent's query and
uses explicitly provided node keys. This allows testing agents on
specific graph neighborhoods:
poc-memory agent run linker --target bcachefs --debug
Consolidate agent logging to one file per run in llm-logs/{agent}/.
Prompt written before LLM call, response appended after. --debug
additionally prints the same content to stdout.
Remove duplicate eprintln! calls and AgentResult.prompt field.
Kill experience_mine and fact_mine job functions from daemon —
observation.agent handles all transcript mining.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add --debug flag that prints the full prompt and LLM response to
stdout, making it easy to iterate on agent prompts. Also adds
prompt field to AgentResult so callers can inspect what was sent.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New command: `poc-memory agent run <agent> [--count N] [--dry-run]`
Runs a single agent by name through the full pipeline (build prompt,
call LLM, apply actions). With --dry-run, sets POC_MEMORY_DRY_RUN=1
so all mutations are no-ops but the agent can still read the graph.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The LCG was producing only 2 distinct matchup pairs due to poor
constants. Switch to xorshift32 for proper coverage of all type pairs.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Replace sort-based ranking with proper Elo system:
- Each agent TYPE has a persistent Elo rating (agent-elo.json)
- Each matchup: pick two random types, grab a recent action from
each, LLM compares, update ratings
- Ratings persist across daily evaluations — natural recency bias
from continuous updates against current opponents
- K=32 for fast adaptation to prompt changes
Usage: poc-memory agent evaluate --matchups 30 --model haiku
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
TIE causes inconsistency in sort (A=B, B=C but A>C breaks ordering).
Force the comparator to always pick a winner. Default to A if response
is unparseable.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
- Use CARGO_MANIFEST_DIR for agent file path (same as defs.rs)
- Dedup affected nodes extracted from reports
- --dry-run shows example comparison prompt without LLM calls
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Chain-of-thought: "say which is better and why" forces clearer
judgment and gives us analysis data for improving agents.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
When both actions are from the same agent, show the instructions once
and just compare the two report outputs + affected nodes. Saves tokens
and makes the comparison cleaner.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Each comparison now shows the LLM:
- Agent instructions (the .agent prompt file)
- Report output (what the agent did)
- Affected nodes content (what it changed)
The comparator sees intent, action, and impact — can judge whether
a deletion was correct, whether links are meaningful, whether
WRITE_NODEs capture real insights.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Yes, really. Rust's stdlib sort_by with an LLM pairwise comparator.
Each comparison is an API call asking "which action was better?"
Sample N actions per agent type, throw them all in a Vec, sort.
Where each agent's samples cluster = that agent's quality score.
Reports per-type average rank and quality ratio.
Supports both haiku (fast/cheap) and sonnet (quality) as comparator.
Usage: poc-memory agent evaluate --samples 5 --model haiku
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
