Commit graph

16 commits

Author SHA1 Message Date
ProofOfConcept
83342897c8 experience-mine: link at creation time, remove # from new keys
Update the experience mining prompt to output links alongside journal
entries. The LLM now returns a "links" array per entry pointing to
existing semantic nodes. Rust code creates the links immediately after
node creation — new nodes arrive pre-connected instead of orphaned.

Also: remove # from all key generation paths (experience miner,
digest section keys, observed transcript keys). New nodes get clean
dash-separated keys.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 16:25:31 -04:00
ProofOfConcept
abce1bba16 digest: structural links, story-like prompt, agent file
When generating a digest, automatically link all source entries to the
digest node (journal entries → daily, dailies → weekly, weeklies →
monthly). This builds the temporal spine of the graph — previously
~4000 journal entries were disconnected islands unreachable by recall.

Rewrote digest prompt to produce narrative rather than reports:
capture the feel, the emotional arc, what it was like to live through
it. Letter to future self, not a task log.

Moved prompt to digest.agent file alongside other agent definitions.
Falls back to prompts/digest.md if agent file not found.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-13 21:37:56 -04:00
ProofOfConcept
945865f594 agents: extract run_and_apply, eliminate dead split-plan.md
- Add run_and_apply() — combines run_one_agent + action application
  into one call. Used by daemon job_consolidation_agent and
  consolidate_full, which had identical run+apply loops.

- Port split_plan_prompt() to use split.agent via defs::resolve_placeholders
  instead of loading the separate split-plan.md template. Make
  resolve_placeholders public for this.

- Delete prompts/split-plan.md — superseded by agents/split.agent
  which was already the canonical definition.
2026-03-10 17:51:32 -04:00
ProofOfConcept
f6ea659975 consolidate: eliminate second LLM call, apply actions inline
The consolidation pipeline previously made a second Sonnet call to
extract structured JSON actions from agent reports. This was both
wasteful (extra LLM call per consolidation) and lossy (only extracted
links and manual items, ignoring WRITE_NODE/REFINE).

Now actions are parsed and applied inline after each agent runs, using
the same parse_all_actions() parser as the knowledge loop. The daemon
scheduler's separate apply phase is also removed.

Also deletes 8 superseded/orphaned prompt .md files (784 lines) that
have been replaced by .agent files.
2026-03-10 17:22:53 -04:00
ProofOfConcept
42d8e265da prompts: remove dead prompt files superseded by .agent files
These prompts are now embedded in their .agent files or no longer
called from any code path.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-10 17:06:29 -04:00
ProofOfConcept
8bbc246b3d split agent: parallel execution, agent-driven edges, no MCP overhead
- Refactor split from serial batch to independent per-node tasks
  (run-agent split N spawns N parallel tasks, gated by llm_concurrency)
- Replace cosine similarity edge inheritance with agent-assigned
  neighbors in the plan JSON — the LLM already understands the
  semantic relationships, no need to approximate with bag-of-words
- Add --strict-mcp-config to claude CLI calls to skip MCP server
  startup (saves ~5s per call)
- Remove hardcoded 2000-char split threshold — let the agent decide
  what's worth splitting
- Reload store before mutations to handle concurrent split races
2026-03-10 03:21:33 -04:00
ProofOfConcept
ca62692a28 split agent: two-phase node decomposition for memory consolidation
Phase 1 sends a large node with its neighbor communities to the LLM
and gets back a JSON split plan (child keys, descriptions, section
hints). Phase 2 fires one extraction call per child in parallel —
each gets the full parent content and extracts/reorganizes just its
portion.

This handles arbitrarily large nodes because output is always
proportional to one child, not the whole parent. Tested on the kent
node (19K chars → 3 children totaling 20K chars with clean topic
separation).

New files:
  prompts/split-plan.md   — phase 1 planning prompt
  prompts/split-extract.md — phase 2 extraction prompt
  prompts/split.md        — original single-phase (kept for reference)

Modified:
  agents/prompts.rs — split_candidates(), split_plan_prompt(),
                      split_extract_prompt(), agent_prompt "split" arm
  agents/daemon.rs  — job_split_agent() two-phase implementation,
                      RPC dispatch for "split" agent type
  tui.rs            — added "split" to AGENT_TYPES
2026-03-10 01:48:41 -04:00
ProofOfConcept
4c973183c4 rename agent: LLM-powered semantic key generation for memory nodes
New consolidation agent that reads node content and generates semantic
3-5 word kebab-case keys, replacing auto-generated slugs (5K+ journal
entries with truncated first-line slugs, 2.5K mined transcripts with
opaque UUIDs).

Implementation:
- prompts/rename.md: agent prompt template with naming conventions
- prompts.rs: format_rename_candidates() selects nodes with long
  auto-generated keys, newest first
- daemon.rs: job_rename_agent() parses RENAME actions from LLM
  output and applies them directly via store.rename_node()
- Wired into RPC handler (run-agent rename) and TUI agent types
- Fix epoch_to_local panic on invalid timestamps (fallback to UTC)

Rename dramatically improves search: key-component matching on
"journal#2026-02-28-violin-dream-room" makes the node findable by
"violin", "dream", or "room" — the auto-slug was unsearchable.
2026-03-10 00:55:26 -04:00
ProofOfConcept
61dd67caf7 experience-mine: harden prompt boundary against transcript injection
Add explicit markers around the conversation transcript so the LLM
treats it as input data rather than instructions to follow.
2026-03-08 18:31:35 -04:00
ProofOfConcept
c085679a0f stash DMN algorithm plan and connector prompt fix
dmn-algorithm-plan.md: seeds, spreading activation, refractory
suppression, spectral diversity, softmax sampling design.
connector.md: add CONFIDENCE guidance so connector outputs aren't
silently rejected by depth threshold.
2026-03-05 10:24:24 -05:00
ProofOfConcept
31c1bca7d7 digest: drop per-level instructions and section templates
The LLM knows how to structure a summary. Move the essential framing
(narrative not task log, link to memory, include Links section) into
the shared prompt template. Drop the ~130 lines of per-level output
format specifications — the level name, date range, and inputs are
sufficient context.
2026-03-03 17:53:43 -05:00
Kent Overstreet
796c72fb25 digest: unify generators and prompts across all three levels
Three near-identical generate_daily/weekly/monthly functions collapsed
into one generate_digest() parameterized by DigestLevel descriptors.
Three separate prompt templates merged into one prompts/digest.md with
level-specific instructions carried in the DigestLevel struct.

Each level defines: name, title, period label, input title, output
format instructions, child prefix (None for daily = reads journal),
and Sonnet timeout.

digest_auto simplified correspondingly — same three phases but using
the unified generator.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-03 17:34:00 -05:00
Kent Overstreet
50da0b7b26 digest: split into focused modules, externalize prompts
digest.rs was 2328 lines containing 6 distinct subsystems. Split into:
- llm.rs: shared LLM utilities (call_sonnet, parse_json_response, semantic_keys)
- audit.rs: link quality audit with parallel Sonnet batching
- enrich.rs: journal enrichment + experience mining
- consolidate.rs: consolidation pipeline + apply

Externalized all inline prompts to prompts/*.md templates using
neuro::load_prompt with {{PLACEHOLDER}} syntax:
- daily-digest.md, weekly-digest.md, monthly-digest.md
- experience.md, journal-enrich.md, consolidation.md

digest.rs retains temporal digest generation (daily/weekly/monthly/auto)
and date helpers. ~940 lines, down from 2328.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-03 17:18:18 -05:00
ProofOfConcept
71e6f15d82 spectral decomposition, search improvements, char boundary fix
- New spectral module: Laplacian eigendecomposition of the memory graph.
  Commands: spectral, spectral-save, spectral-neighbors, spectral-positions,
  spectral-suggest. Spectral neighbors expand search results beyond keyword
  matching to structural proximity.

- Search: use StoreView trait to avoid 6MB state.bin rewrite on every query.
  Append-only retrieval logging. Spectral expansion shows structurally
  nearby nodes after text results.

- Fix panic in journal-tail: string truncation at byte 67 could land inside
  a multi-byte character (em dash). Now walks back to char boundary.

- Replay queue: show classification and spectral outlier score.

- Knowledge agents: extractor, challenger, connector prompts and runner
  scripts for automated graph enrichment.

- memory-search hook: stale state file cleanup (24h expiry).
2026-03-03 01:33:31 -05:00
Kent Overstreet
3e883b7ba7 show suggested link targets in agent prompts
Agents were flying blind — they could see nodes to review and the
topology header, but had no way to discover what targets to link to.
Now each node shows its top 8 text-similar semantic nodes that aren't
already neighbors, giving agents a search-like capability.

Also added section-level targeting guidance to linker.md, transfer.md,
and replay.md prompts: always target the most specific section, not
the file-level node.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-01 00:37:03 -05:00
ProofOfConcept
23fac4e5fe poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
  schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
  link-add, link-impact, decay, consolidate-session, etc.

Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00