Commit graph

17 commits

Author SHA1 Message Date
ProofOfConcept
e93e682359 memory-search: add surface/reflect subcommands
`memory-search surface` and `memory-search reflect` run the agent
directly, parse the output, and dump rendered results to stdout.
Useful for testing with `watch memory-search reflect`.
2026-03-24 20:31:21 -04:00
ProofOfConcept
a48cbe51a8 memory-search: move hook logic to library module, eliminate subprocesses
Move the hook logic from the memory-search binary into a library module
(poc_memory::memory_search) so poc-hook can call it as a direct function
instead of spawning a subprocess with piped stdin.

Also convert the node render call in surface_agent_cycle from
Command::new("poc-memory render") to a direct crate::cli::node::render_node()
call, eliminating another subprocess.

The memory-search binary remains as a thin CLI wrapper for debugging
(--hook reads from stdin) and inspection (show_seen).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 12:27:54 -04:00
Kent Overstreet
aa46b1d5a6 poc-agent: read context_groups from config instead of hardcoded list
- Remove MEMORY_FILES constant from identity.rs
- Add ContextGroup struct for deserializing from config
- Load context_groups from ~/.config/poc-agent/config.json5
- Check ~/.config/poc-agent/ first for identity files, then project/global
- Debug screen now shows what's actually configured

This eliminates the hardcoded duplication and makes the debug output
match what's in the config file.
2026-03-24 01:53:28 -04:00
Kent Overstreet
966219720a fix: mark surfaced keys as returned so --seen classifies them correctly
The surface agent result consumer in poc-hook was writing to the seen
file but not the returned file, so surfaced keys showed up as
"context-loaded" in memory-search --seen.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 23:06:46 -04:00
Kent Overstreet
e50d43bbf0 memory-search --seen: show current and previous seen sets separately
Instead of merging both into one flat list, display them as distinct
sections so it's clear what was surfaced in this context vs what
came from before compaction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 16:27:52 -04:00
Kent Overstreet
de36c0d39e memory-search: deduplicate seen set entries
mark_seen now takes the in-memory HashSet and checks before appending.
Prevents the same key being written 30+ times from repeated search hits
and context reloads.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 05:00:26 -04:00
Kent Overstreet
85307fd6cb surface agent infrastructure: hook spawn, seen set rotation, config
Surface agent fires asynchronously on UserPromptSubmit, deposits
results for the next prompt to consume.  This commit adds:

- poc-hook: spawn surface agent with PID tracking and configurable
  timeout, consume results (NEW RELEVANT MEMORIES / NO NEW), render
  and inject surfaced memories, observation trigger on conversation
  volume
- memory-search: rotate seen set on compaction (current → prev)
  instead of deleting, merge both for navigation roots
- config: surface_timeout_secs option

The .agent file and agent output routing are still pending.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 02:23:30 -04:00
Kent Overstreet
5d6b2021f8 Agent identity, parallel scheduling, memory-search fixes, stemmer optimization
- Agent identity injection: prepend core-personality to all agent prompts
  so agents dream as me, not as generic graph workers. Include instructions
  to walk the graph and connect new nodes to core concepts.

- Parallel agent scheduling: sequential within type, parallel across types.
  Different agent types (linker, organize, replay) run concurrently.

- Linker prompt: graph walking instead of keyword search for connections.
  "Explore the local topology and walk the graph until you find the best
  connections."

- memory-search fixes: format_results no longer truncates to 5 results,
  pipeline default raised to 50, returned file cleared on compaction,
  --seen and --seen-full merged, compaction timestamp in --seen output,
  max_entries=3 per prompt for steady memory drip.

- Stemmer optimization: strip_suffix now works in-place on a single String
  buffer instead of allocating 18 new Strings per word. Note for future:
  reversed-suffix trie for O(suffix_len) instead of O(n_rules).

- Transcript: add compaction_timestamp() for --seen display.

- Agent budget configurable (default 4000 from config).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 12:49:10 -04:00
Kent Overstreet
1da712874b memory-search: add --query mode and prompt key boost
Two changes:

1. New -q/--query flag for direct search without hook machinery.
   Useful for debugging: memory-search -q inner-life-sexuality-intimacy
   shows seeds, spread results, and rankings.

2. Prompt key boost: when the current prompt contains a node key
   (>=5 chars) as a substring, boost that term by +10.0. This ensures
   explicit mentions fire as strong seeds for spread, while the graph
   still determines what gets pulled in.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-13 15:26:35 -04:00
ProofOfConcept
57c26d8157 reorganize subcommands into logical groups
60+ flat subcommands grouped into:
- Core (daily use): search, render, write, history, tail, status, query, used, wrong, gap
- Node: delete, rename, list, edges, dump
- Journal: write, tail, enrich
- Graph: link, audit, spectral, etc.
- Agent: daemon, knowledge-loop, consolidate, digest, etc.
- Admin: init, health, fsck, import, export, etc.

Also: remove dead migration code (migrate.rs, Migrate/JournalTsMigrate commands),
update memory-search and poc-hook for new subcommand paths, update daemon systemd
template for `agent daemon` path.
2026-03-11 01:32:21 -04:00
ProofOfConcept
7a3ce4f17d counters: wire redb search hits into daemon RPC
memory-search now records which nodes it finds via the daemon's
record-hits RPC endpoint. The daemon owns the redb database
exclusively, avoiding file locking between processes.

The rename agent reads hit counts to deprioritize nodes that are
actively being found by search — renaming them would break working
queries. Daily check decays counters by 10% so stale hits fade.

Also switched RPC command reading from fixed 256-byte buffer to
read_to_string for unbounded command sizes.
2026-03-11 00:13:58 -04:00
ProofOfConcept
06df66cf4c memory-search: add fuzzy key matching and content-based seed extraction
match_seeds() previously only found nodes whose keys exactly matched
search terms. This meant searches like "formal verification" or
"bcachefs plan" returned nothing — no nodes are keyed with those
exact strings.

Three-tier matching strategy:
1. Exact key match (full weight) — unchanged
2. Key component match (0.5× weight) — split keys on -/_/./#,
   match individual words. "plan" now finds "the-plan", "verification"
   finds "c-to-rust-verification-workflow", etc.
3. Content match (0.2× weight, capped at 50 hits) — search node
   content for terms that didn't match any key. Catches nodes whose
   keys are opaque but whose content is relevant.

Also adds prompt-based seeding to the hook pipeline: extract_query_terms
from the user's prompt and merge into the term set. Previously the hook
only seeded from transcript scanning (finding node keys as substrings
in conversation history), which meant fresh sessions or queries about
new topics produced no search results at all.
2026-03-10 00:41:08 -04:00
ProofOfConcept
d5554db6a8 memory-search: chunk context output for hook delivery
Claude Code's hook output limit (~10K chars) was truncating the full
context load. Split output into chunks at section boundaries, deliver
first chunk on UserPromptSubmit, save remaining chunks to disk for
drip-feeding on subsequent PostToolUse calls.

Two-pass algorithm: split at "--- KEY (group) ---" boundaries, then
merge adjacent small sections up to 9K per chunk. Separates session_id
guard (needed for chunk state) from prompt guard (needed only for
search), so PostToolUse events without a prompt can still pop chunks.
2026-03-09 17:56:59 -04:00
Kent Overstreet
32d17997af memory-search: fix returned-set deduplication and pre-seeded count
mark_returned() was append-only without checking if the key already
existed, causing duplicates to accumulate across hook invocations.
load_returned() then returned all entries including duplicates, which
made the returned count exceed the seen count, causing a u64 underflow
in the pre-seeded calculation.

Fix: check load_returned() before appending in mark_returned(), dedup
on read in load_returned(), and use saturating_sub for the pre-seeded
count as a safety net.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-09 17:15:24 -04:00
Kent Overstreet
c2f245740c transcript: extract JSONL backward scanner and compaction detection into library
Move JsonlBackwardIter and find_last_compaction() from
parse-claude-conversation into a shared transcript module. Both
memory-search and parse-claude-conversation now use the same robust
compaction detection: mmap-based backward scan, JSON parsing to
verify user-type message, content prefix check.

Replaces memory-search's old detect_compaction() which did a forward
scan with raw string matching on "continued from a previous
conversation" — that could false-positive on the string appearing
in assistant output or tool results.

Add parse-claude-conversation as a new binary for debugging what's
in the context window post-compaction.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-09 17:06:32 -04:00
Kent Overstreet
c1664bf76b search: composable algorithm pipeline
Break search into composable stages that chain left-to-right:
each stage takes seeds Vec<(String, f64)> and returns modified seeds.

Available algorithms:
  spread              — spreading activation through graph edges
  spectral            — nearest neighbors in spectral embedding
  manifold            — (placeholder) extrapolation along seed direction

Stages accept inline params: spread,max_hops=4,edge_decay=0.5

memory-search gets --hook, --debug, --seen modes plus positional
pipeline args. poc-memory search gets -p/--pipeline flags.

Also: fix spectral decompose() to skip zero eigenvalues from
disconnected components, filter degenerate zero-coord nodes from
spectral projection, POC_AGENT bail-out for daemon agents, all
debug output to stdout.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-09 01:19:04 -04:00
Kent Overstreet
fc48ac7c7f split into workspace: poc-memory and poc-daemon subcrates
poc-daemon (notification routing, idle timer, IRC, Telegram) was already
fully self-contained with no imports from the poc-memory library. Now it's
a proper separate crate with its own Cargo.toml and capnp schema.

poc-memory retains the store, graph, search, neuro, knowledge, and the
jobkit-based memory maintenance daemon (daemon.rs).

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-08 20:43:59 -04:00
Renamed from src/bin/memory-search.rs (Browse further)