Commit graph

74 commits

Author SHA1 Message Date
Kent Overstreet
e8462af505 Remove .md suffix stripping from key lookups
The strip_md_suffix function was removed but its usages remained,
causing lookups like `identity.md` to fail (stripped to `identity`
which didn't exist). Now keys are used as-is.

Renamed 4 nodes that had .md suffixes to canonical form:
- identity.md → identity
- promotion-work-queue.md-* → promotion-work-queue-*
- patterns.md#* → patterns-*
- practices.md#* → practices-*

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-15 02:08:35 -04:00
Kent Overstreet
6a5b840db3 cli: add 'node restore' command for undeleting nodes
Restores a deleted node to its last non-deleted content with proper
version continuity (version number continues from absolute latest,
content from last live version).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-15 01:40:29 -04:00
Kent Overstreet
6ec7fcb777 store: protected nodes, explicit provenance in mutations
- Add protected_nodes config list - blocks delete/rename of core nodes
- Remove current_provenance() env var lookup, pass provenance explicitly
- delete_node, rename_node, set_link_strength now take provenance param
- Fix new_relation calls in admin.rs to pass "system" provenance

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-15 01:40:18 -04:00
Kent Overstreet
b3d0a3ab25 store: internal locking, remove Arc<Mutex<Store>> wrapper
Store now has internal Mutex for capnp appends and AtomicU64 for
size tracking. All methods take &self. The external Arc<Mutex<Store>>
is replaced with Arc<Store>.

- Store::append_lock protects file appends
- local.rs functions take &Store (not &mut Store)
- access_local() returns Arc<Store>
- All .lock().await calls removed from callers

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 21:49:54 -04:00
Kent Overstreet
4696bb8b7d store: index ops take WriteTransaction, mutations batch properly
Index functions now take &WriteTransaction instead of &Database,
allowing callers to batch multiple index operations in a single
transaction. Store mutations (upsert, delete, rename, etc.) now
begin_write/commit their own transactions, ensuring atomicity.

- replay_relations uses single txn for all relation indexing
- Store::db() exposes Database for callers needing txn control
- Convenience wrappers open their own txn for simple cases

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 21:44:20 -04:00
Kent Overstreet
2548ca059d store: remove Vec<Relation>, dedup uses index iteration
The relations Vec is gone from Store. dedup now iterates via
edges_for_uuid() instead of mutating in-memory Vec — removes/re-adds
edges through the index directly.

Removed load_relations_vec() and clear_relations() — no longer needed.
Added helper methods: edges_for_uuid, index_relation, remove_relation_from_index.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 21:32:48 -04:00
Kent Overstreet
c2de14dcab store: add reindex_relations for after Vec mutations
- Add index::clear_relations() to drop and recreate RELS table
- Add Store::reindex_relations() to rebuild index from Vec
- Call reindex_relations() at end of dedup command

This ensures index stays in sync with Vec after complex mutations
like UUID redirection in dedup. Vec mutations remain for now but
index is correctly updated afterward.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-13 21:24:49 -04:00
Kent Overstreet
58b0947625 admin: convert fsck and dedup reads to use index
- fsck: use for_each_relation for dangling edge detection
  (pruning deferred - needs delete_edge operation)
- dedup: use for_each_relation for edge counting

Remaining Vec uses in dedup mutation section need new index ops:
- redirect_edge: change source/target UUID
- delete_edge_by_uuid: tombstone by UUID

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-13 21:22:52 -04:00
Kent Overstreet
5877fd857a store: remove nodes and uuid_to_key HashMaps
All node access now goes through index → capnp:
- scoring.rs: consolidation_priority, replay_queue, consolidation_plan
- admin.rs: cmd_init, cmd_fsck, cmd_dedup
- engine.rs: run_generator, eval_filter, run_transform
- parser.rs: resolve_field, execute_query

Added Store::remove_from_index() for dedup cleanup.

The relations Vec remains for now (used for graph building).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 19:49:09 -04:00
Kent Overstreet
9309de68fc store: wire up redb updates on mutations
Mutations (upsert_node, upsert_provenance, delete_node, rename_node)
now update redb indices atomically with capnp log appends, under the
same StoreLock.

Also removes dead cmd_import command and the parse.rs module it depended on.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 19:03:09 -04:00
Kent Overstreet
a1accc7cd4 store: remove visit tracking infrastructure
Remove AgentVisit, TranscriptSegment, and all related visit tracking code.
Provenance is what we've been using to track agent interaction with nodes.

Also removes dead fields from Node (state_tag, created).

-349 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:57:12 -04:00
Kent Overstreet
7d49f29fde store: remove dead code and move params to config
Remove:
- score_weight() - never called
- position field on Node - never read (was for export)
- Provenance enum - inline helper for capnp migration
- migrate_transcript_progress + CLI command
- init_from_markdown, import_file, ingest_units
- export command and export_to_markdown
- RetrievalEvent, GapRecord types
- classify_filename, new_transcript_segment

Move spreading activation params to Config:
- default_node_weight, edge_decay, max_hops, min_activation
- Remove Params struct and StoreView::params()

Simplify cmd_init to just seed identity via upsert().
Simplify cmd_import to use parse_units + upsert directly.

-576 lines

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:50:21 -04:00
Kent Overstreet
2caccf875d Replace rkyv/bincode caching with redb indices
Remove three-tier loading (rkyv snapshot, bincode cache, capnp replay)
in favor of direct capnp log replay + redb for indexed access.

- Remove all rkyv derives from types (Node, Relation, enums, etc.)
- Remove Snapshot struct, RKYV_MAGIC, CACHE_MAGIC constants
- Remove load_snapshot_mmap(), save(), save_snapshot()
- Remove MmapView, AnyView from view.rs (keep StoreView trait)
- Simplify Store::load() to just replay capnp logs
- Add db.rs with redb schema: nodes, uuid_to_key, visits, transcript_progress
- Simplify cmd_fsck to just check capnp integrity + graph health

capnp logs remain source of truth; redb indices will be rebuilt on demand.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:30:58 -04:00
Kent Overstreet
09b30d64f2 cmd_fsck: use access_local(), remove dead AnyView::load()
Convert cmd_fsck to async and use access_local() for the cached store.
Still uses Store::load_from_logs() for fresh comparison.

Remove unused AnyView::load() method - was never called.

Remaining Store::load() calls are all internal caching infrastructure:
- persist.rs cached() for CACHED_STORE
- mod.rs access() fallback for STORE_ACCESS

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:09:02 -04:00
Kent Overstreet
f6f330b07b Convert cmd_import, cmd_export, MigrateTranscriptProgress to access_local()
These were the last Store::load() calls that should use the shared store.
Remaining calls are intentional: fsck (needs both cached and fresh),
persist.rs cached() infrastructure, view.rs read-only fallback, and
access() bootstrap path.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:06:58 -04:00
Kent Overstreet
b8db8754be Convert store and CLI to anyhow::Result for cleaner error handling
Replace Result<_, String> with anyhow::Result throughout:
- hippocampus/store module (persist, ops, types, view, mod)
- CLI modules (admin, agent, graph, journal, node)
- Run trait in main.rs

Use .context() and .with_context() instead of .map_err(|e| format!(...))
patterns. Add bail!() for early error returns.

Add access_local() helper in hippocampus/mod.rs that returns
Result<Arc<Mutex<Store>>> for direct local store access.

Fix store access patterns to properly lock Arc<Mutex<Store>> before
accessing fields in mind/unconscious.rs, mind/mod.rs, subconscious/learn.rs,
and hippocampus/memory.rs.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:05:04 -04:00
Kent Overstreet
5db00e083f centralize memory store interface in hippocampus/mod.rs 2026-04-13 17:44:41 -04:00
Kent Overstreet
063cf031d3 journal_tail: return typed Vec<JournalEntry>, remove Store::load from agent
- journal_tail returns Vec<JournalEntry> with key, content, created_at
- load_startup_journal uses typed API, no more direct Store access
- CLI does formatting, hippocampus returns data

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 15:23:10 -04:00
Kent Overstreet
598f0112a4 memory_links: return typed Vec<LinkInfo> with node weights
- hippocampus::memory_links now returns Vec<LinkInfo> with key,
  link_strength, and node_weight for each neighbor
- Unified memory_tool! macro: mut/ref as token, single main rule
- All tools use serde serialize/deserialize for RPC consistency
- jsonargs handlers now work in client mode (RPC to daemon)
- cli/graph.rs formats LinkInfo for display

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 15:12:06 -04:00
Kent Overstreet
359955f838 defs.rs: async conversion, remove block_in_place
Convert resolve(), resolve_placeholders(), run_agent() to async.
Use memory_render/memory_query directly with .await instead of
block_in_place wrappers.

Propagate async to callers:
- config.rs: resolve(), load_session(), reload_for_model()
- identity.rs: load_memory_files(), assemble_context_message()
- oneshot.rs: run_one_agent()
- prompts.rs: agent_prompt()

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 14:56:26 -04:00
Kent Overstreet
fb46ab095d Consolidate memory RPC in tools/memory.rs
- Move memory_rpc(), socket_path(), SocketConn from mcp_server.rs
- Convert remaining callers to typed async API:
  - defs.rs: organize placeholder, run_agent query
  - cli/agent.rs: query resolution (now async)
  - mind/identity.rs: Store context loading
- Re-export socket_path/memory_rpc from mcp_server for compatibility

All external memory access now goes through tools/memory.rs typed API.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 13:39:59 -04:00
Kent Overstreet
5b07a81aa7 CLI/hippocampus: rename core memory functions to memory_*
Aligns function names with tool names for consistency:
- hippocampus: render → memory_render, write → memory_write, etc.
- tools/memory.rs: macro no longer prepends memory_ prefix
- CLI files: use typed async API throughout (graph.rs, journal.rs, admin.rs)

This eliminates the "memory_graph_topology" tool name bug where
graph_* and journal_* tools were incorrectly prefixed.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 13:26:22 -04:00
Kent Overstreet
fa50f1c826 CLI: convert node commands to typed async API
- node.rs: use memory::* typed helpers instead of memory_rpc()
- main.rs: make Run trait async, await all command dispatch
- defs.rs: bridge get_group_content async via block_in_place

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 13:20:04 -04:00
Kent Overstreet
2ab4aef19f CLI: more RPC conversions, delete obsolete commands
- cmd_health: use graph_health RPC
- cmd_topology: new command using graph_topology RPC
- cmd_status: use graph_topology RPC (type counts folded into topology)
- cmd_run_agent: query resolution via memory_query RPC
- Delete cmd_bulk_rename (one-time migration, obsolete)
- Delete cmd_replay_queue, cmd_digest_links (unconscious agents handle)
- format_topology_header: add type counts, takes &Store now

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-12 23:37:05 -04:00
Kent Overstreet
1f6bfb5915 kill cmd_graph, cmd_organize 2026-04-12 23:20:19 -04:00
ProofOfConcept
11f2d5b169 graph_trace, graph_link_impact: convert to RPC tools
Agents can use these to understand graph structure:
- trace: shows node and neighbors grouped by type
- link_impact: analyzes what happens if a link is removed

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 23:16:12 -04:00
ProofOfConcept
f02a23468e graph_normalize_strengths: convert to RPC tool
Agents can use this to check if edge weights are skewed.
Dry run by default, pass apply:true to write changes.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 23:12:42 -04:00
ProofOfConcept
a8d91896a2 graph_communities: new RPC tool, convert cmd_communities
Agents can use graph_communities to discover isolated knowledge
clusters that need better integration.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 23:09:12 -04:00
ProofOfConcept
be9db3fb1a graph: delete cmd_spread, convert cmd_link to RPC
cmd_spread was duplicate of cmd_search/memory_search.
cmd_link now uses memory_links RPC.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 23:07:06 -04:00
Kent Overstreet
70097fa84b kill cli/misc.rs 2026-04-12 23:03:00 -04:00
ProofOfConcept
5a832b1d6c get_group_content: use RPC, delete store-based version
One function that uses memory_rpc (which handles daemon vs local).
Removes 65 lines of duplicate logic.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 23:00:10 -04:00
ProofOfConcept
d1d57267d3 Remove cmd_log and cmd_params
Retrieval log was never used (history covers node log).
Params should come from config, not hardcoded store defaults.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:47:58 -04:00
ProofOfConcept
aff872e101 cmd_search: thin wrapper around memory_search RPC
Remove term matching, pipeline stages, mmap/store paths. Just
pass keys to memory_search and print result. For anything fancy,
use memory_query.

-165 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:44:59 -04:00
ProofOfConcept
8b59becbab cmd_load_context: use RPC instead of Store::load()
Add get_group_content_rpc() which uses memory_query and memory_render
instead of direct store access. The original get_group_content() stays
for the subconscious path which already has a store open.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:32:47 -04:00
Kent Overstreet
a6b93c2255 cli: kill cmd_list_keys
Redundant with query

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:27:31 -04:00
Kent Overstreet
ad59596335 cli: add memory_history, remove dump-json/edges/lookups
- Add memory_history MCP tool for version history
- Convert cmd_history to use memory_rpc
- Add raw parameter to memory_render for editing
- Remove unused: dump-json, list-edges, lookup-bump, lookups
- Fix render_node path in defs.rs/subconscious.rs

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:24:34 -04:00
Kent Overstreet
3e0c6b039f move render_node() to memory.rs 2026-04-12 22:20:31 -04:00
Kent Overstreet
4b4271f618 cli: convert cmd_edit to use memory_rpc
Add raw parameter to memory_render for getting content without
links footer. cmd_edit now uses memory_render(raw=true) to read
and memory_write to save.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:18:28 -04:00
Kent Overstreet
366b17163d cli: convert write/delete/journal-write to use memory_rpc
- cmd_write → memory_write RPC
- cmd_node_delete → new memory_delete MCP tool + RPC
- cmd_journal_write → journal_new RPC

Removes validate_inline_refs and find_current_transcript
(now handled server-side or not needed).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:15:53 -04:00
Kent Overstreet
7842b6fc8b remove legacy feedback commands (used, wrong, gap, etc.)
These were early experiments with manual feedback signals that
never worked well. The scoring system will handle this properly.

Removed:
- CLI: used, wrong, not-relevant, not-useful, gap
- MCP: memory_used
- Store: mark_used, mark_wrong, record_gap, modify_node

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:12:02 -04:00
Kent Overstreet
11b58e6b0b cli: convert simple commands to use memory_rpc
Commands now forward to daemon (or fallback to local store):
- query → memory_query
- journal tail → journal_tail
- graph link-set → memory_link_set
- graph link-add → memory_link_add
- weight-set → memory_weight_set
- node rename → memory_rename

Removes ~50 lines of duplicated store access code.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 21:54:34 -04:00
Kent Overstreet
93fcc32a00 journal tail: use query engine instead of manual filtering
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 02:29:52 -04:00
Kent Overstreet
919749dc67 more dead code deletion
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-12 02:27:05 -04:00
Kent Overstreet
f00532bdb7 TurnResult: remove text field, simplify oneshot loop
- Remove TurnResult.text (was dead code - Agent::turn handles text internally)
- Simplify run_with_backend to just iterate over steps (Agent::turn loops
  for tool calls and handles empty responses internally)
- Change run/run_shared/run_forked_shared to return Result<(), String>
- Remove AgentResult.output field (no callers used it)
- Stub out legacy text-parsing code (audit, compare) that needs redesign
- Update digest.rs to not depend on text return
- Add level parameter to journal_new/journal_update for digest support

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 02:04:50 -04:00
ProofOfConcept
aad227e487 query: unify PEG and engine parsers
PEG parser now handles both expression syntax (degree > 5 | sort degree)
and pipeline syntax (all | type:episodic | sort:timestamp). Deleted
Stage::parse() and helpers from engine.rs — it's now pure execution.

All callers use parse_stages() from parser.rs as the single entry point.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 20:42:58 -04:00
Kent Overstreet
d269f9006d delete dead code 2026-04-10 16:17:28 -04:00
ProofOfConcept
96e573f2e5 Delete similarity module, rewrite module, and all text-similarity code
Text cosine similarity was being used as a crutch for operations
the graph structure should handle: interference detection, orphan
linking, triangle closing, hub differentiation. These are all
graph-structural operations that the agents (linker, extractor)
handle with actual semantic understanding.

Removed: similarity.rs (stemming + cosine), rewrite.rs (orphan
linking, triangle closing, hub differentiation), detect_interference,
and all CLI commands and consolidation steps that used them.

-794 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:44:10 -04:00
ProofOfConcept
aad0cd669a Remove poc-memory daemon and RPC infrastructure
The background daemon and its job orchestration are redundant now that
the consciousness binary handles everything directly. Gut daemon.rs
down to just GraphHealth + compute_graph_health (used by the F4 TUI
screen), remove the DaemonCmd CLI subcommand, strip daemon RPC
fast-paths from cli/agent.rs, and drop the jobkit dependency.

-1330 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 20:07:05 -04:00
Kent Overstreet
929415af3b delete claude code integration 2026-04-09 19:58:07 -04:00
Kent Overstreet
fba8fcc587 Fix UTF-8 slicing panics: use floor_char_boundary for all truncation
Byte-position truncation (&s[..s.len().min(N)]) panics when position
N lands inside a multi-byte character. Fixed in parser debug logging,
API error messages, oneshot response logging, and CLI agent display.

Also fixed tool dispatch permissions (removed global fallback).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 19:35:01 -04:00