The graph-health logic in consolidation_plan_inner computed
reasonable agent counts based on graph metrics (α, Gini, hub
dominance), then immediately overwrote them with an Elo-weighted
flat-budget distribution, or — if no agent-elo.json existed —
with a simple budget/N per type.
Nothing in the codebase writes agent-elo.json; it's external state
that never gets maintained. So the effective behavior was always the
"No Elo ratings — equal distribution" branch, which just bucketed
agent_budget evenly across active agent types and discarded
everything the graph analysis had just decided.
Keep the graph-health allocation (α → linker count, Gini → distill
bump, organize/distill/split proportional). Drop:
- The entire Elo / agent_budget block at the end of
consolidation_plan_inner
- Config.agent_budget field and its default (1000)
- agent_budget: 40 from Kent's config.json5
- The local agent_types binding inside the function — it was only
used by the now-deleted block. Config.agent_types stays; it has
other consumers.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- KEY_TO_UUID now stores weight (30 bytes: uuid+type+ts+deleted+weight)
- UUID_OFFSETS changed to composite key for O(log n) max-offset lookup
- Add NODES_BY_TYPE index for efficient type+date range queries
- Add for_each_key_weight() to StoreView for index-only iteration
- match_seeds uses index-only path when content not needed
- Fix transaction consistency in ops (single txn for related updates)
- rebuild() now records all uuid→offset mappings for version history
- Backwards compatible: old index formats decoded with default weight
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Links clutter context windows. Use memory_links() to see links.
Pass raw=false explicitly if you want the footer.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
The strip_md_suffix function was removed but its usages remained,
causing lookups like `identity.md` to fail (stripped to `identity`
which didn't exist). Now keys are used as-is.
Renamed 4 nodes that had .md suffixes to canonical form:
- identity.md → identity
- promotion-work-queue.md-* → promotion-work-queue-*
- patterns.md#* → patterns-*
- practices.md#* → practices-*
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Restores a deleted node to its last non-deleted content with proper
version continuity (version number continues from absolute latest,
content from last live version).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Add fsck_full(): compares current index with rebuilt, reports zombies/missing
- Add repair_index(): rebuilds index from capnp log
- Index rebuild now uses timestamp (not version) for "latest" detection
Fixes tombstones shadowing restored nodes when version numbers reset
- Add read_node_at_offset_for_key() to handle batch writes correctly
When multiple nodes share an offset, filter by key to get the right one
- Add find_latest_by_key() and find_last_live_version() for restore support
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Remove POC_PROVENANCE env var lookup from new_relation - callers
now pass provenance explicitly. This fixes tracking when the env
var wasn't set correctly.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Store [negated_timestamp:8][key] as value for descending sort
- recent_by_provenance uses index directly, no capnp reads
- Eliminates 24k×5 capnp reads from subconscious snapshots
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Add all_keys() to StoreView, use in build_adjacency instead of
for_each_node (which was ignoring content/weight anyway)
- Add all_key_uuid_pairs() for single-pass uuid mapping
- Extend KEY_TO_UUID to store [uuid:16][node_type:1][timestamp:8]
- for_each_node_meta now reads from index, no capnp needed
- Add NodeType::from_u8() for unpacking
Graph health: 7s → 2s (3.5x faster)
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Store now has internal Mutex for capnp appends and AtomicU64 for
size tracking. All methods take &self. The external Arc<Mutex<Store>>
is replaced with Arc<Store>.
- Store::append_lock protects file appends
- local.rs functions take &Store (not &mut Store)
- access_local() returns Arc<Store>
- All .lock().await calls removed from callers
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Index functions now take &WriteTransaction instead of &Database,
allowing callers to batch multiple index operations in a single
transaction. Store mutations (upsert, delete, rename, etc.) now
begin_write/commit their own transactions, ensuring atomicity.
- replay_relations uses single txn for all relation indexing
- Store::db() exposes Database for callers needing txn control
- Convenience wrappers open their own txn for simple cases
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The relations Vec is gone from Store. dedup now iterates via
edges_for_uuid() instead of mutating in-memory Vec — removes/re-adds
edges through the index directly.
Removed load_relations_vec() and clear_relations() — no longer needed.
Added helper methods: edges_for_uuid, index_relation, remove_relation_from_index.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Add index::clear_relations() to drop and recreate RELS table
- Add Store::reindex_relations() to rebuild index from Vec
- Call reindex_relations() at end of dedup command
This ensures index stays in sync with Vec after complex mutations
like UUID redirection in dedup. Vec mutations remain for now but
index is correctly updated afterward.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Complete redb schema with bidirectional relation indexing:
- RELS multimap: uuid → packed(other_uuid, strength, rel_type, is_outgoing)
- Each edge stored twice (once per endpoint) with direction bit
- pack_rel/unpack_rel for 22-byte packed format
Wired up:
- replay_relations indexes all relations on load
- add_relation indexes new relations
- for_each_relation reads from index (graph building)
- add_link uses index for existence check
- set_link_strength finds/updates edges via index
- cap_degree uses index for degree counting and pruning
- rename_node finds edges by uuid
Vec<Relation> still maintained for remaining uses (normalize_strengths,
graph_health diagnostics). To be removed in follow-up.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
- capnp.rs: remove reference to removed self.nodes field
- parser.rs: comment out tests for not-yet-implemented features
(not-visited filter, recency() in composite sorts)
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
All node access now goes through index → capnp:
- scoring.rs: consolidation_priority, replay_queue, consolidation_plan
- admin.rs: cmd_init, cmd_fsck, cmd_dedup
- engine.rs: run_generator, eval_filter, run_transform
- parser.rs: resolve_field, execute_query
Added Store::remove_from_index() for dedup cleanup.
The relations Vec remains for now (used for graph building).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Add get_node() and contains_key() methods that read via redb index
- Migrate all store/ reads to use index lookup
- Remove HashMap cache updates from mutations (write-through to capnp+index only)
- Remove replay_nodes() - load no longer builds HashMap
- Update db_is_healthy to validate by spot-checking offsets
- Fix set_weight bug: now persists weight changes to capnp
Store.nodes HashMap still exists for code outside store/ module,
but store/ itself no longer uses it.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Consolidate capnp serialization in one place:
- capnp_enum! and capnp_message! macros
- read_text/read_uuid helpers
- Type-to-capnp mappings
- from_capnp_migrate migration impls
types.rs now only has pure Rust types and helpers.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
With singleton Store (one daemon, RPC for clients), there's no concurrent
writers to capnp log. The file-based flock and incremental refresh logic
was for multi-process coordination we no longer need.
-110 lines of dead concurrency code.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Restructure store module with clearer file names:
- persist.rs → capnp.rs (capnp log IO)
- db.rs → index.rs (redb index operations)
redb now stores key → offset mapping, not serialized nodes.
Mutations record the offset after appending to capnp log.
rebuild_index scans capnp log to reconstruct the index.
The HashMap still exists for now; next step is to use the
index for lookups and remove it.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Mutations (upsert_node, upsert_provenance, delete_node, rename_node)
now update redb indices atomically with capnp log appends, under the
same StoreLock.
Also removes dead cmd_import command and the parse.rs module it depended on.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Remove AgentVisit, TranscriptSegment, and all related visit tracking code.
Provenance is what we've been using to track agent interaction with nodes.
Also removes dead fields from Node (state_tag, created).
-349 lines.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Add db: Option<Database> field to Store
- Store::load() opens redb after replaying capnp logs
- Health check compares node count + spot checks keys
- Rebuilds automatically if db is missing, corrupt, or stale
- Make table definitions public for cross-module access
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Remove CACHED_STORE, cached(), is_stale(), set_store() - redundant
- Convert all Store::cached() callers to use access_local()
- Single Store::load() call remains in access() fallback path
All store access now goes through hippocampus::access() / access_local(),
which handles socket connection or local fallback with caching.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Convert cmd_fsck to async and use access_local() for the cached store.
Still uses Store::load_from_logs() for fresh comparison.
Remove unused AnyView::load() method - was never called.
Remaining Store::load() calls are all internal caching infrastructure:
- persist.rs cached() for CACHED_STORE
- mod.rs access() fallback for STORE_ACCESS
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Replace Result<_, String> with anyhow::Result throughout:
- hippocampus/store module (persist, ops, types, view, mod)
- CLI modules (admin, agent, graph, journal, node)
- Run trait in main.rs
Use .context() and .with_context() instead of .map_err(|e| format!(...))
patterns. Add bail!() for early error returns.
Add access_local() helper in hippocampus/mod.rs that returns
Result<Arc<Mutex<Store>>> for direct local store access.
Fix store access patterns to properly lock Arc<Mutex<Store>> before
accessing fields in mind/unconscious.rs, mind/mod.rs, subconscious/learn.rs,
and hippocampus/memory.rs.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- journal_tail returns Vec<JournalEntry> with key, content, created_at
- load_startup_journal uses typed API, no more direct Store access
- CLI does formatting, hippocampus returns data
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- hippocampus::memory_links now returns Vec<LinkInfo> with key,
link_strength, and node_weight for each neighbor
- Unified memory_tool! macro: mut/ref as token, single main rule
- All tools use serde serialize/deserialize for RPC consistency
- jsonargs handlers now work in client mode (RPC to daemon)
- cli/graph.rs formats LinkInfo for display
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Aligns function names with tool names for consistency:
- hippocampus: render → memory_render, write → memory_write, etc.
- tools/memory.rs: macro no longer prepends memory_ prefix
- CLI files: use typed async API throughout (graph.rs, journal.rs, admin.rs)
This eliminates the "memory_graph_topology" tool name bug where
graph_* and journal_* tools were incorrectly prefixed.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
These were early experiments with manual feedback signals that
never worked well. The scoring system will handle this properly.
Removed:
- CLI: used, wrong, not-relevant, not-useful, gap
- MCP: memory_used
- Store: mark_used, mark_wrong, record_gap, modify_node
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
TrackedMutex and TrackedRwLock wrappers that record hold durations
by source location using #[track_caller]. Stats written to
~/.consciousness/lock-stats.json every second, sorted by max hold time.
Re-exported as crate::Mutex so all locks are instrumented. To disable,
swap the re-export back to tokio::sync::Mutex.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- MCP memory_query tool now uses execute_query path instead of
parse_stages, enabling full expression support (content ~, AND/OR,
neighbors, etc.) instead of just Expr::All
- Parser now accepts double-quoted strings ("foo") in addition to
single quotes ('foo')
- Added tests for double-quote syntax
- Removed dead resolve_field_str function from memory.rs
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Adds parsing for weighted sort expressions like:
sort:degree*0.5+isolation*0.3+recency(organize)*0.2
This fixes organize agent which uses composite scoring.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Read max_concurrent from config (llm_concurrency) instead of hardcoding 2
- Add not-visited: and visited: filters to query parser (were in engine
but missing from parser after unification)
The organize agent was stuck in a spawn/fail loop because its query used
not-visited: which the parser didn't recognize.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
PEG parser now handles both expression syntax (degree > 5 | sort degree)
and pipeline syntax (all | type:episodic | sort:timestamp). Deleted
Stage::parse() and helpers from engine.rs — it's now pure execution.
All callers use parse_stages() from parser.rs as the single entry point.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Text cosine similarity was being used as a crutch for operations
the graph structure should handle: interference detection, orphan
linking, triangle closing, hub differentiation. These are all
graph-structural operations that the agents (linker, extractor)
handle with actual semantic understanding.
Removed: similarity.rs (stemming + cosine), rewrite.rs (orphan
linking, triangle closing, hub differentiation), detect_interference,
and all CLI commands and consolidation steps that used them.
-794 lines.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Duplicate key warnings fire on every store load and were writing to
stderr, corrupting the TUI display. Log write warnings and MCP
server failures are similarly routine. Route these to dbglog.
Serious errors (rkyv snapshot failures, store corruption) remain on
stderr — those are real problems the user needs to see.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
