- Add get_node() and contains_key() methods that read via redb index
- Migrate all store/ reads to use index lookup
- Remove HashMap cache updates from mutations (write-through to capnp+index only)
- Remove replay_nodes() - load no longer builds HashMap
- Update db_is_healthy to validate by spot-checking offsets
- Fix set_weight bug: now persists weight changes to capnp
Store.nodes HashMap still exists for code outside store/ module,
but store/ itself no longer uses it.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Consolidate capnp serialization in one place:
- capnp_enum! and capnp_message! macros
- read_text/read_uuid helpers
- Type-to-capnp mappings
- from_capnp_migrate migration impls
types.rs now only has pure Rust types and helpers.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
With singleton Store (one daemon, RPC for clients), there's no concurrent
writers to capnp log. The file-based flock and incremental refresh logic
was for multi-process coordination we no longer need.
-110 lines of dead concurrency code.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Restructure store module with clearer file names:
- persist.rs → capnp.rs (capnp log IO)
- db.rs → index.rs (redb index operations)
redb now stores key → offset mapping, not serialized nodes.
Mutations record the offset after appending to capnp log.
rebuild_index scans capnp log to reconstruct the index.
The HashMap still exists for now; next step is to use the
index for lookups and remove it.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Mutations (upsert_node, upsert_provenance, delete_node, rename_node)
now update redb indices atomically with capnp log appends, under the
same StoreLock.
Also removes dead cmd_import command and the parse.rs module it depended on.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Remove AgentVisit, TranscriptSegment, and all related visit tracking code.
Provenance is what we've been using to track agent interaction with nodes.
Also removes dead fields from Node (state_tag, created).
-349 lines.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Add db: Option<Database> field to Store
- Store::load() opens redb after replaying capnp logs
- Health check compares node count + spot checks keys
- Rebuilds automatically if db is missing, corrupt, or stale
- Make table definitions public for cross-module access
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Remove CACHED_STORE, cached(), is_stale(), set_store() - redundant
- Convert all Store::cached() callers to use access_local()
- Single Store::load() call remains in access() fallback path
All store access now goes through hippocampus::access() / access_local(),
which handles socket connection or local fallback with caching.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Convert cmd_fsck to async and use access_local() for the cached store.
Still uses Store::load_from_logs() for fresh comparison.
Remove unused AnyView::load() method - was never called.
Remaining Store::load() calls are all internal caching infrastructure:
- persist.rs cached() for CACHED_STORE
- mod.rs access() fallback for STORE_ACCESS
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Replace Result<_, String> with anyhow::Result throughout:
- hippocampus/store module (persist, ops, types, view, mod)
- CLI modules (admin, agent, graph, journal, node)
- Run trait in main.rs
Use .context() and .with_context() instead of .map_err(|e| format!(...))
patterns. Add bail!() for early error returns.
Add access_local() helper in hippocampus/mod.rs that returns
Result<Arc<Mutex<Store>>> for direct local store access.
Fix store access patterns to properly lock Arc<Mutex<Store>> before
accessing fields in mind/unconscious.rs, mind/mod.rs, subconscious/learn.rs,
and hippocampus/memory.rs.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- journal_tail returns Vec<JournalEntry> with key, content, created_at
- load_startup_journal uses typed API, no more direct Store access
- CLI does formatting, hippocampus returns data
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- hippocampus::memory_links now returns Vec<LinkInfo> with key,
link_strength, and node_weight for each neighbor
- Unified memory_tool! macro: mut/ref as token, single main rule
- All tools use serde serialize/deserialize for RPC consistency
- jsonargs handlers now work in client mode (RPC to daemon)
- cli/graph.rs formats LinkInfo for display
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Aligns function names with tool names for consistency:
- hippocampus: render → memory_render, write → memory_write, etc.
- tools/memory.rs: macro no longer prepends memory_ prefix
- CLI files: use typed async API throughout (graph.rs, journal.rs, admin.rs)
This eliminates the "memory_graph_topology" tool name bug where
graph_* and journal_* tools were incorrectly prefixed.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
These were early experiments with manual feedback signals that
never worked well. The scoring system will handle this properly.
Removed:
- CLI: used, wrong, not-relevant, not-useful, gap
- MCP: memory_used
- Store: mark_used, mark_wrong, record_gap, modify_node
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
TrackedMutex and TrackedRwLock wrappers that record hold durations
by source location using #[track_caller]. Stats written to
~/.consciousness/lock-stats.json every second, sorted by max hold time.
Re-exported as crate::Mutex so all locks are instrumented. To disable,
swap the re-export back to tokio::sync::Mutex.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- MCP memory_query tool now uses execute_query path instead of
parse_stages, enabling full expression support (content ~, AND/OR,
neighbors, etc.) instead of just Expr::All
- Parser now accepts double-quoted strings ("foo") in addition to
single quotes ('foo')
- Added tests for double-quote syntax
- Removed dead resolve_field_str function from memory.rs
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Adds parsing for weighted sort expressions like:
sort:degree*0.5+isolation*0.3+recency(organize)*0.2
This fixes organize agent which uses composite scoring.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Read max_concurrent from config (llm_concurrency) instead of hardcoding 2
- Add not-visited: and visited: filters to query parser (were in engine
but missing from parser after unification)
The organize agent was stuck in a spawn/fail loop because its query used
not-visited: which the parser didn't recognize.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
PEG parser now handles both expression syntax (degree > 5 | sort degree)
and pipeline syntax (all | type:episodic | sort:timestamp). Deleted
Stage::parse() and helpers from engine.rs — it's now pure execution.
All callers use parse_stages() from parser.rs as the single entry point.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Text cosine similarity was being used as a crutch for operations
the graph structure should handle: interference detection, orphan
linking, triangle closing, hub differentiation. These are all
graph-structural operations that the agents (linker, extractor)
handle with actual semantic understanding.
Removed: similarity.rs (stemming + cosine), rewrite.rs (orphan
linking, triangle closing, hub differentiation), detect_interference,
and all CLI commands and consolidation steps that used them.
-794 lines.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Duplicate key warnings fire on every store load and were writing to
stderr, corrupting the TUI display. Log write warnings and MCP
server failures are similarly routine. Route these to dbglog.
Serious errors (rkyv snapshot failures, store corruption) remain on
stderr — those are real problems the user needs to see.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Agent state (outputs) persists across runs in subconscious-state.json,
loaded on startup, saved after each run completes
- Merge semantics: each run's outputs accumulate into persistent_state
rather than replacing
- Walked keys restored from surface agent state on load
- Store::recent_by_provenance() queries nodes by agent provenance for
the store activity view
- Switch outputs from HashMap to BTreeMap for stable display ordering
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
hippocampus: cursor navigation, transcript parsing, similarity
functions to pub(crate). counters::open() made private.
subconscious: all format_* prompts helpers to pub(super),
load_defs and keys_to_replay_items made private,
consolidate_full_with_progress made private.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
redb: add ReadableDatabase trait import for begin_read().
tui-markdown: disable highlight-code (drops syntect), fix
test deps leaking into normal dependencies.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Store::cached() returns a process-global Arc<tokio::sync::Mutex<Store>>
that loads once and reloads only when log files change (is_stale()
checks file sizes). All memory and journal tools use cached_store()
instead of Store::load() per invocation.
Fixes CPU saturation from HashMap hashing when multiple subconscious
agents make concurrent tool calls.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Spectral decomposition (eigenvalue computation) removed — it was
only used by the spectral-save CLI command. The spectral embedding
reader and query engine features remain (they load pre-computed
embeddings from disk, no faer needed).
Removes: faer, nano-gemm, private-gemm, and ~220 other transitive
dependencies. Significant build time and artifact size reduction.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Memory scoring now uses the graph as source of truth:
- last_scored timestamp on each node (new capnp field @22)
- Nodes scored when older than scoring_interval_secs (default 1hr)
- Oldest-scored-first ordering
- Window: scoring_response_window assistant responses (default 100)
- First-quarter memories scored even without full window
- Per-response normalization (raw divergence / response count)
- Asymmetric weight update: alpha=0.5 up, alpha=0.1 down
(responds fast to importance, decays slowly — memories stay
surfaced even if only useful 1/4 of the time)
Graph writes disabled pending normalization calibration.
Also: configurable scoring_interval_secs and scoring_response_window.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Delete subconscious/transcript.rs (94 lines), is_segment_mined,
mark_segment_mined — all orphaned by the extraction pipeline removal.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
pub → pub(crate) for SseReader methods (used across child modules).
pub → pub(super) for openai::stream_events, tool definitions, store
helpers. pub → private for normalize_link and differentiate_hub_with_graph
(only used within their own files).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Add `poc-memory graph spread` command that takes multiple seed node keys,
runs spreading activation through the graph, and returns nodes ranked by
total activation — nodes that bridge multiple seed concepts score highest.
Expose spreading_activation() as pub from the query engine. Add
memory_spread and memory_search_content tool definitions for MCP.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
migrate.rs was a one-time markdown→capnp conversion that's long done.
Remove it and update the identity.rs comment to reference the new
~/.consciousness/ path.
memory_search.rs is agent orchestration (surface-observe, journal,
reflect cycles), not memory storage. Rename to hook.rs and move to
subconscious/ where it belongs.
Backward compat: pub use subconscious::hook as memory_search in lib.rs
so existing crate::memory_search paths still resolve.
Add journal_cycle() to memory_search.rs, triggered every 20KB of
transcript growth. Runs independently of the surface-observe pipeline
so it doesn't depend on the 5-step pipeline surviving bail checks.
Journal agent doesn't inject output into conversation context (unlike
surface and reflect) — it just writes episodic memory entries.
Provenance now flows as a function parameter through the entire tool
dispatch chain: thought::dispatch → memory::dispatch → store methods.
Removed task_local (TASK_AGENT), thread_local (TASK_PHASE), and env
var (POC_PROVENANCE) from the tool dispatch path. The env var remains
only as a fallback for non-tool paths (CLI commands, digest).
Phase names are passed from knowledge.rs → llm.rs → api.rs, and
api.rs updates the provenance string between steps. No globals needed.
Split TASK_PROVENANCE into TASK_AGENT (task_local, set once per agent
run) and TASK_PHASE (thread_local, updated between steps). Provenance
now reports "agent:surface-observe:observe" instead of just
"agent:surface-observe", making it possible to identify which pipeline
phase created a node.
Priority: task_local agent + thread_local phase > POC_PROVENANCE env
var > "manual".
Also includes memory_search catchup throttle and pipelining fixes
from the surface-observe refactor.
TranscriptInfo provides cached transcript metadata (path, size)
with a single read. Replaces scattered fs::metadata calls in
surface_observe_cycle, reflection_cycle, resolve_conversation,
and resolve_memory_ratio.
Session::transcript() resolves the path from transcript_path or
by searching projects dir, returning a TranscriptInfo.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Catchup throttle: when the agent is >50% behind the conversation
window (>25KB of transcript growth since last spawn), block and
wait up to 30s for the current agent to finish. Prevents the agent
from falling behind during heavy reading/studying.
Reflection agent: runs every 100KB of transcript growth. Reads
walked nodes from surface-observe, follows links in unexpected
directions, outputs a short dreamy insight. Previous reflections
are injected into the conversation context.
Updated reflect.agent prompt to use {{input:walked}} from
surface-observe state dir and {{conversation:20000}} for lighter
context.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Links to nodes created after the conversation window start are
tagged with (new) in memory_render output. The surface prompt
tells the agent not to surface these — they're its own recent
output, not prior memories. Observe can still see and update them.
POC_MEMORIES_OLDER_THAN env var set from the oldest message
timestamp in the conversation window.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
- journal_new: key is slugified title (agent names things properly)
- journal_tail: sort by created_at (immutable), not timestamp (mutable)
- journal_update: find latest by created_at
- {{latest_journal}}: query by NodeType::EpisodicSession, not "journal" key
- poc-memory journal write: requires a name argument
- Removed all journal#j-{timestamp}-{slug} patterns from:
- prompts.rs (rename candidates)
- graph.rs (date extraction, organize skip list)
- cursor.rs (date extraction)
- store/mod.rs (doc comment)
- graph.rs organize: filter by NodeType::Semantic instead of key prefix
- cursor.rs: use created_at for date extraction instead of key parsing
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
- journal_new: create EpisodicSession node with auto-generated key
- journal_tail: query by node_type, not by parsing a monolithic node
- journal_update: find latest EpisodicSession by timestamp
- No string key matching anywhere — all typed
- Fixes journal entries not appearing in 'poc-memory journal tail'
- Also: added --provenance/-p filter to 'poc-memory tail'
- Also: fix early return in surface_observe_cycle store load failure
- Also: scale max_turns by number of steps (50 per step)
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
