Commit graph

22 commits

Author SHA1 Message Date
Kent Overstreet
7d49f29fde store: remove dead code and move params to config
Remove:
- score_weight() - never called
- position field on Node - never read (was for export)
- Provenance enum - inline helper for capnp migration
- migrate_transcript_progress + CLI command
- init_from_markdown, import_file, ingest_units
- export command and export_to_markdown
- RetrievalEvent, GapRecord types
- classify_filename, new_transcript_segment

Move spreading activation params to Config:
- default_node_weight, edge_decay, max_hops, min_activation
- Remove Params struct and StoreView::params()

Simplify cmd_init to just seed identity via upsert().
Simplify cmd_import to use parse_units + upsert directly.

-576 lines

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:50:21 -04:00
Kent Overstreet
6104c63890 Integrate redb into Store::load() with health check
- Add db: Option<Database> field to Store
- Store::load() opens redb after replaying capnp logs
- Health check compares node count + spot checks keys
- Rebuilds automatically if db is missing, corrupt, or stale
- Make table definitions public for cross-module access

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:33:47 -04:00
Kent Overstreet
2caccf875d Replace rkyv/bincode caching with redb indices
Remove three-tier loading (rkyv snapshot, bincode cache, capnp replay)
in favor of direct capnp log replay + redb for indexed access.

- Remove all rkyv derives from types (Node, Relation, enums, etc.)
- Remove Snapshot struct, RKYV_MAGIC, CACHE_MAGIC constants
- Remove load_snapshot_mmap(), save(), save_snapshot()
- Remove MmapView, AnyView from view.rs (keep StoreView trait)
- Simplify Store::load() to just replay capnp logs
- Add db.rs with redb schema: nodes, uuid_to_key, visits, transcript_progress
- Simplify cmd_fsck to just check capnp integrity + graph health

capnp logs remain source of truth; redb indices will be rebuilt on demand.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:30:58 -04:00
Kent Overstreet
1d88293ccf Remove Store::cached(), consolidate on access_local()
- Remove CACHED_STORE, cached(), is_stale(), set_store() - redundant
- Convert all Store::cached() callers to use access_local()
- Single Store::load() call remains in access() fallback path

All store access now goes through hippocampus::access() / access_local(),
which handles socket connection or local fallback with caching.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:11:58 -04:00
Kent Overstreet
09b30d64f2 cmd_fsck: use access_local(), remove dead AnyView::load()
Convert cmd_fsck to async and use access_local() for the cached store.
Still uses Store::load_from_logs() for fresh comparison.

Remove unused AnyView::load() method - was never called.

Remaining Store::load() calls are all internal caching infrastructure:
- persist.rs cached() for CACHED_STORE
- mod.rs access() fallback for STORE_ACCESS

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:09:02 -04:00
Kent Overstreet
b8db8754be Convert store and CLI to anyhow::Result for cleaner error handling
Replace Result<_, String> with anyhow::Result throughout:
- hippocampus/store module (persist, ops, types, view, mod)
- CLI modules (admin, agent, graph, journal, node)
- Run trait in main.rs

Use .context() and .with_context() instead of .map_err(|e| format!(...))
patterns. Add bail!() for early error returns.

Add access_local() helper in hippocampus/mod.rs that returns
Result<Arc<Mutex<Store>>> for direct local store access.

Fix store access patterns to properly lock Arc<Mutex<Store>> before
accessing fields in mind/unconscious.rs, mind/mod.rs, subconscious/learn.rs,
and hippocampus/memory.rs.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-13 18:05:04 -04:00
Kent Overstreet
7842b6fc8b remove legacy feedback commands (used, wrong, gap, etc.)
These were early experiments with manual feedback signals that
never worked well. The scoring system will handle this properly.

Removed:
- CLI: used, wrong, not-relevant, not-useful, gap
- MCP: memory_used
- Store: mark_used, mark_wrong, record_gap, modify_node

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 22:12:02 -04:00
Kent Overstreet
f56fc3a7c7 locks: add process-wide lock hold time tracking
TrackedMutex and TrackedRwLock wrappers that record hold durations
by source location using #[track_caller]. Stats written to
~/.consciousness/lock-stats.json every second, sorted by max hold time.

Re-exported as crate::Mutex so all locks are instrumented. To disable,
swap the re-export back to tokio::sync::Mutex.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 20:27:42 -04:00
ProofOfConcept
3e0d52c451 Redirect noisy warnings to debug log to stop TUI corruption
Duplicate key warnings fire on every store load and were writing to
stderr, corrupting the TUI display. Log write warnings and MCP
server failures are similarly routine. Route these to dbglog.

Serious errors (rkyv snapshot failures, store corruption) remain on
stderr — those are real problems the user needs to see.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 22:46:48 -04:00
Kent Overstreet
edfa1c37f5 Subconscious: persistent agent state, store activity queries
- Agent state (outputs) persists across runs in subconscious-state.json,
  loaded on startup, saved after each run completes
- Merge semantics: each run's outputs accumulate into persistent_state
  rather than replacing
- Walked keys restored from surface agent state on load
- Store::recent_by_provenance() queries nodes by agent provenance for
  the store activity view
- Switch outputs from HashMap to BTreeMap for stable display ordering

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-07 19:03:05 -04:00
Kent Overstreet
a8c239f3de Cache Store in process — stop reloading on every tool call
Store::cached() returns a process-global Arc<tokio::sync::Mutex<Store>>
that loads once and reloads only when log files change (is_stale()
checks file sizes). All memory and journal tools use cached_store()
instead of Store::load() per invocation.

Fixes CPU saturation from HashMap hashing when multiple subconscious
agents make concurrent tool calls.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-07 03:35:08 -04:00
ProofOfConcept
fcd77fb79e training: per-node scoring with graph weight updates
Memory scoring now uses the graph as source of truth:
- last_scored timestamp on each node (new capnp field @22)
- Nodes scored when older than scoring_interval_secs (default 1hr)
- Oldest-scored-first ordering
- Window: scoring_response_window assistant responses (default 100)
- First-quarter memories scored even without full window
- Per-response normalization (raw divergence / response count)
- Asymmetric weight update: alpha=0.5 up, alpha=0.1 down
  (responds fast to importance, decays slowly — memories stay
  surfaced even if only useful 1/4 of the time)

Graph writes disabled pending normalization calibration.

Also: configurable scoring_interval_secs and scoring_response_window.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-05 01:18:47 -04:00
Kent Overstreet
59aaaa5742 drop strip_md_keys() 2026-04-05 01:18:47 -04:00
Kent Overstreet
dae0cc8191 remove more dead transcript mining code
Delete subconscious/transcript.rs (94 lines), is_segment_mined,
mark_segment_mined — all orphaned by the extraction pipeline removal.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 21:45:56 -04:00
Kent Overstreet
91eb9c95cc delete 20 dead public functions across 12 files
Removed functions with zero callers: parse_timestamp_to_epoch,
hash_key, search_weighted_debug, extract_query_terms, format_results,
move_to_neighbor, adjust_edge_strength, update_graph_metrics,
nearest_to_seeds, nystrom_project, chat_completion_stream, cmd_read,
context_message, split_candidates, split_plan_prompt,
split_extract_prompt, log_event_pub, log_verbose, rpc_record_hits,
memory_definitions. -245 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 16:21:01 -04:00
Kent Overstreet
b0e852a05f add unreachable_pub lint, fix all 17 violations
pub → pub(crate) for SseReader methods (used across child modules).
pub → pub(super) for openai::stream_events, tool definitions, store
helpers. pub → private for normalize_link and differentiate_hub_with_graph
(only used within their own files).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 16:15:32 -04:00
ProofOfConcept
92ca2bf2c8 provenance: pass directly through thought::dispatch, remove globals
Provenance now flows as a function parameter through the entire tool
dispatch chain: thought::dispatch → memory::dispatch → store methods.

Removed task_local (TASK_AGENT), thread_local (TASK_PHASE), and env
var (POC_PROVENANCE) from the tool dispatch path. The env var remains
only as a fallback for non-tool paths (CLI commands, digest).

Phase names are passed from knowledge.rs → llm.rs → api.rs, and
api.rs updates the provenance string between steps. No globals needed.
2026-03-27 15:44:39 -04:00
ProofOfConcept
85302c11d4 provenance: track agent phase, use task_local + thread_local
Split TASK_PROVENANCE into TASK_AGENT (task_local, set once per agent
run) and TASK_PHASE (thread_local, updated between steps). Provenance
now reports "agent:surface-observe:observe" instead of just
"agent:surface-observe", making it possible to identify which pipeline
phase created a node.

Priority: task_local agent + thread_local phase > POC_PROVENANCE env
var > "manual".

Also includes memory_search catchup throttle and pipelining fixes
from the surface-observe refactor.
2026-03-27 15:11:17 -04:00
ProofOfConcept
eac59b423e journal: remove all stringly-typed key patterns, use NodeType
- journal_new: key is slugified title (agent names things properly)
- journal_tail: sort by created_at (immutable), not timestamp (mutable)
- journal_update: find latest by created_at
- {{latest_journal}}: query by NodeType::EpisodicSession, not "journal" key
- poc-memory journal write: requires a name argument
- Removed all journal#j-{timestamp}-{slug} patterns from:
  - prompts.rs (rename candidates)
  - graph.rs (date extraction, organize skip list)
  - cursor.rs (date extraction)
  - store/mod.rs (doc comment)
- graph.rs organize: filter by NodeType::Semantic instead of key prefix
- cursor.rs: use created_at for date extraction instead of key parsing

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-26 19:11:17 -04:00
ProofOfConcept
1e1f17f775 store: link_set upserts instead of erroring on missing link
Creates the link if it doesn't exist, avoiding wasted agent turns
from the link_set/link_add confusion.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-26 14:22:12 -04:00
ProofOfConcept
10932cb67e hippocampus: move MemoryNode + store ops to where they belong
MemoryNode moved from agent/memory.rs to hippocampus/memory.rs — it's
a view over hippocampus data, not agent-specific.

Store operations (set_weight, set_link_strength, add_link) moved into
store/ops.rs. CLI code (cli/graph.rs, cli/node.rs) and agent tools
both call the same store methods now. render_node() delegates to
MemoryNode::from_store().render() — 3 lines instead of 40.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-25 01:55:21 -04:00
ProofOfConcept
d5c0e86700 restructure: hippocampus/ for memory, subconscious/ for agents
hippocampus/ — memory storage, retrieval, and consolidation:
  store, graph, query, similarity, spectral, neuro, counters,
  config, transcript, memory_search, lookups, cursor, migrate

subconscious/ — autonomous agents that process without being asked:
  reflect, surface, consolidate, digest, audit, etc.

All existing crate::X paths preserved via re-exports in lib.rs.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-25 01:05:30 -04:00