Side-by-side model comparison against the current conversation context.
Built on the MindTriggered pattern — F7 drops in as one more
CompareScoring flow next to MemoryScoring / FinetuneScoring.
Motivation: we have the VRAM on the b200 to load two versions of the
same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather
than trust perplexity/KLD numbers on a generic corpus, we can measure
divergence on our actual conversations: for each assistant response,
ask the test model what it would have said given the same prefix, and
eyeball the diffs.
- config.compare.test_backend — names an entry in the existing
backends map to use as the test model. Empty = F7 reports "(unset)"
and does nothing.
- subconscious::compare::{score_compare_candidates, CompareCandidate,
CompareScoringStats, CompareScoring}. For each assistant response,
gen_continuation runs with the test client against the same prefix
the original response saw; pairs stream into
shared.compare_candidates as they complete.
- user::compare::CompareScreen — F7 in the screen list. c/Enter
triggers a run; list/detail layout mirroring F6, detail shows
prior context / original / test-model alternate.
No persistence yet — each F7 run regenerates. Caching via a context
manifest (so we can re-view without re-burning generation) is the
natural follow-up; for now light usage is fine.
Also reusable later for validating finetune checkpoints: same pattern,
swap the test backend for the new checkpoint, watch where it diverges
from the base.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Commit 2989a6afaa ("config: drop dead code") removed
surface_hooks as having "zero external readers" but missed
consciousness-claude/src/hook.rs as a consumer. That crate stopped
building, so poc-hook never ran and no agent cycles (surface-observe,
reflect, journal) fired.
Restore the field with a default of the three hook events we install
(UserPromptSubmit, PostToolUse, Stop), so a fresh install works
without needing to hand-edit config.json5.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
These are identity settings, not memory-graph settings. Sat inside the
\`memory\` section only because that's where Config started life. Move
to AppConfig alongside the other top-level stuff.
Readers now pull from \`config::app()\` instead of \`config::get()\`.
subconscious/defs.rs's conversation-building pass still needs Config
for surface_conversation_bytes, so both guards coexist there —
AppConfig's guard is dropped before the per-step await loop so we
don't stall the config-watcher's writer.
show_config picks up the two new fields at the top of its output.
Kent's config already has them hoisted to the top level.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Both config halves (Config for the memory section, AppConfig globally)
are now reloaded whenever ~/.consciousness/config.json5 changes on
disk. So edits from vim, manual tweaks, or F6's own config_writer
calls all land without a restart. No more "reload the daemon to pick
up a config change."
Wires up the previously-unused Config::reload() (Kent flagged it as
"not dead, just not wired"). Pairs it with an AppConfig reload via
install_app(). Both run on the same file-change event.
Implementation:
- notify-debouncer-mini watches the config file's parent directory
(editors usually replace-via-rename, so watching the file itself
misses the new inode). Debounced at 200ms to coalesce the flurry
of events editors produce around a single save.
- Filter for events whose path is the actual config file.
- On match: call reload() for Config, run build_figment + extract for
AppConfig. If AppConfig parsing fails (editor mid-save with partial
content), log and keep the old cached value.
- Watcher runs in its own named thread, fire-and-forget. If startup
fails we just log and move on — worst case is no live reload, not
a crash.
CliArgs + SubCmd both get Clone derives so the watcher can own a
snapshot of the startup args for future reloads. Watcher is kicked
off in user/mod.rs:start() right after load_session.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The graph-health logic in consolidation_plan_inner computed
reasonable agent counts based on graph metrics (α, Gini, hub
dominance), then immediately overwrote them with an Elo-weighted
flat-budget distribution, or — if no agent-elo.json existed —
with a simple budget/N per type.
Nothing in the codebase writes agent-elo.json; it's external state
that never gets maintained. So the effective behavior was always the
"No Elo ratings — equal distribution" branch, which just bucketed
agent_budget evenly across active agent types and discarded
everything the graph analysis had just decided.
Keep the graph-health allocation (α → linker count, Gini → distill
bump, organize/distill/split proportional). Drop:
- The entire Elo / agent_budget block at the end of
consolidation_plan_inner
- Config.agent_budget field and its default (1000)
- agent_budget: 40 from Kent's config.json5
- The local agent_types binding inside the function — it was only
used by the now-deleted block. Config.agent_types stays; it has
other consumers.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Two parallel backend-resolution paths had drifted apart:
- Main chat: AppConfig::resolve_model() → a named BackendConfig in
AppConfig.backends
- Subconscious / oneshot / context_window(): four skip-serde
"cache" fields on Config (memory section) — api_base_url, api_key,
api_model, api_context_window — that used to be populated at
Config::try_load_shared time by walking memory.agent_model →
root.models[name] → root[backend_name]
When we renamed `models` to `backends` and collapsed ModelConfig into
BackendConfig, the latter chain started silently dereferencing
`root.get("models")` → None → no population. Subconscious agents fell
through the "API not configured" guard; context_window() started
returning 0 (since api_context_window default is u64's 0 now that we
don't populate it). It was only visibly working for the main chat.
Collapse to one path:
- Drop Config.agent_model (duplicate of AppConfig.default_backend)
- Drop Config.{api_base_url, api_key, api_model, api_context_window}
— no longer populated, no longer needed
- Drop default_context_window() — nobody reads the field anymore
- Drop the memory-side resolution block in try_load_shared()
- Subconscious (mind/unconscious.rs) and oneshot (agent/oneshot.rs)
now call load_app() + resolve_model(&app.default_backend) just like
the main chat does
- context_window() reads from config::app().backends[default_backend]
.context_window, defaulting to 128k only if the backend doesn't
specify one
Side effect: Kent's config file drops agent_model, api_reasoning,
journal_days, journal_max — all fields whose Rust counterparts are
now gone. (Figment tolerates unknown fields, so leaving them wouldn't
have broken anything, but they were lying about what's configurable.)
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Four Config fields had no external readers, left over from earlier
features that got refactored away:
- journal_days, journal_max — journal rotation knobs that nothing
actually consults
- prompts_dir — the old per-prompt-file directory, obsolete since
prompt_file metadata itself went away in a prior cleanup
- api_reasoning — a reasoning-mode string that used to flow into the
API request, superseded by per-agent reasoning_effort on AgentState
All four were only ever assigned to and never read. Drop them from the
struct, Default impl, and (as appropriate) deserialization defaults.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
AppConfig had one BackendConfig for credentials and a separate
HashMap<String, ModelConfig> for named model entries. In practice each
named model was always paired with exactly one backend's credentials
— the split bought nothing except an extra struct and the awkward
two-lookup shape in resolve_model (find model → get backend creds →
combine).
Merge them: BackendConfig now carries api_key, base_url, model_id,
and context_window. AppConfig has a single
HashMap<String, BackendConfig> backends map and a default_backend
name. resolve_model is one lookup.
ModelConfig struct deleted. default_model renamed to default_backend.
Config shape changes from
backend: { api_key, base_url }
models: { "27b": { model_id, context_window } }
default_model: "27b"
to
backends: { "27b": { api_key, base_url, model_id, context_window } }
default_backend: "27b"
Updated ~/.consciousness/config.json5 to match.
One small side effect: dropped the --api-key / --api-base figment
merge-opts for "backend.*" targets — those would need to know which
backend to target now and there's no sensible default. The CLI flags
still function as post-resolution overrides on the eventual
SessionConfig.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Config had accumulated several obsolete fields, a legacy load path
that was just returning defaults, and multi-backend infrastructure
that's no longer used.
Removed from Config (memory section):
- load_legacy_jsonl() — just returned Config::default(), no callers
- The legacy-fallback branch in load_from_file
- surface_hooks, surface_timeout_secs — zero external readers
- scoring_chunk_tokens + default fn — zero external readers
- The POC_MEMORY_CONFIG env override note in the header comment
(not actually wired up anywhere)
Collapsed multi-backend to single-backend:
- AppConfig used to carry `anthropic: BackendConfig` and
`openrouter: BackendConfig` as required fields plus an optional
`deepinfra`, picked between at runtime by name. Only one is ever
actually used in any deployment. Collapse to a single
`backend: BackendConfig` on AppConfig, drop the multi-backend
match logic in resolve_model, drop the top-level `backend: String`
selector field, drop the `BackendConfig::resolve` fallback path.
- Also drop BackendConfig.model (redundant with ModelConfig.model_id
once multi-backend is gone).
- ModelConfig.backend field goes — there's only one backend now, no
choice to make.
Dead prompt_file machinery:
- ModelConfig.prompt_file, ResolvedModel.prompt_file, SessionConfig
.prompt_file, Agent.prompt_file — nothing in the codebase actually
reads the file these strings name. Just passed around and compared.
Delete the whole string through every struct.
- The "if prompt_file changed on model switch, recompact" branch in
user/chat.rs goes too (never fired usefully).
Dead memory_project plumbing:
- AppConfig.memory_project field, CliArgs.memory_project, the
--memory-project CLI flag, the figment merge target, the show_config
display line. Nothing reads it anywhere.
Dead ContextInfo struct:
- `struct ContextInfo` was never constructed — context_info: None
was the only initializer. The conditional display blocks in
user/context.rs that dereferenced it were dead.
Behavior change: AppConfig::resolve() now requires a non-empty
`models` map and bails with a helpful message if it's missing. The
old fallback ("no models? use top-level backend + PromptConfig to
build a default") path is gone — it was only kept for symmetry with
a mode nobody used.
Config file shape: `deepinfra: {...}` → `backend: {...}`, and
model entries no longer need `backend:` or `prompt_file:`. Updated
~/.consciousness/config.json5 to match.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Previously when append_kvp created a new section or added a key, it
stuffed the "\n " separator into the new kvp's wsc.0 (the whitespace
between its own key and colon) instead of the prior kvp's wsc.3 (the
whitespace after the prior trailing comma). Result looked like:
lsp_servers: [...],
learn
: {generate_alternates
: true,},}
The writer also didn't set any interior whitespace on the new section's
JSONObjectContext, so everything crammed onto one line — `{key: val,}`
compact, not `{\n key: val,\n}` multi-line.
Rewrote the appender as append_kvp_pretty(object, key, value,
inner_indent, outer_indent):
- separator between kvps goes in the prior kvp's wsc.3, or if we're the
first kvp in a fresh object, in the object's own wsc.0 (after its
opening `{`)
- new kvp's wsc.3 carries `,\n<outer_indent>` so the parent's closing
`}` lands correctly indented
- interior indent vs outer indent are both explicit, so we don't have
to rewrite this logic every time we add another nesting level
New tests: new_section_exact_multiline_layout asserts byte-exact
output shape; new_section_and_key_format_cleanly verifies no key wraps
to the next line. Prior tests just substring-matched and happily passed
on the broken output — that's why this shipped in the first place.
Also: dropped the json5 crate dependency. json-five's serde feature
(default) provides the same from_str / to_string API. One fewer
dependency, and the two were doing the same job.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Runtime-mutable settings (F6's threshold knob, the generate-alternates
toggle, anything else that comes along) were ending up as mirrored
fields on MindState — each new config setting grew MindState::new's
signature and added a clone+sync path. Wrong home. MindState is
ephemeral session state, not a config projection.
Give AppConfig the same treatment the memory Config has: install it
into a global RwLock<AppConfig> at startup via load_app, read through
config::app() (returns a read guard), mutate through update_app. The
config_writer functions now write to disk AND update the cache
atomically, so the one-stop-shop call keeps both in sync.
Also while in here:
- learn.generate_alternates moves from a sentinel file
(~/.consciousness/cache/finetune-alternates, "exists = enabled")
into the config under the learn section. On first run with this
build, if the sentinel file still exists Mind::new flips the
config value to true and removes it. Drops
alternates_enabled()/set_alternates().
- Default threshold 0.0000001 → 1.0. With the timestamp filter
removed the previous value was letting essentially everything
through; 1.0 is a sane "nothing gets through unless you actually
want it" default.
- score_finetune_candidates takes generate_alternates as a parameter
instead of reading a global — caller snapshots the config values
once at the top of start_finetune_scoring so the async task
doesn't need to hold the config read lock across awaits.
- MindState.learn_threshold / learn_generate_alternates gone; the
SetLearn* command handlers now just delegate to config_writer.
Kent noted RwLock<Arc<AppConfig>> (the pattern used by the memory
Config global) is pointless here — nobody needs a snapshot-after-
release, reads are short — so this uses a plain RwLock<AppConfig>
and returns a read guard.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Three changes that together reshape the F6 fine-tune-review screen:
1. Finetune scoring reports through the standard agent activity system
instead of a separate finetune_progress String. The previous design
ran an independent progress field that forced a cross-lock dance and
bespoke UI plumbing. start_finetune_scoring now uses start_activity
+ activity.update, so the usual status line and notifications
capture scoring progress uniformly with other background work.
2. MindState gains a FinetuneScoringStats snapshot (responses seen,
above threshold, max divergence, error). The F6 empty screen shows
this instead of a loading message — so after a scoring run that
produced zero candidates, you can see *why* (e.g., max_divergence
below threshold).
3. The divergence threshold is configurable from F6 via +/- hotkeys
(scales by 10×) and persisted to ~/.consciousness/config.json5 via
config_writer::set_learn_threshold. AppConfig grows a learn section
with a threshold field (default 1e-7).
Also: user/mod.rs no longer uses try_lock() for the per-tick
unconscious/mind state sync — we fixed the locking hot paths that
made try_lock necessary, so lock().await is now the right choice.
And subconscious::learn::score_finetune_candidates now returns
(candidates, max_divergence) so the stats can be populated.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Deleted the directory-walking CLAUDE.md/POC.md loader. Identity now
comes entirely from personality_nodes in the memory graph.
Simplified:
- assemble_context_message() takes just personality_nodes
- Removed config_file_count/memory_file_count tracking
- reload_for_model() → reload_context() (no longer model-specific)
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Replace complex context_groups (with ContextGroup struct, ContextSource
enum, labels, keys arrays) with simple string lists:
- personality_nodes: loaded into main session context
- agent_nodes: loaded into subconscious agent context
Removed ~200 lines of code. The distinction between session and agent
context is now just which list you're in, not a per-group flag.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Identity files migrated to memory nodes:
- identity, core-personality, reflections, where-am-i
Removed:
- ContextSource::File enum variant
- File source parsing and handling
- load_memory_file helper function
Config now only supports Store and Journal sources.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The system prompt duplicated what's already in core-personality and
other memory nodes. Moving everything to memory means it's all
trainable data rather than hardcoded strings.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Config struct deserializes from the "memory" subsection of config.json5,
but lsp_servers and mcp_servers are top-level keys. Now explicitly
extracted from the root after initial deserialization.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
New lsp.rs: LspRegistry manages persistent LSP server connections.
Spawns child processes, speaks LSP protocol (Content-Length framed
JSON-RPC over stdio). Server indexes the project once; queries are
cheap.
Tools: lsp_definition, lsp_references, lsp_hover, lsp_symbols,
lsp_callers. Each takes file/line/character, queries the running
language server.
LspRegistry lives on Agent as Option<Arc>, shared across forks.
Still needs: config-driven server startup (like MCP).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
New mcp_client.rs: McpRegistry manages MCP server connections.
Spawns child processes, speaks JSON-RPC 2.0 over stdio. Discovers
tools via tools/list, dispatches calls via tools/call.
dispatch_with_agent falls through to MCP after checking internal
tools. McpRegistry lives on Agent (shared across forks).
Still needs: config-driven server startup, system prompt integration.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Memory scoring now uses the graph as source of truth:
- last_scored timestamp on each node (new capnp field @22)
- Nodes scored when older than scoring_interval_secs (default 1hr)
- Oldest-scored-first ordering
- Window: scoring_response_window assistant responses (default 100)
- First-quarter memories scored even without full window
- Per-response normalization (raw divergence / response count)
- Asymmetric weight update: alpha=0.5 up, alpha=0.1 down
(responds fast to importance, decays slowly — memories stay
surfaced even if only useful 1/4 of the time)
Graph writes disabled pending normalization calibration.
Also: configurable scoring_interval_secs and scoring_response_window.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Disables memory scoring, surface, and observe agents when set.
Useful for testing with external backends (e.g. OpenRouter) where
background agent traffic would be slow and unnecessary.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
journal_tools() was only in memory_and_journal_tools() for
subconscious agents — not in the main tools() registry. Added
so consciousness and MCP server can use journal_new/tail/update.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Mechanical rename: src/agent/ -> src/user/, all crate::agent:: ->
crate::user:: references updated. Binary poc-agent renamed to
consciousness with CLI name and user-facing strings updated.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Split conversation into ~50K token chunks (configurable via
scoring_chunk_tokens in config) for prompt_logprobs calls.
Each chunk ends at an assistant message boundary. Avoids the
~40GB logprobs tensor allocation that OOM'd on full contexts.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
prompts_dir now defaults to ~/.consciousness/prompts instead of
hardcoded repo path. Function pointer cast goes through *const ()
to silence the compiler warning.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Four expect("HOME not set") calls in config.rs and one unwrap()
in admin.rs would panic if HOME wasn't set. Use dirs::home_dir()
consistently for portability.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Stream chunk timeout is now api_stream_timeout_secs in config
(default 60s). Status bar shows total turn time and per-call
time with timeout: "thinking... 45s, 12/60s".
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
model_context_window() now reads from config.api_context_window
instead of guessing from model name strings. is_anthropic_model()
replaced with backend == "anthropic" checks. Dead model field
removed from AgentDef/AgentHeader.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Separate identity files (loaded via source: "file" in context_groups)
from the memory store (data_dir). New identity_dir config field,
defaults to ~/.consciousness/identity/.
Also restrict subconscious agents to memory-only tools — no
filesystem write access. This prevents agents from creating stray
.md files in the memory directory.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- thalamus/src/idle.rs: dream-start.sh path
- src/agent/dmn.rs: telegram/send.sh path
Part of the directory migration to make this an independent project.
- Telegram data: ~/.consciousness/telegram/
- Rate limiter file: ~/.consciousness/cache/
- parse-claude-conversation stash: ~/.consciousness/sessions/
No more /tmp/ for persistent state, no more ~/.claude/ for our data.
The consciousness project should stand independently of Claude Code.
All data, logs, sessions, and agent state now live under
~/.consciousness/ instead of being scattered across ~/.claude/memory/,
/tmp/claude-memory-search/, ~/.config/poc-memory/, and ~/.cache/.
Layout:
~/.consciousness/
*.capnp, *.bin, *.rkyv — store files
sessions/ — per-session state (seen sets, cookies)
logs/ — all logs (hook, agent, debug, dream)
agents/ — agent runtime state (pid files, output)
notifications/ — notification state
cache/ — transient data
Things that stay in ~/.claude/:
- projects/ (Claude Code transcripts)
- hooks/ (Claude Code hook system)
- telegram/ (shared integration)
- irc/ (shared integration)
- settings.json (Claude Code settings)
Debug log moves from /tmp/ to ~/.consciousness/logs/debug.log.
Session state moves from /tmp/claude-memory-search/ to sessions/.
Notifications move from ~/.claude/notifications/ to notifications/.
Both hippocampus/config.rs and agent/config.rs read from the same
config file (~/.config/poc-agent/config.json5). Having two separate
implementations was a footgun — load_context_groups() was duplicated
three times across the codebase.
Merged into src/config.rs:
- Config (memory settings, global get()/reload())
- AppConfig (agent backend/model settings, figment-based loading)
- SessionConfig (resolved agent session, renamed from agent's Config)
- Single ContextGroup/ContextSource definition used everywhere
Eliminated: duplicate load_context_groups(), duplicate ContextGroup
definition in identity.rs, duplicate config file path constants.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
hippocampus/ — memory storage, retrieval, and consolidation:
store, graph, query, similarity, spectral, neuro, counters,
config, transcript, memory_search, lookups, cursor, migrate
subconscious/ — autonomous agents that process without being asked:
reflect, surface, consolidate, digest, audit, etc.
All existing crate::X paths preserved via re-exports in lib.rs.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
No more subcrate nesting — src/, agents/, schema/, defaults/, build.rs
all live at the workspace root. poc-daemon remains as the only workspace
member. Crate name (poc-memory) and all imports unchanged.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
poc-daemon (notification routing, idle timer, IRC, Telegram) was already
fully self-contained with no imports from the poc-memory library. Now it's
a proper separate crate with its own Cargo.toml and capnp schema.
poc-memory retains the store, graph, search, neuro, knowledge, and the
jobkit-based memory maintenance daemon (daemon.rs).
Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
Keys were a vestige of the file-based era. resolve_key() added .md
to lookups while upsert() used bare keys, creating phantom duplicate
nodes (the instructions bug: writes went to "instructions", reads
found "instructions.md").
- Remove .md normalization from resolve_key, strip instead
- Update all hardcoded key patterns (journal.md# → journal#, etc)
- Add strip_md_keys() migration to fsck: renames nodes and relations
- Add broken link detection to health report
- Delete redirect table (no longer needed)
- Update config defaults and config.jsonl
Migration: run `poc-memory fsck` to rename existing keys.
Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
Replace agent_api_key (which didn't work — claude CLI uses OAuth, not
API keys) with agent_config_dir. When configured, sets CLAUDE_CONFIG_DIR
on claude subprocesses so daemon agent work authenticates with separate
OAuth credentials from the interactive session.
Fix daemon not shutting down on SIGTERM: use process::exit(0) after
cleanup so PR_SET_PDEATHSIG kills child claude processes immediately.
Previously the daemon hung waiting for choir threads/subprocesses to
finish. Restart now takes ~20ms instead of timing out.
Also: main.rs now uses `use poc_memory::*` since lib.rs exists.
Add agent_api_key config option. When set, all LLM calls (experience-mine,
fact-mine, consolidation, knowledge-loop, digest) use this key via
ANTHROPIC_API_KEY env var on the claude subprocess, keeping daemon token
usage on a separate quota from interactive sessions.
Config: {"config": {"agent_api_key": "sk-ant-..."}}
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
