Uses JsonlBackwardIter (SIMD memrchr3) to scan the conversation log
newest-first without reading/parsing the whole file. Stops as soon
as the conversation budget is full. Only the kept nodes get
retokenized and pushed into context.
18MB log → only tokenize the ~50 nodes that fit in the budget.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
restore_from_log reads the full log but walks backwards from the tail,
retokenizing each node as it goes. Stops when conversation budget is
full. Only the nodes that fit get pushed into context.
Added AstNode::retokenize() — recomputes token_ids on all leaves
after deserialization (serde skip means they're empty).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
New lsp.rs: LspRegistry manages persistent LSP server connections.
Spawns child processes, speaks LSP protocol (Content-Length framed
JSON-RPC over stdio). Server indexes the project once; queries are
cheap.
Tools: lsp_definition, lsp_references, lsp_hover, lsp_symbols,
lsp_callers. Each takes file/line/character, queries the running
language server.
LspRegistry lives on Agent as Option<Arc>, shared across forks.
Still needs: config-driven server startup (like MCP).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
New mcp_client.rs: McpRegistry manages MCP server connections.
Spawns child processes, speaks JSON-RPC 2.0 over stdio. Discovers
tools via tools/list, dispatches calls via tools/call.
dispatch_with_agent falls through to MCP after checking internal
tools. McpRegistry lives on Agent (shared across forks).
Still needs: config-driven server startup, system prompt integration.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The unconscious trigger holds the tokio mutex during heavy sync work
(store load, graph build, agent creation), blocking the UI tick which
needs the same lock for snapshots. Fix: try_lock in the UI — skip
the update if the trigger is running.
Also: restore_from_log was re-logging every restored node back to the
log file via push()'s auto-log. Added push_no_log() for restore path.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Loading 23K nodes + building graph was blocking consciousness startup.
Now computed on first trigger cycle (runs async from mind loop).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
read_sections and draw_context now use selected_agent() which maps the
selected index to either a subconscious forked_agent or an unconscious
agent Arc. Context title uses selected_agent_name for both types.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- AutoAgent stored on UnconsciousAgent, swapped out for runs, restored
on completion (same pattern as subconscious agents)
- Agent Arc created before spawn and stored on UnconsciousAgent so
the TUI can lock it to read conversation context live
- run_shared() method on AutoAgent for running with a pre-created Agent
- Default tools: memory_tools (not memory_and_journal_tools)
- trigger/spawn_agent made async for Agent::new()
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- Scan agents directory for all .agent files instead of hardcoded list
- Persist enabled state to ~/.consciousness/agent-enabled.json
- Spacebar on F3 agent list toggles selected agent on/off
- Both subconscious and unconscious agents support toggle
- Disabled agents shown dimmed with "off" indicator
- New agents default to disabled (safe default)
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Graph health stats (alpha, gini, cc, episodic ratio, consolidation
plan) now computed directly by the unconscious module on startup and
every 10 minutes, instead of fetching from the poc-memory daemon.
F4 screen renamed to hippocampus, stripped down to just the health
gauges — daemon task list removed (agents now shown on F3).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- AutoAgent.enabled: universal toggle for any auto agent
- Subconscious: should_trigger checks auto.enabled
- Unconscious: simplified from consolidation-plan-driven budgets to
simple loop with cooldown. Static agent list, max 2 concurrent.
- TUI: unconscious agents shown in F3 subconscious screen under
separator, with enabled/running/runs display
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The log records what goes into context, so it belongs under the context
lock. push() now auto-logs conversation entries, eliminating all the
manual lock-state-for-log, drop, lock-context-for-push dances.
- ContextState: new conversation_log field, Clone impl drops it
(forked contexts don't log)
- push(): auto-logs Section::Conversation entries
- push_node, apply_tool_results, collect_results: all simplified
- collect_results: batch nodes under single context lock
- Assistant response logged under context lock after parse completes
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- push_node: notify before dropping state lock instead of relocking
- Mind::run: single lock for timeout + turn_active + has_input;
single lock for turn_handle + complete_turn
- Agent triggers (subconscious/unconscious) spawned as async tasks
so they don't block the select loop
- has_pending_input() peek for DMN sleep guard — don't sleep when
there's user input waiting
- unconscious: merge collect_results into trigger, single store load
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Unconscious agents (organize, linker, distill, etc.) run independently
of the conversation context. They create fresh Agent instances, select
target nodes via their .agent file queries, and are scheduled by the
consolidation plan which analyzes graph health metrics.
Key differences from subconscious agents:
- No fork — standalone agents with fresh context
- Self-selecting — queries in .agent files pick target nodes
- Budget-driven — consolidation plan allocates runs per type
- Max 2 concurrent, 60s min interval between same-type runs
Wired into Mind event loop alongside subconscious trigger/collect.
TUI display not yet implemented.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The file contains both the DMN state machine and the subconscious agent
orchestration. Renaming to match the conceptual grouping — next step is
adding mind/unconscious.rs for the standalone graph maintenance agents
(organize, linker, etc.) that don't need conversation context.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Observe was creating byte-identical nodes under slightly different names
(e.g. april-8-evening-folded-presence, -presence-2, -folded-state)
because it had no visibility into its own prior writes across runs.
Query recent writes by provenance in trigger(), pass through
run_forked_shared/resolve_prompt as {{recently_written}}, and include
the list in the observe phase prompts so the agent knows what it
already recorded.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Subconscious agents (observe, etc.) fork the conscious agent's context
to share the KV cache prefix. When a multi-step agent fills the context
window, compacting blows the KV cache and evicts the step prompts,
leaving the model with no idea what it was doing.
Fix: forked agents set no_compact=true. On overflow, turn() returns the
error immediately (no compact+retry), and run_with_backend catches it
and returns Ok — the output tool has already written results to
Subconscious.state, so collect_results still picks them up.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Creates an Agent from global config (API credentials, system prompt,
identity), overrides tools with the agent's tool set, and runs through
the standard Backend → run_with_backend → Agent::turn() path.
This enables poc-hook spawned agents (surface-observe, journal, etc.)
to work with the completions API instead of the deleted chat API.
Also added Default derive to CliArgs for config loading.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Ported the old trim_entries logic to the new AstNode types:
- Phase 1: Dedup Memory nodes by key (keep last), drop DMN entries
- Phase 2: While over budget, evict lowest-scored memory (if memories
> 50% of conv tokens) or oldest conversation entry
- Phase 3: Snap to User message boundary at start
Called from compact() which runs on startup and on /compact.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Memory nodes in the conversation section are now counted separately:
sys X% id Y% jnl Z% mem W% conv V% = NK/MK
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
token_ids are not serialized (serde skip), so deserialized nodes had
0 tokens. The custom Deserialize impl recomputes tokens from the body
text, restoring the invariant at the reconstruction boundary. No
separate recompute step needed.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
poc-daemon was using find_claude_pane() which scans ALL tmux panes
for a 'claude' process, potentially finding unrelated sessions.
Now only uses the pane ID set by poc-hook via the user/response
RPC calls. If no pane is set yet, injection is skipped.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
collect_results now checks existing Memory nodes in the conversation
before surfacing. Prevents the same memory from being pushed every
time the surface agent runs.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The output tool closure writes directly to Subconscious.state,
so auto.outputs is always empty. collect_results now reads surface,
reflection, and thalamus keys from self.state.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
ToolHandler is now Arc<dyn Fn(...)> supporting closures that capture
state. The output tool is created during init_output_tool() as a
closure capturing Arc<Mutex<Subconscious>>, writing directly to
Subconscious.state. No more POC_AGENT_OUTPUT_DIR filesystem hack.
- All tool handlers wrapped in Arc::new()
- Tool is Clone (not Copy) — .copied() → .cloned()
- Subconscious wrapped in Arc<Mutex<>> on Mind
- Dead filesystem-based output() function removed
- memory_tools returns 11 items (output removed from static list)
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
- ToolHandler changed to Arc<dyn Fn(...)> (supports closures)
- Subconscious wrapped in Arc<Mutex<>> on Mind
- init_output_tool() pushes output tool closure capturing the Arc
- Output removed from static memory_tools()
- Most tool handlers wrapped in Arc::new() but some have paren issues
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
These were wrong approaches — replacing with proper closure-based
output tool that writes directly to shared Subconscious state.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Forked agents don't have POC_AGENT_OUTPUT_DIR set. The output tool
now returns success regardless — forked agents extract output values
from the AST via run_with_backend. Subprocess agents still write
to disk when the dir is set.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The old dispatch_tools intercepted output() calls and stored them in
auto.outputs. The new Agent::turn() dispatches normally, so output()
was hitting the filesystem path (which fails without POC_AGENT_OUTPUT_DIR).
Now run_with_backend scans the conversation AST after each tool turn
and extracts output() call arguments into auto.outputs. collect_results
in dmn.rs reads these to surface memories and inject reflections.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Byte-position truncation (&s[..s.len().min(N)]) panics when position
N lands inside a multi-byte character. Fixed in parser debug logging,
API error messages, oneshot response logging, and CLI agent display.
Also fixed tool dispatch permissions (removed global fallback).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
When an agent context is present, only dispatch tools in the agent's
tool list. The global fallback was bypassing per-agent tool
restrictions — a subconscious agent could call bash, edit, or any
tool even if its .agent file only allowed memory tools.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Parameter values like ["key1", "key2"] were being wrapped as strings
instead of parsed as JSON arrays. Tools expecting array arguments
(like memory_search) got a string containing the array literal.
Now tries serde_json::from_str first, falls back to String.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Qwen's chat template renders tool results as:
<|im_start|>user\n<tool_response>\n{content}\n</tool_response><|im_end|>
We were rendering as:
<|im_start|>tool\n{content}<|im_end|>
The model never saw <|im_start|>tool in training, so it ignored our
tool results and looped retrying the same call. Found by comparing
our tokenization against vLLM's /tokenize endpoint with chat messages.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
compact() was clearing tool definitions from the system section on
startup — now leaves system section untouched (set once by new()).
Added context token count to parser done log for diagnosing the
subconscious agent loop issue.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
compact() cleared and rebuilt the system section but only pushed the
system prompt — tool definitions were lost. Since new() sets up the
system section correctly (prompt + tools), compact() now only reloads
identity and journal, leaving system untouched.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Logs full response text when no tool calls detected, tool call
bodies when found. Per-agent log files for debugging subconscious
agent parsing issues.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Logs full text length, <tool_call> tag count, and tool call details
on stream completion. Helps diagnose parsing issues with subconscious
agents.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Was checking trim but storing untrimmed. Now stores the trimmed
version — no leading/trailing whitespace in the AST.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Content between tags (e.g. newlines between </think> and <tool_call>)
was creating empty Content nodes. Now trimmed before creating the node.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Parser skips Thinking nodes that are just whitespace. Conscious screen
now shows assistant children (Content, Thinking, ToolCall) as nested
tree items via recursive node_to_view. Nodes get timestamped in
push_node and on assistant branch creation.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The parser can't reliably split model-produced token IDs at tag
boundaries (<think>, <tool_call>) because BPE tokens can span across
tags. Instead, each leaf gets re-encoded from its text content via
the local tokenizer. This gives clean token boundaries aligned with
semantic structure — better for budgeting and potentially for the
model during fine-tuning.
Also skip serializing token_ids to conversation log (they're cached
state, recomputed on construction).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
The parser mutates the AST directly but doesn't write to the
conversation log. The turn loop now logs the completed assistant
branch after the parser handle resolves successfully.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
sync_from_agent now detects changed entries by comparing token counts
(cheap proxy for content changes during streaming). Changed entries
get popped and re-pushed. Extracted push_routed/pop_routed helpers.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
