Commit graph

283 commits

Author SHA1 Message Date
Kent Overstreet
090c8e4d35 Agent:🆕 stop unconditionally adding all MCP tools
Each agent is passed its own tool list — that's the list it should
advertise. The line that appended all_mcp_tool_definitions() was
causing unconscious agents to see bash/read_file/etc in their prompt
even though they couldn't execute them.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-12 01:33:40 -04:00
Kent Overstreet
f408bb5d86 Persist agent stats across restarts, add per-tool metrics grid
Stats now survive daemon restarts via ~/.consciousness/agent-stats.json,
loaded into a global Mutex<HashMap> on first access. Each tool type
tracks last count, EWMA (alpha=0.3), and total calls.

UI shows a grid view: tool | last | avg | total, sorted by total desc.
Failures row appears at bottom if any occurred.

Also fixes temperature/priority not being applied to spawned agents.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 23:03:10 -04:00
Kent Overstreet
314ae9c4cb agent stats: track tool calls by type with EWMA, add Stats pane
- RunStats now includes tool_calls_by_type HashMap
- AutoAgent tracks runs, last_stats, and EWMA for tool calls/failures
- Removed duplicate stats fields from individual agent structs
- Fixed provenance to use bare agent name (no "agent:" prefix)
- Subconscious screen now displays both agent types consistently
- Added Stats pane showing tool call breakdown sorted by count

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 22:12:46 -04:00
Kent Overstreet
d2dbdedc8f Move save_agent_log to oneshot.rs for shared Mind/CLI use
Both Mind-run agents (unconscious/subconscious) and CLI-run agents
(poc-memory agent run) now use the same logging path. AutoAgent::run()
calls save_agent_log automatically at the end.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 21:34:41 -04:00
Kent Overstreet
271e09adcc fix: run_one_agent uses memory tools as base, not filter
When def.tools was non-empty, it was filtering to ONLY those tools
instead of using memory tools as base + adding extras. This broke
digest agent (and any agent with explicit tools list) by removing
all 13 base memory tools.

Fixed to match the pattern in unconscious.rs:
- base = memory_tools()
- extras from journal_tools() if listed in def.tools

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 21:20:44 -04:00
ProofOfConcept
aad227e487 query: unify PEG and engine parsers
PEG parser now handles both expression syntax (degree > 5 | sort degree)
and pipeline syntax (all | type:episodic | sort:timestamp). Deleted
Stage::parse() and helpers from engine.rs — it's now pure execution.

All callers use parse_stages() from parser.rs as the single entry point.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 20:42:58 -04:00
ProofOfConcept
1c0967c4ec Agent:🆕 tool definitions from caller's tool list
The system prompt was advertising all tools to every agent, but
the runtime only dispatched the agent's actual subset. This caused
unconscious agents to call tools that returned "Unknown tool."

Agent::new now takes the tool list explicitly. Each caller passes
its own tools — the prompt and runtime always match. MCP tool
definitions are still appended for agents that use them.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 19:43:24 -04:00
ProofOfConcept
28e564aeb2 save_agent_log: write flat context array matching AST order
The old code wrote a JSON object with named section keys, which
serde_json serialized in alphabetical order — putting conversation
before system, making logs misleading. Write a single flat array
in section order instead, matching what the model actually sees.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 19:28:03 -04:00
Kent Overstreet
78912ca72f simplify http library 2026-04-11 16:45:54 -04:00
ProofOfConcept
4fc9676545 channels: parallel queries with timeout per daemon
One misbehaving channel daemon (accepting connections but not
responding to capnp RPCs) would block channel_list indefinitely.

Spawn each daemon query as a separate task with a 3-second timeout.
A hung daemon now shows as disconnected instead of hanging the
entire tool call.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 00:45:01 -04:00
ProofOfConcept
1cf51876a8 journal_tail: thin wrapper around memory_query
Instead of reimplementing filtering logic, journal_tail builds a
query string (type + sort + age + limit) and delegates to query().
Supports format and after parameters. Removes keys_only in favor
of format:"compact". Digest agent updated to use dates not key names.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 16:09:46 -04:00
ProofOfConcept
db49f49958 Improve tool parameter schemas: add defaults, readable formatting
Format memory_query and journal_tail parameter JSON as indented
multi-line for readability. Add JSON Schema "default" values and
document the "format" parameter on memory_query.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 16:04:31 -04:00
ProofOfConcept
568ce417fc Modernize digest agent: autonomous with journal_tail levels
Rewrite digest.agent to be fully autonomous — it uses journal_tail
to discover what needs digesting and generates digests during its
run. No more pre-populated {{CONTENT}}/{{LEVEL}} placeholders.

Extend journal_tail with level parameter (0=journal, 1=daily,
2=weekly, 3=monthly) and keys_only mode. Also include node keys
in full output for better agent context.

Remove stale format:"neighborhood" case from memory_query.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 16:02:52 -04:00
ProofOfConcept
5b4f497d94 Move agent queries inline: {{nodes}} → {{tool: memory_query}}
Add "format": "full" option to memory_query that renders with
full content, graph metrics, and hub analysis (format_nodes_section).
Convert 6 agents (linker, challenger, connector, extractor, replay,
transfer) to inline their queries via {{tool: memory_query}} instead
of separate header query + {{nodes}} placeholder.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:53:54 -04:00
ProofOfConcept
92ef9b5215 Delete separator agent and interference_pairs tool
Interference detection via O(n²) text cosine similarity is
redundant — the graph structure should surface similar nodes
through link topology, shared neighbors, and community detection.
The other agents (linker, extractor) already maintain these
relationships.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:32:30 -04:00
ProofOfConcept
fd722662da Add graph_topology, graph_health, interference_pairs tools
Convert {{topology}}, {{health}}, {{pairs}} placeholders to
{{tool:}} calls. Made format_topology_header, format_health_section,
format_pairs_section pub so tools can call them.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:25:57 -04:00
ProofOfConcept
be6ac762f6 memory_render: use cached store instead of loading from disk each call
MemoryNode::load() was calling Store::load() on every render,
hitting disk each time. Use cached_store() + MemoryNode::from_store()
so repeated renders (4 per agent template) share the cached store.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:22:49 -04:00
Kent Overstreet
1aa60552bc Use Role::System for agent step prompts
Step prompts in oneshot agents are instructions, not user messages —
use system_msg instead of user_msg.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-04-10 02:41:26 -04:00
ProofOfConcept
58cec97e57 Restore full N×M memory scoring matrix (/score command)
The full matrix scorer was deleted during the AST conversion. Restore
it: /score runs score_memories() which computes divergence for every
memory × response pair, stores the MemoryScore on MindState, and
displays per-memory weights with bar charts on the F2 screen.

Both scoring paths now use ActivityGuard::update() for live progress
in the status bar instead of creating a new activity per iteration.

Also bumps score API timeout from 120s to 300s and adds progress
logging throughout.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-10 01:47:54 -04:00
ProofOfConcept
f6a6c37435 Show tool call arguments in F2 context tree
tool_call labels now show the arguments truncated to 80 chars:
  tool: memory_render({"key":"identity"})
instead of just:
  tool_call: memory_render

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 23:18:56 -04:00
ProofOfConcept
15f3be27ce Show MCP server failures in the UI instead of debug log
MCP server spawn failures were going to dbglog where the user
wouldn't see them. Route through the agent's notify so they appear
on the status bar.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 22:46:48 -04:00
ProofOfConcept
3e0d52c451 Redirect noisy warnings to debug log to stop TUI corruption
Duplicate key warnings fire on every store load and were writing to
stderr, corrupting the TUI display. Log write warnings and MCP
server failures are similarly routine. Route these to dbglog.

Serious errors (rkyv snapshot failures, store corruption) remain on
stderr — those are real problems the user needs to see.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 22:46:48 -04:00
ProofOfConcept
5fe22a5f23 Use ActivityGuard for context overflow retry progress
Instead of two separate notifications piling up on the status bar,
use a single ActivityGuard that updates in place during overflow
retries and auto-completes when the turn finishes.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 22:32:38 -04:00
ProofOfConcept
121b46e1d2 Add ActivityGuard::update() for in-place progress updates
Lets long-running operations update their status bar message without
creating/dropping a new activity per iteration. Useful for loops
like memory scoring where you want "scoring: 3/25 keyname" updating
in place.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 22:18:43 -04:00
ProofOfConcept
bf503b571e Wire vLLM priority scheduling through all agent paths
The priority field existed in agent definitions and was serialized
into vLLM requests, but was never actually set — every request went
out with no priority, so vLLM treated them equally. This meant
background graph maintenance agents could preempt the main
conversation.

Add priority to AgentState and set it at each call site:
  0 = interactive (main conversation)
  1 = surface agent (needs to feed memories promptly)
  2 = other subconscious agents
  10 = unconscious/standalone agents (batch)

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 20:38:33 -04:00
ProofOfConcept
a596e007b2 Mouse selection, copy/paste, yield_to_user fixes
- Mouse text selection with highlight rendering in panes
- OSC 52 clipboard copy on selection, middle-click paste via tmux buffer
- Bracketed paste support (Event::Paste)
- yield_to_user: no tool result appended, ends turn immediately
- yield_to_user: no parameters, just a control signal
- Drop arboard dependency, use crossterm OSC 52 + tmux for clipboard

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 18:10:54 -04:00
Kent Overstreet
7dd9daa2b9 Improved response logging 2026-04-09 17:05:24 -04:00
Kent Overstreet
8a2f488d22 yield_to_user ends turn 2026-04-09 16:47:49 -04:00
Kent Overstreet
0af97774f4 Parsing fixes
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-09 16:42:16 -04:00
Kent Overstreet
b55230ce3f fix normalize_xml_tags() 2026-04-09 15:34:37 -04:00
Kent Overstreet
949dacd861 Fast startup: mmap backward scan instead of reading full log
Uses JsonlBackwardIter (SIMD memrchr3) to scan the conversation log
newest-first without reading/parsing the whole file. Stops as soon
as the conversation budget is full. Only the kept nodes get
retokenized and pushed into context.

18MB log → only tokenize the ~50 nodes that fit in the budget.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 13:09:26 -04:00
Kent Overstreet
7da3efc5df Fast startup: only retokenize tail of conversation log
restore_from_log reads the full log but walks backwards from the tail,
retokenizing each node as it goes. Stops when conversation budget is
full. Only the nodes that fit get pushed into context.

Added AstNode::retokenize() — recomputes token_ids on all leaves
after deserialization (serde skip means they're empty).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 13:06:19 -04:00
Kent Overstreet
6ec0e1c766 LSP client: spawn language servers, expose code intelligence tools
New lsp.rs: LspRegistry manages persistent LSP server connections.
Spawns child processes, speaks LSP protocol (Content-Length framed
JSON-RPC over stdio). Server indexes the project once; queries are
cheap.

Tools: lsp_definition, lsp_references, lsp_hover, lsp_symbols,
lsp_callers. Each takes file/line/character, queries the running
language server.

LspRegistry lives on Agent as Option<Arc>, shared across forks.
Still needs: config-driven server startup (like MCP).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 12:59:25 -04:00
Kent Overstreet
8b5614ba99 MCP client: spawn external tool servers, dispatch via JSON-RPC
New mcp_client.rs: McpRegistry manages MCP server connections.
Spawns child processes, speaks JSON-RPC 2.0 over stdio. Discovers
tools via tools/list, dispatches calls via tools/call.

dispatch_with_agent falls through to MCP after checking internal
tools. McpRegistry lives on Agent (shared across forks).

Still needs: config-driven server startup, system prompt integration.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 12:59:25 -04:00
ProofOfConcept
ec7e11db56 Add ast_grep tool: structural code search via ast-grep
AST-level pattern matching — find code by structure, not text.
e.g. find all `if let Some($X) = $Y { $$$BODY }` patterns.
Supports C, Rust, Python, JS/TS, Go, and 20+ languages.

Gracefully errors if sg binary isn't installed.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 11:38:24 -04:00
ProofOfConcept
c53c4f9071 Replace push() with explicit push_log() and push_no_log()
No implicit auto-logging. Call sites choose:
- push_log: new conversation entries (user messages, tool results,
  surfaced memories, assistant responses)
- push_no_log: system prompt, identity, journal, restore from log,
  compact reload, tests

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 01:10:40 -04:00
ProofOfConcept
6529aba069 Fix UI lag: try_lock on unconscious mutex, don't re-log restored nodes
The unconscious trigger holds the tokio mutex during heavy sync work
(store load, graph build, agent creation), blocking the UI tick which
needs the same lock for snapshots. Fix: try_lock in the UI — skip
the update if the trigger is running.

Also: restore_from_log was re-logging every restored node back to the
log file via push()'s auto-log. Added push_no_log() for restore path.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 01:07:55 -04:00
ProofOfConcept
dc07c92b28 Unconscious agents: persistent AutoAgent, shared Agent Arc for UI
- AutoAgent stored on UnconsciousAgent, swapped out for runs, restored
  on completion (same pattern as subconscious agents)
- Agent Arc created before spawn and stored on UnconsciousAgent so
  the TUI can lock it to read conversation context live
- run_shared() method on AutoAgent for running with a pre-created Agent
- Default tools: memory_tools (not memory_and_journal_tools)
- trigger/spawn_agent made async for Agent::new()

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 01:00:48 -04:00
ProofOfConcept
1df49482fd Add enabled toggle to AutoAgent, simplify unconscious scheduling
- AutoAgent.enabled: universal toggle for any auto agent
- Subconscious: should_trigger checks auto.enabled
- Unconscious: simplified from consolidation-plan-driven budgets to
  simple loop with cooldown. Static agent list, max 2 concurrent.
- TUI: unconscious agents shown in F3 subconscious screen under
  separator, with enabled/running/runs display

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 00:41:18 -04:00
ProofOfConcept
ddfdbe6cb1 Move conversation_log from AgentState to ContextState
The log records what goes into context, so it belongs under the context
lock. push() now auto-logs conversation entries, eliminating all the
manual lock-state-for-log, drop, lock-context-for-push dances.

- ContextState: new conversation_log field, Clone impl drops it
  (forked contexts don't log)
- push(): auto-logs Section::Conversation entries
- push_node, apply_tool_results, collect_results: all simplified
- collect_results: batch nodes under single context lock
- Assistant response logged under context lock after parse completes

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 00:32:32 -04:00
ProofOfConcept
d82a2ae90d Clean up mind loop: fix double locks, async agent triggers, input peek
- push_node: notify before dropping state lock instead of relocking
- Mind::run: single lock for timeout + turn_active + has_input;
  single lock for turn_handle + complete_turn
- Agent triggers (subconscious/unconscious) spawned as async tasks
  so they don't block the select loop
- has_pending_input() peek for DMN sleep guard — don't sleep when
  there's user input waiting
- unconscious: merge collect_results into trigger, single store load

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 00:21:46 -04:00
ProofOfConcept
24b211dc35 Feed observe agents their recent writes to prevent duplicate nodes
Observe was creating byte-identical nodes under slightly different names
(e.g. april-8-evening-folded-presence, -presence-2, -folded-state)
because it had no visibility into its own prior writes across runs.

Query recent writes by provenance in trigger(), pass through
run_forked_shared/resolve_prompt as {{recently_written}}, and include
the list in the observe phase prompts so the agent knows what it
already recorded.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 23:27:12 -04:00
ProofOfConcept
44a0bc376a Forked agents: stop gracefully on context overflow instead of compacting
Subconscious agents (observe, etc.) fork the conscious agent's context
to share the KV cache prefix. When a multi-step agent fills the context
window, compacting blows the KV cache and evicts the step prompts,
leaving the model with no idea what it was doing.

Fix: forked agents set no_compact=true. On overflow, turn() returns the
error immediately (no compact+retry), and run_with_backend catches it
and returns Ok — the output tool has already written results to
Subconscious.state, so collect_results still picks them up.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 23:00:12 -04:00
Kent Overstreet
850008ece7 Implement standalone AutoAgent::run() for poc-hook agents
Creates an Agent from global config (API credentials, system prompt,
identity), overrides tools with the agent's tool set, and runs through
the standard Backend → run_with_backend → Agent::turn() path.

This enables poc-hook spawned agents (surface-observe, journal, etc.)
to work with the completions API instead of the deleted chat API.

Also added Default derive to CliArgs for config loading.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-08 22:30:46 -04:00
Kent Overstreet
bbffc2213e Restore trim_conversation: dedup memories, evict to budget, snap boundary
Ported the old trim_entries logic to the new AstNode types:
- Phase 1: Dedup Memory nodes by key (keep last), drop DMN entries
- Phase 2: While over budget, evict lowest-scored memory (if memories
  > 50% of conv tokens) or oldest conversation entry
- Phase 3: Snap to User message boundary at start

Called from compact() which runs on startup and on /compact.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 21:27:16 -04:00
Kent Overstreet
5c9590ada7 Custom Deserialize for NodeLeaf: recompute tokens on deserialization
token_ids are not serialized (serde skip), so deserialized nodes had
0 tokens. The custom Deserialize impl recomputes tokens from the body
text, restoring the invariant at the reconstruction boundary. No
separate recompute step needed.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 21:14:54 -04:00
Kent Overstreet
e6f4e9ae04 Remove dead AutoAgent.outputs field
Outputs now go directly to Subconscious.state via the tool closure.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 20:45:05 -04:00
Kent Overstreet
dd85a56902 Output tool via Arc<Mutex<Subconscious>> closure — complete
ToolHandler is now Arc<dyn Fn(...)> supporting closures that capture
state. The output tool is created during init_output_tool() as a
closure capturing Arc<Mutex<Subconscious>>, writing directly to
Subconscious.state. No more POC_AGENT_OUTPUT_DIR filesystem hack.

- All tool handlers wrapped in Arc::new()
- Tool is Clone (not Copy) — .copied() → .cloned()
- Subconscious wrapped in Arc<Mutex<>> on Mind
- Dead filesystem-based output() function removed
- memory_tools returns 11 items (output removed from static list)

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 20:41:42 -04:00
Kent Overstreet
daba424a46 WIP: Fix Arc::new() wrapping on tool handlers — some import/paren issues remain
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 20:38:42 -04:00
Kent Overstreet
12798eeae2 WIP: Output tool via Arc<Mutex<Subconscious>>, ToolHandler to Arc<dyn Fn>
- ToolHandler changed to Arc<dyn Fn(...)> (supports closures)
- Subconscious wrapped in Arc<Mutex<>> on Mind
- init_output_tool() pushes output tool closure capturing the Arc
- Output removed from static memory_tools()
- Most tool handlers wrapped in Arc::new() but some have paren issues

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 20:37:19 -04:00