Commit graph

558 commits

Author SHA1 Message Date
Kent Overstreet
809679b6ce delete dead flat-file journal tool and ephemeral stripping
Journal entries are written to the memory graph via journal_new/
journal_update, not appended to a flat file. Remove thought/journal.rs
(67 lines), strip_ephemeral_tool_calls (55 lines), default_journal_path,
and all wiring. -141 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:35:56 -04:00
Kent Overstreet
aceaf0410e delete dead flat-file journal code from thought/context.rs
Journal entries are loaded from the memory graph store, not from the
flat journal file. Remove build_context_window, plan_context,
render_journal_text, assemble_context, truncate_at_section,
find_journal_cutoff, parse_journal*, ContextPlan, and stale TODOs.
Keep JournalEntry, default_journal_path (write path), and the live
context management functions. -363 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:31:12 -04:00
Kent Overstreet
214806cb90 move context functions from agent/context.rs to thought/context.rs
trim_conversation moved to thought/context.rs where model_context_window,
msg_token_count, is_context_overflow, is_stream_error already lived.
Delete the duplicate agent/context.rs (94 lines).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:28:00 -04:00
Kent Overstreet
01bfbc0dad move journal types from agent/journal.rs to thought/context.rs
JournalEntry, parse_journal, parse_journal_text, parse_header_timestamp,
and default_journal_path consolidated into thought/context.rs. Delete
the duplicate agent/journal.rs (235 lines). Update all references.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:25:07 -04:00
Kent Overstreet
e0a54a3b43 save request payload on any API error, not just timeouts
Serialize request JSON before send_and_check so it's available
for both HTTP errors and stream errors. Extracted save logic
into save_failed_request helper on SseReader.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:19:26 -04:00
Kent Overstreet
64dbcbf061 unify memory tracking: entries are the single source of truth
Memory tool results (memory_render) are now pushed as
ConversationEntry::Memory with the node key, instead of plain
Messages. Remove loaded_nodes from ContextState — the debug
screen reads memory info from Memory entries in the conversation.

Surfaced memories from surface-observe are pushed as separate
Memory entries, reflections as separate system-reminder messages.
User input is no longer polluted with hook output.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 14:56:02 -04:00
Kent Overstreet
a21cf31ad2 unify conversation persistence to append-only jsonl
Log ConversationEntry (with Memory/Message typing) instead of
raw Message. restore_from_log reads typed entries directly,
preserving Memory vs Message distinction across restarts.

Remove current.json snapshot and save_session — the append-only
log is the single source of truth. Remove dead read_all and
message_count methods. Add push_entry for logging typed entries.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 14:31:19 -04:00
Kent Overstreet
1f7b585d41 remove Anthropic backend, add request logging on timeout
Delete anthropic.rs (713 lines) — we only use OpenAI-compatible
endpoints (vLLM, OpenRouter). Simplify ApiClient to store base_url
directly instead of Backend enum.

SseReader now stores the serialized request payload and saves it
to ~/.consciousness/logs/failed-request-{ts}.json on stream timeout,
so failed requests can be replayed with curl for debugging.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 14:13:23 -04:00
Kent Overstreet
078dcf22d0 cleanup: remove model name string matching
model_context_window() now reads from config.api_context_window
instead of guessing from model name strings. is_anthropic_model()
replaced with backend == "anthropic" checks. Dead model field
removed from AgentDef/AgentHeader.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 14:09:54 -04:00
Kent Overstreet
47c6694b10 Remove dead code: old context builder, plan_context, journal parsing
Removed from context.rs: ContextPlan, plan_context,
render_journal_text, assemble_context, truncate_at_section,
find_journal_cutoff, parse_msg_timestamp. All replaced by
trim_conversation + journal from memory graph.

Removed from tui.rs: most_recent_file, format_duration
(filesystem scanning leftovers).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 03:40:35 -04:00
Kent Overstreet
e9e47eb798 Replace build_context_window with trim_conversation
build_context_window loaded journal from a stale flat file and
assembled the full context. Now journal comes from the memory graph
and context is assembled on the fly. All that's needed is trimming
the conversation to fit the budget.

trim_conversation accounts for identity, journal, and reserve
tokens, then drops oldest conversation messages until it fits.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 03:35:28 -04:00
Kent Overstreet
87add36cdd Fix: don't overwrite journal during restore/compaction
The restore and compaction paths called build_context_window which
reads from the stale flat journal file, overwriting the journal we
loaded from the memory graph. Preserve the graph-loaded journal
across these operations.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 03:33:04 -04:00
Kent Overstreet
b9e3568385 ConversationEntry enum: typed memory vs conversation messages
Replace untyped message list with ConversationEntry enum:
- Message(Message) — regular conversation turn
- Memory { key, message } — memory content with preserved message
  for KV cache round-tripping

Budget counts memory vs conversation by matching on enum variant.
Debug screen labels memory entries with [memory: key]. No heuristic
tool-name scanning.

Custom serde: Memory serializes with a memory_key field alongside
the message fields, deserializes by checking for the field.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 03:26:00 -04:00
Kent Overstreet
eb4dae04cb Compute ContextBudget on demand from typed sources
Remove cached context_budget field and measure_budget(). Budget
is computed on demand via budget() which calls
ContextState::budget(). Each bucket counted from its typed source.
Memory split from conversation by identifying memory tool calls.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 03:07:45 -04:00
Kent Overstreet
acdfbeeac3 Align debug screen and budget with conversation-only messages
context.messages is conversation-only now — remove conv_start
scanning. Memory counted from loaded_nodes (same as debug screen).
No subtraction heuristics.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 02:56:28 -04:00
Kent Overstreet
5e781e9ae4 Fix budget counting: remove stale refresh_context_message
refresh_context_message was injecting personality into conversation
messages (assuming fixed positions that no longer exist). Replaced
with refresh_context_state which just re-measures and publishes.

conv_tokens now subtracts mem_tokens since memory tool results are
in the conversation message list.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 02:52:59 -04:00
Kent Overstreet
a0aacfc552 Move conversation messages into ContextState
ContextState now owns everything in the context window:
system_prompt, personality, journal, working_stack, loaded_nodes,
and conversation messages. No duplication — each piece exists once
in its typed form.

assemble_api_messages() renders the full message list on the fly
from typed sources. measure_budget() counts each bucket from its
source directly. push_context() removed — identity/journal are
never pushed as messages.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 02:47:32 -04:00
Kent Overstreet
4580f5dade measure_budget: count from typed sources, not message scanning
Identity tokens from system_prompt + personality vec. Journal
from journal entries vec. Memory from loaded_nodes. Conversation
is the remainder. No string prefix matching.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 02:32:26 -04:00
Kent Overstreet
4bdc7ae112 Journal budget: count from structured data, not string matching
Count journal tokens directly from Vec<JournalEntry> instead of
scanning message text for prefix strings. Type system, not string
typing.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 02:29:48 -04:00
Kent Overstreet
5526a26d4c Journal: store as structured Vec<JournalEntry>, not String
Keep journal entries as structured data in ContextState. Render
to text only when building the context message. Debug screen reads
the structured entries directly — no parsing ## headers back out.

Compaction paths temporarily parse the string from build_context_window
back to entries (to be cleaned up when compaction is reworked).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 02:21:45 -04:00
Kent Overstreet
42f1e888c4 Journal: flat 5% context window budget, skip plan_context
Render journal entries directly with ## headers instead of going
through the plan_context/render_journal_text pipeline. 5% of
model context window (~6500 tokens for Qwen 128K). Simpler and
predictable.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 02:00:14 -04:00
Kent Overstreet
7776d87d53 Journal: walk backwards with token budget, not load-all
Iterate journal entries backwards from the conversation cutoff,
accumulating within ~10K token budget (~8% of context window).
Stops when budget is full, keeps at least one entry. Much more
efficient than loading all entries and trimming.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 01:50:36 -04:00
Kent Overstreet
e4285ba75f Load journal from memory graph, not flat file
Replace flat-file journal parser with direct store query for
EpisodicSession nodes. Filter journal entries to only those older
than the oldest conversation message (plus one overlap entry to
avoid gaps). Falls back to 20 recent entries when no conversation
exists yet.

Fixes: poc-agent context window showing 0 journal entries.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 01:48:16 -04:00
Kent Overstreet
c814ed1345 Split hook.rs: core orchestration -> subconscious.rs
subconscious::subconscious — AgentCycleState, AgentInfo, AgentSnapshot,
  SavedAgentState, format_agent_output, cycle methods. Core agent
  lifecycle independent of Claude Code.

subconscious::hook — Claude Code hook: context loading, chunking,
  seen-set management, run_agent_cycles (serialized state entry point).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 01:37:51 -04:00
Kent Overstreet
fbc8572840 Serialized AgentCycleState for Claude Code hook path
SavedAgentState (JSON) persists agent pid/phase/log_path across
hook invocations. The Claude Code hook loads saved state, runs
cycles, saves back. Pids are liveness-checked with kill(pid, 0)
on load. No more scan_pid_files for agent lifecycle tracking.

poc-agent keeps everything in memory (child handles). The hook
path uses serialized state. Same AgentCycleState, different
persistence model.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 01:31:59 -04:00
Kent Overstreet
90d2717423 Use own state for spawn decisions, not pid file scanning
AgentCycleState tracks its own children — agent_running() checks
child handles instead of scan_pid_files(). poll_children() reaps
completed processes. No filesystem scanning for agent lifecycle.

The Claude Code hook path will need serialized AgentCycleState
to persist across invocations (next step).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 01:26:58 -04:00
Kent Overstreet
9ac50bd999 Track agent child processes, reap on completion
spawn_agent returns Child handle + log_path. AgentCycleState stores
the Child, polls with try_wait() on each trigger to detect completion.
No more filesystem scanning to track agent lifecycle.

AgentSnapshot (Clone) sent to TUI for display. AgentInfo holds the
Child handle and stays in the state.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 01:20:03 -04:00
Kent Overstreet
54ea7824d8 Fix agent log path: only set state on spawn, not scan
Agent state (pid, phase, log_path) only updates when we spawn an
agent. The scan_pid_files path no longer calls update_agent —
it just logs. This prevents the scan path from clearing log_path
with None on subsequent triggers.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 01:15:37 -04:00
Kent Overstreet
a90bd4fd47 Agent log screen: show agent output, not hook log
spawn_agent() now returns SpawnResult { pid, log_path } so the
log path is known at spawn time. No more filesystem scanning.
AgentInfo carries log_path, TUI reads it directly.

F2 → Enter shows the actual agent log (stdout/stderr from the
poc-memory agent process), not the hook orchestration log.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 01:04:54 -04:00
Kent Overstreet
1c190a3925 Wire AgentCycleState through runner and TUI
Runner owns AgentCycleState, calls trigger() on each user message
instead of the old run_hook() JSON round-trip. Sends AgentUpdate
messages to TUI after each cycle.

TUI F2 screen reads agent state from messages instead of scanning
the filesystem on every frame. HookSession::from_fields() lets
poc-agent construct sessions without JSON serialization.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 00:52:57 -04:00
Kent Overstreet
d097c8e067 AgentCycleState: persistent state for agent orchestration
Move agent cycle functions from free functions to methods on
AgentCycleState. The struct tracks per-agent pid/phase and the
log file handle. trigger() runs all three cycles and updates
last_output.

Claude Code hook path creates a temporary AgentCycleState per call.
poc-agent will own one persistently and share it with the TUI.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 00:47:52 -04:00
Kent Overstreet
55a037f4c7 Rename Session -> HookSession
The hook's Session is not the same as poc-agent's session concept.
Rename to avoid confusion now that poc-agent will create HookSessions
to call into the agent cycle.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 00:42:25 -04:00
Kent Overstreet
a0245c1279 Refactor hook: split agent orchestration from formatting
- Remove POC_AGENT early return (was from old claude -p era)
- Split hook into run_agent_cycles() -> AgentCycleOutput (returns
  memory keys + reflection) and format_agent_output() (renders for
  Claude Code injection). poc-agent can call run_agent_cycles
  directly and handle output its own way.
- Fix UTF-8 panic in runner.rs display_buf slicing (floor_char_boundary)
- Add priority debug label to API requests
- Wire up F2 agents screen: live pid status, output files, hook log
  tail, arrow key navigation, Enter for log detail view

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 00:32:23 -04:00
Kent Overstreet
c72eb4d528 vLLM priority scheduling for agents
Thread request priority through the API call chain to vLLM's
priority scheduler. Lower value = higher priority, with preemption.

Priority is set per-agent in the .agent header:
- interactive (runner): 0 (default, highest)
- surface-observe: 1 (near-realtime, watches conversation)
- all other agents: 10 (batch, default if not specified)

Requires vLLM started with --scheduling-policy priority.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 23:21:39 -04:00
Kent Overstreet
503e2995c1 Add memory_query to journal agent whitelist
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:25:48 -04:00
Kent Overstreet
c7b0620323 Give journal agent search, render, used tools for linking
Journal needs to find nodes (memory_search), read them
(memory_render), and track seen set (memory_used) to make
informed links. Still no memory_write — node creation is
observe's job.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:25:22 -04:00
Kent Overstreet
e013ec778e Add memory_link_add to journal agent whitelist
Journal entries need to link to relevant memory nodes for graph
connectivity. Added memory_link_add to the journal agent's tool
whitelist alongside the journal tools.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:23:02 -04:00
Kent Overstreet
4c9005a1a5 Set journal agent tool whitelist to journal-only tools
Journal agent now only gets journal_tail, journal_new, journal_update.
Cannot create duplicate memory nodes via memory_write.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:20:28 -04:00
Kent Overstreet
916f14a092 Log effective tool list, not just whitelist
Shows the actual tool names each agent will receive after
whitelist filtering, so logs are accurate regardless of whether
tools is empty (all) or specified.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:20:00 -04:00
Kent Overstreet
8eabeab8eb Tool whitelist from agent header filters native tools
The tools field in agent headers now filters which native tools
the agent receives. Empty = all tools (default). Non-empty =
whitelist. Journal agent can list only journal_tail/journal_new/
journal_update. Log shows actual tool names instead of "no tools".

Threaded tools list through call_api_with_tools → sync wrapper →
llm caller.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:18:42 -04:00
Kent Overstreet
834247fa53 Split journal tools from default definitions, expose to all for now
journal_definitions() separated from definitions() in memory.rs.
All agents get memory + journal tools via memory_and_journal_definitions().
TODO: implement per-agent tool whitelist from header to properly
restrict journal tools to journal agent only.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:12:14 -04:00
Kent Overstreet
4173f5ac5d Remove Bash(poc-memory:*) from all agent configs
Agents must use native tool dispatch, not bash, for correct
provenance tracking. Bash access was leftover from old architecture.
All 12 agents cleaned up.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:03:44 -04:00
Kent Overstreet
d932a90018 Restrict journal agent to journal-only tools
Remove journal tool from memory-instructions-core (only the journal
agent should write journal entries). Add explicit instruction to
journal agent: only use journal_tail/journal_new/journal_update,
not memory_write/render/search.

Prevents the journal agent from creating duplicate memory nodes
about events that surface-observe is already recording.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 15:01:42 -04:00
Kent Overstreet
f9e0c008d9 Compact agent logs by default, verbose with POC_AGENT_VERBOSE
Skip full prompt logging and truncate tool results in normal mode.
Logs now show: header, tool calls with one-line results, response
text. Set POC_AGENT_VERBOSE=1 for full prompts and results.

Makes agent logs scannable at a glance instead of walls of text.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 10:28:15 -04:00
Kent Overstreet
8714a15e1c Remove model field from all agent configs
Agents are routed to Qwen by the runner, not by per-agent model
fields. The "model":"sonnet" was leftover from the Claude API days
and no longer used.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-01 10:15:19 -04:00
Kent Overstreet
64b2f327f9 surface-observe: tighten observe phase to be more factual
Reframe the observe role as librarian — factual, specific, organized.
Record what happened and why. Reflection belongs in the journal;
observe is for memory.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-31 23:09:51 -04:00
Kent Overstreet
3d62f27dfb memory: rename memory_spread → memory_search, remove keyword search
memory_search is now spreading activation — the natural way to search
a graph. Give it seed node keys and it finds conceptually related nodes.

The old keyword-based memory_search and memory_search_content are
removed; memory_query can do everything they did.

Simpler tool set, better defaults. Agents don't need to be told "use
spread not search" — search IS spread now.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-31 20:25:00 -04:00
Kent Overstreet
a837e3f2e4 surface-observe: strongly prefer memory_spread over memory_search
The agent was defaulting to keyword searches despite instructions to
use spreading activation first. Reframe instructions positively:
memory_spread is the default mode of operation. Search is available
for finding specific nodes by name.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-31 20:19:00 -04:00
Kent Overstreet
ebc29a3674 memory: add dispatch handlers for memory_spread and memory_search_content
The new tool definitions broke surface-observe because they had no
corresponding dispatch handlers — the agent runner saw unknown tools
and ran with no tools at all.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-31 18:40:15 -04:00
Kent Overstreet
081d40f306 surface-observe: use spreading activation, watch for behavioral patterns
Update surface-observe agent instructions to use memory_spread as the
primary search strategy — cast a wide net from conversation themes before
drilling in with graph walks.

Add explicit instruction to watch for behavioral patterns (avoidance,
rushing, explaining away data) and surface relevant feedback memories
in the moment.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-31 18:21:35 -04:00