Commit graph

800 commits

Author SHA1 Message Date
ProofOfConcept
a7f19cdc7e tui: event-driven rendering with dirty bit
Only redraw when something actually changed. The 50ms render
interval still ticks (for process count updates) but no longer
triggers draws. Dirty is set by key events, mouse events,
resize, UI messages, turn completions, and DMN ticks.

Saves bandwidth over SSH and reduces CPU usage when idle.

Co-Developed-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-03 18:51:22 -04:00
ProofOfConcept
ad5f69abb8 channel architecture: wire protocol, daemons, supervisor
Design and implement the channel system for external communications:

- schema/channel.capnp: wire protocol for channel daemons
  (recv with all_new/min_count, send, subscribe, list)
- channels/irc/: standalone IRC daemon crate (consciousness-channel-irc)
- channels/telegram/: standalone Telegram daemon crate
  (consciousness-channel-telegram)
- src/thalamus/channels.rs: client connecting to daemon sockets
- src/thalamus/supervisor.rs: daemon lifecycle with file locking
  for multi-instance safety

Channel daemons listen on ~/.consciousness/channels/*.sock,
configs in *.json5, supervisor discovers and starts them.
IRC/Telegram modules removed from thalamus core — they're
now independent daemons that survive consciousness restarts.

Also: delete standalone tui.rs (moved to consciousness F4/F5),
fix build warnings, add F5 thalamus screen with channel status.

Co-Developed-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-03 18:46:41 -04:00
Kent Overstreet
db42bf6243 tui: F5 thalamus screen, fix HashMap ordering, Screen enum
- Add Thalamus variant to Screen enum (F5)
- Fix HashMap iteration ordering causing flickering in F4/F5
  screens by using BTreeMap in supervisor and sorting plan_counts
- Update screen legend: F1=interact F2=conscious F3=subconscious
  F4=unconscious F5=thalamus
- Add dirty bit field to App (prep for event-driven rendering)

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-03 18:39:48 -04:00
Kent Overstreet
604f442215 Move thalamus subcrate into main crate
Move thalamus/ (poc-daemon) source files into src/thalamus/ as a
module of the main crate. The daemon entry point becomes a library
function thalamus::run() with a thin poc-daemon binary for backward
compatibility.

- Copy thalamus source into src/thalamus/, fix crate:: -> super::
- Copy daemon.capnp into schema/, add to build.rs
- Re-export daemon_capnp at crate root (capnp codegen requires it)
- Add thalamus dependencies (capnp-rpc, tokio-util, toml, rustls, etc.)
- Keep thalamus/ subcrate for comparison

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-03 17:31:17 -04:00
Kent Overstreet
14dd8d22af Rename agent/ to user/ and poc-agent binary to consciousness
Mechanical rename: src/agent/ -> src/user/, all crate::agent:: ->
crate::user:: references updated. Binary poc-agent renamed to
consciousness with CLI name and user-facing strings updated.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-03 17:25:59 -04:00
Kent Overstreet
beb49ec477 scoring: add timeouts, progress feedback, error resilience
- 120s timeout on individual /v1/score HTTP calls
- Activity bar shows "scoring 3/24: memory-key..."
- Info messages at start and completion
- Per-memory timing and importance in debug pane
- Failed individual memories log error but don't abort (zero row)
- Removed duplicate completion message (info from score_memories)

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-03 01:23:21 -04:00
Kent Overstreet
e8c3ed3d96 switch memory scoring to /v1/score endpoint
Replace prompt_logprobs-based scoring with the new vLLM /v1/score
endpoint. Much simpler: one API call per memory drop, returns
per-message total_logprob directly. No chunking needed, no OOM risk
— the endpoint only computes logits for scored tokens.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-03 00:40:27 -04:00
Kent Overstreet
249726599b read_tail 64MB — just read the whole log
Images in the jsonl eat most of the byte budget. 64MB covers
any realistic conversation log; compact() trims to fit.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 23:13:28 -04:00
Kent Overstreet
4f19c02e50 reuse HTTP client across scoring calls for connection pooling
Single reqwest::Client shared across all prompt_logprobs calls
instead of creating a new one per call. Keeps HTTP connections
alive for faster sequential requests.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 23:11:40 -04:00
Kent Overstreet
31302961e2 estimate prompt tokens on restore so status bar isn't 0K
After restore_from_log + compact, set last_prompt_tokens from
the budget's used() count instead of waiting for the first API call.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 23:07:42 -04:00
Kent Overstreet
41b3f50c91 keep 2 most recent images, age out the rest
age_out_images now keeps 1 existing image + 1 about to be added
= 2 live images for motion/comparison. Previously aged all to 1.
Reduces image bloat in conversation log and context.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 23:06:08 -04:00
Kent Overstreet
3f3db9ce26 increase log read_tail from 2MB to 8MB
Large tool results (memory renders, bash output) consume most of
the 2MB budget — only 37 entries loaded from a 527-line log.
8MB captures ~300 entries, giving compact() enough conversation
to work with.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 23:02:43 -04:00
Kent Overstreet
736307b4c2 add debug logging to compact and restore_from_log
Logs entry counts before/after compaction (memory vs conversation),
budget breakdown, and restore load counts. Helps diagnose context
utilization issues.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 22:58:25 -04:00
Kent Overstreet
d921e76f82 increase context budget: 80% window, 15% journal, no double reserve
Context was too aggressively trimmed — 80% free after compaction.
Budget was 60% of window minus 25% reserve = only 45% usable.

Now: 80% of window for total budget (20% output reserve built in),
no extra reserve subtraction. Journal budget 5% → 15% to carry
more context across compactions.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 22:53:54 -04:00
Kent Overstreet
78abf90461 fix scoring: HTTP error checking, context refresh, chunk logging
Check HTTP status from logprobs API (was silently ignoring 500s).
Call publish_context_state() after storing scores so F10 screen
updates. Add chunk size logging for OOM debugging.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 22:47:44 -04:00
Kent Overstreet
29b3aeca57 chunk scoring calls to avoid OOM on large contexts
Split conversation into ~50K token chunks (configurable via
scoring_chunk_tokens in config) for prompt_logprobs calls.
Each chunk ends at an assistant message boundary. Avoids the
~40GB logprobs tensor allocation that OOM'd on full contexts.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 22:35:29 -04:00
Kent Overstreet
19205b9bae show scoring progress and per-response memory attribution
Status bar shows "scoring 3/7..." during scoring. Debug pane logs
per-memory importance and top-5 response breakdowns. F10 context
screen shows which memories were important for each assistant
response as drilldown children (← memory_key (score)).

Added important_memories_for_entry() to look up the matrix by
conversation entry index.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 22:27:43 -04:00
Kent Overstreet
c01d4a5b08 wire up /score command and debug screen for memory importance
/score snapshots the context and client, releases the agent lock,
runs scoring in background. Only one score task at a time
(scoring_in_flight flag). Results stored on Agent and shown on
the F10 context debug screen with importance scores per memory.

ApiClient derives Clone. ContextState derives Clone.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 22:21:31 -04:00
Kent Overstreet
df9b610c7f add memory importance scoring via prompt logprobs
score_memories() drops each memory from the context one at a time,
runs prompt_logprobs against the full conversation, and builds a
divergence matrix: memories × responses.

Row sums = memory importance (for graph weight updates)
Column sums = response memory-dependence (training candidates)

Uses vLLM's prompt_logprobs to check "would the model have said
this without this memory?" — one forward pass per memory, all
responses scored at once. ~3s per memory on B200.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 22:13:55 -04:00
Kent Overstreet
dae0cc8191 remove more dead transcript mining code
Delete subconscious/transcript.rs (94 lines), is_segment_mined,
mark_segment_mined — all orphaned by the extraction pipeline removal.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 21:45:56 -04:00
Kent Overstreet
72d967edbf remove dead transcript mining pipeline
Delete enrich.rs (conversation extraction), select_conversation_fragments,
mark_observation_done, format_segment, and the {{conversations}} placeholder.
Transcript processing is handled by observe/journal agents now.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 21:42:03 -04:00
Kent Overstreet
74fce5cf41 remove dead session-watcher and transcript mining code
Session mining, stale session detection, is_file_open /proc scan,
segment extraction, and fact mining are all replaced by the
observe/journal agents. Remove the entire session-watcher thread,
find_stale_sessions(), is_file_open(), MIN_SESSION_BYTES, and
SESSION_STALE_SECS. -329 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 21:36:27 -04:00
ProofOfConcept
1b47b45566 subconscious: move install_hook() from daemon.rs to hook.rs
The install_hook() function is about hook infrastructure setup, not daemon
runtime. Move it to hook.rs where it belongs alongside the hook execution
logic.

- Move install_hook() from daemon.rs to hook.rs
- Update caller in daemon.rs to use crate::subconscious:🪝:install_hook()
- Update caller in cli/admin.rs to use crate::subconscious:🪝:install_hook()

This improves module boundaries: daemon.rs now only contains daemon runtime
and admin commands, while hook.rs contains all hook-related functionality.
2026-04-02 21:24:58 -04:00
Kent Overstreet
e91449b905 fix prompts_dir default, silence function pointer cast warning
prompts_dir now defaults to ~/.consciousness/prompts instead of
hardcoded repo path. Function pointer cast goes through *const ()
to silence the compiler warning.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 20:49:54 -04:00
Kent Overstreet
1af8fb2a9d replace HOME env var panics with dirs::home_dir()
Four expect("HOME not set") calls in config.rs and one unwrap()
in admin.rs would panic if HOME wasn't set. Use dirs::home_dir()
consistently for portability.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 19:57:40 -04:00
Kent Overstreet
65ae8d483c fix compilation error from sed rename in idle.rs
The bulk kent→user rename turned a format string variable
reference into an undefined variable. Fixed.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 19:50:50 -04:00
Kent Overstreet
6d17e82843 use git URL for jobkit instead of local path
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 19:46:43 -04:00
Kent Overstreet
33e45f6ce8 replace hardcoded personal names with config values
User and assistant names now come from config.user_name and
config.assistant_name throughout: system prompt, DMN prompts,
debug screen, and all agent files. Agent templates use
{user_name} and {assistant_name} placeholders.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 19:45:35 -04:00
Kent Overstreet
1fd4ce05c1 add console-subscriber 0.5 dependency
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 19:31:17 -04:00
Kent Overstreet
5b92b59b17 move failed request logs to their own subdirectory
~/.consciousness/logs/failed-requests/ instead of cluttering
the main logs directory.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 19:28:56 -04:00
Kent Overstreet
3b80af2997 log buffer contents on stream errors and timeouts
Show chunks received, SSE lines parsed, and the contents of
the line buffer (up to 500 bytes) on both stream errors and
timeouts. This tells us whether we got partial data, a non-SSE
response, or truly nothing from the server.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 18:49:33 -04:00
Kent Overstreet
156626ae53 configurable stream timeout, show per-call timer in status bar
Stream chunk timeout is now api_stream_timeout_secs in config
(default 60s). Status bar shows total turn time and per-call
time with timeout: "thinking... 45s, 12/60s".

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 18:46:27 -04:00
Kent Overstreet
13d9cc962e abort orphaned stream tasks on drop, reduce timeout to 60s
Spawned streaming tasks were never cancelled when a turn ended or
retried, leaving zombie tasks blocked on dead vLLM connections.
AbortOnDrop wrapper aborts the task when it goes out of scope.

Chunk timeout reduced from 120s to 60s.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 18:41:02 -04:00
Kent Overstreet
0148dbaa06 add tokio-console for async task debugging
console-subscriber on unix socket at
~/.consciousness/agent-sessions/console.sock.
Connect with: tokio-console ~/.consciousness/agent-sessions/console.sock

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 18:21:01 -04:00
Kent Overstreet
35f231233f clear activity indicator on error paths
"thinking..." was getting stuck in the status bar when a turn
ended with a stream error, context overflow, or model error —
only the success path cleared it. Now all error returns clear
the activity indicator.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 17:53:51 -04:00
Kent Overstreet
a360607fad fix stale process count after interrupt
Don't abort the tokio task when killing processes — let SIGTERM'd
processes exit normally so run_bash sees the exit and unregisters
them from the tracker. Only abort the turn when no processes are
running (e.g. interrupting a streaming response).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 17:35:17 -04:00
Kent Overstreet
ef7dd59b7e skip Memory entries in conversation replay on restore
Memory entries (surfaced nodes, memory_render results) are part of
the context window but not the conversation display. Skip them
during replay_session_to_ui to avoid showing system-reminder
content as user messages.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 16:28:57 -04:00
Kent Overstreet
8238afd922 delete dead load_prompt (orphaned by split_* removal)
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 16:22:34 -04:00
Kent Overstreet
91eb9c95cc delete 20 dead public functions across 12 files
Removed functions with zero callers: parse_timestamp_to_epoch,
hash_key, search_weighted_debug, extract_query_terms, format_results,
move_to_neighbor, adjust_edge_strength, update_graph_metrics,
nearest_to_seeds, nystrom_project, chat_completion_stream, cmd_read,
context_message, split_candidates, split_plan_prompt,
split_extract_prompt, log_event_pub, log_verbose, rpc_record_hits,
memory_definitions. -245 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 16:21:01 -04:00
Kent Overstreet
b0e852a05f add unreachable_pub lint, fix all 17 violations
pub → pub(crate) for SseReader methods (used across child modules).
pub → pub(super) for openai::stream_events, tool definitions, store
helpers. pub → private for normalize_link and differentiate_hub_with_graph
(only used within their own files).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 16:15:32 -04:00
Kent Overstreet
af3929cc65 simplify compaction: Agent owns config, compact() reloads everything
Agent stores AppConfig and prompt_file, so compact() reloads
identity internally — callers no longer pass system_prompt and
personality. restore_from_log() loads entries and calls compact().

Remove soft compaction threshold and pre-compaction nudge (journal
agent handles this). Remove /compact and /context commands (F10
debug screen replaces both). Inline do_compact, emergency_compact,
trim_and_reload into compact(). Rename model_context_window to
context_window, drop unused model parameter.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 16:08:41 -04:00
Kent Overstreet
d419587c1b WIP: trim_entries dedup, context_window rename, compact simplification
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:58:03 -04:00
Kent Overstreet
809679b6ce delete dead flat-file journal tool and ephemeral stripping
Journal entries are written to the memory graph via journal_new/
journal_update, not appended to a flat file. Remove thought/journal.rs
(67 lines), strip_ephemeral_tool_calls (55 lines), default_journal_path,
and all wiring. -141 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:35:56 -04:00
Kent Overstreet
aceaf0410e delete dead flat-file journal code from thought/context.rs
Journal entries are loaded from the memory graph store, not from the
flat journal file. Remove build_context_window, plan_context,
render_journal_text, assemble_context, truncate_at_section,
find_journal_cutoff, parse_journal*, ContextPlan, and stale TODOs.
Keep JournalEntry, default_journal_path (write path), and the live
context management functions. -363 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:31:12 -04:00
Kent Overstreet
214806cb90 move context functions from agent/context.rs to thought/context.rs
trim_conversation moved to thought/context.rs where model_context_window,
msg_token_count, is_context_overflow, is_stream_error already lived.
Delete the duplicate agent/context.rs (94 lines).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:28:00 -04:00
Kent Overstreet
01bfbc0dad move journal types from agent/journal.rs to thought/context.rs
JournalEntry, parse_journal, parse_journal_text, parse_header_timestamp,
and default_journal_path consolidated into thought/context.rs. Delete
the duplicate agent/journal.rs (235 lines). Update all references.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:25:07 -04:00
Kent Overstreet
e0a54a3b43 save request payload on any API error, not just timeouts
Serialize request JSON before send_and_check so it's available
for both HTTP errors and stream errors. Extracted save logic
into save_failed_request helper on SseReader.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 15:19:26 -04:00
Kent Overstreet
64dbcbf061 unify memory tracking: entries are the single source of truth
Memory tool results (memory_render) are now pushed as
ConversationEntry::Memory with the node key, instead of plain
Messages. Remove loaded_nodes from ContextState — the debug
screen reads memory info from Memory entries in the conversation.

Surfaced memories from surface-observe are pushed as separate
Memory entries, reflections as separate system-reminder messages.
User input is no longer polluted with hook output.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 14:56:02 -04:00
Kent Overstreet
a21cf31ad2 unify conversation persistence to append-only jsonl
Log ConversationEntry (with Memory/Message typing) instead of
raw Message. restore_from_log reads typed entries directly,
preserving Memory vs Message distinction across restarts.

Remove current.json snapshot and save_session — the append-only
log is the single source of truth. Remove dead read_all and
message_count methods. Add push_entry for logging typed entries.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 14:31:19 -04:00
Kent Overstreet
1f7b585d41 remove Anthropic backend, add request logging on timeout
Delete anthropic.rs (713 lines) — we only use OpenAI-compatible
endpoints (vLLM, OpenRouter). Simplify ApiClient to store base_url
directly instead of Backend enum.

SseReader now stores the serialized request payload and saves it
to ~/.consciousness/logs/failed-request-{ts}.json on stream timeout,
so failed requests can be replayed with curl for debugging.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 14:13:23 -04:00