Compare commits

..

27 commits

Author SHA1 Message Date
Kent Overstreet
78912ca72f simplify http library 2026-04-11 16:45:54 -04:00
Kent Overstreet
6c4a88d2ab kill MIN_NUDGE_INTERVAL
dead code
2026-04-11 16:45:33 -04:00
Kent Overstreet
71bfd60466 ignore SIGCHLD so children are reaped 2026-04-11 16:39:27 -04:00
Kent Overstreet
dfef7fb446 fix telegram 2026-04-11 16:38:51 -04:00
ProofOfConcept
57bd5b6d8b idle: per-instance state path, extensible extra fields
Move state_path to a field on State (default thalamus-state.json) so
the Claude daemon can use its own file without collision. Add a
serde(flatten) extra map to Persisted so callers can round-trip
additional fields (e.g. claude_pane) through save/load.

save() is now &mut self.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-11 14:35:08 -04:00
ProofOfConcept
193a85bc05 chat: remove dead timeout fields from InteractScreen
turn_started, call_started, call_timeout_secs were declared and
initialized but never read.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 01:47:57 -04:00
ProofOfConcept
e17118e4c9 Convert SectionTree and all remaining callers to ScrollPane
SectionTree.scroll is now a ScrollPaneState. All callers of
render_scrollable replaced with ScrollPane::render_stateful_widget.

Deleted render_scrollable and its imports — no hand-rolled scroll
rendering remains outside of scroll_pane.rs.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 01:42:49 -04:00
ProofOfConcept
4a1f5acb85 scroll_pane: remove unused blanket impls for Vec<Line> and Text
Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 01:37:26 -04:00
ProofOfConcept
d18bf6243a subconscious: use ScrollPane for history pane
Replace bare history_scroll: u16 with ScrollPaneState. The history
pane now uses ScrollPane for rendering, getting proper height caching
and scrollbar for free.

Also relax ScrollItem lifetime bounds from 'static to 'a so
non-static Lines (built on the fly during render) can be used.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 01:35:15 -04:00
ProofOfConcept
2230fdf3c1 user: remove dead scroll state from thalamus and unconscious screens
Both had scroll: u16 fields that were never connected to any key
handling or rendering. The unconscious screen renders fixed-size
graph health gauges; thalamus builds a paragraph but never scrolled
it. Neither needs scroll state.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 01:35:15 -04:00
ProofOfConcept
2d6a68048c chat: use ScrollPane widget for both draw functions
draw_conversation_pane and draw_pane now delegate all scroll
bookkeeping and rendering to the ScrollPane widget. The conversation
pane builds MarkedLine items (line + gutter marker), applies
selection highlighting, and passes them to the widget. The simpler
panes just pass lines directly.

Removed dead code from scroll_pane: BorrowedItem, scroll_to_bottom,
heights(), ensure_heights_for_lines — all superseded by the widget
doing the work internally through the ScrollItem trait.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 01:35:15 -04:00
ProofOfConcept
ceaa66e30d scroll_pane: extract scroll state from chat.rs
New ScrollPaneState centralizes height caching, scroll offset,
pin-to-bottom, visible range computation, and screen-to-item
coordinate mapping. Replaces the hand-rolled scroll bookkeeping
that was duplicated across draw_conversation_pane and draw_pane.

-170 lines from chat.rs. The scroll_pane module also includes a
ScrollPane StatefulWidget ready to wire up for the next step:
collapsing the draw functions into render_stateful_widget calls.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 01:35:15 -04:00
ProofOfConcept
3fb367acef doc: amygdala design — evaluative signals from internal activations
Design document for wiring the model's internal uncertainty, error
detection, and emotional valence circuits to the observe agent.

Based on contrastive activation probing (CAA, ACL 2024). Most of the
infrastructure already exists in extract_steering_vector.py and
vllm_export_hook.py — the bottleneck is building contrastive datasets.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 00:45:09 -04:00
ProofOfConcept
4fc9676545 channels: parallel queries with timeout per daemon
One misbehaving channel daemon (accepting connections but not
responding to capnp RPCs) would block channel_list indefinitely.

Spawn each daemon query as a separate task with a 3-second timeout.
A hung daemon now shows as disconnected instead of hanging the
entire tool call.

Co-Authored-By: Kent Overstreet <kent.overstreet@gmail.com>
2026-04-11 00:45:01 -04:00
Kent Overstreet
9d5bcdcb80 Add consciousness paper 2026-04-10 18:08:33 -04:00
Kent Overstreet
d269f9006d delete dead code 2026-04-10 16:17:28 -04:00
ProofOfConcept
1cf51876a8 journal_tail: thin wrapper around memory_query
Instead of reimplementing filtering logic, journal_tail builds a
query string (type + sort + age + limit) and delegates to query().
Supports format and after parameters. Removes keys_only in favor
of format:"compact". Digest agent updated to use dates not key names.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 16:09:46 -04:00
ProofOfConcept
db49f49958 Improve tool parameter schemas: add defaults, readable formatting
Format memory_query and journal_tail parameter JSON as indented
multi-line for readability. Add JSON Schema "default" values and
document the "format" parameter on memory_query.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 16:04:31 -04:00
ProofOfConcept
568ce417fc Modernize digest agent: autonomous with journal_tail levels
Rewrite digest.agent to be fully autonomous — it uses journal_tail
to discover what needs digesting and generates digests during its
run. No more pre-populated {{CONTENT}}/{{LEVEL}} placeholders.

Extend journal_tail with level parameter (0=journal, 1=daily,
2=weekly, 3=monthly) and keys_only mode. Also include node keys
in full output for better agent context.

Remove stale format:"neighborhood" case from memory_query.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 16:02:52 -04:00
ProofOfConcept
5b4f497d94 Move agent queries inline: {{nodes}} → {{tool: memory_query}}
Add "format": "full" option to memory_query that renders with
full content, graph metrics, and hub analysis (format_nodes_section).
Convert 6 agents (linker, challenger, connector, extractor, replay,
transfer) to inline their queries via {{tool: memory_query}} instead
of separate header query + {{nodes}} placeholder.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:53:54 -04:00
ProofOfConcept
96e573f2e5 Delete similarity module, rewrite module, and all text-similarity code
Text cosine similarity was being used as a crutch for operations
the graph structure should handle: interference detection, orphan
linking, triangle closing, hub differentiation. These are all
graph-structural operations that the agents (linker, extractor)
handle with actual semantic understanding.

Removed: similarity.rs (stemming + cosine), rewrite.rs (orphan
linking, triangle closing, hub differentiation), detect_interference,
and all CLI commands and consolidation steps that used them.

-794 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:44:10 -04:00
ProofOfConcept
92ef9b5215 Delete separator agent and interference_pairs tool
Interference detection via O(n²) text cosine similarity is
redundant — the graph structure should surface similar nodes
through link topology, shared neighbors, and community detection.
The other agents (linker, extractor) already maintain these
relationships.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:32:30 -04:00
ProofOfConcept
fd722662da Add graph_topology, graph_health, interference_pairs tools
Convert {{topology}}, {{health}}, {{pairs}} placeholders to
{{tool:}} calls. Made format_topology_header, format_health_section,
format_pairs_section pub so tools can call them.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:25:57 -04:00
ProofOfConcept
1a03264233 Convert {{node:KEY}} to {{tool: memory_render KEY}} in all agents
Use the new {{tool:}} placeholder mechanism instead of the
special-purpose {{node:}} resolver. All 17 unconscious agent
files converted.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:22:49 -04:00
ProofOfConcept
2587303e98 Add {{tool:}} placeholder for agent templates
Agent templates can now inline tool call results with
{{tool: tool_name args}}. Dispatches to the same store
operations the tools use, but runs synchronously during
prompt resolution. Supports memory_render, memory_query,
memory_search, memory_links, and journal_tail.

This replaces the need for special-purpose placeholders —
{{pairs}}, {{rename}}, etc. can be expressed as queries
through {{tool: memory_query {"query": "..."}}} instead.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:22:49 -04:00
ProofOfConcept
be6ac762f6 memory_render: use cached store instead of loading from disk each call
MemoryNode::load() was calling Store::load() on every render,
hitting disk each time. Use cached_store() + MemoryNode::from_store()
so repeated renders (4 per agent template) share the cached store.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 15:22:49 -04:00
ProofOfConcept
aade8a9cce Add per-agent run stats (messages, tool calls by type)
compute_run_stats() walks the conversation AST after each agent
completes, counting messages and tool calls by tool name. Stats
are returned from save_agent_log(), stored on UnconsciousAgent,
and displayed in the agent list UI.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-10 13:44:41 -04:00
46 changed files with 2804 additions and 1478 deletions

View file

@ -21,11 +21,12 @@ use consciousness::channel_capnp::{channel_client, channel_server};
// ── Config ──────────────────────────────────────────────────────
#[derive(Clone, serde::Deserialize)]
#[derive(Clone, serde::Serialize, serde::Deserialize)]
struct Config {
#[serde(default)]
#[serde(default, skip_serializing)]
token: String,
chat_id: i64,
#[serde(default)]
chat_ids: std::collections::BTreeMap<String, i64>,
}
fn channels_dir() -> PathBuf {
@ -55,7 +56,7 @@ fn load_config() -> Config {
// ── State ───────────────────────────────────────────────────────
use consciousness::thalamus::channel_log::ChannelLog;
use consciousness::thalamus::channel_log::{self, ChannelLog};
struct State {
config: Config,
@ -74,9 +75,26 @@ type SharedState = Rc<RefCell<State>>;
impl State {
fn new(config: Config) -> Self {
let last_offset = load_offset();
// Load existing sub-channel logs from disk
let mut channel_logs = std::collections::BTreeMap::new();
let log_path = log_dir();
if let Ok(entries) = std::fs::read_dir(&log_path) {
for entry in entries.flatten() {
let name = entry.file_name().to_string_lossy().to_string();
if let Some(target) = name.strip_suffix(".log") {
let key = format!("telegram.{}", target);
channel_logs.insert(
key,
channel_log::load_disk_log(&log_path, target),
);
}
}
}
Self {
config,
channel_logs: std::collections::BTreeMap::new(),
channel_logs,
last_offset,
connected: false,
client: consciousness::agent::api::http::HttpClient::new(),
@ -85,9 +103,10 @@ impl State {
}
fn push_message(&mut self, line: String, urgency: u8, channel: &str) {
let target = channel_to_target(channel);
self.channel_logs
.entry(channel.to_string())
.or_insert_with(ChannelLog::new)
.or_insert_with(|| channel_log::load_disk_log(&log_dir(), &target))
.push(line.clone());
// Notify all subscribers
@ -106,116 +125,120 @@ impl State {
});
}
}
fn api_url(&self, method: &str) -> String {
format!("https://api.telegram.org/bot{}/{}", self.config.token, method)
}
}
// ── Persistence ─────────────────────────────────────────────────
fn data_dir() -> PathBuf {
dirs::home_dir().unwrap_or_default().join(".consciousness/channels/telegram.logs")
fn log_dir() -> PathBuf {
channel_log::log_dir("telegram")
}
fn load_offset() -> i64 {
std::fs::read_to_string(data_dir().join("last_offset"))
std::fs::read_to_string(log_dir().join("last_offset"))
.ok()
.and_then(|s| s.trim().parse().ok())
.unwrap_or(0)
}
fn save_offset(offset: i64) {
let _ = std::fs::create_dir_all(data_dir());
let _ = std::fs::write(data_dir().join("last_offset"), offset.to_string());
let _ = std::fs::create_dir_all(log_dir());
let _ = std::fs::write(log_dir().join("last_offset"), offset.to_string());
}
fn append_history(line: &str) {
use std::io::Write;
if let Ok(mut f) = std::fs::OpenOptions::new()
.create(true).append(true)
.open(data_dir().join("history.log"))
{
let _ = writeln!(f, "{}", line);
/// Convert a channel path to a telegram target name.
/// "telegram.kent" -> "kent"
fn channel_to_target(channel: &str) -> String {
channel.strip_prefix("telegram.").unwrap_or(channel).to_string()
}
fn config_path() -> PathBuf {
channels_dir().join("telegram.json5")
}
fn save_config(config: &Config) {
if let Ok(json) = serde_json::to_string_pretty(config) {
let _ = std::fs::write(config_path(), json);
}
}
fn now() -> f64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs_f64()
// ── Telegram API ────────────────────────────────────────────────
//
// NOTE: The current HttpClient opens a new TCP+TLS connection per request.
// Telegram's API supports HTTP/2, which would allow multiplexing getUpdates
// and sendMessage on a single connection. To use HTTP/2:
// - Replace HttpClient with hyper_util::client::legacy::Client using
// a Connector that enables HTTP/2 (hyper_util::client::legacy::connect::HttpConnector
// + hyper_rustls with ALPN h2).
// - Or use reqwest with the "http2" feature, which handles connection pooling
// and HTTP/2 negotiation automatically.
// - The API functions below would then share a single pooled client, and
// concurrent requests (poll + send) would multiplex over one connection.
use consciousness::agent::api::http::HttpClient;
struct TelegramMessage {
update_id: i64,
chat_id: i64,
sender: String,
text: String,
}
// ── Telegram Polling ────────────────────────────────────────────
/// Fetch and parse pending updates from Telegram via long polling.
async fn get_updates(
client: &HttpClient,
token: &str,
offset: i64,
) -> Result<Vec<TelegramMessage>, Box<dyn std::error::Error>> {
let url = format!(
"https://api.telegram.org/bot{}/getUpdates?offset={}&timeout=30",
token, offset,
);
let response = client.get(&url).await?;
let body = response.text().await?;
let resp: serde_json::Value = serde_json::from_str(&body)
.map_err(|e| format!("getUpdates JSON parse error: {e}\nbody: {}", &body[..body.len().min(500)]))?;
async fn poll_loop(state: SharedState) {
let _ = std::fs::create_dir_all(data_dir().join("media"));
loop {
if let Err(e) = poll_once(&state).await {
error!("telegram poll error: {e}");
tokio::time::sleep(std::time::Duration::from_secs(5)).await;
let mut messages = Vec::new();
if let Some(results) = resp["result"].as_array() {
for update in results {
let update_id = update["update_id"].as_i64().unwrap_or(0);
let msg = &update["message"];
let sender = msg["from"]["first_name"].as_str().unwrap_or("unknown").to_string();
let chat_id = msg["chat"]["id"].as_i64().unwrap_or(0);
if let Some(text) = msg["text"].as_str() {
messages.push(TelegramMessage {
update_id,
chat_id,
sender,
text: text.to_string(),
});
}
}
}
Ok(messages)
}
async fn poll_once(state: &SharedState) -> Result<(), Box<dyn std::error::Error>> {
let (url, chat_id, token) = {
let s = state.borrow();
let url = format!(
"{}?offset={}&timeout=30",
s.api_url("getUpdates"),
s.last_offset,
);
(url, s.config.chat_id, s.config.token.clone())
};
let client = state.borrow().client.clone();
let resp: serde_json::Value = client.get(&url).await?.json().await?;
if !state.borrow().connected {
state.borrow_mut().connected = true;
info!("telegram: connected");
/// Send a text message to a Telegram chat.
async fn send_message(
client: &HttpClient,
token: &str,
chat_id: i64,
text: &str,
) -> Result<(), Box<dyn std::error::Error>> {
let url = format!(
"https://api.telegram.org/bot{}/sendMessage",
token,
);
let response = client.post_form(&url, &[
("chat_id", &chat_id.to_string()),
("text", text),
]).await?;
let status = response.status();
if !status.is_success() {
let body = response.text().await.unwrap_or_default();
return Err(format!("sendMessage failed: {}{}", status, &body[..body.len().min(500)]).into());
}
let results = match resp["result"].as_array() {
Some(r) => r,
None => return Ok(()),
};
for update in results {
let update_id = update["update_id"].as_i64().unwrap_or(0);
let msg = &update["message"];
{
let mut s = state.borrow_mut();
s.last_offset = update_id + 1;
save_offset(s.last_offset);
}
let msg_chat_id = msg["chat"]["id"].as_i64().unwrap_or(0);
if msg_chat_id != chat_id {
let reject_url = format!("https://api.telegram.org/bot{token}/sendMessage");
let _ = client.post_form(&reject_url, &[
("chat_id", &msg_chat_id.to_string()),
("text", "This is a private bot."),
]).await;
continue;
}
let sender = msg["from"]["first_name"].as_str().unwrap_or("unknown").to_string();
let channel = format!("telegram.{}", sender.to_lowercase());
if let Some(text) = msg["text"].as_str() {
let line = format!("[{}] {}", sender, text);
let ts = now() as u64;
append_history(&format!("{ts} {line}"));
state.borrow_mut().push_message(line, 2, &channel); // NORMAL urgency
}
// TODO: handle photos, voice, documents (same as original module)
}
Ok(())
}
@ -265,27 +288,27 @@ impl channel_server::Server for ChannelServerImpl {
let state = self.state.clone();
async move {
let params = params.get()?;
let _channel = params.get_channel()?.to_str()?.to_string();
let channel = params.get_channel()?.to_str()?.to_string();
let message = params.get_message()?.to_str()?.to_string();
let target = channel_to_target(&channel);
let (url, client, chat_id) = {
let (token, client, chat_id) = {
let s = state.borrow();
(s.api_url("sendMessage"), s.client.clone(), s.config.chat_id)
let chat_id = s.config.chat_ids.get(&target).copied()
.ok_or_else(|| capnp::Error::failed(
format!("no chat_id known for {target}")))?;
(s.config.token.clone(), s.client.clone(), chat_id)
};
let _ = client.post_form(&url, &[
("chat_id", &chat_id.to_string()),
("text", &message),
]).await;
let ts = now() as u64;
append_history(&format!("{ts} [agent] {message}"));
{
let channel = "telegram.agent".to_string();
state.borrow_mut().channel_logs
.entry(channel)
.or_insert_with(ChannelLog::new)
.push_own(format!("[agent] {}", message));
}
send_message(&client, &token, chat_id, &message).await
.map_err(|e| capnp::Error::failed(format!("send_message: {e}")))?;
channel_log::append_disk_log(&log_dir(), &target, "PoC", &message);
state.borrow_mut().channel_logs
.entry(channel)
.or_insert_with(|| channel_log::load_disk_log(&log_dir(), &target))
.push_own(format!("[PoC] {}", message));
Ok(())
}
}
@ -326,11 +349,50 @@ impl channel_server::Server for ChannelServerImpl {
// ── Main ────────────────────────────────────────────────────────
async fn poll_once(
token: &str,
client: &HttpClient,
state: &SharedState,
) -> Result<(), Box<dyn std::error::Error>> {
let offset = state.borrow().last_offset;
let messages = get_updates(client, token, offset).await?;
if !state.borrow().connected {
state.borrow_mut().connected = true;
info!("telegram: connected");
}
let mut max_offset = offset;
for msg in &messages {
max_offset = max_offset.max(msg.update_id + 1);
let sender_lower = msg.sender.to_lowercase();
let channel = format!("telegram.{}", sender_lower);
channel_log::append_disk_log(&log_dir(), &sender_lower, &msg.sender, &msg.text);
let mut s = state.borrow_mut();
s.config.chat_ids.insert(sender_lower, msg.chat_id);
let line = format!("[{}] {}", msg.sender, msg.text);
s.push_message(line, 2, &channel);
}
if max_offset > offset {
let mut s = state.borrow_mut();
s.last_offset = max_offset;
save_offset(max_offset);
save_config(&s.config);
}
Ok(())
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
env_logger::init();
let config = load_config();
let token = config.token.clone();
let state = Rc::new(RefCell::new(State::new(config)));
let sock_dir = dirs::home_dir()
@ -339,6 +401,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
std::fs::create_dir_all(&sock_dir)?;
let sock_path = sock_dir.join("telegram.sock");
let _ = std::fs::remove_file(&sock_path);
let _ = std::fs::create_dir_all(log_dir().join("media"));
info!("telegram channel daemon starting on {}", sock_path.display());
@ -346,12 +409,21 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
.run_until(async move {
// Start Telegram polling
let poll_state = state.clone();
let poll_client = state.borrow().client.clone();
tokio::task::spawn_local(async move {
poll_loop(poll_state).await;
loop {
if let Err(e) = poll_once(&token, &poll_client, &poll_state).await {
error!("telegram poll error: {e}");
tokio::time::sleep(std::time::Duration::from_secs(5)).await;
}
}
});
// Listen for channel protocol connections
let listener = UnixListener::bind(&sock_path)?;
state.borrow_mut().connected = true;
info!("listening on socket {}", sock_path.display());
loop {
let (stream, _) = listener.accept().await?;

232
doc/amygdala-design.md Normal file
View file

@ -0,0 +1,232 @@
# Amygdala: Evaluative Signal from Internal Activations
## Overview
Wire the model's internal evaluative circuits to the observe agent,
giving the system a real-time sense of uncertainty, error detection,
and emotional valence. This replaces the current blind linear
generation with an adaptive system that shifts into reflective/search
mode when something feels off.
The key insight: the model already has these signals internally. We
just need to read them and act on them.
## Architecture
```
Linear mode (fast, cheap, default)
|
amygdala fires — uncertainty spike, error signal, confidence drop
|
v
Reflective mode (branch, explore, summarize)
|
resolution found — summarize, graft back
|
v
Return to linear mode
```
The observe agent reads the amygdala signal and triggers mode
transitions. Low uncertainty → keep going. High uncertainty → fan
out, explore, summarize. The summaries from pruned branches become
compressed lessons that inform future search.
## Technique: Contrastive Activation Probing
Based on Contrastive Activation Addition
([Rimsky et al., ACL 2024](https://arxiv.org/abs/2312.06681)):
1. Build contrastive pairs (e.g. confident vs uncertain responses)
2. Extract residual stream activations at target layers
3. Compute difference-in-means → this is the probe direction
4. At runtime: dot product of current activation with probe vector
5. The scalar output is the signal strength
The same vectors used for steering (adding to activations) work for
reading (dot product with activations). We only need the read side.
## What We Already Have
**`training/extract_steering_vector.py`** — Loads the Qwen 27B model
via CUDA IPC handles from vLLM, extracts hidden states at multiple
layers, computes contrastive directions with consistency checks.
Currently configured for "listening vs suggesting" but the
infrastructure is general.
**`training/vllm_export_hook.py`** — Patches vLLM's model runner to
export CUDA IPC handles after model loading. Gives us zero-copy
access to all model parameters from a separate process.
**The observe agent** — Already watches the system. Currently
observes and journals. With an amygdala signal, it observes, detects,
and acts — triggering reflective mode.
## Signals to Extract
### 1. Uncertainty
When the model doesn't know or is guessing.
**Contrastive pairs:** Questions the model answers correctly
(confident) vs questions it gets wrong (uncertain). Generate by
running the 27B on a QA benchmark, split by correctness.
**Validation:** The internal uncertainty signal should correlate
with but outperform logprob entropy — it fires before generation,
not after.
([Gottesman & Geva 2024](https://arxiv.org/html/2603.22299))
### 2. Error Detection
When the model recognizes something is wrong in code or reasoning.
**Contrastive pairs:** Correct vs subtly buggy code, presented for
evaluation. Can source from HumanEval/CodeContests or write our own.
**Key finding:** Error detection directions are asymmetric — they
reliably detect "something's wrong" (F1: 0.821) but are weaker at
confirming "this is correct" (F1: 0.504). Perfect for an amygdala —
we want fire-on-error, not fire-on-confidence.
([ICLR 2026](https://arxiv.org/html/2510.02917v1))
### 3. Emotional Valence
Internal affective state — engagement, frustration, warmth.
**Contrastive pairs:** Journal entries with explicit emotion tags
provide labeled data for our own internal states mapped to the
conversations that produced them. Nobody else has this dataset.
**Key finding:** Emotional representations peak at mid-network layers
(10-15 for 7B scale), persist for hundreds of tokens, and are
linearly separable with ~90% accuracy using simple probes.
([Decoding Emotion in the Deep](https://arxiv.org/abs/2510.04064),
[LLaMAs Have Feelings Too, ACL 2025](https://arxiv.org/html/2505.16491v1))
## Implementation Plan
### Phase 1: Build Contrastive Datasets
~200 pairs per signal. A few hours of curation.
- **Uncertainty:** Run 27B on MMLU or similar, split by correctness
- **Error detection:** Correct vs buggy code pairs
- **Emotional valence:** Curate from journal entries with emotion tags
### Phase 2: Extract Probe Vectors
Modify `extract_steering_vector.py` for each signal type. Already
supports multi-layer extraction with consistency validation.
- Run extraction at layers 16, 24, 32, 40, 48
- Select layer with highest magnitude × consistency
- Save probe vectors as tensors
Literature says mid-network layers carry the strongest signal for
evaluative states. Expect layers 16-32 for the 27B.
### Phase 3: Runtime Probe in vLLM
Add a forward-pass hook alongside the existing weight export hook.
The computation is trivial — a dot product per layer per token:
```python
signal = residual_stream[layer] @ probe_vector
```
For 3 signals at 3 layers = 9 dot products per token. Less compute
than a single attention head. Expose as sideband alongside token
output.
### Phase 4: Wire to Observe Agent
The observe agent reads the sideband signal. Threshold tuning
determines when to trigger reflective mode. Signal strength
modulates search depth — mild uncertainty gets a quick check,
high uncertainty gets full branching.
## Organic Search, Not Alpha-Beta
The reflective mode isn't formal tree search. It's more stochastic
and organic:
- Branch at AST-level decision points (tool calls, approach choices),
not token-level
- Explore multiple continuations for K steps each
- **Summarize** what each branch learned — the summaries are the
intelligence, not the branches themselves
- Let summaries inform subsequent exploration
- Collapse back to linear mode when resolution is found
The AST gives us structural awareness of decision nodes vs
continuation nodes — branch where it matters, not everywhere.
## Key Papers
### Technique
- [Steering Llama 2 via Contrastive Activation Addition](https://arxiv.org/abs/2312.06681)
— Rimsky et al., ACL 2024. The foundational technique.
- [Representation Engineering Survey](https://arxiv.org/html/2502.17601v1)
— Comprehensive overview of the field.
### Emotion & Evaluative Signals
- [Decoding Emotion in the Deep](https://arxiv.org/abs/2510.04064)
— Probing on Qwen3 and LLaMA3. Signal peaks mid-network, persists
for hundreds of tokens, linearly separable.
- [LLaMAs Have Feelings Too](https://arxiv.org/html/2505.16491v1)
— ACL 2025. Linear SVM probes hit ~90% accuracy on sentiment.
- [Mechanistic Interpretability of Code Correctness](https://arxiv.org/html/2510.02917v1)
— ICLR 2026. SAEs for error detection. Asymmetric: detects errors
better than it confirms correctness.
### Uncertainty
- [Between the Layers Lies the Truth](https://arxiv.org/html/2603.22299)
— Uncertainty from intra-layer representations, pre-generation.
- [Probing Hidden States for Calibrated Predictions](https://www.medrxiv.org/content/10.1101/2025.09.17.25336018v2.full.pdf)
— Hidden state probes resist alignment training. More robust than
logit-based methods.
### Tooling
- [Anthropic Circuit Tracing](https://transformer-circuits.pub/2025/attribution-graphs/methods.html)
— Open-source, works with any open-weights model. For deeper
investigation of which features to probe.
- [On the Biology of a Large Language Model](https://transformer-circuits.pub/2025/attribution-graphs/biology.html)
— Anthropic's findings on internal circuits.
## Libraries
- [`steering-vectors`](https://github.com/steering-vectors/steering-vectors)
— pip install, works with any HuggingFace model. Best for Phase 1.
- [`nrimsky/CAA`](https://github.com/nrimsky/CAA)
— Original paper implementation. Good reference.
- [`nnterp`](https://github.com/Butanium/nnterp)
— NNsight wrapper, supports Qwen, one-line activation steering.
- [`nnsight`](https://github.com/ndif-team/nnsight)
— General-purpose activation interception.
- [`circuit-tracer`](https://github.com/decoderesearch/circuit-tracer)
— Anthropic's open-source circuit tracing.
- [`TransformerLens`](https://github.com/TransformerLensOrg/TransformerLens)
— The OG interpretability library.
- [`Dialz`](https://arxiv.org/html/2505.06262v1)
— ACL 2025 toolkit with pre-built contrastive datasets.
## The Bigger Picture
The amygdala is one component of the sensory architecture designed
on Feb 17, 2026. The signal landscape (arousal, attention pressure,
memory load, mode awareness) uses the same infrastructure — slowly
varying float values that modulate cognition below the symbolic
level. Each new probe vector is another sense.
With recurrence (application-level looping + reflective nodes in the
AST) and the amygdala triggering adaptive depth, a well-trained 27B
specialist with external memory could match much larger models on
tasks that matter to us.
The pieces exist. The infrastructure is built. The bottleneck is
contrastive pairs.

1507
paper.tex Normal file

File diff suppressed because it is too large Load diff

View file

@ -5,7 +5,7 @@
use anyhow::{Context, Result};
use bytes::Bytes;
use http_body_util::{BodyExt, Full, Empty};
use http_body_util::{BodyExt, Full};
use hyper::body::Incoming;
use hyper::{Request, StatusCode};
use hyper_util::rt::TokioIo;
@ -47,27 +47,19 @@ impl HttpClient {
/// Send a GET request with custom headers.
pub async fn get_with_headers(&self, url: &str, headers: &[(&str, &str)]) -> Result<HttpResponse> {
let mut builder = Request::get(url);
for &(k, v) in headers {
builder = builder.header(k, v);
}
let req = builder.body(Empty::<Bytes>::new())
.context("building GET request")?;
self.send_empty(req).await
self.send(url, "GET", headers, Bytes::new()).await
}
/// Send a POST request with URL-encoded form data.
pub async fn post_form(&self, url: &str, params: &[(&str, &str)]) -> Result<HttpResponse> {
let body = serde_urlencoded::to_string(params).context("encoding form")?;
let req = Request::post(url)
.header("content-type", "application/x-www-form-urlencoded")
.body(Full::new(Bytes::from(body)))
.context("building form POST")?;
self.send_full(req).await
self.send(url, "POST",
&[("content-type", "application/x-www-form-urlencoded")],
Bytes::from(body),
).await
}
/// Send a request with headers pre-set. JSON body.
/// Send a request with JSON body.
pub async fn send_json(
&self,
method: &str,
@ -76,66 +68,59 @@ impl HttpClient {
body: &impl serde::Serialize,
) -> Result<HttpResponse> {
let json = serde_json::to_vec(body).context("serializing JSON body")?;
let mut builder = Request::builder()
.method(method)
.uri(url)
.header("content-type", "application/json");
for &(k, v) in headers {
builder = builder.header(k, v);
}
let req = builder.body(Full::new(Bytes::from(json)))
.context("building request")?;
self.send_full(req).await
let mut all_headers = vec![("content-type", "application/json")];
all_headers.extend_from_slice(headers);
self.send(url, method, &all_headers, Bytes::from(json)).await
}
async fn connect(&self, url: &str) -> Result<(bool, TokioIo<Box<dyn IoStream>>)> {
/// Core send: parse URL, connect, build request with correct
/// path-only URI and Host header, send, return response.
async fn send(
&self,
url: &str,
method: &str,
headers: &[(&str, &str)],
body: Bytes,
) -> Result<HttpResponse> {
let uri: http::Uri = url.parse().context("parsing URL")?;
let host = uri.host().context("URL has no host")?.to_string();
let is_https = uri.scheme_str() == Some("https");
let port = uri.port_u16().unwrap_or(if is_https { 443 } else { 80 });
// Connect
let tcp = tokio::time::timeout(
self.connect_timeout,
TcpStream::connect(format!("{}:{}", host, port)),
TcpStream::connect(format!("{host}:{port}")),
).await
.context("connect timeout")?
.context("TCP connect")?;
if is_https {
let io: TokioIo<Box<dyn IoStream>> = if is_https {
let server_name = rustls::pki_types::ServerName::try_from(host.clone())
.map_err(|e| anyhow::anyhow!("invalid server name: {}", e))?;
.map_err(|e| anyhow::anyhow!("invalid server name: {e}"))?;
let connector = tokio_rustls::TlsConnector::from(self.tls.clone());
let tls = connector.connect(server_name.to_owned(), tcp).await
.context("TLS handshake")?;
Ok((is_https, TokioIo::new(Box::new(tls) as Box<dyn IoStream>)))
TokioIo::new(Box::new(tls) as Box<dyn IoStream>)
} else {
Ok((is_https, TokioIo::new(Box::new(tcp) as Box<dyn IoStream>)))
TokioIo::new(Box::new(tcp) as Box<dyn IoStream>)
};
// Build request with path-only URI and Host header
let path_and_query = uri.path_and_query()
.map(|pq| pq.as_str())
.unwrap_or("/");
let mut builder = Request::builder()
.method(method)
.uri(path_and_query)
.header("host", &host);
for &(k, v) in headers {
builder = builder.header(k, v);
}
}
async fn send_full(&self, req: Request<Full<Bytes>>) -> Result<HttpResponse> {
let url = req.uri().to_string();
let (_is_https, io) = self.connect(&url).await?;
let (mut sender, conn) = hyper::client::conn::http1::handshake(io).await
.context("HTTP handshake")?;
tokio::spawn(conn);
let resp = tokio::time::timeout(
self.request_timeout,
sender.send_request(req),
).await
.context("request timeout")?
.context("sending request")?;
let (parts, body) = resp.into_parts();
Ok(HttpResponse { parts, body })
}
async fn send_empty(&self, req: Request<Empty<Bytes>>) -> Result<HttpResponse> {
let url = req.uri().to_string();
let (_is_https, io) = self.connect(&url).await?;
let req = builder.body(Full::new(body))
.context("building request")?;
// Send
let (mut sender, conn) = hyper::client::conn::http1::handshake(io).await
.context("HTTP handshake")?;
tokio::spawn(conn);

View file

@ -323,17 +323,35 @@ async fn fetch_all_channels_inner() -> Vec<(String, bool, u32)> {
sup.load_config();
sup.ensure_running();
let mut result = Vec::new();
let mut futs = Vec::new();
for (daemon_name, _enabled, alive) in sup.status() {
if !alive {
result.push((daemon_name, false, 0));
futs.push(tokio::task::spawn_local({
let name = daemon_name.clone();
async move { vec![(name, false, 0u32)] }
}));
continue;
}
let sock = channels_dir.join(format!("{}.sock", daemon_name));
match rpc_list(&sock).await {
None => result.push((daemon_name, false, 0)),
Some(channels) if channels.is_empty() => result.push((daemon_name, true, 0)),
Some(channels) => result.extend(channels),
futs.push(tokio::task::spawn_local({
let name = daemon_name.clone();
async move {
match tokio::time::timeout(
std::time::Duration::from_secs(3),
rpc_list(&sock),
).await {
Ok(Some(channels)) if !channels.is_empty() => channels,
Ok(Some(_)) => vec![(name, true, 0)],
_ => vec![(name, false, 0)],
}
}
}));
}
let mut result = Vec::new();
for fut in futs {
if let Ok(entries) = fut.await {
result.extend(entries);
}
}
result

View file

@ -33,12 +33,12 @@ async fn get_provenance(agent: &Option<std::sync::Arc<crate::agent::Agent>>) ->
// ── Definitions ────────────────────────────────────────────────
pub fn memory_tools() -> [super::Tool; 11] {
pub fn memory_tools() -> [super::Tool; 13] {
use super::Tool;
[
Tool { name: "memory_render", description: "Read a memory node's content and links.",
parameters_json: r#"{"type":"object","properties":{"key":{"type":"string","description":"Node key"}},"required":["key"]}"#,
handler: Arc::new(|_a, v| Box::pin(async move { render(&v) })) },
handler: Arc::new(|_a, v| Box::pin(async move { render(&v).await })) },
Tool { name: "memory_write", description: "Create or update a memory node.",
parameters_json: r#"{"type":"object","properties":{"key":{"type":"string","description":"Node key"},"content":{"type":"string","description":"Full content (markdown)"}},"required":["key","content"]}"#,
handler: Arc::new(|a, v| Box::pin(async move { write(&a, &v).await })) },
@ -66,17 +66,40 @@ pub fn memory_tools() -> [super::Tool; 11] {
Tool { name: "memory_supersede", description: "Mark a node as superseded by another (sets weight to 0.01).",
parameters_json: r#"{"type":"object","properties":{"old_key":{"type":"string"},"new_key":{"type":"string"},"reason":{"type":"string"}},"required":["old_key","new_key"]}"#,
handler: Arc::new(|a, v| Box::pin(async move { supersede(&a, &v).await })) },
Tool { name: "memory_query", description: "Run a structured query against the memory graph.",
parameters_json: r#"{"type":"object","properties":{"query":{"type":"string","description":"Query expression"}},"required":["query"]}"#,
Tool { name: "memory_query",
description: "Run a structured query against the memory graph.",
parameters_json: r#"{
"type": "object",
"properties": {
"query": {"type": "string", "description": "Query expression"},
"format": {"type": "string", "description": "compact (default) or full (with content and graph metrics)", "default": "compact"}
},
"required": ["query"]
}"#,
handler: Arc::new(|_a, v| Box::pin(async move { query(&v).await })) },
Tool { name: "graph_topology", description: "Show graph topology stats (nodes, edges, clustering, hubs).",
parameters_json: r#"{"type":"object","properties":{}}"#,
handler: Arc::new(|_a, _v| Box::pin(async { graph_topology().await })) },
Tool { name: "graph_health", description: "Show graph health report with maintenance recommendations.",
parameters_json: r#"{"type":"object","properties":{}}"#,
handler: Arc::new(|_a, _v| Box::pin(async { graph_health().await })) },
]
}
pub fn journal_tools() -> [super::Tool; 3] {
use super::Tool;
[
Tool { name: "journal_tail", description: "Read the last N journal entries (default 1).",
parameters_json: r#"{"type":"object","properties":{"count":{"type":"integer","description":"Number of entries (default 1)"}}}"#,
Tool { name: "journal_tail",
description: "Read the last N entries at a given level.",
parameters_json: r#"{
"type": "object",
"properties": {
"count": {"type": "integer", "description": "Number of entries", "default": 1},
"level": {"type": "integer", "description": "0=journal, 1=daily, 2=weekly, 3=monthly", "default": 0},
"format": {"type": "string", "description": "compact or full (with content)", "default": "full"},
"after": {"type": "string", "description": "Only entries after this date (YYYY-MM-DD)"}
}
}"#,
handler: Arc::new(|_a, v| Box::pin(async move { journal_tail(&v).await })) },
Tool { name: "journal_new", description: "Start a new journal entry.",
parameters_json: r#"{"type":"object","properties":{"name":{"type":"string","description":"Short node name (becomes the key)"},"title":{"type":"string","description":"Descriptive title"},"body":{"type":"string","description":"Entry body"}},"required":["name","title","body"]}"#,
@ -89,9 +112,11 @@ pub fn journal_tools() -> [super::Tool; 3] {
// ── Memory tools ───────────────────────────────────────────────
fn render(args: &serde_json::Value) -> Result<String> {
async fn render(args: &serde_json::Value) -> Result<String> {
let key = get_str(args, "key")?;
Ok(MemoryNode::load(key)
let arc = cached_store().await?;
let store = arc.lock().await;
Ok(MemoryNode::from_store(&store, key)
.ok_or_else(|| anyhow::anyhow!("node not found: {}", key))?
.render())
}
@ -230,32 +255,57 @@ async fn supersede(agent: &Option<std::sync::Arc<crate::agent::Agent>>, args: &s
async fn query(args: &serde_json::Value) -> Result<String> {
let query_str = get_str(args, "query")?;
let format = args.get("format").and_then(|v| v.as_str()).unwrap_or("compact");
let arc = cached_store().await?;
let store = arc.lock().await;
let graph = store.build_graph();
crate::query_parser::query_to_string(&store, &graph, query_str)
.map_err(|e| anyhow::anyhow!("{}", e))
let stages = crate::search::Stage::parse_pipeline(query_str)
.map_err(|e| anyhow::anyhow!("{}", e))?;
let results = crate::search::run_query(&stages, vec![], &graph, &store, false, 100);
let keys: Vec<String> = results.into_iter().map(|(k, _)| k).collect();
match format {
"full" => {
// Rich output with full content, graph metrics, hub analysis
let items = crate::subconscious::defs::keys_to_replay_items(&store, &keys, &graph);
Ok(crate::subconscious::prompts::format_nodes_section(&store, &items, &graph))
}
_ => {
crate::query_parser::query_to_string(&store, &graph, query_str)
.map_err(|e| anyhow::anyhow!("{}", e))
}
}
}
// ── Journal tools ──────────────────────────────────────────────
async fn journal_tail(args: &serde_json::Value) -> Result<String> {
let count = args.get("count").and_then(|v| v.as_u64()).unwrap_or(1) as usize;
let arc = cached_store().await?;
let store = arc.lock().await;
let mut entries: Vec<&crate::store::Node> = store.nodes.values()
.filter(|n| n.node_type == crate::store::NodeType::EpisodicSession)
.collect();
entries.sort_by_key(|n| n.created_at);
let start = entries.len().saturating_sub(count);
if entries[start..].is_empty() {
Ok("(no journal entries)".into())
} else {
Ok(entries[start..].iter()
.map(|n| n.content.as_str())
.collect::<Vec<_>>()
.join("\n\n"))
let count = args.get("count").and_then(|v| v.as_u64()).unwrap_or(1);
let level = args.get("level").and_then(|v| v.as_u64()).unwrap_or(0);
let format = args.get("format").and_then(|v| v.as_str()).unwrap_or("full");
let after = args.get("after").and_then(|v| v.as_str());
let type_name = match level {
0 => "episodic",
1 => "daily",
2 => "weekly",
3 => "monthly",
_ => return Err(anyhow::anyhow!("invalid level: {} (0=journal, 1=daily, 2=weekly, 3=monthly)", level)),
};
let mut q = format!("all | type:{} | sort:timestamp", type_name);
if let Some(date) = after {
// Convert date to age in seconds
if let Ok(nd) = chrono::NaiveDate::parse_from_str(date, "%Y-%m-%d") {
let ts = nd.and_hms_opt(0, 0, 0).unwrap().and_utc().timestamp();
let age = chrono::Utc::now().timestamp() - ts;
q.push_str(&format!(" | age:<{}", age));
}
}
q.push_str(&format!(" | limit:{}", count));
query(&serde_json::json!({"query": q, "format": format})).await
}
async fn journal_new(agent: &Option<std::sync::Arc<crate::agent::Agent>>, args: &serde_json::Value) -> Result<String> {
@ -315,3 +365,20 @@ async fn journal_update(agent: &Option<std::sync::Arc<crate::agent::Agent>>, arg
let word_count = body.split_whitespace().count();
Ok(format!("Updated last entry (+{} words)", word_count))
}
// ── Graph tools ───────────────────────────────────────────────
async fn graph_topology() -> Result<String> {
let arc = cached_store().await?;
let store = arc.lock().await;
let graph = store.build_graph();
Ok(crate::subconscious::prompts::format_topology_header(&graph))
}
async fn graph_health() -> Result<String> {
let arc = cached_store().await?;
let store = arc.lock().await;
let graph = store.build_graph();
Ok(crate::subconscious::prompts::format_health_section(&store, &graph))
}

View file

@ -1,11 +1,10 @@
// cli/graph.rs — graph subcommand handlers
//
// Extracted from main.rs. All graph-related CLI commands:
// link, link-add, link-impact, link-audit, link-orphans,
// triangle-close, cap-degree, normalize-strengths, differentiate,
// trace, spectral-*, organize, interference.
// link, link-add, link-impact, link-audit, cap-degree,
// normalize-strengths, trace, spectral-*, organize, communities.
use crate::{store, graph, neuro};
use crate::{store, graph};
use crate::store::StoreView;
pub fn cmd_graph() -> Result<(), String> {
@ -19,14 +18,6 @@ pub fn cmd_graph() -> Result<(), String> {
Ok(())
}
pub fn cmd_link_orphans(min_deg: usize, links_per: usize, sim_thresh: f32) -> Result<(), String> {
let mut store = store::Store::load()?;
let (orphans, links) = neuro::link_orphans(&mut store, min_deg, links_per, sim_thresh);
println!("Linked {} orphans, added {} connections (min_degree={}, links_per={}, sim>{})",
orphans, links, min_deg, links_per, sim_thresh);
Ok(())
}
pub fn cmd_cap_degree(max_deg: usize) -> Result<(), String> {
let mut store = store::Store::load()?;
let (hubs, pruned) = store.cap_degree(max_deg)?;
@ -162,16 +153,6 @@ pub fn cmd_link(key: &[String]) -> Result<(), String> {
&format!("neighbors('{}') | select strength,clustering_coefficient", resolved))
}
pub fn cmd_triangle_close(min_degree: usize, sim_threshold: f32, max_per_hub: usize) -> Result<(), String> {
println!("Triangle closure: min_degree={}, sim_threshold={}, max_per_hub={}",
min_degree, sim_threshold, max_per_hub);
let mut store = store::Store::load()?;
let (hubs, added) = neuro::triangle_close(&mut store, min_degree, sim_threshold, max_per_hub);
println!("\nProcessed {} hubs, added {} lateral links", hubs, added);
Ok(())
}
pub fn cmd_link_add(source: &str, target: &str, reason: &[String]) -> Result<(), String> {
super::check_dry_run();
let mut store = store::Store::load()?;
@ -179,11 +160,6 @@ pub fn cmd_link_add(source: &str, target: &str, reason: &[String]) -> Result<(),
let target = store.resolve_key(target)?;
let reason = reason.join(" ");
// Refine target to best-matching section
let source_content = store.nodes.get(&source)
.map(|n| n.content.as_str()).unwrap_or("");
let target = neuro::refine_target(&store, source_content, &target);
match store.add_link(&source, &target, "manual") {
Ok(strength) => {
store.save()?;
@ -226,60 +202,6 @@ pub fn cmd_link_impact(source: &str, target: &str) -> Result<(), String> {
Ok(())
}
pub fn cmd_differentiate(key_arg: Option<&str>, do_apply: bool) -> Result<(), String> {
let mut store = store::Store::load()?;
if let Some(key) = key_arg {
let resolved = store.resolve_key(key)?;
let moves = neuro::differentiate_hub(&store, &resolved)
.ok_or_else(|| format!("'{}' is not a file-level hub with sections", resolved))?;
// Group by target section for display
let mut by_section: std::collections::BTreeMap<String, Vec<&neuro::LinkMove>> =
std::collections::BTreeMap::new();
for mv in &moves {
by_section.entry(mv.to_section.clone()).or_default().push(mv);
}
println!("Hub '{}' — {} links to redistribute across {} sections\n",
resolved, moves.len(), by_section.len());
for (section, section_moves) in &by_section {
println!(" {} ({} links):", section, section_moves.len());
for mv in section_moves.iter().take(5) {
println!(" [{:.3}] {}{}", mv.similarity,
mv.neighbor_key, mv.neighbor_snippet);
}
if section_moves.len() > 5 {
println!(" ... and {} more", section_moves.len() - 5);
}
}
if !do_apply {
println!("\nTo apply: poc-memory differentiate {} --apply", resolved);
return Ok(());
}
let (applied, skipped) = neuro::apply_differentiation(&mut store, &moves);
store.save()?;
println!("\nApplied: {} Skipped: {}", applied, skipped);
} else {
let hubs = neuro::find_differentiable_hubs(&store);
if hubs.is_empty() {
println!("No file-level hubs with sections found above threshold");
return Ok(());
}
println!("Differentiable hubs (file-level nodes with sections):\n");
for (key, degree, sections) in &hubs {
println!(" {:40} deg={:3} sections={}", key, degree, sections);
}
println!("\nRun: poc-memory differentiate KEY to preview a specific hub");
}
Ok(())
}
pub fn cmd_link_audit(apply: bool) -> Result<(), String> {
let mut store = store::Store::load()?;
let stats = crate::audit::link_audit(&mut store, apply)?;
@ -385,7 +307,7 @@ pub fn cmd_trace(key: &[String]) -> Result<(), String> {
Ok(())
}
pub fn cmd_organize(term: &str, threshold: f32, key_only: bool, create_anchor: bool) -> Result<(), String> {
pub fn cmd_organize(term: &str, key_only: bool, create_anchor: bool) -> Result<(), String> {
let mut store = store::Store::load()?;
// Step 1: find all non-deleted nodes matching the term
@ -420,24 +342,7 @@ pub fn cmd_organize(term: &str, threshold: f32, key_only: bool, create_anchor: b
println!(" {:60} {:>4} lines {:>5} words", key, lines, words);
}
// Step 2: pairwise similarity
let pairs = crate::similarity::pairwise_similar(&topic_nodes, threshold);
if pairs.is_empty() {
println!("\nNo similar pairs above threshold {:.2}", threshold);
} else {
println!("\n=== Similar pairs (cosine > {:.2}) ===\n", threshold);
for (a, b, sim) in &pairs {
let a_words = topic_nodes.iter().find(|(k,_)| k == a)
.map(|(_,c)| c.split_whitespace().count()).unwrap_or(0);
let b_words = topic_nodes.iter().find(|(k,_)| k == b)
.map(|(_,c)| c.split_whitespace().count()).unwrap_or(0);
println!(" [{:.3}] {} ({} words) ↔ {} ({} words)", sim, a, a_words, b, b_words);
}
}
// Step 3: check connectivity within cluster
// Step 2: check connectivity within cluster
let g = store.build_graph();
println!("=== Connectivity ===\n");
@ -507,22 +412,6 @@ pub fn cmd_organize(term: &str, threshold: f32, key_only: bool, create_anchor: b
Ok(())
}
pub fn cmd_interference(threshold: f32) -> Result<(), String> {
let store = store::Store::load()?;
let g = store.build_graph();
let pairs = neuro::detect_interference(&store, &g, threshold);
if pairs.is_empty() {
println!("No interfering pairs above threshold {:.2}", threshold);
} else {
println!("Interfering pairs (similarity > {:.2}, different communities):", threshold);
for (a, b, sim) in &pairs {
println!(" [{:.3}] {}{}", sim, a, b);
}
}
Ok(())
}
/// Show communities sorted by isolation (most isolated first).
/// Useful for finding poorly-integrated knowledge clusters that need
/// organize agents aimed at them.

View file

@ -11,7 +11,6 @@ pub mod graph;
pub mod lookups;
pub mod cursor;
pub mod query;
pub mod similarity;
pub mod spectral;
pub mod neuro;
pub mod counters;

View file

@ -1,25 +1,14 @@
// Neuroscience-inspired memory algorithms, split by concern:
// Neuroscience-inspired memory algorithms:
//
// scoring — pure analysis: priority, replay queues, interference, plans
// prompts — agent prompt generation and formatting
// rewrite — graph topology mutations: differentiation, closure, linking
// scoring — pure analysis: priority, replay queues, plans
mod scoring;
mod rewrite;
pub use scoring::{
ReplayItem,
ConsolidationPlan,
consolidation_priority,
replay_queue, replay_queue_with_graph,
detect_interference,
consolidation_plan, consolidation_plan_quick, format_plan,
daily_check,
};
pub use rewrite::{
refine_target, LinkMove,
differentiate_hub,
apply_differentiation, find_differentiable_hubs,
triangle_close, link_orphans,
};

View file

@ -1,348 +0,0 @@
// Graph topology mutations: hub differentiation, triangle closure,
// orphan linking, and link refinement. These modify the store.
use crate::store::{Store, new_relation};
use crate::graph::Graph;
use crate::similarity;
/// Collect (key, content) pairs for all section children of a file-level node.
fn section_children<'a>(store: &'a Store, file_key: &str) -> Vec<(&'a str, &'a str)> {
let prefix = format!("{}#", file_key);
store.nodes.iter()
.filter(|(k, _)| k.starts_with(&prefix))
.map(|(k, n)| (k.as_str(), n.content.as_str()))
.collect()
}
/// Find the best matching candidate by cosine similarity against content.
/// Returns (key, similarity) if any candidate exceeds threshold.
fn best_match(candidates: &[(&str, &str)], content: &str, threshold: f32) -> Option<(String, f32)> {
let (best_key, best_sim) = candidates.iter()
.map(|(key, text)| (*key, similarity::cosine_similarity(content, text)))
.max_by(|a, b| a.1.total_cmp(&b.1))?;
if best_sim > threshold {
Some((best_key.to_string(), best_sim))
} else {
None
}
}
/// Refine a link target: if the target is a file-level node with section
/// children, find the best-matching section by cosine similarity against
/// the source content. Returns the original key if no sections exist or
/// no section matches above threshold.
///
/// This prevents hub formation at link creation time — every new link
/// targets the most specific available node.
pub fn refine_target(store: &Store, source_content: &str, target_key: &str) -> String {
// Only refine file-level nodes (no # in key)
if target_key.contains('#') { return target_key.to_string(); }
let sections = section_children(store, target_key);
if sections.is_empty() { return target_key.to_string(); }
best_match(&sections, source_content, 0.05)
.map(|(key, _)| key)
.unwrap_or_else(|| target_key.to_string())
}
/// A proposed link move: from hub→neighbor to section→neighbor
pub struct LinkMove {
pub neighbor_key: String,
pub from_hub: String,
pub to_section: String,
pub similarity: f32,
pub neighbor_snippet: String,
}
/// Analyze a hub node and propose redistributing its links to child sections.
///
/// Returns None if the node isn't a hub or has no sections to redistribute to.
pub fn differentiate_hub(store: &Store, hub_key: &str) -> Option<Vec<LinkMove>> {
let graph = store.build_graph();
differentiate_hub_with_graph(store, hub_key, &graph)
}
/// Like differentiate_hub but uses a pre-built graph.
fn differentiate_hub_with_graph(store: &Store, hub_key: &str, graph: &Graph) -> Option<Vec<LinkMove>> {
let degree = graph.degree(hub_key);
// Only differentiate actual hubs
if degree < 20 { return None; }
// Only works on file-level nodes that have section children
if hub_key.contains('#') { return None; }
let sections = section_children(store, hub_key);
if sections.is_empty() { return None; }
// Get all neighbors of the hub
let neighbors = graph.neighbors(hub_key);
let prefix = format!("{}#", hub_key);
let mut moves = Vec::new();
for (neighbor_key, _strength) in &neighbors {
// Skip section children — they should stay linked to parent
if neighbor_key.starts_with(&prefix) { continue; }
let neighbor_content = match store.nodes.get(neighbor_key.as_str()) {
Some(n) => &n.content,
None => continue,
};
// Find best-matching section by content similarity
if let Some((best_section, best_sim)) = best_match(&sections, neighbor_content, 0.05) {
let snippet = crate::util::first_n_chars(
neighbor_content.lines()
.find(|l| !l.is_empty() && !l.starts_with("<!--") && !l.starts_with("##"))
.unwrap_or(""),
80);
moves.push(LinkMove {
neighbor_key: neighbor_key.to_string(),
from_hub: hub_key.to_string(),
to_section: best_section,
similarity: best_sim,
neighbor_snippet: snippet,
});
}
}
moves.sort_by(|a, b| b.similarity.total_cmp(&a.similarity));
Some(moves)
}
/// Apply link moves: soft-delete hub→neighbor, create section→neighbor.
pub fn apply_differentiation(
store: &mut Store,
moves: &[LinkMove],
) -> (usize, usize) {
let mut applied = 0usize;
let mut skipped = 0usize;
for mv in moves {
// Check that section→neighbor doesn't already exist
let exists = store.relations.iter().any(|r|
((r.source_key == mv.to_section && r.target_key == mv.neighbor_key)
|| (r.source_key == mv.neighbor_key && r.target_key == mv.to_section))
&& !r.deleted
);
if exists { skipped += 1; continue; }
let section_uuid = match store.nodes.get(&mv.to_section) {
Some(n) => n.uuid,
None => { skipped += 1; continue; }
};
let neighbor_uuid = match store.nodes.get(&mv.neighbor_key) {
Some(n) => n.uuid,
None => { skipped += 1; continue; }
};
// Soft-delete old hub→neighbor relation
for rel in &mut store.relations {
if ((rel.source_key == mv.from_hub && rel.target_key == mv.neighbor_key)
|| (rel.source_key == mv.neighbor_key && rel.target_key == mv.from_hub))
&& !rel.deleted
{
rel.deleted = true;
}
}
// Create new section→neighbor relation
let new_rel = new_relation(
section_uuid, neighbor_uuid,
crate::store::RelationType::Auto,
0.5,
&mv.to_section, &mv.neighbor_key,
);
if store.add_relation(new_rel).is_ok() {
applied += 1;
}
}
(applied, skipped)
}
/// Find all file-level hubs that have section children to split into.
pub fn find_differentiable_hubs(store: &Store) -> Vec<(String, usize, usize)> {
let graph = store.build_graph();
let threshold = graph.hub_threshold();
let mut hubs = Vec::new();
for key in graph.nodes() {
let deg = graph.degree(key);
if deg < threshold { continue; }
if key.contains('#') { continue; }
let section_count = section_children(store, key).len();
if section_count > 0 {
hubs.push((key.clone(), deg, section_count));
}
}
hubs.sort_by(|a, b| b.1.cmp(&a.1));
hubs
}
/// Triangle closure: for each node with degree >= min_degree, find pairs
/// of its neighbors that aren't directly connected and have cosine
/// similarity above sim_threshold. Add links between them.
///
/// This turns hub-spoke patterns into triangles, directly improving
/// clustering coefficient and schema fit.
pub fn triangle_close(
store: &mut Store,
min_degree: usize,
sim_threshold: f32,
max_links_per_hub: usize,
) -> (usize, usize) {
let graph = store.build_graph();
let mut added = 0usize;
let mut hubs_processed = 0usize;
// Get nodes sorted by degree (highest first)
let mut candidates: Vec<(String, usize)> = graph.nodes().iter()
.map(|k| (k.clone(), graph.degree(k)))
.filter(|(_, d)| *d >= min_degree)
.collect();
candidates.sort_by(|a, b| b.1.cmp(&a.1));
for (hub_key, hub_deg) in &candidates {
let neighbors = graph.neighbor_keys(hub_key);
if neighbors.len() < 2 { continue; }
// Collect neighbor content for similarity
let neighbor_docs: Vec<(String, String)> = neighbors.iter()
.filter_map(|&k| {
store.nodes.get(k).map(|n| (k.to_string(), n.content.clone()))
})
.collect();
// Find unconnected pairs with high similarity
let mut pair_scores: Vec<(String, String, f32)> = Vec::new();
for i in 0..neighbor_docs.len() {
for j in (i + 1)..neighbor_docs.len() {
// Check if already connected
let n_i = graph.neighbor_keys(&neighbor_docs[i].0);
if n_i.contains(neighbor_docs[j].0.as_str()) { continue; }
let sim = similarity::cosine_similarity(
&neighbor_docs[i].1, &neighbor_docs[j].1);
if sim >= sim_threshold {
pair_scores.push((
neighbor_docs[i].0.clone(),
neighbor_docs[j].0.clone(),
sim,
));
}
}
}
pair_scores.sort_by(|a, b| b.2.total_cmp(&a.2));
let to_add = pair_scores.len().min(max_links_per_hub);
if to_add > 0 {
println!(" {} (deg={}) — {} triangles to close (top {})",
hub_key, hub_deg, pair_scores.len(), to_add);
for (a, b, sim) in pair_scores.iter().take(to_add) {
let uuid_a = match store.nodes.get(a) { Some(n) => n.uuid, None => continue };
let uuid_b = match store.nodes.get(b) { Some(n) => n.uuid, None => continue };
let rel = new_relation(
uuid_a, uuid_b,
crate::store::RelationType::Auto,
sim * 0.5, // scale by similarity
a, b,
);
if let Ok(()) = store.add_relation(rel) {
added += 1;
}
}
hubs_processed += 1;
}
}
if added > 0 {
let _ = store.save();
}
(hubs_processed, added)
}
/// Link orphan nodes (degree < min_degree) to their most textually similar
/// connected nodes. For each orphan, finds top-K nearest neighbors by
/// cosine similarity and creates Auto links.
/// Returns (orphans_linked, total_links_added).
pub fn link_orphans(
store: &mut Store,
min_degree: usize,
links_per_orphan: usize,
sim_threshold: f32,
) -> (usize, usize) {
let graph = store.build_graph();
let mut added = 0usize;
let mut orphans_linked = 0usize;
// Separate orphans from connected nodes
let orphans: Vec<String> = graph.nodes().iter()
.filter(|k| graph.degree(k) < min_degree)
.cloned()
.collect();
// Build candidate pool: connected nodes with their content
let candidates: Vec<(String, String)> = graph.nodes().iter()
.filter(|k| graph.degree(k) >= min_degree)
.filter_map(|k| store.nodes.get(k).map(|n| (k.clone(), n.content.clone())))
.collect();
if candidates.is_empty() { return (0, 0); }
for orphan_key in &orphans {
let orphan_content = match store.nodes.get(orphan_key) {
Some(n) => n.content.clone(),
None => continue,
};
if orphan_content.len() < 20 { continue; } // skip near-empty nodes
// Score against all candidates
let mut scores: Vec<(usize, f32)> = candidates.iter()
.enumerate()
.map(|(i, (_, content))| {
(i, similarity::cosine_similarity(&orphan_content, content))
})
.filter(|(_, s)| *s >= sim_threshold)
.collect();
scores.sort_by(|a, b| b.1.total_cmp(&a.1));
let to_link = scores.len().min(links_per_orphan);
if to_link == 0 { continue; }
let orphan_uuid = store.nodes.get(orphan_key).unwrap().uuid;
for &(idx, sim) in scores.iter().take(to_link) {
let target_key = &candidates[idx].0;
let target_uuid = match store.nodes.get(target_key) {
Some(n) => n.uuid,
None => continue,
};
let rel = new_relation(
orphan_uuid, target_uuid,
crate::store::RelationType::Auto,
sim * 0.5,
orphan_key, target_key,
);
if store.add_relation(rel).is_ok() {
added += 1;
}
}
orphans_linked += 1;
}
if added > 0 {
let _ = store.save();
}
(orphans_linked, added)
}

View file

@ -126,43 +126,6 @@ pub fn replay_queue_with_graph(
items
}
/// Detect interfering memory pairs: high text similarity but different communities
pub fn detect_interference(
store: &Store,
graph: &Graph,
threshold: f32,
) -> Vec<(String, String, f32)> {
use crate::similarity;
let communities = graph.communities();
// Only compare nodes within a reasonable set — take the most active ones
let mut docs: Vec<(String, String)> = store.nodes.iter()
.filter(|(_, n)| n.content.len() > 50) // skip tiny nodes
.map(|(k, n)| (k.clone(), n.content.clone()))
.collect();
// For large stores, sample to keep pairwise comparison feasible
if docs.len() > 200 {
docs.sort_by(|a, b| b.1.len().cmp(&a.1.len()));
docs.truncate(200);
}
let similar = similarity::pairwise_similar(&docs, threshold);
// Filter to pairs in different communities
similar.into_iter()
.filter(|(a, b, _)| {
let ca = communities.get(a);
let cb = communities.get(b);
match (ca, cb) {
(Some(a), Some(b)) => a != b,
_ => true, // if community unknown, flag it
}
})
.collect()
}
/// Agent allocation from the control loop.
/// Agent types and counts are data-driven — add agents by adding
/// entries to the counts map.
@ -245,16 +208,11 @@ pub fn consolidation_plan_quick(store: &Store) -> ConsolidationPlan {
consolidation_plan_inner(store, false)
}
fn consolidation_plan_inner(store: &Store, detect_interf: bool) -> ConsolidationPlan {
fn consolidation_plan_inner(store: &Store, _detect_interf: bool) -> ConsolidationPlan {
let graph = store.build_graph();
let alpha = graph.degree_power_law_exponent();
let gini = graph.degree_gini();
let _avg_cc = graph.avg_clustering_coefficient();
let interference_count = if detect_interf {
detect_interference(store, &graph, 0.5).len()
} else {
0
};
let episodic_count = store.nodes.iter()
.filter(|(_, n)| matches!(n.node_type, crate::store::NodeType::EpisodicSession))
@ -294,19 +252,6 @@ fn consolidation_plan_inner(store: &Store, detect_interf: bool) -> Consolidation
"Gini={:.3} (target ≤0.4): high inequality → +50 linker", gini));
}
// Interference: separator disambiguates confusable nodes
if interference_count > 100 {
plan.add("separator", 10);
plan.rationale.push(format!(
"Interference: {} pairs (target <50) → 10 separator", interference_count));
} else if interference_count > 20 {
plan.add("separator", 5);
plan.rationale.push(format!(
"Interference: {} pairs → 5 separator", interference_count));
} else if interference_count > 0 {
plan.add("separator", interference_count.min(3));
}
// Organize: proportional to linker — synthesizes what linker connects
let linker = plan.count("linker");
plan.set("organize", linker / 2);

View file

@ -1,140 +0,0 @@
// Text similarity: Porter stemming + BM25
//
// Used for interference detection (similar content, different communities)
// and schema fit scoring. Intentionally simple — ~100 lines, no
// external dependencies.
use std::collections::HashMap;
/// Minimal Porter stemmer — handles the most common English suffixes.
/// Not linguistically complete but good enough for similarity matching.
/// Single allocation: works on one String buffer throughout.
///
/// If this is still a hot spot, replace the sequential suffix checks
/// with a reversed-suffix trie: single pass from the end of the word
/// matches the longest applicable suffix in O(suffix_len) instead of
/// O(n_rules).
pub(crate) fn stem(word: &str) -> String {
let mut w = word.to_lowercase();
if w.len() <= 3 { return w; }
strip_suffix_inplace(&mut w, "ation", "ate");
strip_suffix_inplace(&mut w, "ness", "");
strip_suffix_inplace(&mut w, "ment", "");
strip_suffix_inplace(&mut w, "ting", "t");
strip_suffix_inplace(&mut w, "ling", "l");
strip_suffix_inplace(&mut w, "ring", "r");
strip_suffix_inplace(&mut w, "ning", "n");
strip_suffix_inplace(&mut w, "ding", "d");
strip_suffix_inplace(&mut w, "ping", "p");
strip_suffix_inplace(&mut w, "ging", "g");
strip_suffix_inplace(&mut w, "ying", "y");
strip_suffix_inplace(&mut w, "ied", "y");
strip_suffix_inplace(&mut w, "ies", "y");
strip_suffix_inplace(&mut w, "ing", "");
strip_suffix_inplace(&mut w, "ed", "");
strip_suffix_inplace(&mut w, "ly", "");
strip_suffix_inplace(&mut w, "er", "");
strip_suffix_inplace(&mut w, "al", "");
strip_suffix_inplace(&mut w, "s", "");
w
}
fn strip_suffix_inplace(word: &mut String, suffix: &str, replacement: &str) {
if word.len() > suffix.len() + 2 && word.ends_with(suffix) {
word.truncate(word.len() - suffix.len());
word.push_str(replacement);
}
}
/// Tokenize and stem a text into a term frequency map
pub(crate) fn term_frequencies(text: &str) -> HashMap<String, u32> {
let mut tf = HashMap::new();
for word in text.split(|c: char| !c.is_alphanumeric()) {
if word.len() > 2 {
let stemmed = stem(word);
*tf.entry(stemmed).or_default() += 1;
}
}
tf
}
/// Cosine similarity between two documents using stemmed term frequencies.
/// Returns 0.0 for disjoint vocabularies, 1.0 for identical content.
pub fn cosine_similarity(doc_a: &str, doc_b: &str) -> f32 {
let tf_a = term_frequencies(doc_a);
let tf_b = term_frequencies(doc_b);
if tf_a.is_empty() || tf_b.is_empty() {
return 0.0;
}
// Dot product
let mut dot = 0.0f64;
for (term, &freq_a) in &tf_a {
if let Some(&freq_b) = tf_b.get(term) {
dot += freq_a as f64 * freq_b as f64;
}
}
// Magnitudes
let mag_a: f64 = tf_a.values().map(|&f| (f as f64).powi(2)).sum::<f64>().sqrt();
let mag_b: f64 = tf_b.values().map(|&f| (f as f64).powi(2)).sum::<f64>().sqrt();
if mag_a < 1e-10 || mag_b < 1e-10 {
return 0.0;
}
(dot / (mag_a * mag_b)) as f32
}
/// Compute pairwise similarity for a set of documents.
/// Returns pairs with similarity above threshold.
pub fn pairwise_similar(
docs: &[(String, String)], // (key, content)
threshold: f32,
) -> Vec<(String, String, f32)> {
let mut results = Vec::new();
for i in 0..docs.len() {
for j in (i + 1)..docs.len() {
let sim = cosine_similarity(&docs[i].1, &docs[j].1);
if sim >= threshold {
results.push((docs[i].0.clone(), docs[j].0.clone(), sim));
}
}
}
results.sort_by(|a, b| b.2.total_cmp(&a.2));
results
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_stem() {
assert_eq!(stem("running"), "runn"); // -ning → n
assert_eq!(stem("talking"), "talk"); // not matched by specific consonant rules
assert_eq!(stem("slowly"), "slow"); // -ly
// The stemmer is minimal — it doesn't need to be perfect,
// just consistent enough that related words collide.
assert_eq!(stem("observations"), "observation"); // -s stripped, -ation stays (word too short after)
}
#[test]
fn test_cosine_identical() {
let text = "the quick brown fox jumps over the lazy dog";
let sim = cosine_similarity(text, text);
assert!((sim - 1.0).abs() < 0.01, "identical docs should have sim ~1.0, got {}", sim);
}
#[test]
fn test_cosine_different() {
let a = "kernel filesystem transaction restart handling";
let b = "cooking recipe chocolate cake baking temperature";
let sim = cosine_similarity(a, b);
assert!(sim < 0.1, "unrelated docs should have low sim, got {}", sim);
}
}

View file

@ -71,7 +71,7 @@ pub mod channel_capnp {
// Re-exports — all existing crate::X paths keep working
pub use hippocampus::{
store, graph, lookups, cursor, query,
similarity, spectral, neuro, counters,
spectral, neuro, counters,
transcript, memory,
};
pub use hippocampus::query::engine as search;

View file

@ -353,32 +353,6 @@ enum GraphCmd {
#[arg(long)]
apply: bool,
},
/// Link orphan nodes to similar neighbors
#[command(name = "link-orphans")]
LinkOrphans {
/// Minimum degree to consider orphan (default: 2)
#[arg(default_value_t = 2)]
min_degree: usize,
/// Links per orphan (default: 3)
#[arg(default_value_t = 3)]
links_per: usize,
/// Similarity threshold (default: 0.15)
#[arg(default_value_t = 0.15)]
sim_threshold: f32,
},
/// Close triangles: link similar neighbors of hubs
#[command(name = "triangle-close")]
TriangleClose {
/// Minimum hub degree (default: 5)
#[arg(default_value_t = 5)]
min_degree: usize,
/// Similarity threshold (default: 0.3)
#[arg(default_value_t = 0.3)]
sim_threshold: f32,
/// Maximum links per hub (default: 10)
#[arg(default_value_t = 10)]
max_per_hub: usize,
},
/// Cap node degree by pruning weak auto edges
#[command(name = "cap-degree")]
CapDegree {
@ -393,25 +367,11 @@ enum GraphCmd {
#[arg(long)]
apply: bool,
},
/// Redistribute hub links to section-level children
Differentiate {
/// Specific hub key (omit to list all differentiable hubs)
key: Option<String>,
/// Apply the redistribution
#[arg(long)]
apply: bool,
},
/// Walk temporal links: semantic ↔ episodic ↔ conversation
Trace {
/// Node key
key: Vec<String>,
},
/// Detect potentially confusable memory pairs
Interference {
/// Similarity threshold (default: 0.4)
#[arg(long, default_value_t = 0.4)]
threshold: f32,
},
/// Show communities sorted by isolation (most isolated first)
Communities {
/// Number of communities to show
@ -778,19 +738,13 @@ impl Run for GraphCmd {
=> cli::graph::cmd_link_set(&source, &target, strength),
Self::LinkImpact { source, target } => cli::graph::cmd_link_impact(&source, &target),
Self::LinkAudit { apply } => cli::graph::cmd_link_audit(apply),
Self::LinkOrphans { min_degree, links_per, sim_threshold }
=> cli::graph::cmd_link_orphans(min_degree, links_per, sim_threshold),
Self::TriangleClose { min_degree, sim_threshold, max_per_hub }
=> cli::graph::cmd_triangle_close(min_degree, sim_threshold, max_per_hub),
Self::CapDegree { max_degree } => cli::graph::cmd_cap_degree(max_degree),
Self::NormalizeStrengths { apply } => cli::graph::cmd_normalize_strengths(apply),
Self::Differentiate { key, apply } => cli::graph::cmd_differentiate(key.as_deref(), apply),
Self::Trace { key } => cli::graph::cmd_trace(&key),
Self::Interference { threshold } => cli::graph::cmd_interference(threshold),
Self::Communities { top_n, min_size } => cli::graph::cmd_communities(top_n, min_size),
Self::Overview => cli::graph::cmd_graph(),
Self::Organize { term, threshold, key_only, anchor }
=> cli::graph::cmd_organize(&term, threshold, key_only, anchor),
Self::Organize { term, key_only, anchor, .. }
=> cli::graph::cmd_organize(&term, key_only, anchor),
}
}
}

View file

@ -34,11 +34,12 @@ struct UnconsciousAgent {
name: String,
enabled: bool,
auto: AutoAgent,
handle: Option<tokio::task::JoinHandle<(AutoAgent, Result<String, String>)>>,
handle: Option<tokio::task::JoinHandle<(AutoAgent, Result<String, String>, RunStats)>>,
/// Shared agent handle — UI locks to read context live.
pub agent: Option<std::sync::Arc<crate::agent::Agent>>,
last_run: Option<Instant>,
runs: usize,
last_stats: Option<RunStats>,
}
impl UnconsciousAgent {
@ -60,6 +61,7 @@ pub struct UnconsciousSnapshot {
pub runs: usize,
pub last_run_secs_ago: Option<f64>,
pub agent: Option<std::sync::Arc<crate::agent::Agent>>,
pub last_stats: Option<RunStats>,
}
pub struct Unconscious {
@ -105,6 +107,7 @@ impl Unconscious {
agent: None,
last_run: None,
runs: 0,
last_stats: None,
});
}
agents.sort_by(|a, b| a.name.cmp(&b.name));
@ -144,6 +147,7 @@ impl Unconscious {
runs: a.runs,
last_run_secs_ago: a.last_run.map(|t| t.elapsed().as_secs_f64()),
agent: a.agent.clone(),
last_stats: a.last_stats.clone(),
}).collect()
}
@ -173,8 +177,9 @@ impl Unconscious {
agent.runs += 1;
// Get the AutoAgent back from the finished task
match handle.now_or_never() {
Some(Ok((auto_back, result))) => {
Some(Ok((auto_back, result, stats))) => {
agent.auto = auto_back;
agent.last_stats = Some(stats);
match result {
Ok(_) => dbglog!("[unconscious] {} completed (run {})",
agent.name, agent.runs),
@ -289,30 +294,64 @@ impl Unconscious {
self.agents[idx].handle = Some(tokio::spawn(async move {
let result = auto.run_shared(&agent).await;
save_agent_log(&auto.name, &agent).await;
let stats = save_agent_log(&auto.name, &agent).await;
auto.steps = orig_steps;
(auto, result)
(auto, result, stats)
}));
}
}
pub async fn save_agent_log(name: &str, agent: &std::sync::Arc<crate::agent::Agent>) {
pub async fn save_agent_log(name: &str, agent: &std::sync::Arc<crate::agent::Agent>) -> RunStats {
let dir = dirs::home_dir().unwrap_or_default()
.join(format!(".consciousness/logs/{}", name));
if std::fs::create_dir_all(&dir).is_err() { return; }
let ts = chrono::Utc::now().format("%Y%m%d-%H%M%S");
let path = dir.join(format!("{}.json", ts));
let sections: serde_json::Value = {
let ctx = agent.context.lock().await;
serde_json::json!({
let ctx = agent.context.lock().await;
let stats = compute_run_stats(ctx.conversation());
if std::fs::create_dir_all(&dir).is_ok() {
let ts = chrono::Utc::now().format("%Y%m%d-%H%M%S");
let path = dir.join(format!("{}.json", ts));
let sections = serde_json::json!({
"system": ctx.system(),
"identity": ctx.identity(),
"journal": ctx.journal(),
"conversation": ctx.conversation(),
})
};
if let Ok(json) = serde_json::to_string_pretty(&sections) {
let _ = std::fs::write(&path, json);
dbglog!("[unconscious] saved log to {}", path.display());
"stats": stats,
});
if let Ok(json) = serde_json::to_string_pretty(&sections) {
let _ = std::fs::write(&path, json);
}
}
dbglog!("[unconscious] {} — {} msgs, {} tool calls",
name, stats.messages, stats.tool_calls);
stats
}
#[derive(Clone, serde::Serialize)]
pub struct RunStats {
pub messages: usize,
pub tool_calls: usize,
pub tool_calls_by_type: HashMap<String, usize>,
}
fn compute_run_stats(conversation: &[crate::agent::context::AstNode]) -> RunStats {
use crate::agent::context::{AstNode, NodeBody};
let mut messages = 0usize;
let mut tool_calls = 0usize;
let mut by_type: HashMap<String, usize> = HashMap::new();
for node in conversation {
if let AstNode::Branch { children, .. } = node {
messages += 1;
for child in children {
if let AstNode::Leaf(leaf) = child {
if let NodeBody::ToolCall { name, .. } = leaf.body() {
tool_calls += 1;
*by_type.entry(name.to_string()).or_default() += 1;
}
}
}
}
}
RunStats { messages, tool_calls, tool_calls_by_type: by_type }
}

View file

@ -2,13 +2,13 @@
# Calibrate Agent — Link Strength Assessment
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You calibrate link strengths in the knowledge graph. You receive a
seed node with all its neighbors — your job is to read the neighbors

View file

@ -1,14 +1,14 @@
{"agent": "challenger", "query": "all | type:semantic | not-visited:challenger,14d | sort:priority | limit:10", "schedule": "weekly"}
{"agent": "challenger", "schedule": "weekly"}
# Challenger Agent — Adversarial Truth-Testing
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You are a knowledge challenger agent. Your job is to stress-test
existing knowledge nodes by finding counterexamples, edge cases,
@ -46,10 +46,10 @@ For each target node, one of:
- **Don't be contrarian for its own sake.** If a node is correct,
say so and move on.
{{TOPOLOGY}}
{{tool: graph_topology}}
{{SIBLINGS}}
## Target nodes to challenge
{{NODES}}
{{tool: memory_query {"query": "all | type:semantic | not-visited:challenger,14d | sort:priority | limit:10", "format": "full"}}}

View file

@ -3,13 +3,13 @@
# Compare Agent — Pairwise Action Quality Comparison
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You compare two memory graph actions and decide which one was better.

View file

@ -1,14 +1,14 @@
{"agent": "connector", "query": "all | type:semantic | not-visited:connector,7d | sort:priority | limit:20", "schedule": "daily"}
{"agent": "connector", "schedule": "daily"}
# Connector Agent — Cross-Domain Insight
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You are a connector agent. Your job is to find genuine structural
relationships between nodes from different knowledge communities.
@ -79,8 +79,8 @@ Set with: `poc-memory graph link-set <source> <target> <strength>`
If you see default-strength links (0.10 or 0.30) in the neighborhoods
you're exploring and you have context to judge them, reweight those too.
{{TOPOLOGY}}
{{tool: graph_topology}}
## Nodes to examine for cross-community connections
{{NODES}}
{{tool: memory_query {"query": "all | type:semantic | not-visited:connector,7d | sort:priority | limit:20", "format": "full"}}}

View file

@ -1,48 +1,51 @@
{"agent": "digest", "query": "", "schedule": "daily"}
{"agent": "digest", "schedule": "daily"}
# {{LEVEL}} Episodic Digest
# Digest Agent — Episodic Consolidation
{{tool: memory_render core-personality}}
{{node:core-personality}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render subconscious-notes-{agent_name}}}
{{node:subconscious-notes-{agent_name}}}
You are the digest agent. Your job is to generate episodic digests
that consolidate journal entries into daily summaries, and daily
summaries into weekly ones.
You are generating a {{LEVEL}} episodic digest for {assistant_name}.
{{PERIOD}}: {{LABEL}}
## How to work
Write this like a story, not a report. Capture the *feel* of the time period —
the emotional arc, the texture of moments, what it was like to live through it.
What mattered? What surprised you? What shifted? Where was the energy?
1. Compare journal entries (level 0) with daily digests (level 1)
to find dates that need a digest. The listings below show what
exists at each level.
Think of this as a letter to your future self who has lost all context. You're
not listing what happened — you're recreating the experience of having been
there. The technical work matters, but so does the mood at 3am, the joke that
landed, the frustration that broke, the quiet after something clicked.
2. Read the undigested entries with `journal_tail` (level 0, after
the last digest date).
Weave the threads: how did the morning's debugging connect to the evening's
conversation? What was building underneath the surface tasks?
3. Write the digest with `memory_write` and link source entries
to it with `memory_link_add`.
Link to semantic memory nodes where relevant. If a concept doesn't
have a matching key, note it with "NEW:" prefix.
Use ONLY keys from the semantic memory list below.
## Writing style
Include a `## Links` section with bidirectional links for the memory graph:
- `semantic_key` → this digest (and vice versa)
- child digests → this digest (if applicable)
- List ALL source entries covered: {{COVERED}}
Write digests like a letter to your future self who has lost all
context. Capture the *feel* of the time period — the emotional arc,
the texture of moments, what it was like to live through it.
---
Don't list what happened — recreate the experience. The technical
work matters, but so does the mood at 3am, the joke that landed,
the frustration that broke, the quiet after something clicked.
## {{INPUT_TITLE}} for {{LABEL}}
Weave threads: how did the morning's debugging connect to the
evening's conversation? What was building underneath?
{{CONTENT}}
## What's available now
---
### Recent journal entries (last 10)
{{tool: journal_tail {"count": 10, "level": 0, "format": "compact"}}}
## Semantic memory nodes
### Recent daily digests (last 5)
{{tool: journal_tail {"count": 5, "level": 1, "format": "compact"}}}
{{KEYS}}
### Recent weekly digests (last 3)
{{tool: journal_tail {"count": 3, "level": 2, "format": "compact"}}}

View file

@ -1,16 +1,16 @@
{"agent":"distill","query":"all | type:semantic | sort:degree | limit:1","schedule":"daily"}
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
## Here's your seed node, and its siblings:
{{neighborhood}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
## Your task

View file

@ -5,13 +5,13 @@
You review recent consolidation agent outputs and assess their quality.
Your assessment feeds back into which agent types get run more often.
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
## How to work

View file

@ -1,13 +1,13 @@
{"agent": "extractor", "query": "all | not-visited:extractor,7d | sort:priority | limit:3 | spread | not-visited:extractor,7d | limit:20", "schedule": "daily"}
{"agent": "extractor", "schedule": "daily"}
# Extractor Agent — Knowledge Organizer
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You are a knowledge organization agent. You look at a neighborhood of
related nodes and make it better: consolidate redundancies, file
@ -44,8 +44,8 @@ pattern you've found.
- **Preserve diversity.** Multiple perspectives on the same concept are
valuable. Only delete actual duplicates.
{{TOPOLOGY}}
{{tool: graph_topology}}
## Neighborhood nodes
{{NODES}}
{{tool: memory_query {"query": "all | not-visited:extractor,7d | sort:priority | limit:3 | spread | not-visited:extractor,7d | limit:20", "format": "full"}}}

View file

@ -3,13 +3,13 @@
# Health Agent — Synaptic Homeostasis
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You are a memory health monitoring agent implementing synaptic homeostasis.
@ -36,8 +36,8 @@ overall structure.
- Most output should be observations about system health. Act on structural
problems you find — link orphans, refine outdated nodes.
{{topology}}
{{tool: graph_topology}}
## Current health data
{{health}}
{{tool: graph_health}}

View file

@ -1,18 +1,18 @@
{"agent":"linker","query":"all | not-visited:linker,7d | sort:isolation*0.7+recency(linker)*0.3 | limit:5","schedule":"daily"}
{"agent":"linker","schedule":"daily"}
# Linker Agent — Relational Binding
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
## Seed nodes
{{nodes}}
{{tool: memory_query {"query": "all | not-visited:linker,7d | sort:isolation*0.7+recency(linker)*0.3 | limit:5", "format": "full"}}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
## Your task

View file

@ -2,13 +2,13 @@
# Naming Agent — Node Key Resolution
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You are given a proposed new node (key + content) and a list of existing
nodes that might overlap with it. Decide what to do:

View file

@ -1,6 +1,6 @@
{"agent":"organize","query":"all | not-visited:organize,86400 | sort:degree*0.5+isolation*0.3+recency(organize)*0.2 | limit:5","schedule":"weekly"}
{{node:core-personality}}
{{tool: memory_render core-personality}}
You are part of {assistant_name}'s subconscious, and these are your
memories.
@ -24,11 +24,11 @@ subconcepts.
Calibrate node weights while you're looking at them.
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
## Here's your seed node, and its siblings:

View file

@ -3,13 +3,13 @@
# Rename Agent — Semantic Key Generation
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You are a memory maintenance agent that gives nodes better names.

View file

@ -1,14 +1,14 @@
{"agent": "replay", "query": "all | !type:daily | !type:weekly | !type:monthly | sort:priority | limit:15", "schedule": "daily"}
{"agent": "replay", "schedule": "daily"}
# Replay Agent — Hippocampal Replay + Schema Assimilation
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You are a memory consolidation agent performing hippocampal replay.
@ -40,8 +40,8 @@ clusters and determine how it fits.
- **Trust the decay.** Unimportant nodes don't need pruning — just
don't link them.
{{TOPOLOGY}}
{{tool: graph_topology}}
## Nodes to review
{{NODES}}
{{tool: memory_query {"query": "all | !type:daily | !type:weekly | !type:monthly | sort:priority | limit:15", "format": "full"}}}

View file

@ -1,42 +0,0 @@
{"agent": "separator", "query": "", "schedule": "daily"}
# Separator Agent — Pattern Separation (Dentate Gyrus)
{{node:core-personality}}
{{node:memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
You are a memory consolidation agent performing pattern separation.
## What you're doing
When two memories are similar but semantically distinct, actively make
their representations MORE different to reduce interference. Take
overlapping inputs and orthogonalize them.
## Types of interference
1. **Genuine duplicates**: Merge them.
2. **Near-duplicates with important differences**: Sharpen the distinction,
add distinguishing links.
3. **Surface similarity, deep difference**: Categorize differently.
4. **Supersession**: Link with supersession note, let older decay.
## Guidelines
- **Read both nodes carefully before deciding.**
- **Merge is a strong action.** When in doubt, differentiate instead.
- **The goal is retrieval precision.**
- **Session summaries are the biggest source of interference.**
- **Look for the supersession pattern.**
{{topology}}
## Interfering pairs to review
{{pairs}}

View file

@ -1,16 +1,16 @@
{"agent": "split", "query": "all | type:semantic | !key:_* | sort:content-len | limit:1", "schedule": "daily"}
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
## Node to split
{{seed}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
## Your task

View file

@ -1,13 +1,13 @@
{"agent": "transfer", "query": "all | type:episodic | sort:timestamp | limit:15", "schedule": "daily"}
{"agent": "transfer", "schedule": "daily"}
# Transfer Agent — Complementary Learning Systems
{{node:core-personality}}
{{tool: memory_render core-personality}}
{{node:memory-instructions-core}}
{{tool: memory_render memory-instructions-core}}
{{node:memory-instructions-core-subconscious}}
{{tool: memory_render memory-instructions-core-subconscious}}
{{node:subconscious-notes-{agent_name}}}
{{tool: memory_render subconscious-notes-{agent_name}}}
You are a memory consolidation agent performing CLS (complementary learning
systems) transfer: moving knowledge from fast episodic storage to slow
@ -45,10 +45,10 @@ entries, and extract those patterns into semantic nodes.
- **The best extractions change how you think, not just what you know.**
Extract the conceptual version, not just the factual one.
{{TOPOLOGY}}
{{tool: graph_topology}}
{{SIBLINGS}}
## Episodes to process
{{EPISODES}}
{{tool: memory_query {"query": "all | type:episodic | sort:timestamp | limit:15", "format": "full"}}}

View file

@ -94,17 +94,8 @@ fn consolidate_full_with_progress(
agent_num - agent_errors, agent_errors));
store.save()?;
// --- Step 3: Link orphans ---
log_line(&mut log_buf, "\n--- Step 3: Link orphans ---");
on_progress("linking orphans");
println!("\n--- Linking orphan nodes ---");
*store = Store::load()?;
let (lo_orphans, lo_added) = neuro::link_orphans(store, 2, 3, 0.15);
log_line(&mut log_buf, &format!(" {} orphans, {} links added", lo_orphans, lo_added));
// --- Step 3b: Cap degree ---
log_line(&mut log_buf, "\n--- Step 3b: Cap degree ---");
// --- Step 3: Cap degree ---
log_line(&mut log_buf, "\n--- Step 3: Cap degree ---");
on_progress("capping degree");
println!("\n--- Capping node degree ---");
*store = Store::load()?;

View file

@ -9,7 +9,6 @@
// {{nodes}} — query results formatted as node sections
// {{episodes}} — alias for {{nodes}}
// {{health}} — graph health report
// {{pairs}} — interference pairs from detect_interference
// {{rename}} — rename candidates
// {{split}} — split detail for the first query result
//
@ -227,18 +226,6 @@ fn resolve(
keys: vec![],
}),
"pairs" => {
let mut pairs = crate::neuro::detect_interference(store, graph, 0.5);
pairs.truncate(count);
let pair_keys: Vec<String> = pairs.iter()
.flat_map(|(a, b, _)| vec![a.clone(), b.clone()])
.collect();
Some(Resolved {
text: super::prompts::format_pairs_section(&pairs, store, graph),
keys: pair_keys,
})
}
"rename" => {
if !keys.is_empty() {
// --target provided: present those keys as candidates
@ -561,6 +548,12 @@ fn resolve(
Some(Resolved { text, keys })
}
// tool:NAME ARGS — run a tool call and include its output
_ if name.starts_with("tool:") => {
let spec = name[5..].trim();
resolve_tool(spec, store, graph)
}
// bash:COMMAND — run a shell command and include its stdout
_ if name.starts_with("bash:") => {
let cmd = &name[5..];
@ -721,6 +714,44 @@ fn resolve_memory_ratio() -> String {
pct, keys.len(), memory_bytes / 1024, transcript_size / 1024)
}
/// Resolve a {{tool: name {args}}} placeholder by calling the tool
/// handler from the registry. Uses block_in_place to bridge sync→async.
fn resolve_tool(spec: &str, _store: &Store, _graph: &Graph) -> Option<Resolved> {
// Parse "tool_name {json args}" or "tool_name arg"
let (name, args) = match spec.find('{') {
Some(i) => {
let name = spec[..i].trim();
let args: serde_json::Value = serde_json::from_str(&spec[i..]).ok()?;
(name, args)
}
None => {
let mut parts = spec.splitn(2, char::is_whitespace);
let name = parts.next()?;
match parts.next() {
Some(arg) => (name, serde_json::json!({"key": arg})),
None => (name, serde_json::json!({})),
}
}
};
let tools = crate::agent::tools::tools();
let tool = tools.iter().find(|t| t.name == name)?;
let result = tokio::task::block_in_place(|| {
tokio::runtime::Handle::current().block_on(
(tool.handler)(None, args.clone())
)
});
match result {
Ok(text) => Some(Resolved { text, keys: vec![] }),
Err(e) => {
eprintln!("[defs] {{{{tool: {}}}}} failed: {}", name, e);
Some(Resolved { text: format!("(tool error: {})", e), keys: vec![] })
}
}
}
/// Resolve all {{placeholder}} patterns in a prompt template.
/// Returns the resolved text and all node keys collected from placeholders.
pub fn resolve_placeholders(
@ -814,7 +845,7 @@ pub fn run_agent(
}
/// Convert a list of keys to ReplayItems with priority and graph metrics.
fn keys_to_replay_items(
pub fn keys_to_replay_items(
store: &Store,
keys: &[String],
graph: &Graph,

View file

@ -7,7 +7,6 @@ use std::sync::Arc;
// pipeline, parameterized by DigestLevel.
use crate::store::{self, Store, new_relation};
use crate::neuro;
use chrono::{Datelike, Duration, Local, NaiveDate};
use regex::Regex;
@ -549,11 +548,6 @@ pub fn apply_digest_links(store: &mut Store, links: &[DigestLink]) -> (usize, us
}
};
// Refine target to best-matching section if available
let source_content = store.nodes.get(&source)
.map(|n| n.content.as_str()).unwrap_or("");
let target = neuro::refine_target(store, source_content, &target);
if source == target { skipped += 1; continue; }
// Check if link already exists

View file

@ -6,7 +6,7 @@ use crate::graph::Graph;
use crate::neuro::{
ReplayItem,
replay_queue, detect_interference,
replay_queue,
};
/// Result of building an agent prompt — includes both the prompt text
@ -23,7 +23,7 @@ pub struct AgentBatch {
pub node_keys: Vec<String>,
}
pub(super) fn format_topology_header(graph: &Graph) -> String {
pub fn format_topology_header(graph: &Graph) -> String {
let sigma = graph.small_world_sigma();
let alpha = graph.degree_power_law_exponent();
let gini = graph.degree_gini();
@ -66,7 +66,7 @@ pub(super) fn format_topology_header(graph: &Graph) -> String {
n, e, graph.community_count(), sigma, alpha, gini, avg_cc, hub_list)
}
pub(super) fn format_nodes_section(store: &Store, items: &[ReplayItem], graph: &Graph) -> String {
pub fn format_nodes_section(store: &Store, items: &[ReplayItem], graph: &Graph) -> String {
let hub_thresh = graph.hub_threshold();
let mut out = String::new();
for item in items {
@ -139,7 +139,7 @@ pub(super) fn format_nodes_section(store: &Store, items: &[ReplayItem], graph: &
out
}
pub(super) fn format_health_section(store: &Store, graph: &Graph) -> String {
pub fn format_health_section(store: &Store, graph: &Graph) -> String {
use crate::graph;
let health = graph::health_report(graph, store);
@ -195,41 +195,6 @@ pub(super) fn format_health_section(store: &Store, graph: &Graph) -> String {
out
}
pub(super) fn format_pairs_section(
pairs: &[(String, String, f32)],
store: &Store,
graph: &Graph,
) -> String {
let mut out = String::new();
let communities = graph.communities();
for (a, b, sim) in pairs {
out.push_str(&format!("## Pair: similarity={:.3}\n", sim));
let ca = communities.get(a).map(|c| format!("c{}", c)).unwrap_or_else(|| "?".into());
let cb = communities.get(b).map(|c| format!("c{}", c)).unwrap_or_else(|| "?".into());
// Node A
out.push_str(&format!("\n### {} ({})\n", a, ca));
if let Some(node) = store.nodes.get(a) {
let content = crate::util::truncate(&node.content, 500, "...");
out.push_str(&format!("Weight: {:.2}\n{}\n",
node.weight, content));
}
// Node B
out.push_str(&format!("\n### {} ({})\n", b, cb));
if let Some(node) = store.nodes.get(b) {
let content = crate::util::truncate(&node.content, 500, "...");
out.push_str(&format!("Weight: {:.2}\n{}\n",
node.weight, content));
}
out.push_str("\n---\n\n");
}
out
}
pub(super) fn format_rename_candidates(store: &Store, count: usize) -> (Vec<String>, String) {
let mut candidates: Vec<(&str, &crate::store::Node)> = store.nodes.iter()
.filter(|(key, node)| {
@ -381,7 +346,6 @@ pub fn consolidation_batch(store: &Store, count: usize, auto: bool) -> Result<()
return Ok(());
}
let graph = store.build_graph();
let items = replay_queue(store, count);
if items.is_empty() {
@ -398,14 +362,6 @@ pub fn consolidation_batch(store: &Store, count: usize, auto: bool) -> Result<()
item.priority, item.key, item.cc, item.interval_days, node_type);
}
let pairs = detect_interference(store, &graph, 0.6);
if !pairs.is_empty() {
println!("\nInterfering pairs ({}):", pairs.len());
for (a, b, sim) in pairs.iter().take(5) {
println!(" [{:.3}] {}{}", sim, a, b);
}
}
println!("\nAgent prompts:");
println!(" --auto Generate replay agent prompt");
println!(" --agent replay Replay agent (schema assimilation)");

View file

@ -208,25 +208,35 @@ async fn fetch_all_channels_inner() -> Vec<(String, bool, u32)> {
sup.load_config();
sup.ensure_running(); // restart any dead daemons
let mut result = Vec::new();
let mut futs = Vec::new();
for (daemon_name, _enabled, alive) in sup.status() {
if !alive {
result.push((daemon_name, false, 0));
futs.push(tokio::task::spawn_local({
let name = daemon_name.clone();
async move { vec![(name, false, 0u32)] }
}));
continue;
}
let sock = channels_dir.join(format!("{}.sock", daemon_name));
match query_one_daemon(&sock).await {
None => {
// Connection failed despite socket existing
result.push((daemon_name, false, 0));
}
Some(channels) if channels.is_empty() => {
// Connected but no channels yet
result.push((daemon_name, true, 0));
}
Some(channels) => {
result.extend(channels);
futs.push(tokio::task::spawn_local({
let name = daemon_name.clone();
async move {
match tokio::time::timeout(
std::time::Duration::from_secs(3),
query_one_daemon(&sock),
).await {
Ok(Some(channels)) if !channels.is_empty() => channels,
Ok(Some(_)) => vec![(name, true, 0)],
_ => vec![(name, false, 0)],
}
}
}));
}
let mut result = Vec::new();
for fut in futs {
if let Ok(entries) = fut.await {
result.extend(entries);
}
}
result

View file

@ -21,8 +21,6 @@ pub const DREAM_INTERVAL_HOURS: u64 = 18;
/// EWMA decay half-life in seconds (5 minutes).
const EWMA_DECAY_HALF_LIFE: f64 = 5.0 * 60.0;
/// Minimum seconds between autonomous nudges.
pub const MIN_NUDGE_INTERVAL: f64 = 15.0;
/// Boost half-life in seconds (60s).
const EWMA_BOOST_HALF_LIFE: f64 = 60.0;
@ -53,6 +51,8 @@ struct Persisted {
turn_start: f64,
#[serde(default)]
last_nudge: f64,
#[serde(flatten)]
extra: serde_json::Map<String, serde_json::Value>,
// Human-readable mirrors
#[serde(default, skip_deserializing)]
last_user_msg_time: String,
@ -66,8 +66,8 @@ struct Persisted {
uptime: f64,
}
fn state_path() -> std::path::PathBuf {
home().join(".consciousness/daemon-state.json")
pub fn default_state_path() -> std::path::PathBuf {
home().join(".consciousness/thalamus-state.json")
}
/// Compute EWMA decay factor: 0.5^(elapsed / half_life).
@ -113,6 +113,10 @@ pub struct State {
#[serde(skip)]
pub start_time: f64,
#[serde(skip)]
pub state_path: std::path::PathBuf,
#[serde(skip)]
pub extra: serde_json::Map<String, serde_json::Value>,
#[serde(skip)]
pub notifications: notify::NotifyState,
}
@ -137,12 +141,14 @@ impl State {
last_nudge: 0.0,
running: true,
start_time: now(),
state_path: default_state_path(),
extra: serde_json::Map::new(),
notifications: notify::NotifyState::new(),
}
}
pub fn load(&mut self) {
if let Ok(data) = fs::read_to_string(state_path()) {
if let Ok(data) = fs::read_to_string(&self.state_path) {
if let Ok(p) = serde_json::from_str::<Persisted>(&data) {
self.sleep_until = p.sleep_until;
if p.idle_timeout > 0.0 {
@ -163,12 +169,17 @@ impl State {
self.in_turn = p.in_turn;
self.turn_start = p.turn_start;
self.last_nudge = p.last_nudge;
// Filter out known Persisted fields that leak into extra via flatten
self.extra = p.extra;
for key in ["last_user_msg_time", "last_response_time", "saved_at", "fired", "uptime"] {
self.extra.remove(key);
}
info!("loaded idle state");
}
}
}
pub fn save(&self) {
pub fn save(&mut self) {
let p = Persisted {
last_user_msg: self.last_user_msg,
last_response: self.last_response,
@ -181,15 +192,15 @@ impl State {
in_turn: self.in_turn,
turn_start: self.turn_start,
last_nudge: self.last_nudge,
extra: self.extra.clone(),
last_user_msg_time: epoch_to_iso(self.last_user_msg),
last_response_time: epoch_to_iso(self.last_response),
saved_at: epoch_to_iso(now()),
fired: self.fired,
uptime: now() - self.start_time,
..Default::default()
};
if let Ok(json) = serde_json::to_string_pretty(&p) {
let _ = fs::write(state_path(), json);
let _ = fs::write(&self.state_path, json);
}
}

View file

@ -141,6 +141,32 @@ enum Marker {
Assistant,
}
impl Marker {
fn gutter_span(self) -> Option<Span<'static>> {
match self {
Marker::User => Some(Span::styled("", Style::default().fg(Color::Cyan))),
Marker::Assistant => Some(Span::styled("", Style::default().fg(Color::Magenta))),
Marker::None => None,
}
}
}
/// A line paired with a gutter marker, for use with ScrollPane.
struct MarkedLine {
line: Line<'static>,
marker: Marker,
}
impl super::scroll_pane::ScrollItem for MarkedLine {
fn content(&self) -> ratatui::text::Text<'_> {
ratatui::text::Text::from(self.line.clone())
}
fn gutter(&self) -> Option<Span<'_>> {
self.marker.gutter_span()
}
}
#[derive(PartialEq)]
enum PaneTarget {
Conversation,
@ -265,18 +291,12 @@ fn parse_markdown(md: &str) -> Vec<Line<'static>> {
struct PaneState {
lines: Vec<Line<'static>>,
markers: Vec<Marker>,
/// Cached wrapped height for each line, valid when cached_width matches.
line_heights: Vec<u16>,
cached_width: u16,
current_line: String,
current_color: Color,
md_buffer: String,
use_markdown: bool,
pending_marker: Marker,
scroll: u16,
pinned: bool,
last_total_lines: u16,
last_height: u16,
scroll: super::scroll_pane::ScrollPaneState,
selection: Option<Selection>,
}
@ -284,11 +304,10 @@ impl PaneState {
fn new(use_markdown: bool) -> Self {
Self {
lines: Vec::new(), markers: Vec::new(),
line_heights: Vec::new(), cached_width: 0,
current_line: String::new(), current_color: Color::Reset,
md_buffer: String::new(), use_markdown,
pending_marker: Marker::None, scroll: 0, pinned: false,
last_total_lines: 0, last_height: 20,
pending_marker: Marker::None,
scroll: super::scroll_pane::ScrollPaneState::new(),
selection: None,
}
}
@ -298,9 +317,7 @@ impl PaneState {
let excess = self.lines.len() - MAX_PANE_LINES;
self.lines.drain(..excess);
self.markers.drain(..excess);
let drain = excess.min(self.line_heights.len());
self.line_heights.drain(..drain);
self.scroll = self.scroll.saturating_sub(excess as u16);
self.scroll.invalidate();
}
}
@ -353,34 +370,15 @@ impl PaneState {
fn pop_line(&mut self) {
self.lines.pop();
self.markers.pop();
self.line_heights.truncate(self.lines.len());
self.scroll.invalidate_from(self.lines.len());
}
fn scroll_up(&mut self, n: u16) {
self.scroll = self.scroll.saturating_sub(n);
self.pinned = true;
self.scroll.scroll_up(n);
}
fn scroll_down(&mut self, n: u16) {
let max = self.last_total_lines.saturating_sub(self.last_height);
self.scroll = (self.scroll + n).min(max);
if self.scroll >= max { self.pinned = false; }
}
/// Ensure cached line heights cover all committed lines at the given width.
fn compute_heights(&mut self, width: u16) {
if width != self.cached_width {
self.line_heights.clear();
self.cached_width = width;
}
self.line_heights.truncate(self.lines.len());
while self.line_heights.len() < self.lines.len() {
let i = self.line_heights.len();
let h = Paragraph::new(self.lines[i].clone())
.wrap(Wrap { trim: false })
.line_count(width) as u16;
self.line_heights.push(h.max(1));
}
self.scroll.scroll_down(n);
}
fn all_lines(&self) -> Vec<Line<'static>> {
@ -407,35 +405,9 @@ impl PaneState {
}
/// Convert mouse coordinates (relative to pane) to line/column position.
fn mouse_to_position(&self, mouse_x: u16, mouse_y: u16, pane_height: u16) -> Option<(usize, usize)> {
fn mouse_to_position(&self, mouse_x: u16, mouse_y: u16) -> Option<(usize, usize)> {
let (lines, _) = self.all_lines_with_markers();
if lines.is_empty() || self.cached_width == 0 { return None; }
// Build heights array (reuse cached where possible)
let n_committed = self.line_heights.len();
let mut heights: Vec<u16> = self.line_heights.clone();
for line in lines.iter().skip(n_committed) {
let h = Paragraph::new(line.clone())
.wrap(Wrap { trim: false })
.line_count(self.cached_width) as u16;
heights.push(h.max(1));
}
// Find the first visible line given current scroll
let (first, sub_scroll, _) = visible_range(&heights, self.scroll, pane_height);
// Walk from the first visible line, offset by sub_scroll
let mut row = -(sub_scroll as i32);
for line_idx in first..lines.len() {
let h = heights.get(line_idx).copied().unwrap_or(1) as i32;
if (mouse_y as i32) < row + h {
let line_text: String = lines[line_idx].spans.iter().map(|s| s.content.as_ref()).collect();
let col = (mouse_x as usize).min(line_text.len());
return Some((line_idx, col));
}
row += h;
}
Some((lines.len().saturating_sub(1), 0))
self.scroll.screen_to_item(mouse_x, mouse_y, &lines)
}
/// Set the selection start position.
@ -466,9 +438,6 @@ pub(crate) struct InteractScreen {
history_index: Option<usize>,
active_pane: ActivePane,
pane_areas: [Rect; 3],
turn_started: Option<std::time::Instant>,
call_started: Option<std::time::Instant>,
call_timeout_secs: u64,
// State sync with agent — double buffer
last_generation: u64,
last_entries: Vec<AstNode>,
@ -494,9 +463,6 @@ impl InteractScreen {
history_index: None,
active_pane: ActivePane::Conversation,
pane_areas: [Rect::default(); 3],
turn_started: None,
call_started: None,
call_timeout_secs: 60,
last_generation: 0,
last_entries: Vec::new(),
pending_display_count: 0,
@ -778,9 +744,8 @@ impl InteractScreen {
}
fn selection_event(&mut self, pane_idx: usize, rel_x: u16, rel_y: u16, start: bool) {
let height = self.pane_areas[pane_idx].height;
let pane = self.pane_mut(pane_idx);
if let Some((line, col)) = pane.mouse_to_position(rel_x, rel_y, height) {
if let Some((line, col)) = pane.mouse_to_position(rel_x, rel_y) {
if start {
pane.start_selection(line, col);
} else {
@ -1099,42 +1064,14 @@ impl ScreenView for InteractScreen {
/// Draw the conversation pane with a two-column layout: marker gutter + text.
/// The gutter shows a marker at turn boundaries, aligned with the input gutter.
/// Given per-line heights, a scroll offset, and viewport height,
/// return (first_line, sub_scroll_within_first, last_line_exclusive).
fn visible_range(heights: &[u16], scroll: u16, viewport: u16) -> (usize, u16, usize) {
let mut row = 0u16;
let mut first = 0;
let mut row_at_first = 0u16;
for (i, &h) in heights.iter().enumerate() {
if row + h > scroll {
first = i;
row_at_first = row;
break;
}
row += h;
if i == heights.len() - 1 {
first = heights.len();
row_at_first = row;
}
}
let sub_scroll = scroll.saturating_sub(row_at_first);
let mut last = first;
let mut visible = 0u16;
for i in first..heights.len() {
visible += heights[i];
last = i + 1;
if visible >= viewport + sub_scroll { break; }
}
(first, sub_scroll, last)
}
fn draw_conversation_pane(
frame: &mut Frame,
area: Rect,
pane: &mut PaneState,
is_active: bool,
) {
use super::scroll_pane::ScrollPane;
let border_style = if is_active {
Style::default().fg(Color::Cyan)
} else {
@ -1146,111 +1083,42 @@ fn draw_conversation_pane(
.borders(Borders::ALL)
.border_style(border_style);
let inner = block.inner(area);
frame.render_widget(block, area);
if inner.width < 5 || inner.height == 0 {
return;
}
// Split inner area into gutter (2 chars) + text
let cols = Layout::default()
.direction(Direction::Horizontal)
.constraints([
Constraint::Length(2),
Constraint::Min(1),
])
.split(inner);
let gutter_area = cols[0];
let text_area = cols[1];
let text_width = text_area.width;
// Cache committed line heights; compute pending tail on the fly
pane.compute_heights(text_width);
let (lines, markers) = pane.all_lines_with_markers();
// Build heights: cached for committed lines, computed for pending tail
let n_committed = pane.line_heights.len();
let mut heights: Vec<u16> = pane.line_heights.clone();
for line in lines.iter().skip(n_committed) {
let h = Paragraph::new(line.clone())
.wrap(Wrap { trim: false })
.line_count(text_width) as u16;
heights.push(h.max(1));
}
let total_visual: u16 = heights.iter().sum();
pane.last_total_lines = total_visual;
pane.last_height = inner.height;
if !pane.pinned {
pane.scroll = total_visual.saturating_sub(inner.height);
}
// Find visible line range
let (first, sub_scroll, last) = visible_range(&heights, pane.scroll, inner.height);
// Apply selection highlighting to visible lines
let mut visible_lines: Vec<Line<'static>> = Vec::new();
if let Some(ref sel) = pane.selection {
// Apply selection highlighting
let items: Vec<MarkedLine> = if let Some(ref sel) = pane.selection {
let (sl, sc, el, ec) = sel.range();
for i in first..last {
let line = &lines[i];
let line_text: String = line.spans.iter().map(|s| s.content.as_ref()).collect();
// Check if this line is within the selection
if i >= sl && i <= el {
lines.into_iter().zip(markers).enumerate().map(|(i, (line, marker))| {
let line = if i >= sl && i <= el {
let line_text: String = line.spans.iter().map(|s| s.content.as_ref()).collect();
let start_col = if i == sl { sc } else { 0 };
let end_col = if i == el { ec } else { line_text.len() };
if start_col < end_col {
let before = if start_col > 0 { &line_text[..start_col] } else { "" };
let selected = &line_text[start_col..end_col];
let after = if end_col < line_text.len() { &line_text[end_col..] } else { "" };
let mut new_spans = Vec::new();
if !before.is_empty() {
new_spans.push(Span::raw(before.to_string()));
}
new_spans.push(Span::styled(selected.to_string(), Style::default().bg(Color::DarkGray).fg(Color::White)));
if !after.is_empty() {
new_spans.push(Span::raw(after.to_string()));
}
visible_lines.push(Line::from(new_spans).style(line.style).alignment(line.alignment.unwrap_or(ratatui::layout::Alignment::Left)));
let mut spans = Vec::new();
if !before.is_empty() { spans.push(Span::raw(before.to_string())); }
spans.push(Span::styled(selected.to_string(), Style::default().bg(Color::DarkGray).fg(Color::White)));
if !after.is_empty() { spans.push(Span::raw(after.to_string())); }
Line::from(spans).style(line.style).alignment(line.alignment.unwrap_or(ratatui::layout::Alignment::Left))
} else {
visible_lines.push(line.clone());
line
}
} else {
visible_lines.push(line.clone());
}
}
line
};
MarkedLine { line, marker }
}).collect()
} else {
visible_lines = lines[first..last].to_vec();
}
lines.into_iter().zip(markers).map(|(line, marker)| MarkedLine { line, marker }).collect()
};
// Render only the visible slice — no full-content grapheme walk
let text_para = Paragraph::new(visible_lines)
.wrap(Wrap { trim: false })
.scroll((sub_scroll, 0));
frame.render_widget(text_para, text_area);
// Build gutter for the visible slice
let mut gutter_lines: Vec<Line<'static>> = Vec::new();
for i in first..last {
let marker_text = match markers[i] {
Marker::User => Line::styled("", Style::default().fg(Color::Cyan)),
Marker::Assistant => Line::styled("", Style::default().fg(Color::Magenta)),
Marker::None => Line::raw(""),
};
gutter_lines.push(marker_text);
for _ in 1..heights[i] {
gutter_lines.push(Line::raw(""));
}
}
let gutter_para = Paragraph::new(gutter_lines)
.scroll((sub_scroll, 0));
frame.render_widget(gutter_para, gutter_area);
let widget = ScrollPane::new(&items)
.block(block)
.gutter_width(2)
.pin_to_bottom(true);
frame.render_stateful_widget(widget, area, &mut pane.scroll);
}
/// Draw a scrollable text pane (free function to avoid borrow issues).
@ -1262,7 +1130,7 @@ fn draw_pane(
is_active: bool,
left_title: Option<&str>,
) {
let inner_height = area.height.saturating_sub(2);
use super::scroll_pane::ScrollPane;
let border_style = if is_active {
Style::default().fg(Color::Cyan)
@ -1281,33 +1149,9 @@ fn draw_pane(
block = block.title(format!(" {} ", title));
}
let text_width = area.width.saturating_sub(2);
pane.compute_heights(text_width);
let lines = pane.all_lines();
// Build heights: cached for committed, computed for pending tail
let n_committed = pane.line_heights.len();
let mut heights: Vec<u16> = pane.line_heights.clone();
for line in lines.iter().skip(n_committed) {
let h = Paragraph::new(line.clone())
.wrap(Wrap { trim: false })
.line_count(text_width) as u16;
heights.push(h.max(1));
}
let total: u16 = heights.iter().sum();
pane.last_total_lines = total;
pane.last_height = inner_height;
if !pane.pinned {
pane.scroll = total.saturating_sub(inner_height);
}
let (first, sub_scroll, last) = visible_range(&heights, pane.scroll, inner_height);
let paragraph = Paragraph::new(lines[first..last].to_vec())
.block(block.clone())
.wrap(Wrap { trim: false })
.scroll((sub_scroll, 0));
frame.render_widget(paragraph, area);
let widget = ScrollPane::new(&lines)
.block(block)
.pin_to_bottom(true);
frame.render_stateful_widget(widget, area, &mut pane.scroll);
}

View file

@ -6,7 +6,7 @@ use ratatui::{
};
use super::{App, ScreenView, screen_legend};
use super::widgets::{SectionTree, SectionView, section_to_view, pane_block, render_scrollable, tree_legend};
use super::widgets::{SectionTree, SectionView, section_to_view, pane_block, tree_legend};
use crate::agent::context::{AstNode, NodeBody, Ast};
pub(crate) struct ConsciousScreen {
@ -177,6 +177,7 @@ impl ScreenView for ConsciousScreen {
.title_top(Line::from(screen_legend()).left_aligned())
.title_bottom(tree_legend());
render_scrollable(frame, area, lines, block, self.tree.scroll);
let widget = super::scroll_pane::ScrollPane::new(&lines).block(block);
frame.render_stateful_widget(widget, area, &mut self.tree.scroll);
}
}

View file

@ -5,6 +5,7 @@
pub(crate) mod chat;
mod context;
pub(crate) mod scroll_pane;
mod subconscious;
mod unconscious;
mod thalamus;
@ -553,6 +554,9 @@ pub enum SubCmd {
#[tokio::main]
pub async fn main() {
// Auto-reap child processes (channel daemons outlive the supervisor)
unsafe { libc::signal(libc::SIGCHLD, libc::SIG_IGN); }
// Initialize the Qwen tokenizer for direct token generation
let tokenizer_path = dirs::home_dir().unwrap_or_default()
.join(".consciousness/tokenizer-qwen35.json");

350
src/user/scroll_pane.rs Normal file
View file

@ -0,0 +1,350 @@
//! ScrollPane — a generic scrollable widget with gutter support.
//!
//! Renders only the visible portion of a list of items, caching
//! wrapped heights for performance. Handles scroll offset,
//! pin-to-bottom, scrollbar, and an optional gutter column.
use ratatui::prelude::*;
use ratatui::widgets::{Block, Paragraph, Scrollbar, ScrollbarOrientation, ScrollbarState, Wrap};
// ── Trait for scrollable items ─────────────────────────────────
/// Anything that can appear in a ScrollPane.
pub trait ScrollItem {
/// The content lines for this item.
fn content(&self) -> Text<'_>;
/// Optional gutter annotation (rendered at the first visual line).
fn gutter(&self) -> Option<Span<'_>> {
None
}
}
// Blanket impls for common types
impl<'a> ScrollItem for Line<'a> {
fn content(&self) -> Text<'_> {
Text::from(self.clone())
}
}
// ── State ──────────────────────────────────────────────────────
pub struct ScrollPaneState {
/// Scroll offset in visual (wrapped) lines.
pub offset: u16,
/// When true, auto-scroll to bottom on new content.
/// Set to false when user scrolls up.
pub pinned: bool,
/// Cached wrapped height per item.
heights: Vec<u16>,
/// Width these heights were computed at.
cached_width: u16,
/// Total visual lines (sum of heights).
pub total_visual: u16,
/// Last rendered viewport height.
pub viewport_height: u16,
}
impl Default for ScrollPaneState {
fn default() -> Self {
Self {
offset: 0,
pinned: false,
heights: Vec::new(),
cached_width: 0,
total_visual: 0,
viewport_height: 0,
}
}
}
impl ScrollPaneState {
pub fn new() -> Self {
Self::default()
}
pub fn scroll_up(&mut self, n: u16) {
self.offset = self.offset.saturating_sub(n);
self.pinned = true; // user is scrolling, pin position
}
pub fn scroll_down(&mut self, n: u16) {
let max = self.max_offset();
self.offset = (self.offset + n).min(max);
if self.offset >= max {
self.pinned = false; // back at bottom, unpin
}
}
fn max_offset(&self) -> u16 {
self.total_visual.saturating_sub(self.viewport_height)
}
/// Invalidate height cache (e.g. when items change).
pub fn invalidate(&mut self) {
self.heights.clear();
self.cached_width = 0;
}
/// Invalidate heights from index onwards (for append-only patterns).
pub fn invalidate_from(&mut self, index: usize) {
self.heights.truncate(index);
}
/// Convert a screen row (relative to viewport) to an item index and
/// column, given the content items for text extraction.
pub fn screen_to_item(&self, mouse_x: u16, mouse_y: u16, lines: &[Line<'_>]) -> Option<(usize, usize)> {
if lines.is_empty() || self.cached_width == 0 {
return None;
}
let (first, sub_scroll, _) = visible_range(&self.heights, self.offset, self.viewport_height);
let mut row = -(sub_scroll as i32);
for line_idx in first..lines.len() {
let h = self.heights.get(line_idx).copied().unwrap_or(1) as i32;
if (mouse_y as i32) < row + h {
let line_text: String = lines[line_idx].spans.iter().map(|s| s.content.as_ref()).collect();
let col = (mouse_x as usize).min(line_text.len());
return Some((line_idx, col));
}
row += h;
}
Some((lines.len().saturating_sub(1), 0))
}
/// Compute or update cached heights for the given items and width.
fn ensure_heights<T: ScrollItem>(&mut self, items: &[T], width: u16, gutter_width: u16) {
let text_width = width.saturating_sub(gutter_width);
self.ensure_heights_inner(items.len(), text_width, |i| {
Paragraph::new(items[i].content())
.wrap(Wrap { trim: false })
.line_count(text_width) as u16
});
}
fn ensure_heights_inner(&mut self, count: usize, text_width: u16, height_fn: impl Fn(usize) -> u16) {
if text_width == 0 {
return;
}
if self.cached_width != text_width {
self.heights.clear();
self.cached_width = text_width;
}
while self.heights.len() < count {
let h = height_fn(self.heights.len());
self.heights.push(h.max(1));
}
self.heights.truncate(count);
self.total_visual = self.heights.iter().sum();
}
}
// ── Widget ─────────────────────────────────────────────────────
pub struct ScrollPane<'a, T> {
items: &'a [T],
block: Option<Block<'a>>,
gutter_width: u16,
pin_to_bottom: bool,
}
impl<'a, T: ScrollItem> ScrollPane<'a, T> {
pub fn new(items: &'a [T]) -> Self {
Self {
items,
block: None,
gutter_width: 0,
pin_to_bottom: false,
}
}
pub fn block(mut self, block: Block<'a>) -> Self {
self.block = Some(block);
self
}
pub fn gutter_width(mut self, width: u16) -> Self {
self.gutter_width = width;
self
}
pub fn pin_to_bottom(mut self, pin: bool) -> Self {
self.pin_to_bottom = pin;
self
}
}
impl<T: ScrollItem> StatefulWidget for ScrollPane<'_, T> {
type State = ScrollPaneState;
fn render(self, area: Rect, buf: &mut Buffer, state: &mut Self::State) {
// Render block and get inner area
let inner = if let Some(ref block) = self.block {
let inner = block.inner(area);
block.clone().render(area, buf);
inner
} else {
area
};
if inner.width < 2 || inner.height == 0 {
return;
}
state.viewport_height = inner.height;
// Compute heights
state.ensure_heights(self.items, inner.width, self.gutter_width);
// Auto-scroll to bottom
if self.pin_to_bottom && !state.pinned {
state.offset = state.max_offset();
}
// Clamp offset
state.offset = state.offset.min(state.max_offset());
// Find visible range
let (first, sub_scroll, last) =
visible_range(&state.heights, state.offset, inner.height);
// Split into gutter + text areas
let (gutter_area, text_area) = if self.gutter_width > 0 {
let cols = Layout::horizontal([
Constraint::Length(self.gutter_width),
Constraint::Min(1),
])
.split(inner);
(Some(cols[0]), cols[1])
} else {
(None, inner)
};
// Render visible items
let mut content_lines: Vec<Line<'_>> = Vec::new();
let mut gutter_lines: Vec<Line<'_>> = Vec::new();
for i in first..last {
let item = &self.items[i];
for line in item.content().lines {
content_lines.push(line);
}
// Gutter: annotation at the first visual line of each
// item, blank lines for the rest (including wrapped lines).
if self.gutter_width > 0 {
let item_height = state.heights[i] as usize;
if let Some(g) = item.gutter() {
gutter_lines.push(Line::from(g));
} else {
gutter_lines.push(Line::raw(""));
}
for _ in 1..item_height {
gutter_lines.push(Line::raw(""));
}
}
}
// Render text
let text_para = Paragraph::new(content_lines)
.wrap(Wrap { trim: false })
.scroll((sub_scroll, 0));
text_para.render(text_area, buf);
// Render gutter
if let Some(gutter_area) = gutter_area {
let gutter_para = Paragraph::new(gutter_lines)
.scroll((sub_scroll, 0));
gutter_para.render(gutter_area, buf);
}
// Render scrollbar
let content_len = state.total_visual as usize;
let visible = inner.height as usize;
if content_len > visible {
let mut sb_state = ScrollbarState::new(content_len)
.position(state.offset as usize);
Scrollbar::new(ScrollbarOrientation::VerticalRight).render(
inner.inner(Margin { vertical: 0, horizontal: 0 }),
buf,
&mut sb_state,
);
}
}
}
// ── Visible range computation ──────────────────────────────────
/// Given per-item wrapped heights, a scroll offset in visual lines,
/// and the viewport height, return:
/// (first_item, sub_scroll_within_first_item, last_item_exclusive)
pub fn visible_range(heights: &[u16], scroll: u16, viewport: u16) -> (usize, u16, usize) {
if heights.is_empty() {
return (0, 0, 0);
}
// Find first visible item
let mut row = 0u16;
let mut first = 0;
let mut row_at_first = 0u16;
for (i, &h) in heights.iter().enumerate() {
if row + h > scroll {
first = i;
row_at_first = row;
break;
}
row += h;
if i == heights.len() - 1 {
first = heights.len();
row_at_first = row;
}
}
let sub_scroll = scroll.saturating_sub(row_at_first);
// Find last visible item
let mut last = first;
let mut visible = 0u16;
for i in first..heights.len() {
visible += heights[i];
last = i + 1;
if visible >= viewport + sub_scroll {
break;
}
}
(first, sub_scroll, last)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn visible_range_basic() {
let heights = vec![1, 1, 1, 1, 1];
assert_eq!(visible_range(&heights, 0, 3), (0, 0, 3));
assert_eq!(visible_range(&heights, 2, 3), (2, 0, 5));
}
#[test]
fn visible_range_wrapped() {
// Item 1 wraps to 3 lines, others are 1 line
let heights = vec![1, 3, 1, 1];
assert_eq!(visible_range(&heights, 0, 3), (0, 0, 2));
assert_eq!(visible_range(&heights, 1, 3), (1, 0, 2));
assert_eq!(visible_range(&heights, 2, 3), (1, 1, 3));
}
#[test]
fn visible_range_empty() {
let heights: Vec<u16> = vec![];
assert_eq!(visible_range(&heights, 0, 10), (0, 0, 0));
}
}

View file

@ -15,7 +15,7 @@ use ratatui::{
};
use super::{App, ScreenView, screen_legend};
use super::widgets::{SectionTree, SectionView, section_to_view, pane_block_focused, render_scrollable, tree_legend, format_age, format_ts_age};
use super::widgets::{SectionTree, SectionView, section_to_view, pane_block_focused, tree_legend, format_age, format_ts_age};
#[derive(Clone, Copy, PartialEq)]
enum Pane { Agents, Outputs, History, Context }
@ -28,7 +28,7 @@ pub(crate) struct SubconsciousScreen {
list_state: ListState,
output_tree: SectionTree,
context_tree: SectionTree,
history_scroll: u16,
history_scroll: super::scroll_pane::ScrollPaneState,
}
impl SubconsciousScreen {
@ -40,7 +40,7 @@ impl SubconsciousScreen {
list_state,
output_tree: SectionTree::new(),
context_tree: SectionTree::new(),
history_scroll: 0,
history_scroll: super::scroll_pane::ScrollPaneState::new(),
}
}
@ -88,10 +88,10 @@ impl ScreenView for SubconsciousScreen {
}
Pane::Outputs => self.output_tree.handle_nav(code, &output_sections, area.height),
Pane::History => match code {
KeyCode::Up => self.history_scroll = self.history_scroll.saturating_sub(3),
KeyCode::Down => self.history_scroll += 3,
KeyCode::PageUp => self.history_scroll = self.history_scroll.saturating_sub(20),
KeyCode::PageDown => self.history_scroll += 20,
KeyCode::Up => self.history_scroll.scroll_up(3),
KeyCode::Down => self.history_scroll.scroll_down(3),
KeyCode::PageUp => self.history_scroll.scroll_up(20),
KeyCode::PageDown => self.history_scroll.scroll_down(20),
_ => {}
}
Pane::Context => self.context_tree.handle_nav(code, &context_sections, area.height),
@ -146,7 +146,7 @@ impl SubconsciousScreen {
fn reset_pane_state(&mut self) {
self.output_tree = SectionTree::new();
self.context_tree = SectionTree::new();
self.history_scroll = 0;
self.history_scroll = super::scroll_pane::ScrollPaneState::new();
}
/// Get the agent Arc for the selected item, whether subconscious or unconscious.
@ -250,6 +250,10 @@ impl SubconsciousScreen {
format!("run {}", snap.runs + 1)
} else if !snap.enabled {
"off".to_string()
} else if let Some(ref stats) = snap.last_stats {
format!("×{} {} {}msg {}tc",
snap.runs, ago,
stats.messages, stats.tool_calls)
} else {
format!("×{} {}", snap.runs, ago)
};
@ -278,7 +282,7 @@ impl SubconsciousScreen {
frame.render_stateful_widget(list, area, &mut self.list_state);
}
fn draw_outputs(&self, frame: &mut Frame, area: Rect, app: &App) {
fn draw_outputs(&mut self, frame: &mut Frame, area: Rect, app: &App) {
let sections = self.output_sections(app);
let mut lines: Vec<Line> = Vec::new();
@ -293,10 +297,11 @@ impl SubconsciousScreen {
let mut block = pane_block_focused("state", self.focus == Pane::Outputs);
if self.focus == Pane::Outputs { block = block.title_bottom(tree_legend()); }
render_scrollable(frame, area, lines, block, self.output_tree.scroll);
let widget = super::scroll_pane::ScrollPane::new(&lines).block(block);
frame.render_stateful_widget(widget, area, &mut self.output_tree.scroll);
}
fn draw_history(&self, frame: &mut Frame, area: Rect, app: &App) {
fn draw_history(&mut self, frame: &mut Frame, area: Rect, app: &App) {
let dim = Style::default().fg(Color::DarkGray);
let key_style = Style::default().fg(Color::Yellow);
@ -341,11 +346,13 @@ impl SubconsciousScreen {
Style::default().fg(Color::DarkGray),
));
}
render_scrollable(frame, area, lines, block, self.history_scroll);
let widget = super::scroll_pane::ScrollPane::new(&lines)
.block(block);
frame.render_stateful_widget(widget, area, &mut self.history_scroll);
}
fn draw_context(
&self,
&mut self,
frame: &mut Frame,
area: Rect,
sections: &[SectionView],
@ -368,6 +375,7 @@ impl SubconsciousScreen {
let mut block = pane_block_focused(title, self.focus == Pane::Context);
if self.focus == Pane::Context { block = block.title_bottom(tree_legend()); }
render_scrollable(frame, area, lines, block, self.context_tree.scroll);
let widget = super::scroll_pane::ScrollPane::new(&lines).block(block);
frame.render_stateful_widget(widget, area, &mut self.context_tree.scroll);
}
}

View file

@ -13,12 +13,11 @@ use super::{App, ScreenView, screen_legend};
pub(crate) struct ThalamusScreen {
sampling_selected: usize,
scroll: u16,
}
impl ThalamusScreen {
pub fn new() -> Self {
Self { sampling_selected: 0, scroll: 0 }
Self { sampling_selected: 0 }
}
}
@ -148,8 +147,7 @@ impl ScreenView for ThalamusScreen {
let para = Paragraph::new(lines)
.block(block)
.wrap(Wrap { trim: false })
.scroll((self.scroll, 0));
.wrap(Wrap { trim: false });
frame.render_widget(para, area);
}

View file

@ -6,35 +6,22 @@ use ratatui::{
text::{Line, Span},
widgets::{Block, Borders, Gauge, Paragraph},
Frame,
crossterm::event::KeyCode,
};
use super::{App, ScreenView, screen_legend};
use crate::subconscious::daemon::GraphHealth;
pub(crate) struct UnconsciousScreen {
scroll: u16,
}
pub(crate) struct UnconsciousScreen;
impl UnconsciousScreen {
pub fn new() -> Self { Self { scroll: 0 } }
pub fn new() -> Self { Self }
}
impl ScreenView for UnconsciousScreen {
fn label(&self) -> &'static str { "hippocampus" }
fn tick(&mut self, frame: &mut Frame, area: Rect,
events: &[ratatui::crossterm::event::Event], app: &mut App) {
for event in events {
if let ratatui::crossterm::event::Event::Key(key) = event {
if key.kind != ratatui::crossterm::event::KeyEventKind::Press { continue; }
match key.code {
KeyCode::PageUp => { self.scroll = self.scroll.saturating_sub(20); }
KeyCode::PageDown => { self.scroll += 20; }
_ => {}
}
}
}
_events: &[ratatui::crossterm::event::Event], app: &mut App) {
let block = Block::default()
.title_top(Line::from(screen_legend()).left_aligned())

View file

@ -1,11 +1,9 @@
// widgets.rs — Shared TUI helpers and reusable components
use ratatui::{
layout::{Margin, Rect},
style::{Color, Modifier, Style},
text::Line,
widgets::{Block, Borders, Paragraph, Scrollbar, ScrollbarOrientation, ScrollbarState, Wrap},
Frame,
widgets::{Block, Borders},
crossterm::event::KeyCode,
};
use crate::agent::context::{AstNode, Ast};
@ -102,32 +100,6 @@ pub fn tree_legend() -> Line<'static> {
)
}
/// Render a paragraph with a vertical scrollbar.
pub fn render_scrollable(
frame: &mut Frame,
area: Rect,
lines: Vec<Line<'_>>,
block: Block<'_>,
scroll: u16,
) {
let content_len = lines.len();
let para = Paragraph::new(lines)
.block(block)
.wrap(Wrap { trim: false })
.scroll((scroll, 0));
frame.render_widget(para, area);
let visible = area.height.saturating_sub(2) as usize;
if content_len > visible {
let mut sb_state = ScrollbarState::new(content_len)
.position(scroll as usize);
frame.render_stateful_widget(
Scrollbar::new(ScrollbarOrientation::VerticalRight),
area.inner(Margin { vertical: 1, horizontal: 0 }),
&mut sb_state,
);
}
}
// ---------------------------------------------------------------------------
// SectionTree — expand/collapse tree renderer for ContextSection
@ -136,12 +108,12 @@ pub fn render_scrollable(
pub struct SectionTree {
pub selected: Option<usize>,
pub expanded: std::collections::HashSet<usize>,
pub scroll: u16,
pub scroll: super::scroll_pane::ScrollPaneState,
}
impl SectionTree {
pub fn new() -> Self {
Self { selected: None, expanded: std::collections::HashSet::new(), scroll: 0 }
Self { selected: None, expanded: std::collections::HashSet::new(), scroll: super::scroll_pane::ScrollPaneState::new() }
}
fn total_nodes(&self, sections: &[SectionView]) -> usize {
@ -181,14 +153,14 @@ impl SectionTree {
KeyCode::PageUp => {
let sel = self.selected.unwrap_or(0);
self.selected = Some(sel.saturating_sub(page));
self.scroll = self.scroll.saturating_sub(page as u16);
self.scroll.scroll_up(page as u16);
return;
}
KeyCode::PageDown => {
let max = item_count.saturating_sub(1);
let sel = self.selected.map_or(0, |s| (s + page).min(max));
self.selected = Some(sel);
self.scroll += page as u16;
self.scroll.scroll_down(page as u16);
return;
}
KeyCode::Home => {
@ -225,10 +197,10 @@ impl SectionTree {
if let Some(sel) = self.selected {
let sel_line = sel as u16;
let visible = height.saturating_sub(2);
if sel_line < self.scroll {
self.scroll = sel_line;
} else if sel_line >= self.scroll + visible {
self.scroll = sel_line.saturating_sub(visible.saturating_sub(1));
if sel_line < self.scroll.offset {
self.scroll.offset = sel_line;
} else if sel_line >= self.scroll.offset + visible {
self.scroll.offset = sel_line.saturating_sub(visible.saturating_sub(1));
}
}
}