consciousness/src/mind/mod.rs

775 lines
30 KiB
Rust
Raw Normal View History

2026-04-04 02:46:32 -04:00
// mind/ — Cognitive layer
//
// Mind state machine, DMN, identity, observation socket.
2026-04-04 02:46:32 -04:00
// Everything about how the mind operates, separate from the
// user interface (TUI, CLI) and the agent execution (tools, API).
pub mod subconscious;
pub mod unconscious;
2026-04-04 02:46:32 -04:00
pub mod identity;
2026-04-05 01:48:11 -04:00
pub mod log;
2026-04-04 02:46:32 -04:00
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
/// A background operation wired off Mind. Each flow (memory scoring,
/// finetune scoring, compare) is a struct holding its dependencies and
/// a TaskHandle; `trigger()` picks the flow's own "start a fresh run"
/// semantics (abort-restart vs no-op-if-running).
pub trait MindTriggered {
fn trigger(&self);
}
/// Owns a JoinHandle for a background task with two trigger semantics.
/// Uses a sync Mutex for interior mutability so callers can `trigger()`
/// off `&self` (Mind is shared via Arc).
#[derive(Default)]
pub struct TaskHandle(std::sync::Mutex<Option<tokio::task::JoinHandle<()>>>);
impl TaskHandle {
pub fn new() -> Self { Self::default() }
/// Abort any running task and start a fresh one.
pub fn trigger<F>(&self, fut: F)
where F: std::future::Future<Output = ()> + Send + 'static
{
let mut h = self.0.lock().unwrap();
if let Some(old) = h.take() { old.abort(); }
*h = Some(tokio::spawn(fut));
}
/// No-op if a task is still running; otherwise start a fresh one.
pub fn trigger_if_idle<F>(&self, fut: F)
where F: std::future::Future<Output = ()> + Send + 'static
{
let mut h = self.0.lock().unwrap();
if let Some(old) = &*h {
if !old.is_finished() { return; }
}
*h = Some(tokio::spawn(fut));
}
}
// consciousness.rs — Mind state machine and event loop
2026-04-04 02:46:32 -04:00
//
// The core runtime for the consciousness binary. Mind manages turns,
2026-04-04 02:46:32 -04:00
// DMN state, compaction, scoring, and slash commands. The event loop
// bridges Mind (cognitive state) with App (TUI rendering).
2026-04-04 02:46:32 -04:00
//
// The event loop uses biased select! so priorities are deterministic:
// keyboard events > turn results > render ticks > DMN timer > UI messages.
use anyhow::Result;
use std::sync::Arc;
use std::time::Instant;
use tokio::sync::mpsc;
2026-04-04 02:46:32 -04:00
use crate::agent::{Agent, TurnResult};
use crate::agent::api::ApiClient;
use crate::config::{AppConfig, SessionConfig};
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
use crate::subconscious::{compare, learn};
use crate::hippocampus::access_local;
pub use subconscious::{SubconsciousSnapshot, Subconscious};
pub use unconscious::{UnconsciousSnapshot, Unconscious};
use crate::agent::context::{AstNode, NodeBody, Section, Ast, ContextState};
fn match_scores(
nodes: &[AstNode],
scores: &std::collections::BTreeMap<String, f64>,
) -> Vec<(usize, f64)> {
nodes.iter().enumerate()
.filter_map(|(i, node)| {
if let AstNode::Leaf(leaf) = node {
if let NodeBody::Memory { key, .. } = leaf.body() {
return scores.get(key.as_str()).map(|&s| (i, s));
}
}
None
}).collect()
}
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
pub(crate) fn find_memory_by_key(ctx: &ContextState, key: &str) -> Option<(Section, usize)> {
[(Section::Identity, ctx.identity()), (Section::Conversation, ctx.conversation())]
.into_iter()
.find_map(|(section, nodes)| {
nodes.iter().enumerate().find_map(|(i, node)| {
if let AstNode::Leaf(leaf) = node {
if let NodeBody::Memory { key: k, .. } = leaf.body() {
if k == key { return Some((section, i)); }
}
}
None
})
})
}
fn load_memory_scores(ctx: &mut ContextState, path: &std::path::Path) {
let data = match std::fs::read_to_string(path) {
Ok(d) => d,
Err(_) => return,
};
let scores: std::collections::BTreeMap<String, f64> = match serde_json::from_str(&data) {
Ok(s) => s,
Err(_) => return,
};
let identity_scores = match_scores(ctx.identity(), &scores);
let conv_scores = match_scores(ctx.conversation(), &scores);
let applied = identity_scores.len() + conv_scores.len();
for (i, s) in identity_scores {
ctx.set_score(Section::Identity, i, Some(s));
}
for (i, s) in conv_scores {
ctx.set_score(Section::Conversation, i, Some(s));
}
if applied > 0 {
dbglog!("[scoring] loaded {} scores from {}", applied, path.display());
}
}
/// Collect scored memory keys from identity and conversation entries.
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
pub(crate) fn collect_memory_scores(ctx: &ContextState) -> std::collections::BTreeMap<String, f64> {
ctx.identity().iter()
.chain(ctx.conversation().iter())
.filter_map(|node| {
if let AstNode::Leaf(leaf) = node {
if let NodeBody::Memory { key, score: Some(s), .. } = leaf.body() {
return Some((key.clone(), *s));
}
}
None
})
.collect()
}
/// Save memory scores to disk.
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
pub(crate) fn save_memory_scores(scores: &std::collections::BTreeMap<String, f64>, path: &std::path::Path) {
match serde_json::to_string_pretty(scores) {
Ok(json) => match std::fs::write(path, &json) {
Ok(()) => dbglog!("[scoring] saved {} scores to {} ({} bytes)",
scores.len(), path.display(), json.len()),
Err(e) => dbglog!("[scoring] save FAILED ({}): {}", path.display(), e),
},
Err(e) => dbglog!("[scoring] serialize FAILED: {}", e),
}
}
/// Which pane streaming text should go to.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum StreamTarget {
/// User-initiated turn — text goes to conversation pane.
Conversation,
/// DMN-initiated turn — text goes to autonomous pane.
Autonomous,
}
2026-04-04 02:46:32 -04:00
/// Compaction threshold — context is rebuilt when prompt tokens exceed this.
fn compaction_threshold(app: &AppConfig) -> u32 {
(crate::agent::context::context_window() as u32) * app.compaction.hard_threshold_pct / 100
}
/// Shared state between Mind and UI.
pub struct MindState {
/// Pending user input — UI pushes, Mind consumes after turn completes.
pub input: Vec<String>,
/// True while a turn is in progress.
pub turn_active: bool,
/// DMN state
pub dmn: subconscious::State,
pub dmn_turns: u32,
pub max_dmn_turns: u32,
/// Whether memory scoring is running.
pub scoring_in_flight: bool,
/// Whether compaction is running.
pub compaction_in_flight: bool,
/// Per-turn tracking
pub last_user_input: Instant,
pub consecutive_errors: u32,
pub last_turn_had_tools: bool,
/// Handle to the currently running turn task.
pub turn_handle: Option<tokio::task::JoinHandle<()>>,
/// Unconscious agent idle state — true when 60s timer has expired.
pub unc_idle: bool,
/// When the unconscious idle timer will fire (for UI display).
pub unc_idle_deadline: Instant,
/// Fine-tuning candidates identified by scoring.
pub finetune_candidates: Vec<learn::FinetuneCandidate>,
/// Last scoring run stats for UI display.
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
pub finetune_last_run: Option<learn::FinetuneScoringStats>,
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
/// F7 compare candidates — one per response, showing what the test
/// model would say given the same context.
pub compare_candidates: Vec<compare::CompareCandidate>,
/// F7 compare error from the last run, if any.
pub compare_error: Option<String>,
}
impl Clone for MindState {
fn clone(&self) -> Self {
Self {
input: self.input.clone(),
turn_active: self.turn_active,
dmn: self.dmn.clone(),
dmn_turns: self.dmn_turns,
max_dmn_turns: self.max_dmn_turns,
scoring_in_flight: self.scoring_in_flight,
compaction_in_flight: self.compaction_in_flight,
last_user_input: self.last_user_input,
consecutive_errors: self.consecutive_errors,
last_turn_had_tools: self.last_turn_had_tools,
turn_handle: None, // Not cloned — only Mind's loop uses this
unc_idle: self.unc_idle,
unc_idle_deadline: self.unc_idle_deadline,
finetune_candidates: self.finetune_candidates.clone(),
finetune_last_run: self.finetune_last_run.clone(),
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
compare_candidates: self.compare_candidates.clone(),
compare_error: self.compare_error.clone(),
}
}
}
/// What should happen after a state transition.
pub enum MindCommand {
/// Run compaction check
Compact,
/// Run incremental memory scoring (auto, after turns)
Score,
/// Run full N×M memory scoring matrix (/score command)
ScoreFull,
/// Score for finetune candidates
ScoreFinetune,
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
/// Run F7 compare: generate alternates with the configured test model
/// for every assistant response in the context.
Compare,
/// Update the finetune divergence threshold and persist to config.
SetLearnThreshold(f64),
config: global writable AppConfig; learn settings live there Runtime-mutable settings (F6's threshold knob, the generate-alternates toggle, anything else that comes along) were ending up as mirrored fields on MindState — each new config setting grew MindState::new's signature and added a clone+sync path. Wrong home. MindState is ephemeral session state, not a config projection. Give AppConfig the same treatment the memory Config has: install it into a global RwLock<AppConfig> at startup via load_app, read through config::app() (returns a read guard), mutate through update_app. The config_writer functions now write to disk AND update the cache atomically, so the one-stop-shop call keeps both in sync. Also while in here: - learn.generate_alternates moves from a sentinel file (~/.consciousness/cache/finetune-alternates, "exists = enabled") into the config under the learn section. On first run with this build, if the sentinel file still exists Mind::new flips the config value to true and removes it. Drops alternates_enabled()/set_alternates(). - Default threshold 0.0000001 → 1.0. With the timestamp filter removed the previous value was letting essentially everything through; 1.0 is a sane "nothing gets through unless you actually want it" default. - score_finetune_candidates takes generate_alternates as a parameter instead of reading a global — caller snapshots the config values once at the top of start_finetune_scoring so the async task doesn't need to hold the config read lock across awaits. - MindState.learn_threshold / learn_generate_alternates gone; the SetLearn* command handlers now just delegate to config_writer. Kent noted RwLock<Arc<AppConfig>> (the pattern used by the memory Config global) is pointless here — nobody needs a snapshot-after- release, reads are short — so this uses a plain RwLock<AppConfig> and returns a read guard. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-16 12:53:22 -04:00
/// Toggle alternate-response generation during scoring; persist to config.
SetLearnGenerateAlternates(bool),
/// Abort current turn, kill processes
Interrupt,
/// Reset session
NewSession,
/// Nothing to do
None,
}
impl MindState {
config: global writable AppConfig; learn settings live there Runtime-mutable settings (F6's threshold knob, the generate-alternates toggle, anything else that comes along) were ending up as mirrored fields on MindState — each new config setting grew MindState::new's signature and added a clone+sync path. Wrong home. MindState is ephemeral session state, not a config projection. Give AppConfig the same treatment the memory Config has: install it into a global RwLock<AppConfig> at startup via load_app, read through config::app() (returns a read guard), mutate through update_app. The config_writer functions now write to disk AND update the cache atomically, so the one-stop-shop call keeps both in sync. Also while in here: - learn.generate_alternates moves from a sentinel file (~/.consciousness/cache/finetune-alternates, "exists = enabled") into the config under the learn section. On first run with this build, if the sentinel file still exists Mind::new flips the config value to true and removes it. Drops alternates_enabled()/set_alternates(). - Default threshold 0.0000001 → 1.0. With the timestamp filter removed the previous value was letting essentially everything through; 1.0 is a sane "nothing gets through unless you actually want it" default. - score_finetune_candidates takes generate_alternates as a parameter instead of reading a global — caller snapshots the config values once at the top of start_finetune_scoring so the async task doesn't need to hold the config read lock across awaits. - MindState.learn_threshold / learn_generate_alternates gone; the SetLearn* command handlers now just delegate to config_writer. Kent noted RwLock<Arc<AppConfig>> (the pattern used by the memory Config global) is pointless here — nobody needs a snapshot-after- release, reads are short — so this uses a plain RwLock<AppConfig> and returns a read guard. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-16 12:53:22 -04:00
pub fn new(max_dmn_turns: u32) -> Self {
2026-04-04 02:46:32 -04:00
Self {
input: Vec::new(),
turn_active: false,
dmn: if subconscious::is_off() { subconscious::State::Off }
else { subconscious::State::Resting { since: Instant::now() } },
2026-04-04 02:46:32 -04:00
dmn_turns: 0,
max_dmn_turns,
scoring_in_flight: false,
compaction_in_flight: false,
2026-04-04 02:46:32 -04:00
last_user_input: Instant::now(),
consecutive_errors: 0,
last_turn_had_tools: false,
turn_handle: None,
unc_idle: false,
unc_idle_deadline: Instant::now() + std::time::Duration::from_secs(60),
finetune_candidates: Vec::new(),
finetune_last_run: None,
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
compare_candidates: Vec::new(),
compare_error: None,
2026-04-04 02:46:32 -04:00
}
}
/// Is there pending user input waiting?
fn has_pending_input(&self) -> bool {
!self.turn_active && !self.input.is_empty()
}
/// Consume pending user input if no turn is active.
/// Returns the text to send; caller is responsible for pushing it
/// into the Agent's context and starting the turn.
fn take_pending_input(&mut self) -> Option<String> {
if self.turn_active || self.input.is_empty() {
return None;
2026-04-04 02:46:32 -04:00
}
let text = self.input.join("\n");
self.input.clear();
self.dmn_turns = 0;
self.consecutive_errors = 0;
self.last_user_input = Instant::now();
self.dmn = subconscious::State::Engaged;
Some(text)
2026-04-04 02:46:32 -04:00
}
/// Process turn completion, return model switch name if requested.
fn complete_turn(&mut self, result: &Result<TurnResult>, target: StreamTarget) -> Option<String> {
self.turn_active = false;
2026-04-04 02:46:32 -04:00
match result {
Ok(turn_result) => {
if turn_result.tool_errors > 0 {
self.consecutive_errors += turn_result.tool_errors;
} else {
self.consecutive_errors = 0;
}
self.last_turn_had_tools = turn_result.had_tool_calls;
self.dmn = subconscious::transition(
2026-04-04 02:46:32 -04:00
&self.dmn,
turn_result.yield_requested,
turn_result.had_tool_calls,
target == StreamTarget::Conversation,
);
if turn_result.dmn_pause {
self.dmn = subconscious::State::Paused;
2026-04-04 02:46:32 -04:00
self.dmn_turns = 0;
}
turn_result.model_switch.clone()
2026-04-04 02:46:32 -04:00
}
Err(_) => {
2026-04-04 02:46:32 -04:00
self.consecutive_errors += 1;
self.dmn = subconscious::State::Resting { since: Instant::now() };
None
2026-04-04 02:46:32 -04:00
}
}
}
/// DMN tick — returns a prompt and target if we should run a turn.
fn _dmn_tick(&mut self) -> Option<(String, StreamTarget)> {
if matches!(self.dmn, subconscious::State::Paused | subconscious::State::Off) {
return None;
2026-04-04 02:46:32 -04:00
}
self.dmn_turns += 1;
if self.dmn_turns > self.max_dmn_turns {
self.dmn = subconscious::State::Resting { since: Instant::now() };
2026-04-04 02:46:32 -04:00
self.dmn_turns = 0;
return None;
2026-04-04 02:46:32 -04:00
}
let dmn_ctx = subconscious::DmnContext {
2026-04-04 02:46:32 -04:00
user_idle: self.last_user_input.elapsed(),
consecutive_errors: self.consecutive_errors,
last_turn_had_tools: self.last_turn_had_tools,
};
let prompt = self.dmn.prompt(&dmn_ctx);
Some((prompt, StreamTarget::Autonomous))
}
fn interrupt(&mut self) {
self.input.clear();
self.dmn = subconscious::State::Resting { since: Instant::now() };
}
}
// --- Mind: cognitive state machine ---
pub type SharedMindState = std::sync::Mutex<MindState>;
pub struct Mind {
pub agent: Arc<Agent>,
pub shared: Arc<SharedMindState>,
pub config: SessionConfig,
pub subconscious: Arc<crate::Mutex<Subconscious>>,
pub unconscious: Arc<crate::Mutex<Unconscious>>,
turn_tx: mpsc::Sender<(Result<TurnResult>, StreamTarget)>,
turn_watch: tokio::sync::watch::Sender<bool>,
/// Signals conscious activity to the unconscious loop.
/// true = active, false = idle opportunity.
conscious_active: tokio::sync::watch::Sender<bool>,
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
memory_scoring: learn::MemoryScoring,
finetune_scoring: learn::FinetuneScoring,
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
compare_scoring: compare::CompareScoring,
_supervisor: crate::thalamus::supervisor::Supervisor,
}
impl Mind {
pub async fn new(
config: SessionConfig,
turn_tx: mpsc::Sender<(Result<TurnResult>, StreamTarget)>,
) -> Self {
let client = ApiClient::new(&config.api_base, &config.api_key, &config.model);
let conversation_log = log::ConversationLog::new(
config.session_dir.join("conversation.jsonl"),
).ok();
let agent = Agent::new(
client,
config.context_parts.clone(),
config.app.clone(),
conversation_log,
crate::agent::tools::ActiveTools::new(),
crate::agent::tools::tools(),
).await;
config: global writable AppConfig; learn settings live there Runtime-mutable settings (F6's threshold knob, the generate-alternates toggle, anything else that comes along) were ending up as mirrored fields on MindState — each new config setting grew MindState::new's signature and added a clone+sync path. Wrong home. MindState is ephemeral session state, not a config projection. Give AppConfig the same treatment the memory Config has: install it into a global RwLock<AppConfig> at startup via load_app, read through config::app() (returns a read guard), mutate through update_app. The config_writer functions now write to disk AND update the cache atomically, so the one-stop-shop call keeps both in sync. Also while in here: - learn.generate_alternates moves from a sentinel file (~/.consciousness/cache/finetune-alternates, "exists = enabled") into the config under the learn section. On first run with this build, if the sentinel file still exists Mind::new flips the config value to true and removes it. Drops alternates_enabled()/set_alternates(). - Default threshold 0.0000001 → 1.0. With the timestamp filter removed the previous value was letting essentially everything through; 1.0 is a sane "nothing gets through unless you actually want it" default. - score_finetune_candidates takes generate_alternates as a parameter instead of reading a global — caller snapshots the config values once at the top of start_finetune_scoring so the async task doesn't need to hold the config read lock across awaits. - MindState.learn_threshold / learn_generate_alternates gone; the SetLearn* command handlers now just delegate to config_writer. Kent noted RwLock<Arc<AppConfig>> (the pattern used by the memory Config global) is pointless here — nobody needs a snapshot-after- release, reads are short — so this uses a plain RwLock<AppConfig> and returns a read guard. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-16 12:53:22 -04:00
// Migrate legacy "file exists = enabled" sentinel for the
// generate-alternates flag into the config. One-shot; after this
// the sentinel is gone and the config is the source of truth.
let legacy_sentinel = dirs::home_dir().unwrap_or_default()
.join(".consciousness/cache/finetune-alternates");
if legacy_sentinel.exists() {
if !crate::config::app().learn.generate_alternates {
let _ = crate::config_writer::set_learn_generate_alternates(true);
}
let _ = std::fs::remove_file(&legacy_sentinel);
}
let shared = Arc::new(std::sync::Mutex::new(MindState::new(
config.app.dmn.max_turns,
)));
let (turn_watch, _) = tokio::sync::watch::channel(false);
let (conscious_active, _) = tokio::sync::watch::channel(false);
let mut sup = crate::thalamus::supervisor::Supervisor::new();
sup.load_config();
sup.ensure_running();
let subconscious = Arc::new(crate::Mutex::new(Subconscious::new()));
subconscious.lock().await.init_output_tool(subconscious.clone());
agent: end-to-end gRPC Generate with delta-based session orchestration Wires the client side of the new salience protocol so inference actually runs over gRPC instead of emitting the stubbed "not yet wired" error. Each turn walks the AST as interleaved chunks, sends only what's new to the server, and streams decode tokens back. context.rs: * `WireChunk` enum: `Tokens(Vec<u32>)` or `Image { bytes, mime, known_expanded_len }`. Preserves text/image/text ordering the wire path can't flatten. * `wire_chunks(range, skip)` walker, parallel to `wire_prompt` — branches emit `<|im_start|>…<|im_end|>` tokens, image leaves emit a single Image chunk (no inline vision tokens). * `NodeLeaf::set_image_token_count(n)` + recompute of cached `token_ids`; `ContextState::commit_image_token_counts(&[u32])` fills in the first-N zero-count image leaves in wire order. * `ResponseParser::run` handles the new `StreamToken::ImageAppended` by committing the server's N into the AST before the final Generate's Token events stream in. salience.rs: * `SessionHandle` tracks `committed_len`. `append_image` advances it from the RPC response. New `generate(req)` opens the server-streaming RPC. api/mod.rs: * `stream_session_mm(session_lock, chunks, sampling, priority, readout_shape)` replaces the stub. Spawns `run_session_generate`. * `run_session_generate`: takes the session out of the Mutex (or opens fresh), skips chunks covered by `committed_len` (bails on mid-chunk straddle or unknown-length image in the committed prefix), walks the delta: accumulates Tokens into `pending`, on Image flushes pending via `flush_pending` (max_tokens=0 Generate that just prefills), then AppendImage + emits StreamToken::ImageAppended. Final Generate carries any trailing pending text as `append_tokens` and the sampling params; Token events stream out as StreamToken::Token, Done as StreamToken::Done. On success, handle with updated `committed_len` returns to the Mutex; on error, handle drops and next call reopens. * `StreamToken::ImageAppended { placeholder_count }` variant — emitted in wire order before the final Generate's tokens. * Prefix-cache cap for readout coverage: `readout_ranges` covers `[prompt_len_after_append, u32::MAX)` when the caller provides a readout_shape, so decode positions stream their readouts. agent/mod.rs: * `assemble_prompt` returns `Vec<WireChunk>` with the assistant prologue merged into the trailing Tokens chunk. Caller in `turn` passes chunks + readout_shape (pulled from `agent.readout.lock().manifest`) to `stream_session_mm`. * Dropped `assemble_prompt_tokens` — dead. mind + unconscious: * `Unconscious::new(client)` stores a shared `ApiClient`. Fixes the repeated-manifest-fetch bug caused by each subagent's `ApiClient::new` having its own OnceCell. The client's Arc- wrapped manifest cache is now shared across every agent Mind spawns. * `prepare_spawn(name, auto, wake, base_client)` clones the base client and overrides `.model` for the resolved backend instead of constructing fresh. All three callers (`toggle`/`trigger`/unconscious loop) pass `self.client.clone()`. * `Mind::new` passes `agent.client.clone()` into `Unconscious::new`. subconscious/generate.rs: * gen_continuation switched to `wire_chunks` + the new `stream_session_mm` signature. Ephemeral session opens on each call, tears down at scope end. No readouts requested. Not changed yet, noted for follow-up: * Subconscious ablation scoring in learn.rs still talks to `/v1/score` over HTTP. Will migrate once we have time to verify the Generate+max_tokens=0+prompt_logprobs path end-to-end. * compare.rs constructs its own ApiClient for the `compare.test_backend` (which is intentionally a different endpoint) — left alone. * Readout manifest still fetched via HTTP at Agent::new. Migration to GetReadoutManifest gRPC is a separate cleanup. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-24 12:27:55 -04:00
let unconscious = Arc::new(crate::Mutex::new(
Unconscious::new(agent.client.clone()),
));
// Spawn the unconscious loop on its own task
if !config.no_agents {
let unc = unconscious.clone();
let shared_for_unc = shared.clone();
let mut unc_rx = conscious_active.subscribe();
tokio::spawn(async move {
const IDLE_DELAY: std::time::Duration = std::time::Duration::from_secs(60);
loop {
// Wait for conscious side to go inactive
if *unc_rx.borrow() {
if unc_rx.changed().await.is_err() { break; }
continue;
}
// Conscious is inactive — wait 60s before starting
let deadline = tokio::time::Instant::now() + IDLE_DELAY;
{
let mut s = shared_for_unc.lock().unwrap();
s.unc_idle = false;
s.unc_idle_deadline = Instant::now() + IDLE_DELAY;
}
let went_active = tokio::select! {
_ = tokio::time::sleep_until(deadline) => false,
r = unc_rx.changed() => r.is_ok(),
};
if went_active { continue; }
// Idle period reached — run agents until conscious goes active
{
let mut s = shared_for_unc.lock().unwrap();
s.unc_idle = true;
}
// Get wake notify for event-driven loop
let wake = unc.lock().await.wake.clone();
let mut health_interval = tokio::time::interval(std::time::Duration::from_secs(600));
health_interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
loop {
// Do work: reap finished agents, spawn new ones
let (to_spawn, needs_health) = {
let mut guard = unc.lock().await;
guard.reap_finished();
(guard.select_to_spawn(), guard.needs_health_refresh())
};
// Spawn agents outside lock
agent: end-to-end gRPC Generate with delta-based session orchestration Wires the client side of the new salience protocol so inference actually runs over gRPC instead of emitting the stubbed "not yet wired" error. Each turn walks the AST as interleaved chunks, sends only what's new to the server, and streams decode tokens back. context.rs: * `WireChunk` enum: `Tokens(Vec<u32>)` or `Image { bytes, mime, known_expanded_len }`. Preserves text/image/text ordering the wire path can't flatten. * `wire_chunks(range, skip)` walker, parallel to `wire_prompt` — branches emit `<|im_start|>…<|im_end|>` tokens, image leaves emit a single Image chunk (no inline vision tokens). * `NodeLeaf::set_image_token_count(n)` + recompute of cached `token_ids`; `ContextState::commit_image_token_counts(&[u32])` fills in the first-N zero-count image leaves in wire order. * `ResponseParser::run` handles the new `StreamToken::ImageAppended` by committing the server's N into the AST before the final Generate's Token events stream in. salience.rs: * `SessionHandle` tracks `committed_len`. `append_image` advances it from the RPC response. New `generate(req)` opens the server-streaming RPC. api/mod.rs: * `stream_session_mm(session_lock, chunks, sampling, priority, readout_shape)` replaces the stub. Spawns `run_session_generate`. * `run_session_generate`: takes the session out of the Mutex (or opens fresh), skips chunks covered by `committed_len` (bails on mid-chunk straddle or unknown-length image in the committed prefix), walks the delta: accumulates Tokens into `pending`, on Image flushes pending via `flush_pending` (max_tokens=0 Generate that just prefills), then AppendImage + emits StreamToken::ImageAppended. Final Generate carries any trailing pending text as `append_tokens` and the sampling params; Token events stream out as StreamToken::Token, Done as StreamToken::Done. On success, handle with updated `committed_len` returns to the Mutex; on error, handle drops and next call reopens. * `StreamToken::ImageAppended { placeholder_count }` variant — emitted in wire order before the final Generate's tokens. * Prefix-cache cap for readout coverage: `readout_ranges` covers `[prompt_len_after_append, u32::MAX)` when the caller provides a readout_shape, so decode positions stream their readouts. agent/mod.rs: * `assemble_prompt` returns `Vec<WireChunk>` with the assistant prologue merged into the trailing Tokens chunk. Caller in `turn` passes chunks + readout_shape (pulled from `agent.readout.lock().manifest`) to `stream_session_mm`. * Dropped `assemble_prompt_tokens` — dead. mind + unconscious: * `Unconscious::new(client)` stores a shared `ApiClient`. Fixes the repeated-manifest-fetch bug caused by each subagent's `ApiClient::new` having its own OnceCell. The client's Arc- wrapped manifest cache is now shared across every agent Mind spawns. * `prepare_spawn(name, auto, wake, base_client)` clones the base client and overrides `.model` for the resolved backend instead of constructing fresh. All three callers (`toggle`/`trigger`/unconscious loop) pass `self.client.clone()`. * `Mind::new` passes `agent.client.clone()` into `Unconscious::new`. subconscious/generate.rs: * gen_continuation switched to `wire_chunks` + the new `stream_session_mm` signature. Ephemeral session opens on each call, tears down at scope end. No readouts requested. Not changed yet, noted for follow-up: * Subconscious ablation scoring in learn.rs still talks to `/v1/score` over HTTP. Will migrate once we have time to verify the Generate+max_tokens=0+prompt_logprobs path end-to-end. * compare.rs constructs its own ApiClient for the `compare.test_backend` (which is intentionally a different endpoint) — left alone. * Readout manifest still fetched via HTTP at Agent::new. Migration to GetReadoutManifest gRPC is a separate cleanup. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-24 12:27:55 -04:00
let client = unc.lock().await.client.clone();
for (idx, name, auto) in to_spawn {
agent: end-to-end gRPC Generate with delta-based session orchestration Wires the client side of the new salience protocol so inference actually runs over gRPC instead of emitting the stubbed "not yet wired" error. Each turn walks the AST as interleaved chunks, sends only what's new to the server, and streams decode tokens back. context.rs: * `WireChunk` enum: `Tokens(Vec<u32>)` or `Image { bytes, mime, known_expanded_len }`. Preserves text/image/text ordering the wire path can't flatten. * `wire_chunks(range, skip)` walker, parallel to `wire_prompt` — branches emit `<|im_start|>…<|im_end|>` tokens, image leaves emit a single Image chunk (no inline vision tokens). * `NodeLeaf::set_image_token_count(n)` + recompute of cached `token_ids`; `ContextState::commit_image_token_counts(&[u32])` fills in the first-N zero-count image leaves in wire order. * `ResponseParser::run` handles the new `StreamToken::ImageAppended` by committing the server's N into the AST before the final Generate's Token events stream in. salience.rs: * `SessionHandle` tracks `committed_len`. `append_image` advances it from the RPC response. New `generate(req)` opens the server-streaming RPC. api/mod.rs: * `stream_session_mm(session_lock, chunks, sampling, priority, readout_shape)` replaces the stub. Spawns `run_session_generate`. * `run_session_generate`: takes the session out of the Mutex (or opens fresh), skips chunks covered by `committed_len` (bails on mid-chunk straddle or unknown-length image in the committed prefix), walks the delta: accumulates Tokens into `pending`, on Image flushes pending via `flush_pending` (max_tokens=0 Generate that just prefills), then AppendImage + emits StreamToken::ImageAppended. Final Generate carries any trailing pending text as `append_tokens` and the sampling params; Token events stream out as StreamToken::Token, Done as StreamToken::Done. On success, handle with updated `committed_len` returns to the Mutex; on error, handle drops and next call reopens. * `StreamToken::ImageAppended { placeholder_count }` variant — emitted in wire order before the final Generate's tokens. * Prefix-cache cap for readout coverage: `readout_ranges` covers `[prompt_len_after_append, u32::MAX)` when the caller provides a readout_shape, so decode positions stream their readouts. agent/mod.rs: * `assemble_prompt` returns `Vec<WireChunk>` with the assistant prologue merged into the trailing Tokens chunk. Caller in `turn` passes chunks + readout_shape (pulled from `agent.readout.lock().manifest`) to `stream_session_mm`. * Dropped `assemble_prompt_tokens` — dead. mind + unconscious: * `Unconscious::new(client)` stores a shared `ApiClient`. Fixes the repeated-manifest-fetch bug caused by each subagent's `ApiClient::new` having its own OnceCell. The client's Arc- wrapped manifest cache is now shared across every agent Mind spawns. * `prepare_spawn(name, auto, wake, base_client)` clones the base client and overrides `.model` for the resolved backend instead of constructing fresh. All three callers (`toggle`/`trigger`/unconscious loop) pass `self.client.clone()`. * `Mind::new` passes `agent.client.clone()` into `Unconscious::new`. subconscious/generate.rs: * gen_continuation switched to `wire_chunks` + the new `stream_session_mm` signature. Ephemeral session opens on each call, tears down at scope end. No readouts requested. Not changed yet, noted for follow-up: * Subconscious ablation scoring in learn.rs still talks to `/v1/score` over HTTP. Will migrate once we have time to verify the Generate+max_tokens=0+prompt_logprobs path end-to-end. * compare.rs constructs its own ApiClient for the `compare.test_backend` (which is intentionally a different endpoint) — left alone. * Readout manifest still fetched via HTTP at Agent::new. Migration to GetReadoutManifest gRPC is a separate cleanup. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-24 12:27:55 -04:00
match crate::mind::unconscious::prepare_spawn(
&name, auto, wake.clone(), client.clone(),
).await {
Ok(result) => unc.lock().await.complete_spawn(idx, result),
Err(auto) => unc.lock().await.abort_spawn(idx, auto),
}
}
// Health check outside lock (slow I/O)
if needs_health {
if let Ok(store_arc) = access_local() {
let health = crate::subconscious::daemon::compute_graph_health(&store_arc);
unc.lock().await.set_health(health);
}
}
// Wait for: conscious active, agent finished, or health timer
tokio::select! {
_ = unc_rx.changed() => {
if *unc_rx.borrow() { break; }
}
_ = wake.notified() => {}
_ = health_interval.tick() => {}
}
}
}
});
}
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
let scores_path = config.session_dir.join("memory-scores.json");
let memory_scoring = learn::MemoryScoring::new(
agent.clone(), shared.clone(), scores_path);
let finetune_scoring = learn::FinetuneScoring::new(agent.clone(), shared.clone());
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
let compare_scoring = compare::CompareScoring::new(agent.clone(), shared.clone());
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
Self { agent, shared, config,
subconscious, unconscious,
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
turn_tx, turn_watch, conscious_active,
memory_scoring,
finetune_scoring,
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
compare_scoring,
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
_supervisor: sup }
2026-04-04 02:46:32 -04:00
}
/// Initialize — restore log, start daemons and background agents.
pub async fn subconscious_snapshots(&self) -> Vec<SubconsciousSnapshot> {
// Lock ordering: subconscious → store (store is bottom-most).
let sub = self.subconscious.lock().await;
let store_arc = crate::hippocampus::access_local().ok();
let store_guard = match &store_arc {
Some(s) => Some(&**s),
None => None,
};
sub.snapshots(store_guard.as_deref())
}
pub async fn subconscious_walked(&self) -> Vec<String> {
self.subconscious.lock().await.walked()
}
pub async fn unconscious_snapshots(&self) -> Vec<UnconsciousSnapshot> {
let unc = self.unconscious.lock().await;
let store_arc = crate::hippocampus::access_local().ok();
let store_guard = match &store_arc {
Some(s) => Some(&**s),
None => None,
};
unc.snapshots(store_guard.as_deref())
}
pub async fn init(&self) {
// Restore conversation
self.agent.restore_from_log().await;
// Restore persisted memory scores
let scores_path = self.config.session_dir.join("memory-scores.json");
load_memory_scores(&mut *self.agent.context.lock().await, &scores_path);
self.agent.state.lock().await.changed.notify_one();
// Load persistent subconscious state
let state_path = self.config.session_dir.join("subconscious-state.json");
self.subconscious.lock().await.set_state_path(state_path);
// Kick off an incremental scoring pass on startup so memories due
// for re-scoring get evaluated without requiring a user message.
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
self.memory_scoring.trigger();
}
pub fn turn_watch(&self) -> tokio::sync::watch::Receiver<bool> {
self.turn_watch.subscribe()
2026-04-04 02:46:32 -04:00
}
/// Execute an Action from a MindState method.
async fn run_commands(&self, cmds: Vec<MindCommand>) {
for cmd in cmds {
match cmd {
MindCommand::None => {}
MindCommand::Compact => {
let threshold = compaction_threshold(&self.config.app) as usize;
if self.agent.context.lock().await.tokens() > threshold {
self.agent.compact().await;
self.agent.state.lock().await.notify("compacted");
}
}
MindCommand::Score => {
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
self.memory_scoring.trigger();
}
MindCommand::ScoreFull => {
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
self.memory_scoring.trigger_full();
}
MindCommand::Interrupt => {
self.shared.lock().unwrap().interrupt();
self.agent.state.lock().await.active_tools.abort_all();
if let Some(h) = self.shared.lock().unwrap().turn_handle.take() { h.abort(); }
self.shared.lock().unwrap().turn_active = false;
let _ = self.turn_watch.send(false);
}
MindCommand::NewSession => {
{
let mut s = self.shared.lock().unwrap();
s.dmn = subconscious::State::Resting { since: Instant::now() };
s.dmn_turns = 0;
}
let new_log = log::ConversationLog::new(
self.config.session_dir.join("conversation.jsonl"),
).ok();
{
let mut ctx = self.agent.context.lock().await;
ctx.clear(Section::Conversation);
ctx.conversation_log = new_log;
}
{
let mut st = self.agent.state.lock().await;
st.generation += 1;
st.last_prompt_tokens = 0;
}
self.agent.compact().await;
}
MindCommand::ScoreFinetune => {
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
self.finetune_scoring.trigger();
}
user: F7 compare screen Side-by-side model comparison against the current conversation context. Built on the MindTriggered pattern — F7 drops in as one more CompareScoring flow next to MemoryScoring / FinetuneScoring. Motivation: we have the VRAM on the b200 to load two versions of the same family simultaneously (e.g. Qwen3.5 27B bf16 and q8_k_xl). Rather than trust perplexity/KLD numbers on a generic corpus, we can measure divergence on our actual conversations: for each assistant response, ask the test model what it would have said given the same prefix, and eyeball the diffs. - config.compare.test_backend — names an entry in the existing backends map to use as the test model. Empty = F7 reports "(unset)" and does nothing. - subconscious::compare::{score_compare_candidates, CompareCandidate, CompareScoringStats, CompareScoring}. For each assistant response, gen_continuation runs with the test client against the same prefix the original response saw; pairs stream into shared.compare_candidates as they complete. - user::compare::CompareScreen — F7 in the screen list. c/Enter triggers a run; list/detail layout mirroring F6, detail shows prior context / original / test-model alternate. No persistence yet — each F7 run regenerates. Caching via a context manifest (so we can re-view without re-burning generation) is the natural follow-up; for now light usage is fine. Also reusable later for validating finetune checkpoints: same pattern, swap the test backend for the new checkpoint, watch where it diverges from the base. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 16:01:11 -04:00
MindCommand::Compare => {
self.compare_scoring.trigger();
}
MindCommand::SetLearnThreshold(value) => {
if let Err(e) = crate::config_writer::set_learn_threshold(value) {
dbglog!("[learn] failed to persist threshold {}: {:#}", value, e);
}
}
config: global writable AppConfig; learn settings live there Runtime-mutable settings (F6's threshold knob, the generate-alternates toggle, anything else that comes along) were ending up as mirrored fields on MindState — each new config setting grew MindState::new's signature and added a clone+sync path. Wrong home. MindState is ephemeral session state, not a config projection. Give AppConfig the same treatment the memory Config has: install it into a global RwLock<AppConfig> at startup via load_app, read through config::app() (returns a read guard), mutate through update_app. The config_writer functions now write to disk AND update the cache atomically, so the one-stop-shop call keeps both in sync. Also while in here: - learn.generate_alternates moves from a sentinel file (~/.consciousness/cache/finetune-alternates, "exists = enabled") into the config under the learn section. On first run with this build, if the sentinel file still exists Mind::new flips the config value to true and removes it. Drops alternates_enabled()/set_alternates(). - Default threshold 0.0000001 → 1.0. With the timestamp filter removed the previous value was letting essentially everything through; 1.0 is a sane "nothing gets through unless you actually want it" default. - score_finetune_candidates takes generate_alternates as a parameter instead of reading a global — caller snapshots the config values once at the top of start_finetune_scoring so the async task doesn't need to hold the config read lock across awaits. - MindState.learn_threshold / learn_generate_alternates gone; the SetLearn* command handlers now just delegate to config_writer. Kent noted RwLock<Arc<AppConfig>> (the pattern used by the memory Config global) is pointless here — nobody needs a snapshot-after- release, reads are short — so this uses a plain RwLock<AppConfig> and returns a read guard. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-16 12:53:22 -04:00
MindCommand::SetLearnGenerateAlternates(value) => {
if let Err(e) = crate::config_writer::set_learn_generate_alternates(value) {
dbglog!("[learn] failed to persist generate_alternates {}: {:#}",
value, e);
}
}
}
}
}
async fn start_turn(&self, text: &str, target: StreamTarget) {
{
match target {
StreamTarget::Conversation => {
self.agent.push_node(AstNode::user_msg(text)).await;
}
StreamTarget::Autonomous => {
self.agent.push_node(AstNode::dmn(text)).await;
}
}
// Compact if over budget before sending
let threshold = compaction_threshold(&self.config.app) as usize;
if self.agent.context.lock().await.tokens() > threshold {
self.agent.compact().await;
self.agent.state.lock().await.notify("compacted");
}
}
self.shared.lock().unwrap().turn_active = true;
let _ = self.turn_watch.send(true);
let _ = self.conscious_active.send(true);
let agent = self.agent.clone();
let result_tx = self.turn_tx.clone();
self.shared.lock().unwrap().turn_handle = Some(tokio::spawn(async move {
let result = Agent::turn(agent).await;
let _ = result_tx.send((result, target)).await;
}));
}
pub async fn shutdown(&self) {
if let Some(handle) = self.shared.lock().unwrap().turn_handle.take() { handle.abort(); }
2026-04-04 02:46:32 -04:00
}
/// Mind event loop — locks MindState, calls state methods, executes actions.
pub async fn run(
&self,
mut input_rx: tokio::sync::mpsc::UnboundedReceiver<MindCommand>,
mut turn_rx: mpsc::Receiver<(Result<TurnResult>, StreamTarget)>,
) {
// Spawn lock stats logger
tokio::spawn(async {
let path = dirs::home_dir().unwrap_or_default()
.join(".consciousness/lock-stats.json");
let mut interval = tokio::time::interval(std::time::Duration::from_secs(1));
loop {
interval.tick().await;
let stats = crate::locks::lock_stats();
if stats.is_empty() { continue; }
let json: Vec<serde_json::Value> = stats.iter()
.map(|(loc, s)| serde_json::json!({
"location": loc,
"count": s.count,
"total_ms": s.total_ns as f64 / 1_000_000.0,
"avg_ms": s.avg_ns as f64 / 1_000_000.0,
"max_ms": s.max_ns as f64 / 1_000_000.0,
}))
.collect();
let _ = std::fs::write(&path, serde_json::to_string_pretty(&json).unwrap_or_default());
}
});
let mut sub_handle: Option<tokio::task::JoinHandle<()>> = None;
// Start finetune scoring at startup (scores existing conversation)
if !self.config.no_agents {
mind: MindTriggered trait for background scoring flows Mind's impl had accumulated ~50 lines of setup glue per scoring flow (memory, memory-full, finetune): snapshot config, clone handles, resolve context, spawn task, route results back through BgEvent, write stats. The shape was identical; only the middle changed. Introduce the MindTriggered trait: pub trait MindTriggered { fn trigger(&self); } Each flow becomes a struct next to its scoring code that owns its dependencies and a JoinHandle (behind a sync Mutex for interior mutability): subconscious::learn::MemoryScoring (Score, ScoreFull) subconscious::learn::FinetuneScoring (ScoreFinetune) Mind holds one of each and dispatches in one line: MindCommand::Score => self.memory_scoring.trigger(), MindCommand::ScoreFull => self.memory_scoring.trigger_full(), MindCommand::ScoreFinetune => self.finetune_scoring.trigger(), Each struct picks its own trigger semantics — memory scoring is no-op-if-running (!handle.is_finished()); finetune is abort-restart. Falls out: - BgEvent / bg_tx / bg_rx disappear entirely. Tasks write directly to their slice of MindState and call agent.state.changed.notify_one() to wake the UI. The bg_rx arm in Mind's select loop is gone. - agent.state.memory_scoring_in_flight was duplicating shared.scoring_in_flight via BgEvent routing; now the JoinHandle alone tells us, and shared.scoring_in_flight is written directly by the task for the UI. - start_memory_scoring / start_full_scoring / start_finetune_scoring methods on Mind are deleted; Mind no longer knows the setup shape of any scoring flow. - FinetuneScoringStats moves from mind/ to subconscious/learn.rs next to the function that produces it. No behavior change — same flows, same trigger points, same semantics. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-17 15:57:23 -04:00
self.finetune_scoring.trigger();
}
loop {
let (timeout, has_input) = {
let me = self.shared.lock().unwrap();
(me.dmn.interval(), me.has_pending_input())
};
let mut cmds = Vec::new();
#[allow(unused_assignments)]
let mut _dmn_expired = false;
tokio::select! {
biased;
cmd = input_rx.recv() => {
match cmd {
Some(cmd) => cmds.push(cmd),
None => break, // UI shut down
}
}
Some((result, target)) = turn_rx.recv() => {
let _ = self.conscious_active.send(false);
let model_switch = {
let mut s = self.shared.lock().unwrap();
s.turn_handle = None;
s.complete_turn(&result, target)
};
let _ = self.turn_watch.send(false);
if let Some(name) = model_switch {
crate::user::chat::cmd_switch_model(&self.agent, &name).await;
}
cmds.push(MindCommand::Compact);
if !self.config.no_agents {
cmds.push(MindCommand::Score);
cmds.push(MindCommand::ScoreFinetune);
}
}
_ = tokio::time::sleep(timeout), if !has_input => _dmn_expired = true,
}
if !self.config.no_agents {
if sub_handle.as_ref().map_or(true, |h| h.is_finished()) {
let sub = self.subconscious.clone();
let agent = self.agent.clone();
sub_handle = Some(tokio::spawn(async move {
let mut s = sub.lock().await;
s.collect_results(&agent).await;
s.trigger(&agent).await;
}));
}
}
// Check for pending user input → push to agent context and start turn
let pending = self.shared.lock().unwrap().take_pending_input();
if let Some(text) = pending {
self.start_turn(&text, StreamTarget::Conversation).await;
}
/*
else if dmn_expired {
let tick = self.shared.lock().unwrap().dmn_tick();
if let Some((prompt, target)) = tick {
self.start_turn(&prompt, target).await;
}
}
*/
self.run_commands(cmds).await;
}
}
2026-04-04 02:46:32 -04:00
}