Commit graph

60 commits

Author SHA1 Message Date
ProofOfConcept
b115cec096 Run UI on a dedicated OS thread
The UI event loop was running on the same tokio runtime as inference,
tool execution, and background agents. When the runtime was busy, the
UI's select loop couldn't wake up to render — causing visible latency
and input lag.

Give the UI its own OS thread with a dedicated single-threaded tokio
runtime. The mind loop stays on the main runtime. Cross-runtime
communication (channels, watch, Notify) works unchanged.

Also drops the tokio-scoped dependency, which was only used to scope
the two tasks together.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 20:31:07 -04:00
ProofOfConcept
aad0cd669a Remove poc-memory daemon and RPC infrastructure
The background daemon and its job orchestration are redundant now that
the consciousness binary handles everything directly. Gut daemon.rs
down to just GraphHealth + compute_graph_health (used by the F4 TUI
screen), remove the DaemonCmd CLI subcommand, strip daemon RPC
fast-paths from cli/agent.rs, and drop the jobkit dependency.

-1330 lines.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 20:07:05 -04:00
Kent Overstreet
929415af3b delete claude code integration 2026-04-09 19:58:07 -04:00
ProofOfConcept
a596e007b2 Mouse selection, copy/paste, yield_to_user fixes
- Mouse text selection with highlight rendering in panes
- OSC 52 clipboard copy on selection, middle-click paste via tmux buffer
- Bracketed paste support (Event::Paste)
- yield_to_user: no tool result appended, ends turn immediately
- yield_to_user: no parameters, just a control signal
- Drop arboard dependency, use crossterm OSC 52 + tmux for clipboard

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 18:10:54 -04:00
ProofOfConcept
ec7e11db56 Add ast_grep tool: structural code search via ast-grep
AST-level pattern matching — find code by structure, not text.
e.g. find all `if let Some($X) = $Y { $$$BODY }` patterns.
Supports C, Rust, Python, JS/TS, Go, and 20+ languages.

Gracefully errors if sg binary isn't installed.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-09 11:38:24 -04:00
Kent Overstreet
9fb9c2b2cb Add serde derives to AST types, enable chrono serde feature
Prep for wiring context_new.rs into the codebase: AstNode, NodeLeaf,
NodeBody, Role all derive Serialize/Deserialize for conversation log
persistence.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 14:17:40 -04:00
Kent Overstreet
e9765799c4 Move tool definitions into ContextState as system entries
Tool definitions are now pushed as a ContextEntry in the system
section at Agent construction time, formatted in the Qwen chat
template style. They're tokenized, scored, and treated like any
other context entry.

assemble_prompt_tokens() no longer takes a tools parameter —
tools are already in the context. This prepares for the switch
to /v1/completions where tools aren't a separate API field.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 11:36:33 -04:00
Kent Overstreet
67e3228c32 Kill tiktoken — all token counting now uses Qwen 3.5 tokenizer
Remove tiktoken-rs dependency, CoreBPE field on Agent, and the
msg_token_count() function. All tokenization now goes through the
global HuggingFace tokenizer in agent/tokenizer.rs.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 11:25:28 -04:00
Kent Overstreet
5e4067c04f Replace token counting with token generation via HuggingFace tokenizer
Add agent/tokenizer.rs with global Qwen 3.5 tokenizer that generates
actual token IDs including chat template wrapping. ContextEntry now
stores token_ids: Vec<u32> instead of tokens: usize — the count is
derived from the length.

ContextEntry::new() tokenizes automatically via the global tokenizer.
ContextSection::push_entry() takes a raw ConversationEntry and
tokenizes it. set_message() re-tokenizes without needing an external
tokenizer parameter.

Token IDs include the full chat template: <|im_start|>role\ncontent
<|im_end|>\n — so concatenating token_ids across entries produces a
ready-to-send prompt for vLLM's /v1/completions endpoint.

The old tiktoken CoreBPE is now unused on Agent (will be removed in
a followup). Token counts are now exact for Qwen 3.5 instead of the
~85-90% approximation from cl100k_base.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 11:20:03 -04:00
Kent Overstreet
d7c93ffdf1 Upgrade redb 2 → 4, slim down tui-markdown
redb: add ReadableDatabase trait import for begin_read().
tui-markdown: disable highlight-code (drops syntect), fix
test deps leaking into normal dependencies.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-07 13:06:02 -04:00
Kent Overstreet
61b0a43cf5 Use tui-markdown fork — tracing fully eliminated
Point to koverstreet/tui-markdown which replaces tracing with log.
tracing is now completely gone from the dependency tree.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-07 13:00:29 -04:00
Kent Overstreet
3625764ca5 organize cargo deps 2026-04-07 12:54:23 -04:00
Kent Overstreet
1cf4f504c0 Kill reqwest — minimal HTTP client on raw hyper + tokio-rustls
New src/agent/api/http.rs: ~240 lines, supports GET/POST, JSON/form
bodies, SSE streaming via chunk(), TLS via rustls. No tracing dep.

Removes reqwest from the main crate and telegram channel crate.
Cargo.lock drops ~900 lines of transitive dependencies.

tracing now only pulled in by tui-markdown.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-07 12:50:40 -04:00
Kent Overstreet
a421c3c9f3 Upgrade capnp 0.20 → 0.25, capnp-rpc 0.20 → 0.25
RPC trait methods changed from &mut self to self: Rc<Self> and
return types from Promise<(), Error> to impl Future<Output = Result<...>>.

Updated all Server impls across 6 files: DaemonImpl (rpc.rs),
NotifyForwarder (channels.rs), and ChannelServerImpl in all channel
crates (irc, telegram, tmux, socat). Local pry! macro replaces
capnp_rpc::pry to match the new impl Future return type.

Warning-clean workspace build.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-07 12:29:44 -04:00
Kent Overstreet
8c1fef3c69 poc-daemon: subscribe to channel notifications, drop config.rs
Wire poc-daemon into channel daemon notifications via subscribe_all().
Channel notifications (IRC, telegram, tmux) now flow through the
existing notification pipeline instead of the dead module system.

Remove claude/config.rs — daemon config is fully covered by
channel config files in ~/.consciousness/channels/.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 12:58:46 -04:00
Kent Overstreet
93bc49959c deps: replace tracing with log+env_logger, drop console-subscriber
Switch all tracing::{info,warn,error} to log::{info,warn,error}.
Replace tracing_subscriber::fmt::init() with env_logger::init().
Drop tracing, tracing-subscriber, tracing-appender as direct deps.
Drop console feature from jobkit (was pulling in console-subscriber
which pulled in tracing-subscriber).

tracing still compiled as transitive dep of reqwest and tui-markdown,
but our code no longer depends on it.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 06:54:11 -04:00
Kent Overstreet
917960cb76 deps: remove faer (224 transitive crates)
Spectral decomposition (eigenvalue computation) removed — it was
only used by the spectral-save CLI command. The spectral embedding
reader and query engine features remain (they load pre-computed
embeddings from disk, no faer needed).

Removes: faer, nano-gemm, private-gemm, and ~220 other transitive
dependencies. Significant build time and artifact size reduction.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 06:39:47 -04:00
Kent Overstreet
c1a5638be5 deps: switch reqwest to rustls, drop aws-lc-sys
Use rustls instead of default native-tls (aws-lc-sys) for HTTPS.
Saves ~80 MB of build artifacts. Applied to both main crate and
telegram channel daemon.

Also: tracing default-features = false (Kent's edit).

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 06:31:50 -04:00
Kent Overstreet
8d045a3e6b use ratatui::crossterm re-exports, add event-stream feature
All crossterm imports go through ratatui::crossterm. Direct crossterm
dep kept only for the event-stream feature flag (EventStream for
async terminal input).

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 06:22:31 -04:00
Kent Overstreet
2ab4fd1c92 Trim unused deps
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 06:06:38 -04:00
Kent Overstreet
7b75296457 Update reqwest
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 06:02:25 -04:00
Kent Overstreet
3f79bba27a Update json5
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 06:01:24 -04:00
Kent Overstreet
5eaba3c951 mind: use tokio-scoped for Mind/UI loop lifetimes
Both event loops borrow &mind through a scoped spawn — no Arc on
Mind needed. Interior Arcs on agent/shared stay (background spawns
need 'static), but event_loop::run() now takes &Arc refs instead
of cloned Arcs.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 05:58:51 -04:00
Kent Overstreet
7123c9166d channels: add socat daemon for generic stream channels
consciousness-channel-socat listens on a unix socket for incoming
connections, turning each into a bidirectional text channel. Also
supports outbound connections via the open RPC (tcp: or unix:).

Two sockets:
  socat.sock        — capnp RPC (channel protocol)
  socat.stream.sock — data (incoming connections become channels)

No config file needed. The simplest possible channel daemon.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-05 01:18:47 -04:00
ProofOfConcept
1ef137fb3a channels: add tmux pane channel daemon
Standalone daemon that streams tmux pane output via pipe-pane
(no polling). Each configured pane becomes a channel "tmux.<label>"
accessible through the standard channel.capnp protocol.

- pipe-pane streams PTY output directly to FIFOs
- Async readers push new lines into ChannelLogs
- send works via tmux send-keys
- Cleanup disconnects pipe-pane on daemon exit

Config: ~/.consciousness/channels/tmux.json5
Socket: ~/.consciousness/channels/tmux.sock

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-04 18:19:21 -04:00
ProofOfConcept
61deb7d488 delete dead code: channel-test, mcp-schema, cmd_mcp_schema
channel-test was a debug tool, mcp-schema was superseded by
consciousness-mcp, cmd_mcp_schema in cli/misc.rs was the old
poc-memory subcommand.

Co-Developed-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-03 20:52:40 -04:00
ProofOfConcept
e104a16f61 consciousness-mcp: full MCP server in Rust
Replaces the Python MCP bridge. Single binary speaks JSON-RPC
over stdio, exposes 14 tools:
- 10 memory tools (delegate to poc-memory CLI)
- channel_list, channel_recv, channel_send, channel_notifications

No external dependencies beyond serde_json. Channel tools use
capnp RPC to talk to daemon sockets directly.

Co-Developed-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-03 20:30:07 -04:00
ProofOfConcept
56fc3a20d8 move mcp-schema to standalone binary in src/claude/
mcp-schema is Claude Code glue — extract from poc-memory
subcommand to src/claude/mcp-schema.rs standalone binary.
Update Python MCP bridge to call the new binary.

Co-Developed-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-03 20:22:48 -04:00
Kent Overstreet
dd7f1e3f86 move Claude Code-specific code from thalamus/ to claude/
Separates the Claude-specific daemon (idle timer, tmux pane detection,
prompt injection, RPC server, session hooks) from the universal
infrastructure (channels, supervisor, notify, daemon protocol).

thalamus/ now contains only substrate-independent code: the channel
client/supervisor, notification system, daemon_capnp protocol, and
shared helpers (now(), home()).

claude/ contains: idle.rs, tmux.rs, context.rs, rpc.rs, config.rs,
hook.rs (moved from subconscious/), plus the daemon CLI and server
startup code from thalamus/mod.rs.

All re-exports preserved for backward compatibility.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-03 19:26:24 -04:00
ProofOfConcept
48b8ba73d8 delete old thalamus subcrate
All code was already merged into src/thalamus/. The poc-daemon
binary is now a thin wrapper in src/bin/poc-daemon.rs.
2026-04-03 19:03:12 -04:00
ProofOfConcept
ad5f69abb8 channel architecture: wire protocol, daemons, supervisor
Design and implement the channel system for external communications:

- schema/channel.capnp: wire protocol for channel daemons
  (recv with all_new/min_count, send, subscribe, list)
- channels/irc/: standalone IRC daemon crate (consciousness-channel-irc)
- channels/telegram/: standalone Telegram daemon crate
  (consciousness-channel-telegram)
- src/thalamus/channels.rs: client connecting to daemon sockets
- src/thalamus/supervisor.rs: daemon lifecycle with file locking
  for multi-instance safety

Channel daemons listen on ~/.consciousness/channels/*.sock,
configs in *.json5, supervisor discovers and starts them.
IRC/Telegram modules removed from thalamus core — they're
now independent daemons that survive consciousness restarts.

Also: delete standalone tui.rs (moved to consciousness F4/F5),
fix build warnings, add F5 thalamus screen with channel status.

Co-Developed-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-04-03 18:46:41 -04:00
Kent Overstreet
604f442215 Move thalamus subcrate into main crate
Move thalamus/ (poc-daemon) source files into src/thalamus/ as a
module of the main crate. The daemon entry point becomes a library
function thalamus::run() with a thin poc-daemon binary for backward
compatibility.

- Copy thalamus source into src/thalamus/, fix crate:: -> super::
- Copy daemon.capnp into schema/, add to build.rs
- Re-export daemon_capnp at crate root (capnp codegen requires it)
- Add thalamus dependencies (capnp-rpc, tokio-util, toml, rustls, etc.)
- Keep thalamus/ subcrate for comparison

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-03 17:31:17 -04:00
Kent Overstreet
14dd8d22af Rename agent/ to user/ and poc-agent binary to consciousness
Mechanical rename: src/agent/ -> src/user/, all crate::agent:: ->
crate::user:: references updated. Binary poc-agent renamed to
consciousness with CLI name and user-facing strings updated.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-03 17:25:59 -04:00
Kent Overstreet
6d17e82843 use git URL for jobkit instead of local path
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 19:46:43 -04:00
Kent Overstreet
1fd4ce05c1 add console-subscriber 0.5 dependency
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 19:31:17 -04:00
Kent Overstreet
0148dbaa06 add tokio-console for async task debugging
console-subscriber on unix socket at
~/.consciousness/agent-sessions/console.sock.
Connect with: tokio-console ~/.consciousness/agent-sessions/console.sock

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-02 18:21:01 -04:00
ProofOfConcept
29ce56845d remove old poc-agent directory
Source merged into src/agent/ — these are dead copies.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-25 01:06:57 -04:00
ProofOfConcept
d5c0e86700 restructure: hippocampus/ for memory, subconscious/ for agents
hippocampus/ — memory storage, retrieval, and consolidation:
  store, graph, query, similarity, spectral, neuro, counters,
  config, transcript, memory_search, lookups, cursor, migrate

subconscious/ — autonomous agents that process without being asked:
  reflect, surface, consolidate, digest, audit, etc.

All existing crate::X paths preserved via re-exports in lib.rs.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-25 01:05:30 -04:00
ProofOfConcept
998b71e52c flatten: move poc-memory contents to workspace root
No more subcrate nesting — src/, agents/, schema/, defaults/, build.rs
all live at the workspace root. poc-daemon remains as the only workspace
member. Crate name (poc-memory) and all imports unchanged.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-25 00:54:12 -04:00
ProofOfConcept
891cca57f8 merge poc-agent into poc-memory as agent/ module
Eliminates the circular dependency between poc-agent and poc-memory by
moving all poc-agent source into poc-memory/src/agent/. The poc-agent
binary now builds from poc-memory/src/bin/poc-agent.rs using library
imports. All poc_agent:: references updated to crate::agent::.

poc-agent/ directory kept for now (removed from workspace members).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-03-25 00:52:41 -04:00
Kent Overstreet
a321f87db6 build: add tokio_unstable and codegen-units to cargo config
console-subscriber (used by jobkit's console feature) requires tokio
to be built with --cfg tokio_unstable. Move this and codegen-units=6
from RUSTFLAGS env var to .cargo/config.toml so per-project cargo
config actually works (env var RUSTFLAGS overrides config.toml).

Also remove invalid frame-pointer keys from Cargo.toml profile
sections — frame pointers are already handled via -Cforce-frame-pointers
in the config.toml rustflags.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 15:04:38 -04:00
Kent Overstreet
869a2fbc38 observation agent rewrite, edit command, daemon fixes
- observation.agent: rewritten to navigate graph and prefer refining
  existing nodes over creating new ones. Identity-framed prompt,
  goals over rules.
- poc-memory edit: opens node in $EDITOR, writes back on save,
  no-op if unchanged
- daemon: remove extra_workers (jobkit tokio migration dropped it),
  remove sequential chaining of same-type agents (in-flight exclusion
  is sufficient)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 23:51:06 -04:00
Kent Overstreet
1b48e57f34 Remove jobkit-daemon from workspace members
jobkit-daemon is now an external git dependency with its own repo.
The local clone was only needed temporarily to fix a broken
Cargo.toml in the remote.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 22:59:21 -04:00
Kent Overstreet
0a62832fe3 Upgrade workspace to edition 2024, add --local flag to agent run
Edition 2024 changes:
- gen is reserved: rename variable in query/engine.rs
- set_var is unsafe: wrap in unsafe block in cli/agent.rs
- match ergonomics: add explicit & in spectral.rs filter closure

New --local flag for `poc-memory agent run` bypasses the daemon and
runs the agent directly in-process. Useful for testing agent prompt
changes without waiting in the daemon queue.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 22:44:36 -04:00
ProofOfConcept
420a777eba extract jobkit-daemon library from poc-memory daemon
Create jobkit-daemon crate with generic daemon infrastructure:
- event_log: JSONL append with size-based rotation
- socket: Unix domain socket RPC client and server with signal handling
- status: JSON status file read/write

Migrate daemon.rs to use the library:
- Worker pool setup via Daemon::new()
- Socket loop + signal handling via Daemon::run()
- RPC handlers as registered closures
- Logging, status writing, send_rpc all delegate to library

Migrate tui.rs to use socket::send_rpc() instead of inline UnixStream.

daemon.rs: 1952 → 1806 lines (-146), old status_socket_loop removed.
tui.rs: socket boilerplate removed.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 02:40:30 -04:00
ProofOfConcept
5024cf7002 enable frame pointers and debug info in release builds
So we can profile with perf when the daemon spins.
2026-03-12 18:11:09 -04:00
Kent Overstreet
fc48ac7c7f split into workspace: poc-memory and poc-daemon subcrates
poc-daemon (notification routing, idle timer, IRC, Telegram) was already
fully self-contained with no imports from the poc-memory library. Now it's
a proper separate crate with its own Cargo.toml and capnp schema.

poc-memory retains the store, graph, search, neuro, knowledge, and the
jobkit-based memory maintenance daemon (daemon.rs).

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-08 20:43:59 -04:00
Kent Overstreet
908f8c9e52 docs: split README into component docs, update jobkit dep
- Break README into README.md (overview), docs/daemon.md (pipeline
  stages, diagnostics, common issues), docs/notifications.md
  (notification daemon, IRC/Telegram modules)
- Update jobkit dependency from local path to git URL

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-07 13:56:09 -05:00
Kent Overstreet
aa24c40a1c Extract lib.rs, inline search in memory-search hook
Create lib.rs so all binaries can share library code directly instead
of shelling out to poc-memory. memory-search now calls search::search()
and store::Store::load() in-process instead of Command::new("poc-memory").

The load-context call still shells out (needs get_group_content moved
from main.rs to a library module).

Also: add search::format_results(), deduplicate extract_query_terms.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 22:23:03 -05:00
ProofOfConcept
d0080698f3 cli: switch to clap, add notify-timeout, improve status display
Replace manual arg parsing with clap derive for the full command set.
Single source of truth for command names, args, and help text.

Add notify_timeout (default 2min) — controls how long after last
response before notifications inject via tmux instead of waiting
for the hook. Separate from idle_timeout (5min) which controls
autonomous prompts.

Improve `poc-daemon status` to show both timers with elapsed/configured
and block reason, replacing the terse one-liner.

Add new Status fields over capnp: idleTimeout, notifyTimeout,
sinceActivity, sinceUser, blockReason.

ExecStart in poc-daemon.service now uses `daemon` subcommand.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-05 21:32:27 -05:00