Add journal_cycle() to memory_search.rs, triggered every 20KB of
transcript growth. Runs independently of the surface-observe pipeline
so it doesn't depend on the 5-step pipeline surviving bail checks.
Journal agent doesn't inject output into conversation context (unlike
surface and reflect) — it just writes episodic memory entries.
Split TASK_PROVENANCE into TASK_AGENT (task_local, set once per agent
run) and TASK_PHASE (thread_local, updated between steps). Provenance
now reports "agent:surface-observe:observe" instead of just
"agent:surface-observe", making it possible to identify which pipeline
phase created a node.
Priority: task_local agent + thread_local phase > POC_PROVENANCE env
var > "manual".
Also includes memory_search catchup throttle and pipelining fixes
from the surface-observe refactor.
TranscriptInfo provides cached transcript metadata (path, size)
with a single read. Replaces scattered fs::metadata calls in
surface_observe_cycle, reflection_cycle, resolve_conversation,
and resolve_memory_ratio.
Session::transcript() resolves the path from transcript_path or
by searching projects dir, returning a TranscriptInfo.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Catchup throttle: when the agent is >50% behind the conversation
window (>25KB of transcript growth since last spawn), block and
wait up to 30s for the current agent to finish. Prevents the agent
from falling behind during heavy reading/studying.
Reflection agent: runs every 100KB of transcript growth. Reads
walked nodes from surface-observe, follows links in unexpected
directions, outputs a short dreamy insight. Previous reflections
are injected into the conversation context.
Updated reflect.agent prompt to use {{input:walked}} from
surface-observe state dir and {{conversation:20000}} for lighter
context.
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
- journal_new: create EpisodicSession node with auto-generated key
- journal_tail: query by node_type, not by parsing a monolithic node
- journal_update: find latest EpisodicSession by timestamp
- No string key matching anywhere — all typed
- Fixes journal entries not appearing in 'poc-memory journal tail'
- Also: added --provenance/-p filter to 'poc-memory tail'
- Also: fix early return in surface_observe_cycle store load failure
- Also: scale max_turns by number of steps (50 per step)
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
- Remove unused now_secs(), parse_json_response, any_alive, Regex import
- Signal handler: replace Mutex with AtomicPtr<c_char> for signal safety
(Mutex::lock in a signal handler can deadlock if main thread holds it)
- PidGuard Drop reclaims the leaked CString; signal handler just unlinks
- scan_pid_files moved to knowledge.rs as pub helper
- setup_agent_state calls scan_pid_files to clean stale pids on startup
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
- Bail command moved from hardcoded closure to external script
specified in agent JSON header ("bail": "bail-no-competing.sh")
- Runner executes script between steps with pid file path as $1,
cwd = state dir. Non-zero exit stops the pipeline.
- PID files simplified to just the phase name (no JSON) for easy
bash inspection (cat pid-*)
- scan_pid_files helper deduplicates pid scanning logic
- Timeout check uses file mtime instead of embedded timestamp
- PID file cleaned up on bail/error (not just success)
- output() tool validates key names (rejects pid-*, /, ..)
- Agent log files append instead of truncate
- Fixed orphaned derive and doc comment on AgentStep/AgentDef
- Phase written after bail check passes, not before
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
- AgentStep with phase labels (=== PROMPT phase:name ===)
- PID files in state dir (pid-{PID} with JSON phase/timestamp)
- Built-in bail check: between steps, bail if other pid files exist
- surface_observe_cycle replaces surface_agent_cycle + journal_agent_cycle
- Reads surface output from state dir instead of parsing stdout
- Pipelining: starts new agent if running one is past surface phase
- link_set upserts (creates link if missing)
- Better error message for context window overflow
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
Generic session state (session_id, seen set, state directory) doesn't
belong in the memory search module. Now at crate root, re-exported
from memory_search for backwards compatibility.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
hippocampus/ — memory storage, retrieval, and consolidation:
store, graph, query, similarity, spectral, neuro, counters,
config, transcript, memory_search, lookups, cursor, migrate
subconscious/ — autonomous agents that process without being asked:
reflect, surface, consolidate, digest, audit, etc.
All existing crate::X paths preserved via re-exports in lib.rs.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
