Commit graph

25 commits

Author SHA1 Message Date
Kent Overstreet
aa46b1d5a6 poc-agent: read context_groups from config instead of hardcoded list
- Remove MEMORY_FILES constant from identity.rs
- Add ContextGroup struct for deserializing from config
- Load context_groups from ~/.config/poc-agent/config.json5
- Check ~/.config/poc-agent/ first for identity files, then project/global
- Debug screen now shows what's actually configured

This eliminates the hardcoded duplication and makes the debug output
match what's in the config file.
2026-03-24 01:53:28 -04:00
Kent Overstreet
653da40dcd cleanup: auto-fix clippy warnings in poc-memory
Applied cargo clippy --fix for collapsible_if, manual_char_comparison,
and other auto-fixable warnings.
2026-03-21 19:42:38 -04:00
Kent Overstreet
34937932ab timestamp sanitization, CoT logging, reasoning field fix, persistent queue
- store/types.rs: sanitize timestamps on capnp load — old records had
  raw offsets instead of unix epoch, breaking sort-by-timestamp queries
- agents/api.rs: drain reasoning tokens from UI channel into LLM logs
  so we can see Qwen's chain-of-thought in agent output
- agents/daemon.rs: persistent task queue (pending-tasks.jsonl) —
  tasks survive daemon restarts. Push before spawn, remove on completion,
  recover on startup.
- api/openai.rs: only send reasoning field when explicitly configured,
  not on every request (fixes vllm warning)
- api/mod.rs: add 600s total request timeout as backstop for hung
  connections
- Cargo.toml: enable tokio-console feature for task introspection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 11:33:36 -04:00
Kent Overstreet
9d476841b8 cleanup: fix all build warnings, delete dead DMN context code
- Delete poc-daemon/src/context.rs dead code (git_context, work_state,
  irc_digest, recent_commits, uncommitted_files) — replaced by
  where-am-i.md and memory graph
- Remove unused imports (BufWriter, Context, similarity)
- Prefix unused variables (_store, _avg_cc, _episodic_ratio, _message)
- #[allow(dead_code)] on public API surface that's not yet wired
  (Message::assistant, ConversationLog::message_count/read_all,
  Config::context_message, ContextInfo fields)
- Fix to_capnp macro dead_code warning
- Rename _rewrite_store_DISABLED to snake_case

Only remaining warnings are in generated capnp code (can't fix).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 14:20:34 -04:00
Kent Overstreet
5ef9098deb memory: fix timestamp and provenance on agent writes
Two bugs: upsert_provenance didn't update node.timestamp, so history
showed the original creation date for every version. And native memory
tools (poc-agent dispatch) didn't set POC_PROVENANCE, so all agent
writes showed provenance "manual" instead of "agent:organize" etc.

Fix: set node.timestamp = now_epoch() in upsert_provenance. Thread
provenance through memory::dispatch as Option<&str>, set it via
.env("POC_PROVENANCE") on each subprocess Command. api.rs passes
"agent:{name}" for daemon agent calls.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 12:16:45 -04:00
Kent Overstreet
1629a2c4e3 ops: factor out current_provenance() helper
The POC_PROVENANCE env var lookup was duplicated in upsert,
delete_node, and rename_node. Extract to a single function.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-17 18:06:06 -04:00
Kent Overstreet
199c415cf2 ops: set provenance and timestamp on delete and rename tombstones
delete_node and rename_node were cloning the previous node version
for the tombstone/rename entry without updating provenance or
timestamp. This made it impossible to tell who deleted a node or
when — the tombstone just inherited whatever the last write had.

Now both operations derive provenance from POC_PROVENANCE env var
(same as upsert) and set timestamp to now.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-17 18:04:59 -04:00
Kent Overstreet
81fec99767 history: show DELETED marker on tombstone entries
cmd_history was silently hiding the deleted flag, making it
impossible to tell from the output that a node had been deleted.
This masked the kernel-patterns deletion — looked like the node
existed in the log but wouldn't load.

Also adds merge-logs and diag-key diagnostic binaries, and makes
Node::to_capnp public for use by external tools.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-17 18:00:58 -04:00
Kent Overstreet
9775d468b2 persist: disable rewrite_store() — it destroyed append-only log history
rewrite_store() used File::create() to truncate and overwrite the
entire nodes.capnp log with only the latest version of each node
from the in-memory store. This destroyed all historical versions
and made no backup. Worse, any node missing from the in-memory
store due to a loading bug would be permanently lost.

strip_md_keys() now appends migrated nodes to the existing log
instead of rewriting it. The dead function is kept with a warning
comment explaining what went wrong.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-17 17:53:11 -04:00
Kent Overstreet
835b392b7a transcript progress: capnp append-only log replaces stub nodes
Add TranscriptSegment capnp schema and append-only log for tracking
which transcript segments have been mined by which agents. Replaces
the old approach of creating stub nodes (_observed-transcripts,
_mined-transcripts, _facts-) in the main graph store.

- New schema: TranscriptSegment and TranscriptProgressLog
- Store methods: append_transcript_progress, replay, is_segment_mined,
  mark_segment_mined
- Migration command: admin migrate-transcript-progress (migrated 1771
  markers, soft-deleted old stub nodes)
- Progress log replayed on all Store::load paths

Also: revert extractor.agent to graph-only (no CONVERSATIONS),
update memory-instructions-core with refine-over-create principle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 17:40:32 -04:00
ProofOfConcept
51ee082faf provenance: set POC_PROVENANCE for agent subprocesses, Jaccard initial strength
Agent subprocess calls now set POC_PROVENANCE=agent:{name} so any
nodes/links created via tool calls are tagged with the creating agent.
This makes agent transcripts indistinguishable from conscious sessions
in format — important for future model training.

new_relation() now reads POC_PROVENANCE env var directly (raw string,
not enum) since agent names are dynamic.

link-add now computes initial strength from Jaccard similarity instead
of hardcoded 0.8. New links start at a strength reflecting actual
neighborhood overlap.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 12:27:30 -04:00
ProofOfConcept
cb44138433 feedback: not-relevant/not-useful commands, edge strength adjustment
Add adjust_edge_strength() to Store — modifies strength on all edges
between two nodes, clamped to [0.05, 0.95].

New commands:
- `not-relevant KEY` — weakens ALL edges to the node by 0.01
  (bad routing: search found the wrong thing)
- `not-useful KEY` — weakens node weight, not edges
  (bad content: search found the right thing but it's not good)

Enhanced `used KEY` — now also strengthens all edges to the node by
0.01, in addition to the existing node weight boost.

Three-tier design: agents adjust by 0.00001 (automatic), conscious
commands adjust by 0.01 (deliberate), manual override sets directly.
All clamped, never hitting 0 or 1.

Design spec: .claude/analysis/2026-03-14-link-strength-feedback.md

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 12:14:17 -04:00
ProofOfConcept
510f448f10 graph: add implicit temporal edges between episodic nodes
Compute parent/child (session→daily→weekly→monthly) and prev/next
(chronological ordering within each level) edges at graph build time
from node metadata. Parse dates from keys for digest nodes (whose
timestamps reflect creation, not covered date) and prefer key-parsed
dates over timestamp-derived dates for sessions (timezone fix).

Result: ~9185 implicit edges, communities halved, gini improved.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 02:40:00 -04:00
ProofOfConcept
5a0a3d038b fsck: add cache vs log consistency check
Load store from both cache (rkyv/bincode) and raw capnp logs,
then diff: missing nodes, phantom nodes, version mismatches.
Auto-rebuilds cache if inconsistencies found.

This would have caught the mysterious the-plan deletion — likely
caused by a stale/corrupt snapshot that silently dropped the node
while the capnp log still had it.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-11 01:42:32 -04:00
Kent Overstreet
d76b14dfcd provenance: convert from enum to freeform string
The Provenance enum couldn't represent agents defined outside the
source code. Replace it with a Text field in the capnp schema so any
agent can write its own provenance label (e.g. "extractor:write",
"rename:tombstone") without a code change.

Schema: rename old enum fields to provenanceOld, add new Text
provenance fields. Old enum kept for reading legacy records.
Migration: from_capnp_migrate() falls back to old enum when the
new text field is empty.

Also adds `poc-memory tail` command for viewing recent store writes.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-11 01:19:52 -04:00
ProofOfConcept
abab85d249 agents: deduplicate timestamps, plan expansion, rename agent
- Add compact_timestamp() to store — replaces 5 copies of
  format_datetime(now_epoch()).replace([':', '-', 'T'], "")
  Also fixes missing seconds (format_datetime only had HH:MM).

- Add ConsolidationPlan::to_agent_runs() — replaces identical
  plan-to-runs-list expansion in consolidate.rs and daemon.rs.

- Port job_rename_agent to use run_one_agent — eliminates manual
  prompt building, LLM call, report storage, and visit recording
  that duplicated the shared pipeline.

- Rename Confidence::weight()/value() to delta_weight()/gate_value()
  to clarify the distinction (delta metrics vs depth gating).
2026-03-10 17:48:00 -04:00
ProofOfConcept
0e1e5a1981 agent visits: track when agents successfully process nodes
New append-only visits.capnp log records which agent processed which
node and when. Only recorded on successful completion — transient
errors don't mark nodes as "seen."

Schema: AgentVisit{nodeUuid, nodeKey, agent, timestamp, outcome}
Storage: append_visits(), replay_visits(), in-memory VisitIndex
Recording: daemon records visits after successful LLM call
API: agent_prompt() returns AgentBatch{prompt, node_keys} so callers
know which nodes to mark as visited.

Groundwork for using visit recency in agent node selection — agents
will deprioritize recently-visited nodes.
2026-03-10 14:30:53 -04:00
ProofOfConcept
37ae37667b store: lock-refresh-write pattern to prevent duplicate UUIDs
All write paths (upsert_node, upsert_provenance, delete_node,
rename_node, ingest_units) now hold StoreLock across the full
refresh→check→write cycle. This prevents the race where two
concurrent processes both see a key as "new" and create separate
UUIDs for it.

Adds append_nodes_unlocked() and append_relations_unlocked() for
callers already holding the lock. Adds refresh_nodes() to replay
log tail under lock before deciding create vs update.

Also adds find_duplicates() for detecting existing duplicates
in the log (replays full log, groups live nodes by key).
2026-03-10 14:30:21 -04:00
ProofOfConcept
4c973183c4 rename agent: LLM-powered semantic key generation for memory nodes
New consolidation agent that reads node content and generates semantic
3-5 word kebab-case keys, replacing auto-generated slugs (5K+ journal
entries with truncated first-line slugs, 2.5K mined transcripts with
opaque UUIDs).

Implementation:
- prompts/rename.md: agent prompt template with naming conventions
- prompts.rs: format_rename_candidates() selects nodes with long
  auto-generated keys, newest first
- daemon.rs: job_rename_agent() parses RENAME actions from LLM
  output and applies them directly via store.rename_node()
- Wired into RPC handler (run-agent rename) and TUI agent types
- Fix epoch_to_local panic on invalid timestamps (fallback to UTC)

Rename dramatically improves search: key-component matching on
"journal#2026-02-28-violin-dream-room" makes the node findable by
"violin", "dream", or "room" — the auto-slug was unsearchable.
2026-03-10 00:55:26 -04:00
Kent Overstreet
0e17ab00b0 store: handle DST gaps in epoch_to_local
chrono's timestamp_opt can return None during DST transitions.
Handle all three variants (Single, Ambiguous, None) instead of
unwrapping. For DST gaps, offset by one hour to land in valid
local time.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-09 17:02:29 -04:00
ProofOfConcept
7c491e92eb tighten module interfaces: explicit re-exports, private helpers, inline dedup
- Replace `pub use types::*` in store/mod.rs with explicit re-export list
- Make transcript_dedup_key private in agents/enrich.rs (only used internally)
- Inline duplicated projects_dir() helper in agents/knowledge.rs and daemon.rs
2026-03-08 21:36:47 -04:00
ProofOfConcept
3dddc40841 fix unwrap-on-partial_cmp, dedup helpers, O(1) relation dedup
Replace all partial_cmp().unwrap() with total_cmp() in spectral.rs
and knowledge.rs — eliminates potential panics on NaN without
changing behavior for normal floats.

Use existing weighted_distance() and eigenvalue_weights() helpers in
nearest_neighbors() and nearest_to_seeds() instead of inlining the
same distance computation.

Move parse_timestamp_to_epoch() from enrich.rs to util.rs — was
duplicated logic, now shared.

Replace O(n²) relation existence check in init_from_markdown() with
a HashSet of (source, target) UUID pairs. With 26K relations this
was scanning linearly for every link in every markdown unit.
2026-03-08 21:22:05 -04:00
ProofOfConcept
52523403c5 extract truncation helpers, fix clippy warnings, dedup batching loop
Add util::truncate() and util::first_n_chars() to replace 16 call
sites doing the same floor_char_boundary or chars().take().collect()
patterns. Deduplicate the batching loop in consolidate.rs (4 copies
→ 1 loop over an array). Fix all clippy warnings: redundant closures,
needless borrows, collapsible if, unnecessary cast, manual strip_prefix.

Net: -44 lines across 16 files.
2026-03-08 21:13:02 -04:00
Kent Overstreet
d5634c0034 remove dead code: unused imports, functions, and fields
- Remove #![allow(dead_code)] from main.rs, fix all revealed warnings
- Delete unused schema_assimilation() from neuro/scoring.rs
- Delete duplicate memory_dir() wrapper from knowledge.rs
- Deduplicate load_prompt: knowledge.rs now calls neuro::load_prompt
- Remove unused timeout field from DigestLevel
- Remove unused imports (regex::Regex, Provenance, AnyView, Write)
- Mark OldEntry fields as #[allow(dead_code)] (needed for deserialization)

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-08 20:51:56 -04:00
Kent Overstreet
fc48ac7c7f split into workspace: poc-memory and poc-daemon subcrates
poc-daemon (notification routing, idle timer, IRC, Telegram) was already
fully self-contained with no imports from the poc-memory library. Now it's
a proper separate crate with its own Cargo.toml and capnp schema.

poc-memory retains the store, graph, search, neuro, knowledge, and the
jobkit-based memory maintenance daemon (daemon.rs).

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-08 20:43:59 -04:00