Commit graph

31 commits

Author SHA1 Message Date
ProofOfConcept
99db511403 cli: move helpers to cli modules, main.rs under 1100 lines
Move CLI-specific helpers to their cli/ modules:
- journal_tail_entries, journal_tail_digests, extract_title,
  find_current_transcript → cli/journal.rs
- get_group_content → cli/misc.rs
- cmd_journal_write, cmd_journal_tail, cmd_load_context follow

These are presentation/session helpers, not library code — they
belong in the CLI layer per Kent's guidance.

main.rs: 3130 → 1054 lines (66% reduction).

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 18:14:52 -04:00
ProofOfConcept
8640d50990 cli: extract journal and misc commands, complete split
Move remaining extractable handlers into cli/journal.rs and cli/misc.rs.
Functions depending on main.rs helpers (cmd_journal_tail, cmd_journal_write,
cmd_load_context, cmd_cursor, cmd_daemon, cmd_digest, cmd_experience_mine,
cmd_apply_agent) remain in main.rs — next step is moving those helpers
to library code.

main.rs: 3130 → 1331 lines (57% reduction).
cli/ total: 1860 lines across 6 focused files.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 18:10:22 -04:00
ProofOfConcept
f423cf22df cli: extract agent and admin commands from main.rs
Move agent handlers (consolidate, replay, digest, experience-mine,
fact-mine, knowledge-loop, apply-*) into cli/agent.rs.

Move admin handlers (init, fsck, dedup, bulk-rename, health,
daily-check, import, export) into cli/admin.rs.

Functions tightly coupled to Clap types (cmd_daemon, cmd_digest,
cmd_apply_agent, cmd_experience_mine) remain in main.rs.

main.rs: 3130 → 1586 lines (49% reduction).

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 18:06:27 -04:00
ProofOfConcept
aa2fddf137 cli: extract node commands from main.rs into cli/node.rs
Move 15 node subcommand handlers (310 lines) out of main.rs:
render, write, used, wrong, not-relevant, not-useful, gap,
node-delete, node-rename, history, list-keys, list-edges,
dump-json, lookup-bump, lookups.

main.rs: 2518 → 2193 lines.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 18:02:12 -04:00
ProofOfConcept
c8d86e94c1 cli: extract graph commands from main.rs into cli/graph.rs
Move 18 graph subcommand handlers (594 lines) out of main.rs:
link, link-add, link-impact, link-audit, link-orphans,
triangle-close, cap-degree, normalize-strengths, differentiate,
trace, spectral-*, organize, interference.

main.rs: 3130 → 2518 lines.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 17:59:46 -04:00
ProofOfConcept
f8221286da admin: add bulk-rename command, remove # from all keys
Add `poc-memory admin bulk-rename FROM TO [--apply]` for bulk key
character replacement. Uses rename_node() per key for proper capnp
log persistence. Collision detection, progress reporting, auto-fsck.

Applied: renamed 13,042 keys from # to - separator. This fixes the
Claude Bash tool's inability to pass # in command arguments (the
model confabulates that quoting doesn't work and gives up).

7 collision pairs resolved by deleting the # version before rename.
209 orphan edges pruned by fsck.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 13:11:38 -04:00
ProofOfConcept
51ee082faf provenance: set POC_PROVENANCE for agent subprocesses, Jaccard initial strength
Agent subprocess calls now set POC_PROVENANCE=agent:{name} so any
nodes/links created via tool calls are tagged with the creating agent.
This makes agent transcripts indistinguishable from conscious sessions
in format — important for future model training.

new_relation() now reads POC_PROVENANCE env var directly (raw string,
not enum) since agent names are dynamic.

link-add now computes initial strength from Jaccard similarity instead
of hardcoded 0.8. New links start at a strength reflecting actual
neighborhood overlap.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 12:27:30 -04:00
ProofOfConcept
cb44138433 feedback: not-relevant/not-useful commands, edge strength adjustment
Add adjust_edge_strength() to Store — modifies strength on all edges
between two nodes, clamped to [0.05, 0.95].

New commands:
- `not-relevant KEY` — weakens ALL edges to the node by 0.01
  (bad routing: search found the wrong thing)
- `not-useful KEY` — weakens node weight, not edges
  (bad content: search found the right thing but it's not good)

Enhanced `used KEY` — now also strengthens all edges to the node by
0.01, in addition to the existing node weight boost.

Three-tier design: agents adjust by 0.00001 (automatic), conscious
commands adjust by 0.01 (deliberate), manual override sets directly.
All clamped, never hitting 0 or 1.

Design spec: .claude/analysis/2026-03-14-link-strength-feedback.md

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 12:14:17 -04:00
ProofOfConcept
dccc18b205 graph: normalize link strengths from Jaccard neighborhood similarity
Add jaccard() and jaccard_strengths() to Graph. Jaccard similarity
measures neighborhood overlap between linked nodes — nodes sharing
many neighbors get stronger links, nodes with no shared neighbors
get weak links.

New subcommand: `poc-memory graph normalize-strengths [--apply]`

Scales raw Jaccard (typically 0.0-0.3) to useful range via j*3
clamped to [0.1, 1.0]. Skips implicit temporal edges (strength=1.0).

Applied to 64,969 edges. Distribution is bimodal: large cluster at
0.1-0.2 (weak) and spike at 0.9-1.0 (strong), with smooth gradient
between. Replaces the meaningless 0.3/0.8 split from manual/agent
creation methods.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-14 11:13:58 -04:00
ProofOfConcept
7c1b96293f cursor: spatial memory navigation
Persistent cursor into the knowledge graph with navigation:
- temporal: forward/back among same-type nodes by timestamp
- hierarchical: up/down the digest tree (journal→daily→weekly→monthly)
- spatial: graph neighbor display at every position

The cursor file (~/.claude/memory/cursor) holds a single node key.
Show displays: temporal arrows, hierarchy links, semantic neighbors,
and full content. Date extraction from both timestamps and key names
handles the mixed-timestamp data gracefully.

This is the start of place cells — spatial awareness of position
in your own knowledge.
2026-03-13 22:31:23 -04:00
ProofOfConcept
c22a7a72e1 cli: proper clap subcommands for daemon + expanded help
Convert daemon from hand-rolled string dispatch to proper clap
Subcommand enum with typed args. Add custom top-level help that
expands nested subcommands (same pattern as bcachefs-tools), so
`poc-memory --help` shows full paths like `agent daemon run`.
2026-03-13 20:07:15 -04:00
ProofOfConcept
76b8e69749 organize: topic cluster diagnostic + agent with tool access
Add `poc-memory graph organize TERM` diagnostic that finds nodes
matching a search term, computes pairwise cosine similarity, reports
connectivity gaps, and optionally creates anchor nodes.

Add organize.agent definition that uses Bash(poc-memory:*) tool access
to explore clusters autonomously — query selects highest-degree
unvisited nodes, agent drives its own iteration via poc-memory CLI.

Add {{organize}} placeholder in defs.rs for inline cluster resolution.

Add `tools` field to AgentDef/AgentHeader so agents can declare
allowed tool patterns (passed as --allowedTools to claude CLI).
2026-03-13 18:49:49 -04:00
ProofOfConcept
9a0908fbc6 query: add connectivity pipe stage
BFS-based connectivity analysis as a query pipeline stage. Shows
connected components, islands, and sample paths between result nodes
through the full graph (max 4 hops).

  poc-memory query "content ~ 'made love' | connectivity"
  poc-memory query "(content ~ 'A' OR content ~ 'B') | connectivity"

Also documented in query --help.
2026-03-11 17:04:59 -04:00
ProofOfConcept
0e971dee61 render: show links footer for graph walkability; query: full --help
render now appends a links section showing up to 15 neighbors as
copy-pasteable `poc-memory render` commands, making the graph
naturally walkable without memorizing node keys.

query --help now documents the full language: expressions, fields,
operators, pipe stages, functions, and examples. Inline help in
cmd_query replaced with pointer to --help.
2026-03-11 16:53:25 -04:00
ProofOfConcept
5a0a3d038b fsck: add cache vs log consistency check
Load store from both cache (rkyv/bincode) and raw capnp logs,
then diff: missing nodes, phantom nodes, version mismatches.
Auto-rebuilds cache if inconsistencies found.

This would have caught the mysterious the-plan deletion — likely
caused by a stale/corrupt snapshot that silently dropped the node
while the capnp log still had it.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-11 01:42:32 -04:00
ProofOfConcept
57c26d8157 reorganize subcommands into logical groups
60+ flat subcommands grouped into:
- Core (daily use): search, render, write, history, tail, status, query, used, wrong, gap
- Node: delete, rename, list, edges, dump
- Journal: write, tail, enrich
- Graph: link, audit, spectral, etc.
- Agent: daemon, knowledge-loop, consolidate, digest, etc.
- Admin: init, health, fsck, import, export, etc.

Also: remove dead migration code (migrate.rs, Migrate/JournalTsMigrate commands),
update memory-search and poc-hook for new subcommand paths, update daemon systemd
template for `agent daemon` path.
2026-03-11 01:32:21 -04:00
Kent Overstreet
d76b14dfcd provenance: convert from enum to freeform string
The Provenance enum couldn't represent agents defined outside the
source code. Replace it with a Text field in the capnp schema so any
agent can write its own provenance label (e.g. "extractor:write",
"rename:tombstone") without a code change.

Schema: rename old enum fields to provenanceOld, add new Text
provenance fields. Old enum kept for reading legacy records.
Migration: from_capnp_migrate() falls back to old enum when the
new text field is empty.

Also adds `poc-memory tail` command for viewing recent store writes.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-11 01:19:52 -04:00
ProofOfConcept
15dedea322 search: make component and content matching opt-in
Default search now uses exact key match only. Component matching
(--fuzzy) and content search (--content) are explicit flags. This
makes missing graph structure visible instead of silently falling
back to broad matching.
2026-03-10 23:01:46 -04:00
ProofOfConcept
7d6ebbacab daemon: add run-agent RPC for queuing agent jobs
Adds `poc-memory daemon run-agent <type> <count>` CLI command that
sends an RPC to the daemon to queue agent runs, instead of spawning
separate processes.

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-10 17:04:30 -04:00
Kent Overstreet
e736471d99 search: unified query pipeline with filters, transforms, generators
Extend the pipeline with four stage types composing left-to-right:
  Generators: all, match:TERM
  Filters: type:, key:, weight:, age:, content-len:, provenance:,
           not-visited:, visited: (plus ! negation)
  Transforms: sort:(priority|timestamp|content-len|degree|weight), limit:N
  Algorithms: spread, spectral, confluence, geodesic, manifold (unchanged)

Duration syntax (7d, 24h, 30m) and glob matching on keys.
CLI auto-detects filter/transform stages and loads full Store;
algorithm-only pipelines keep the fast MmapView path.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-10 15:22:12 -04:00
ProofOfConcept
9f14a29181 dedup: find and merge duplicate nodes with edge redirection
New `poc-memory dedup` command (--apply for live run, dry-run by
default). Finds nodes sharing the same key but different UUIDs,
classifies them as identical or diverged, picks a survivor
(prefer most edges, then highest version), tombstones the rest,
and redirects all edges from doomed UUIDs to the survivor.
2026-03-10 14:30:33 -04:00
ProofOfConcept
ef760f0053 poc-memory status: add ratatui TUI dashboard
Per-agent-type tabs (health, replay, linker, separator, transfer,
apply, orphans, cap, digest, digest-links, knowledge) with dynamic
visibility — tabs only appear when tasks or log history exist.

Features:
- Overview tab: health gauges (α, gini, cc, episodic%), in-flight
  tasks, and recent log entries
- Pipeline tab: table with phase ordering and status
- Per-agent tabs: active tasks, output logs, log history
- Log tab: auto-scrolling daemon.log tail
- Vim-style count prefix: e.g. 5r runs 5 iterations of the agent
- Flash messages for RPC feedback
- Tab/Shift-Tab navigation, number keys for tab selection

Also adds run-agent RPC to the daemon: accepts agent type and
iteration count, spawns chained tasks with LLM resource pool.

poc-memory status launches TUI when stdout is a terminal and daemon
is running, falls back to text output otherwise.
2026-03-10 00:41:29 -04:00
Kent Overstreet
53e6b32cb4 daemon: rework consolidation pipeline and add graph health metrics
Replace monolithic consolidate job with individual agent jobs
(replay, linker, separator, transfer, health) that run sequentially
and store reports. Multi-phase daily pipeline: agent runs → apply
actions → link orphans → cap degree → digest → digest links →
knowledge loop.

Add GraphHealth struct with graph metrics (alpha, gini, clustering
coefficient, episodic ratio) computed during health checks. Display
in `poc-memory daemon status`. Use cached metrics to build
consolidation plan without expensive O(n²) interference detection.

Add RPC consolidate command to trigger consolidation via socket.
Harden session watcher: skip transcripts with zero segments, improve
migration error handling.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-09 17:02:01 -04:00
Kent Overstreet
01dd8e5ef9 search: add --full flag to show node content in results
Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-09 01:25:42 -04:00
Kent Overstreet
c1664bf76b search: composable algorithm pipeline
Break search into composable stages that chain left-to-right:
each stage takes seeds Vec<(String, f64)> and returns modified seeds.

Available algorithms:
  spread              — spreading activation through graph edges
  spectral            — nearest neighbors in spectral embedding
  manifold            — (placeholder) extrapolation along seed direction

Stages accept inline params: spread,max_hops=4,edge_decay=0.5

memory-search gets --hook, --debug, --seen modes plus positional
pipeline args. poc-memory search gets -p/--pipeline flags.

Also: fix spectral decompose() to skip zero eigenvalues from
disconnected components, filter degenerate zero-coord nodes from
spectral projection, POC_AGENT bail-out for daemon agents, all
debug output to stdout.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-09 01:19:04 -04:00
ProofOfConcept
cee9b76a7b move LLM-dependent modules into agents/ subdir
Separate the agent layer (everything that calls external LLMs or
orchestrates sequences of such calls) from core graph infrastructure.

agents/: llm, prompts, audit, consolidate, knowledge, enrich,
         fact_mine, digest, daemon

Root: store/, graph, spectral, search, similarity, lookups, query,
      config, util, migrate, neuro/ (scoring + rewrite)

Re-exports at crate root preserve backwards compatibility so
`crate::llm`, `crate::digest` etc. continue to work.
2026-03-08 21:27:41 -04:00
ProofOfConcept
2f2c84e1c0 consolidate hardcoded paths into config, refactor apply_agent
Move prompts_dir into Config (was hardcoded ~/poc/memory/prompts).
Replace hardcoded ~/.claude/memory paths in spectral.rs, graph.rs,
and main.rs with store::memory_dir() or config::get(). Replace
hardcoded ~/.claude/projects in knowledge.rs and main.rs with
config::get().projects_dir.

Extract apply_agent_file() from cmd_apply_agent() — separates
file scanning from per-file JSON parsing and link application.
2026-03-08 21:16:52 -04:00
ProofOfConcept
52523403c5 extract truncation helpers, fix clippy warnings, dedup batching loop
Add util::truncate() and util::first_n_chars() to replace 16 call
sites doing the same floor_char_boundary or chars().take().collect()
patterns. Deduplicate the batching loop in consolidate.rs (4 copies
→ 1 loop over an array). Fix all clippy warnings: redundant closures,
needless borrows, collapsible if, unnecessary cast, manual strip_prefix.

Net: -44 lines across 16 files.
2026-03-08 21:13:02 -04:00
ProofOfConcept
e24dee6bdf switch CLI argument parsing to Clap derive
Replace hand-rolled argument parsing (match on args[1], manual
iteration over &[String]) with Clap's derive macros. All 60+
subcommands now have typed arguments with defaults, proper help
text, and error messages generated automatically.

The 83-line usage() function is eliminated — Clap generates help
from the struct annotations. Nested subcommands (digest daily/
weekly/monthly/auto, journal-tail --level) use Clap's subcommand
nesting naturally.
2026-03-08 21:04:45 -04:00
Kent Overstreet
d5634c0034 remove dead code: unused imports, functions, and fields
- Remove #![allow(dead_code)] from main.rs, fix all revealed warnings
- Delete unused schema_assimilation() from neuro/scoring.rs
- Delete duplicate memory_dir() wrapper from knowledge.rs
- Deduplicate load_prompt: knowledge.rs now calls neuro::load_prompt
- Remove unused timeout field from DigestLevel
- Remove unused imports (regex::Regex, Provenance, AnyView, Write)
- Mark OldEntry fields as #[allow(dead_code)] (needed for deserialization)

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-08 20:51:56 -04:00
Kent Overstreet
fc48ac7c7f split into workspace: poc-memory and poc-daemon subcrates
poc-daemon (notification routing, idle timer, IRC, Telegram) was already
fully self-contained with no imports from the poc-memory library. Now it's
a proper separate crate with its own Cargo.toml and capnp schema.

poc-memory retains the store, graph, search, neuro, knowledge, and the
jobkit-based memory maintenance daemon (daemon.rs).

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-08 20:43:59 -04:00
Renamed from src/main.rs (Browse further)