Commit graph

11 commits

Author SHA1 Message Date
ProofOfConcept
635da6d3e2 split capnp_store.rs into src/store/ module hierarchy
capnp_store.rs (1772 lines) → four focused modules:
  store/types.rs  — types, macros, constants, path helpers
  store/parse.rs  — markdown parsing (MemoryUnit, parse_units)
  store/view.rs   — StoreView trait, MmapView, AnyView
  store/mod.rs    — Store impl methods, re-exports

new_node/new_relation become free functions in types.rs.
All callers updated: capnp_store:: → store::
2026-03-03 12:56:15 -05:00
ProofOfConcept
ec8b4b2ed2 eliminate schema_fit: it's clustering coefficient
schema_fit was algebraically identical to clustering_coefficient
(both compute 2E/(d*(d-1)) = fraction of connected neighbor pairs).
Remove the redundant function, field, and metrics column.

- Delete schema_fit() and schema_fit_all() from graph.rs
- Remove schema_fit field from Node struct
- Remove avg_schema_fit from MetricsSnapshot (duplicated avg_cc)
- Replace all callers with graph.clustering_coefficient()
- Rename ReplayItem.schema_fit to .cc
- Query: "cc" and "schema_fit" both resolve from graph CC
- Low-CC count folded into health report CC line

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-03-03 12:21:04 -05:00
ProofOfConcept
fb7aa46e03 graph: schema_fit is algebraically identical to clustering_coefficient
Both functions count connected pairs among a node's neighbors:
  cc = 2*triangles / (deg*(deg-1))
  density = inter_edges / (n*(n-1)/2) = 2*inter_edges / (n*(n-1))

Since inter_edges == triangles and n == deg, density == cc.
schema_fit was (density + cc) / 2.0 = (cc + cc) / 2.0 = cc.

Verified empirically: assert!((density - cc).abs() < 1e-6) passed
on all 2401 nodes before this change.

Keep schema_fit as a semantic alias — CC is a graph metric,
schema fit is a cognitive one — but eliminate the redundant O(n²)
pairwise computation that was running for every node.
2026-03-03 12:09:02 -05:00
ProofOfConcept
fa7fe8c14b query: rich QueryResult + toolkit cleanup
QueryResult carries a fields map (BTreeMap<String, Value>) so callers
don't re-resolve fields after queries run. Neighbors queries inject
edge context (strength, rel_type) at construction time.

New public API:
- run_query(): parse + execute + format in one call
- format_value(): format a Value for display
- execute_parsed(): internal, avoids double-parse in run_query

Removed: output_stages(), format_field()

Simplified commands:
- cmd_query, cmd_graph, cmd_link, cmd_list_keys all delegate to run_query
- cmd_experience_mine uses existing find_current_transcript()

Deduplication:
- now_epoch() 3 copies → 1 (capnp_store's public fn)
- hub_threshold → Graph::hub_threshold() method
- eval_node + eval_edge → single eval() with closure for field resolution
- compare() collapsed via Ordering (35 → 15 lines)

Modernization:
- 12 sites of partial_cmp().unwrap_or(Ordering::Equal) → total_cmp()
2026-03-03 12:07:04 -05:00
ProofOfConcept
a36449032c query: peg-based query language for ad-hoc graph exploration
poc-memory query "degree > 15"
poc-memory query "key ~ 'journal.*' AND degree > 10"
poc-memory query "neighbors('identity.md') WHERE strength > 0.5"
poc-memory query "community_id = community('identity.md')" --fields degree,category

Grammar-driven: the peg definition IS the language spec. Supports
boolean logic (AND/OR/NOT), numeric and string comparison, regex
match (~), graph traversal (neighbors() with WHERE), and function
calls (community(), degree()). Output flags: --fields, --sort,
--limit, --count.

New dependency: peg 0.8 (~68KB, 2 tiny deps).
2026-03-03 10:55:30 -05:00
ProofOfConcept
71e6f15d82 spectral decomposition, search improvements, char boundary fix
- New spectral module: Laplacian eigendecomposition of the memory graph.
  Commands: spectral, spectral-save, spectral-neighbors, spectral-positions,
  spectral-suggest. Spectral neighbors expand search results beyond keyword
  matching to structural proximity.

- Search: use StoreView trait to avoid 6MB state.bin rewrite on every query.
  Append-only retrieval logging. Spectral expansion shows structurally
  nearby nodes after text results.

- Fix panic in journal-tail: string truncation at byte 67 could land inside
  a multi-byte character (em dash). Now walks back to char boundary.

- Replay queue: show classification and spectral outlier score.

- Knowledge agents: extractor, challenger, connector prompts and runner
  scripts for automated graph enrichment.

- memory-search hook: stale state file cleanup (24h expiry).
2026-03-03 01:33:31 -05:00
ProofOfConcept
94dbca6018 graph health: fix-categories, cap-degree, link-orphans
Three new tools for structural graph health:

- fix-categories: rule-based recategorization fixing core inflation
  (225 → 26 core nodes). Only identity.md and kent.md stay core;
  everything else reclassified to tech/obs/gen by file prefix rules.

- cap-degree: two-phase degree capping. First prunes weakest Auto
  edges, then prunes Link edges to high-degree targets (they have
  alternative paths). Brought max degree from 919 → 50.

- link-orphans: connects degree-0/1 nodes to most textually similar
  connected nodes via cosine similarity. Linked 614 orphans.

Also: community detection now filters edges below strength 0.3,
preventing weak auto-links from merging unrelated communities.

Pipeline updated: consolidate-full now runs link-orphans + cap-degree
instead of triangle-close (which was counterproductive — densified
hub neighborhoods instead of building bridges).

Net effect: Gini 0.754 → 0.546, max degree 919 → 50.
2026-03-01 08:18:07 -05:00
Kent Overstreet
59cfa2959f fix NaN panics and eliminate redundant graph rebuilds
- All partial_cmp().unwrap() → unwrap_or(Ordering::Equal) to prevent
  NaN panics in sort operations across neuro.rs, graph.rs, similarity.rs
- replay_queue_with_graph: accepts pre-built graph, avoids rebuilding
  in agent_prompt (was building 2-3x per prompt)
- differentiate_hub_with_graph: same pattern for differentiation
- Simplify double-reverse history iteration to slice indexing

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
2026-03-01 00:33:53 -05:00
ProofOfConcept
29d5ed47a1 clippy: fix all warnings across all binaries
- &PathBuf → &Path in memory-search.rs signatures
- Redundant field name in graph.rs struct init
- Add truncate(false) to lock file open
- Derive Default for Store instead of manual impl
- slice::from_ref instead of &[x.clone()]
- rsplit_once instead of split().last()
- str::repeat instead of iter::repeat().take().collect()
- is_none_or instead of map_or(true, ...)
- strip_prefix instead of manual slicing

Zero warnings on `cargo clippy`.
2026-02-28 23:47:11 -05:00
ProofOfConcept
7ee6f9c651 refactor: eliminate date shell-outs, move logic to Store methods
- Replace all 5 `Command::new("date")` calls across 4 files with
  pure Rust time formatting via libc localtime_r
- Add format_date/format_datetime/format_datetime_space helpers to
  capnp_store
- Move import_file, find_journal_node, export_to_markdown, render_file,
  file_sections into Store methods where they belong
- Fix find_current_transcript to search all project dirs instead of
  hardcoding bcachefs-tools path
- Fix double-reference .clone() warnings in cmd_trace
- Fix unused variable warning in neuro.rs

main.rs: 1290 → 1137 lines, zero warnings.
2026-02-28 23:44:44 -05:00
ProofOfConcept
23fac4e5fe poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
  schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
  link-add, link-impact, decay, consolidate-session, etc.

Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation

Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00