poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
// Spreading activation search across the memory graph
|
|
|
|
|
//
|
|
|
|
|
// Same model as the old system but richer: uses graph edge strengths,
|
|
|
|
|
// supports circumscription parameter for blending associative vs
|
|
|
|
|
// causal walks, and benefits from community-aware result grouping.
|
|
|
|
|
|
2026-03-03 12:56:15 -05:00
|
|
|
use crate::store::StoreView;
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
use crate::graph::Graph;
|
|
|
|
|
|
|
|
|
|
use std::collections::{HashMap, HashSet, VecDeque};
|
|
|
|
|
|
|
|
|
|
pub struct SearchResult {
|
|
|
|
|
pub key: String,
|
|
|
|
|
pub activation: f64,
|
|
|
|
|
pub is_direct: bool,
|
|
|
|
|
pub snippet: Option<String>,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/// Spreading activation with circumscription parameter.
|
|
|
|
|
///
|
|
|
|
|
/// circ = 0.0: field mode — all edges (default, broad resonance)
|
|
|
|
|
/// circ = 1.0: causal mode — prefer causal edges
|
|
|
|
|
fn spreading_activation(
|
|
|
|
|
seeds: &[(String, f64)],
|
|
|
|
|
graph: &Graph,
|
2026-03-03 01:33:31 -05:00
|
|
|
store: &impl StoreView,
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
_circumscription: f64,
|
|
|
|
|
) -> Vec<(String, f64)> {
|
2026-03-03 01:33:31 -05:00
|
|
|
let params = store.params();
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
|
|
|
|
|
let mut activation: HashMap<String, f64> = HashMap::new();
|
|
|
|
|
let mut queue: VecDeque<(String, f64, u32)> = VecDeque::new();
|
|
|
|
|
|
|
|
|
|
for (key, act) in seeds {
|
|
|
|
|
let current = activation.entry(key.clone()).or_insert(0.0);
|
|
|
|
|
if *act > *current {
|
|
|
|
|
*current = *act;
|
|
|
|
|
queue.push_back((key.clone(), *act, 0));
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
while let Some((key, act, depth)) = queue.pop_front() {
|
|
|
|
|
if depth >= params.max_hops { continue; }
|
|
|
|
|
|
|
|
|
|
for (neighbor, strength) in graph.neighbors(&key) {
|
2026-03-03 01:33:31 -05:00
|
|
|
let neighbor_weight = store.node_weight(neighbor.as_str());
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
let propagated = act * params.edge_decay * neighbor_weight * strength as f64;
|
|
|
|
|
if propagated < params.min_activation { continue; }
|
|
|
|
|
|
|
|
|
|
let current = activation.entry(neighbor.clone()).or_insert(0.0);
|
|
|
|
|
if propagated > *current {
|
|
|
|
|
*current = propagated;
|
|
|
|
|
queue.push_back((neighbor.clone(), propagated, depth + 1));
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let mut results: Vec<_> = activation.into_iter().collect();
|
2026-03-03 12:07:04 -05:00
|
|
|
results.sort_by(|a, b| b.1.total_cmp(&a.1));
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
results
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/// Full search: find direct hits, spread activation, return ranked results
|
2026-03-03 01:33:31 -05:00
|
|
|
pub fn search(query: &str, store: &impl StoreView) -> Vec<SearchResult> {
|
|
|
|
|
let graph = crate::graph::build_graph_fast(store);
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
let query_lower = query.to_lowercase();
|
|
|
|
|
let query_tokens: Vec<&str> = query_lower.split_whitespace().collect();
|
|
|
|
|
|
|
|
|
|
let mut seeds: Vec<(String, f64)> = Vec::new();
|
|
|
|
|
let mut snippets: HashMap<String, String> = HashMap::new();
|
|
|
|
|
|
2026-03-03 01:33:31 -05:00
|
|
|
store.for_each_node(|key, content, weight| {
|
|
|
|
|
let content_lower = content.to_lowercase();
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
|
|
|
|
|
let exact_match = content_lower.contains(&query_lower);
|
|
|
|
|
let token_match = query_tokens.len() > 1
|
|
|
|
|
&& query_tokens.iter().all(|t| content_lower.contains(t));
|
|
|
|
|
|
|
|
|
|
if exact_match || token_match {
|
2026-03-03 01:33:31 -05:00
|
|
|
let activation = if exact_match { weight as f64 } else { weight as f64 * 0.85 };
|
|
|
|
|
seeds.push((key.to_owned(), activation));
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
|
2026-03-03 01:33:31 -05:00
|
|
|
let snippet: String = content.lines()
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
.filter(|l| {
|
|
|
|
|
let ll = l.to_lowercase();
|
|
|
|
|
if exact_match && ll.contains(&query_lower) { return true; }
|
|
|
|
|
query_tokens.iter().any(|t| ll.contains(t))
|
|
|
|
|
})
|
|
|
|
|
.take(3)
|
|
|
|
|
.map(|l| {
|
|
|
|
|
let t = l.trim();
|
2026-03-08 21:13:02 -04:00
|
|
|
crate::util::truncate(t, 97, "...")
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
})
|
|
|
|
|
.collect::<Vec<_>>()
|
|
|
|
|
.join("\n ");
|
2026-03-03 01:33:31 -05:00
|
|
|
snippets.insert(key.to_owned(), snippet);
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
}
|
2026-03-03 01:33:31 -05:00
|
|
|
});
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
|
|
|
|
|
if seeds.is_empty() {
|
|
|
|
|
return Vec::new();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let direct_hits: HashSet<String> = seeds.iter().map(|(k, _)| k.clone()).collect();
|
|
|
|
|
let raw_results = spreading_activation(&seeds, &graph, store, 0.0);
|
|
|
|
|
|
|
|
|
|
raw_results.into_iter().map(|(key, activation)| {
|
|
|
|
|
let is_direct = direct_hits.contains(&key);
|
|
|
|
|
let snippet = snippets.get(&key).cloned();
|
|
|
|
|
SearchResult { key, activation, is_direct, snippet }
|
|
|
|
|
}).collect()
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/// Extract meaningful search terms from natural language.
|
|
|
|
|
/// Strips common English stop words, returns up to max_terms words.
|
|
|
|
|
pub fn extract_query_terms(text: &str, max_terms: usize) -> String {
|
|
|
|
|
const STOP_WORDS: &[&str] = &[
|
|
|
|
|
"the", "a", "an", "is", "are", "was", "were", "do", "does", "did",
|
|
|
|
|
"have", "has", "had", "will", "would", "could", "should", "can",
|
|
|
|
|
"may", "might", "shall", "been", "being", "to", "of", "in", "for",
|
|
|
|
|
"on", "with", "at", "by", "from", "as", "but", "or", "and", "not",
|
|
|
|
|
"no", "if", "then", "than", "that", "this", "it", "its", "my",
|
|
|
|
|
"your", "our", "we", "you", "i", "me", "he", "she", "they", "them",
|
|
|
|
|
"what", "how", "why", "when", "where", "about", "just", "let",
|
|
|
|
|
"want", "tell", "show", "think", "know", "see", "look", "make",
|
|
|
|
|
"get", "go", "some", "any", "all", "very", "really", "also", "too",
|
|
|
|
|
"so", "up", "out", "here", "there",
|
|
|
|
|
];
|
|
|
|
|
|
|
|
|
|
text.to_lowercase()
|
|
|
|
|
.split(|c: char| !c.is_alphanumeric())
|
|
|
|
|
.filter(|w| !w.is_empty() && w.len() > 2 && !STOP_WORDS.contains(w))
|
|
|
|
|
.take(max_terms)
|
|
|
|
|
.collect::<Vec<_>>()
|
|
|
|
|
.join(" ")
|
|
|
|
|
}
|
2026-03-05 22:23:03 -05:00
|
|
|
|
|
|
|
|
/// Format search results as text lines (for hook consumption).
|
|
|
|
|
pub fn format_results(results: &[SearchResult]) -> String {
|
|
|
|
|
let mut out = String::new();
|
|
|
|
|
for (i, r) in results.iter().enumerate().take(5) {
|
|
|
|
|
let marker = if r.is_direct { "→" } else { " " };
|
|
|
|
|
out.push_str(&format!("{}{:2}. [{:.2}/{:.2}] {}",
|
|
|
|
|
marker, i + 1, r.activation, r.activation, r.key));
|
|
|
|
|
out.push('\n');
|
|
|
|
|
if let Some(ref snippet) = r.snippet {
|
|
|
|
|
out.push_str(&format!(" {}\n", snippet));
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
out
|
|
|
|
|
}
|