poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
fn main() {
|
|
|
|
|
capnpc::CompilerCommand::new()
|
|
|
|
|
.file("schema/memory.capnp")
|
|
|
|
|
.run()
|
2026-04-03 17:31:17 -04:00
|
|
|
.expect("capnp compile failed (memory.capnp)");
|
|
|
|
|
|
|
|
|
|
capnpc::CompilerCommand::new()
|
|
|
|
|
.file("schema/daemon.capnp")
|
|
|
|
|
.run()
|
|
|
|
|
.expect("capnp compile failed (daemon.capnp)");
|
2026-04-03 18:46:14 -04:00
|
|
|
|
|
|
|
|
capnpc::CompilerCommand::new()
|
|
|
|
|
.file("schema/channel.capnp")
|
|
|
|
|
.run()
|
|
|
|
|
.expect("capnp compile failed (channel.capnp)");
|
salience: add gRPC client + TLS plumbing for stateful vllm sessions
Adds the client-side of a stateful gRPC protocol against vllm, plus
the TLS trust machinery so we can talk to self-signed vllm servers.
Protocol (proto/salience.proto):
Bidi-streaming Session RPC carries OpenSession / AppendTokens /
Generate / Cancel from client and SessionReady / PrefillProgress /
Token / GenerateDone / Error from server. Separate Fork unary RPC
for cheap branching (prefix cache shares KV automatically). Plus
ListSessions, CloseSession, GetReadoutManifest admin RPCs.
Per-token readouts ship as packed f32 ([n_layers * n_concepts] per
token, flat). Logprobs use range-selected positions plus a top-k
parameter — empty ranges means no logprobs, any range means emit
sampled-token logprob at those positions, top_k > 0 adds
alternatives.
Client (src/agent/api/salience.rs):
Tonic-generated types under pb::, a connect() helper, with_auth()
for bearer metadata, and a Session handle wrapping the bidi stream:
open() handshakes SessionReady; append() is fire-and-forget;
generate() returns impl Stream<Item = Event> that drains inbound
until Done or terminating Error. One generate at a time per session.
Peak picker (src/agent/salience.rs):
Pure function over ReadoutEntry traces. Per-concept z-score against
trace global stats; contiguous above-threshold regions emit one
peak at the local max. Configurable sigma threshold and min-std
safety floor. Deterministic tie-break on offset then concept name.
12 unit tests covering empty traces, flat channels, single/multi
spikes, contiguous humps, multi-concept independence, trailing
runs, sub-threshold noise, layer-out-of-range, manifest shape
mismatch, and threshold tunability.
TLS (src/agent/api/http.rs):
HttpClient::build now also loads every .pem file under
~/.consciousness/certs/ into the rustls root store — so dropping
a <host>.pem in that directory is enough to trust a new self-
signed server; no code changes per new host. Also installs the
rustls default crypto provider explicitly via OnceLock: tonic's
tls features pulled in both ring and aws-lc-rs on the resolver
path, and rustls 0.23 refuses to auto-pick when either could win.
Build (build.rs, Cargo.toml):
tonic-build generates Rust types from proto/salience.proto at
cargo-build time, using a vendored protoc binary
(protoc-bin-vendored) so no system install is required. New
runtime deps: tonic, prost, async-stream, tokio-stream,
rustls-pemfile.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-23 02:21:07 -04:00
|
|
|
|
|
|
|
|
// Generate salience.v1 gRPC client + message types from proto.
|
|
|
|
|
// Server side (python) is generated separately via grpcio-tools.
|
|
|
|
|
// Use vendored protoc so we don't require a system install.
|
|
|
|
|
let protoc = protoc_bin_vendored::protoc_bin_path()
|
|
|
|
|
.expect("vendored protoc not available for this platform");
|
|
|
|
|
// SAFETY: build script is single-threaded at this point; setting env
|
|
|
|
|
// before invoking tonic_build is the documented way to point it at a
|
|
|
|
|
// non-PATH protoc.
|
|
|
|
|
unsafe { std::env::set_var("PROTOC", protoc); }
|
|
|
|
|
tonic_build::configure()
|
|
|
|
|
.build_server(false)
|
|
|
|
|
.build_client(true)
|
|
|
|
|
.compile_protos(&["proto/salience.proto"], &["proto"])
|
|
|
|
|
.expect("tonic_build compile failed (salience.proto)");
|
|
|
|
|
|
|
|
|
|
println!("cargo:rerun-if-changed=proto/salience.proto");
|
poc-memory v0.4.0: graph-structured memory with consolidation pipeline
Rust core:
- Cap'n Proto append-only storage (nodes + relations)
- Graph algorithms: clustering coefficient, community detection,
schema fit, small-world metrics, interference detection
- BM25 text similarity with Porter stemming
- Spaced repetition replay queue
- Commands: search, init, health, status, graph, categorize,
link-add, link-impact, decay, consolidate-session, etc.
Python scripts:
- Episodic digest pipeline: daily/weekly/monthly-digest.py
- retroactive-digest.py for backfilling
- consolidation-agents.py: 3 parallel Sonnet agents
- apply-consolidation.py: structured action extraction + apply
- digest-link-parser.py: extract ~400 explicit links from digests
- content-promotion-agent.py: promote episodic obs to semantic files
- bulk-categorize.py: categorize all nodes via single Sonnet call
- consolidation-loop.py: multi-round automated consolidation
Co-Authored-By: Kent Overstreet <kent.overstreet@linux.dev>
2026-02-28 22:17:00 -05:00
|
|
|
}
|