waffles/consciousness
1
0
Fork 0
forked from kent/consciousness
Code Pull requests Activity

salience: add gRPC client + TLS plumbing for stateful vllm sessions

Browse source 
Adds the client-side of a stateful gRPC protocol against vllm, plus
the TLS trust machinery so we can talk to self-signed vllm servers.

Protocol (proto/salience.proto):
  Bidi-streaming Session RPC carries OpenSession / AppendTokens /
  Generate / Cancel from client and SessionReady / PrefillProgress /
  Token / GenerateDone / Error from server. Separate Fork unary RPC
  for cheap branching (prefix cache shares KV automatically). Plus
  ListSessions, CloseSession, GetReadoutManifest admin RPCs.

  Per-token readouts ship as packed f32 ([n_layers * n_concepts] per
  token, flat). Logprobs use range-selected positions plus a top-k
  parameter — empty ranges means no logprobs, any range means emit
  sampled-token logprob at those positions, top_k > 0 adds
  alternatives.

Client (src/agent/api/salience.rs):
  Tonic-generated types under pb::, a connect() helper, with_auth()
  for bearer metadata, and a Session handle wrapping the bidi stream:
  open() handshakes SessionReady; append() is fire-and-forget;
  generate() returns impl Stream<Item = Event> that drains inbound
  until Done or terminating Error. One generate at a time per session.

Peak picker (src/agent/salience.rs):
  Pure function over ReadoutEntry traces. Per-concept z-score against
  trace global stats; contiguous above-threshold regions emit one
  peak at the local max. Configurable sigma threshold and min-std
  safety floor. Deterministic tie-break on offset then concept name.
  12 unit tests covering empty traces, flat channels, single/multi
  spikes, contiguous humps, multi-concept independence, trailing
  runs, sub-threshold noise, layer-out-of-range, manifest shape
  mismatch, and threshold tunability.

TLS (src/agent/api/http.rs):
  HttpClient::build now also loads every .pem file under
  ~/.consciousness/certs/ into the rustls root store — so dropping
  a <host>.pem in that directory is enough to trust a new self-
  signed server; no code changes per new host. Also installs the
  rustls default crypto provider explicitly via OnceLock: tonic's
  tls features pulled in both ring and aws-lc-rs on the resolver
  path, and rustls 0.23 refuses to auto-pick when either could win.

Build (build.rs, Cargo.toml):
  tonic-build generates Rust types from proto/salience.proto at
  cargo-build time, using a vendored protoc binary
  (protoc-bin-vendored) so no system install is required. New
  runtime deps: tonic, prost, async-stream, tokio-stream,
  rustls-pemfile.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
This commit is contained in:
Kent Overstreet 2026-04-23 02:21:07 -04:00
commit 08213f9093
15 changed files with 1689 additions and 440 deletions
Download patch file Download diff file Expand all files Collapse all files

17
build.rs
View file

@ -13,4 +13,21 @@ fn main() {
.file("schema/channel.capnp")
.run()
.expect("capnp compile failed (channel.capnp)");
// Generate salience.v1 gRPC client + message types from proto.
// Server side (python) is generated separately via grpcio-tools.
// Use vendored protoc so we don't require a system install.
let protoc = protoc_bin_vendored::protoc_bin_path()
.expect("vendored protoc not available for this platform");
// SAFETY: build script is single-threaded at this point; setting env
// before invoking tonic_build is the documented way to point it at a
// non-PATH protoc.
unsafe { std::env::set_var("PROTOC", protoc); }
tonic_build::configure()
.build_server(false)
.build_client(true)
.compile_protos(&["proto/salience.proto"], &["proto"])
.expect("tonic_build compile failed (salience.proto)");
println!("cargo:rerun-if-changed=proto/salience.proto");
}