transcript: extract JSONL backward scanner and compaction detection into library

Move JsonlBackwardIter and find_last_compaction() from
parse-claude-conversation into a shared transcript module. Both
memory-search and parse-claude-conversation now use the same robust
compaction detection: mmap-based backward scan, JSON parsing to
verify user-type message, content prefix check.

Replaces memory-search's old detect_compaction() which did a forward
scan with raw string matching on "continued from a previous
conversation" — that could false-positive on the string appearing
in assistant output or tool results.

Add parse-claude-conversation as a new binary for debugging what's
in the context window post-compaction.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
This commit is contained in:
Kent Overstreet 2026-03-09 17:06:32 -04:00
parent 0e17ab00b0
commit c2f245740c
4 changed files with 548 additions and 9 deletions

View file

@ -14,6 +14,7 @@ pub mod spectral;
pub mod lookups;
pub mod query;
pub mod migrate;
pub mod transcript;
pub mod neuro;
// Agent layer (LLM-powered operations)