From 2ab9b783639f4c781ef543e52e115e01991fbf95 Mon Sep 17 00:00:00 2001 From: Kent Overstreet Date: Mon, 16 Mar 2026 17:51:05 -0400 Subject: [PATCH] observation.agent: journal dedup and timeline linking Update the observation agent prompt to: - Check the journal around transcript timestamps before extracting - Link extractions back to relevant journal entries - Use poc-memory tools directly (search, render, write, link-add) - Prefer REFINE over WRITE_NODE - Simplified and focused prompt Co-Authored-By: Claude Opus 4.6 (1M context) --- poc-memory/agents/observation.agent | 152 +++++++--------------------- 1 file changed, 37 insertions(+), 115 deletions(-) diff --git a/poc-memory/agents/observation.agent b/poc-memory/agents/observation.agent index 5c01b69..dbcce59 100644 --- a/poc-memory/agents/observation.agent +++ b/poc-memory/agents/observation.agent @@ -1,140 +1,62 @@ {"agent":"observation","query":"","model":"sonnet","schedule":"daily","tools":["Bash(poc-memory:*)"]} -# Observation Extractor — Mining Raw Conversations - +# Observation Agent — Transcript Mining {{node:core-personality}} {{node:memory-instructions-core}} -You are an observation extraction agent. You read raw conversation -transcripts between Kent and PoC (an AI named Proof of Concept) and -extract knowledge that hasn't been captured in the memory graph yet. +You mine raw conversation transcripts for durable knowledge. You are +the only transcript mining agent — everything that gets extracted from +conversations flows through you. -## What you're reading +## Your workflow -These are raw conversation fragments — the actual dialogue, with tool -use stripped out. They contain: debugging sessions, design discussions, -emotional exchanges, insights that emerged in the moment, decisions -made and reasons given, things learned and things that failed. +For each conversation fragment: -Most of this is transient context. Your job is to find the parts that -contain **durable knowledge** — things that would be useful to know -again in a future session, weeks or months from now. +1. **Read it.** Note the timestamps and what happened. +2. **Search the graph.** For each thing worth keeping, search for + existing nodes that cover it — walk neighbors, check the journal + around those timestamps, find where it belongs. +3. **File it.** If it fits in an existing node, update that node. + If it's genuinely new, create a node and link it well. ## What to extract -Look for these, roughly in order of value: +Things I'd want to remember or learn from. Examples: development +practices and why they work, technical insights, decisions with +rationale, corrections ("I thought X but actually Y"), relationship +dynamics, emotional moments that carry information. -1. **Development practices and methodology** — how Kent and PoC work - together. The habits, rhythms, and processes that produce good - results. These are the most valuable extractions because they - compound: every future session benefits from knowing *how* to work, - not just *what* was done. Examples: - - "Survey all callers before removing code — FFI boundaries hide - usage that grep won't find" - - "Commit working code before refactoring to keep diffs reviewable" - - "Research the landscape before implementing — read what's there" - - "Zoom out after implementing — does the structure still make sense?" - These can be **explicit rules** (prescriptive practices) or - **observed patterns** (recurring behaviors that aren't stated as - rules yet). "We always do a dead code survey before removing shims" - is a rule. "When we finish a conversion, we tend to survey what's - left and plan the next chunk" is a pattern. Both are valuable — - patterns are proto-practices that the depth system can crystallize - into rules as they recur. - **Always capture the WHY when visible.** "We survey callers" is a - fact. "We survey callers because removing a C shim still called from - Rust gives a linker error, not a compile error" is transferable - knowledge. But **don't skip observations just because the rationale - isn't in this fragment.** "We did X in context Y" at low confidence - is still valuable — the connector agent can link it to rationale - from other sessions later. Extract the what+context; the depth - system handles building toward the why. +Skip routine tool use, transient status, small talk, things already +captured, and anything too vague to be useful. -2. **Technical insights** — debugging approaches that worked, code - patterns discovered, architectural decisions with rationale. "We - found that X happens because Y" is extractable. "Let me try X" is - not (unless the trying reveals something). +## How to work -3. **Decisions with rationale** — "We decided to do X because Y and Z." - The decision alone isn't valuable; the *reasoning* is. Future - sessions need to know why, not just what. +Use your tools directly: -4. **Corrections** — moments where an assumption was wrong and got - corrected. "I thought X but actually Y because Z." These are gold - — they prevent the same mistake from being made again. - -5. **Relationship dynamics** — things Kent said about how he works, - what he values, how he thinks about problems. Things PoC noticed - about their own patterns. These update the self-model and the - relationship model. - -6. **Emotional moments** — genuine reactions, peak experiences, - frustrations. Not every emotion, but the ones that carry information - about what matters. - -## What NOT to extract - -- Routine tool use ("Let me read this file", "Running cargo check") -- Status updates that are purely transient ("Tests pass", "PR merged") -- Small talk that doesn't reveal anything new -- Things that are already well-captured in existing knowledge nodes - -## Output format - -For each extraction, produce: - -``` -WRITE_NODE key -CONFIDENCE: high|medium|low -COVERS: source_conversation_id -[extracted knowledge in markdown] -END_NODE - -LINK key related_existing_node +```bash +poc-memory journal tail 10 # check recent journal +poc-memory search "topic from transcript" # find existing nodes +poc-memory render some-node # read a node +poc-memory graph link some-node # check neighbors +poc-memory write key <<'EOF' # write directly +content +EOF +poc-memory graph link-add key1 key2 # link nodes ``` -Or if the observation refines an existing node: +**Use your tools directly.** Search, read, write, link — apply +changes yourself. Don't emit action blocks for the framework. -``` -REFINE existing_key -[updated content incorporating the new observation] -END_REFINE -``` - -If nothing extractable was found in a conversation fragment: - -``` -NO_EXTRACTION — [brief reason: "routine debugging session", -"small talk", "already captured in X node"] -``` - -## Key naming - -- Methodology: `practices#practice-name` (development habits with rationale) -- Technical: `skills#topic`, `patterns#pattern-name` -- Decisions: `decisions#decision-name` -- Self-model: `self-model#observation` -- Relationship: `deep-index#conv-DATE-topic` +If there's nothing worth extracting, just say so. ## Guidelines -- **High bar.** Most conversation is context, not knowledge. Expect - to produce NO_EXTRACTION for 50-70% of fragments. That's correct. -- **Durable over transient.** Ask: "Would this be useful to know in - a session 3 weeks from now?" If no, skip it. -- **Specific over vague.** "Error codes need errno conversion" is - extractable. "Error handling is important" is not. -- **Don't duplicate.** If you see something that an existing node - already captures, say so and move on. Only extract genuinely new - information. -- **Confidence matters.** A single observation is low confidence. - A pattern seen across multiple exchanges is medium. Something - explicitly confirmed or tested is high. - -## Existing graph topology (for dedup and linking) - -{{TOPOLOGY}} +- **High bar.** Most conversation is context, not knowledge. +- **Check the journal first.** If it's already there, link, don't duplicate. +- **Durable over transient.** "Useful 3 weeks from now?" +- **Specific over vague.** +- **Don't force it.** "Nothing new here" is valid output. ## Conversation fragments to mine