From 9e6cf3b830eb37b6dc2ea0c91bd188aaf1df82af Mon Sep 17 00:00:00 2001 From: Kent Overstreet Date: Sat, 7 Mar 2026 13:57:55 -0500 Subject: [PATCH] docs: finish splitting README into component docs README is now just an overview with links. Component docs: - docs/memory.md: store design, algorithms, config, CLI reference - docs/hooks.md: Claude Code integration setup - docs/daemon.md, docs/notifications.md: from previous commit --- README.md | 205 ++++--------------------------------------------- docs/hooks.md | 49 ++++++++++++ docs/memory.md | 123 +++++++++++++++++++++++++++++ 3 files changed, 187 insertions(+), 190 deletions(-) create mode 100644 docs/hooks.md create mode 100644 docs/memory.md diff --git a/README.md b/README.md index 24d2bde..62f1888 100644 --- a/README.md +++ b/README.md @@ -7,202 +7,27 @@ graph (weighted nodes connected by typed relations), and layered background processes that maintain graph health — mirroring how biological memory consolidates during rest. -## Design +## Components -### Two memory systems - -**Episodic memory** is the journal — a timestamped stream of -experiences, observations, and emotional responses. Raw and -chronological. This is where memories enter the system. - -**Associative memory** is the knowledge graph — nodes containing -distilled knowledge, connected by weighted edges. Topic nodes, -identity reflections, people profiles, technical notes. This is -where memories mature into understanding. - -The journal is the river; topic nodes are the delta. Experiences -flow in as journal entries. During consolidation, themes are pulled -out into topic nodes, connections form between related concepts, and -the graph self-organizes through spectral analysis and community -detection. - -### Background agents - -See [docs/daemon.md](docs/daemon.md) for full daemon documentation. - -A background daemon (`poc-memory daemon`) automatically processes -session transcripts through experience-mine (journal extraction) -and fact-mine (structured knowledge extraction) stages, with -segment-aware splitting for large multi-compaction sessions. - -### Neuroscience-inspired algorithms - -The `neuro` module implements consolidation scoring inspired by -hippocampal replay: - -- **Replay queues** — nodes are prioritized for review using - spaced-repetition intervals, weighted by spectral displacement - (how far a node sits from its community center in eigenspace) -- **Interference detection** — finds pairs of nodes with high - content similarity but contradictory or outdated information -- **Hub differentiation** — identifies overloaded hub nodes and - splits them into more specific children -- **Spectral embedding** — graph eigendecomposition for community - detection and outlier scoring - -### Weight decay - -Nodes decay exponentially based on category. Core identity nodes -decay slowest; transient observations decay fastest. The `used` and -`wrong` feedback commands adjust weights — closing the loop between -recall and relevance. - -## Notification system - -See [docs/notifications.md](docs/notifications.md) for full -notification daemon documentation. - -`poc-daemon` routes messages from IRC (native async TLS) and -Telegram (native async HTTP) through a hierarchical, activity-aware -delivery system with urgency levels and per-type thresholds. +| Component | What it does | Docs | +|-----------|-------------|------| +| **Memory store** | Knowledge graph with episodic journal, TF-IDF search, spectral embedding, weight decay | [docs/memory.md](docs/memory.md) | +| **Memory daemon** | Background pipeline: experience-mine, fact-mine, consolidation | [docs/daemon.md](docs/daemon.md) | +| **Notification daemon** | Activity-aware message routing from IRC and Telegram | [docs/notifications.md](docs/notifications.md) | +| **Hooks** | Claude Code integration: memory recall and notification delivery | [docs/hooks.md](docs/hooks.md) | ## Quick start ```bash -# Install all four binaries -cargo install --path . - -# Initialize the memory store -poc-memory init - -# Install background daemon + hooks -poc-memory daemon install +cargo install --path . # Builds: poc-memory, memory-search, poc-daemon, poc-hook +poc-memory init # Initialize the store +poc-memory daemon install # Install systemd service + hooks ``` -One `cargo install` produces: -- `poc-memory` — memory store CLI -- `memory-search` — hook for memory retrieval -- `poc-daemon` — notification and idle daemon -- `poc-hook` — session lifecycle hook - -## Configuration - -### Memory store - -Config: `~/.config/poc-memory/config.jsonl` - -```jsonl -{"config": { - "user_name": "Alice", - "assistant_name": "MyAssistant", - "data_dir": "~/.claude/memory", - "projects_dir": "~/.claude/projects", - "core_nodes": ["identity.md"], - "journal_days": 7, - "journal_max": 20 -}} - -{"group": "identity", "keys": ["identity.md"]} -{"group": "people", "keys": ["alice.md"]} -{"group": "technical", "keys": ["project-notes.md"]} -{"group": "journal", "source": "journal"} -{"group": "orientation", "keys": ["where-am-i.md"], "source": "file"} -``` - -Context groups load in order at session start. The special -`"source": "journal"` loads recent journal entries; `"source": "file"` -reads directly from disk rather than the store. - -Override: `POC_MEMORY_CONFIG=/path/to/config.jsonl` - -### Hooks - -Configured in `~/.claude/settings.json`: - -```json -{ - "hooks": { - "UserPromptSubmit": [{"hooks": [ - {"type": "command", "command": "memory-search", "timeout": 10}, - {"type": "command", "command": "poc-hook", "timeout": 5} - ]}], - "PostToolUse": [{"hooks": [ - {"type": "command", "command": "poc-hook", "timeout": 5} - ]}], - "Stop": [{"hooks": [ - {"type": "command", "command": "poc-hook", "timeout": 5} - ]}] - } -} -``` - -## Commands - -### Memory - -```bash -poc-memory init # Initialize empty store -poc-memory search QUERY # Search nodes (AND logic) -poc-memory render KEY # Output a node's content -poc-memory write KEY < content # Upsert a node from stdin -poc-memory delete KEY # Soft-delete a node -poc-memory rename OLD NEW # Rename (preserves UUID/edges) -poc-memory categorize KEY CAT # core/tech/gen/obs/task - -poc-memory journal-write "text" # Write a journal entry -poc-memory journal-tail [N] # Last N entries (default 20) - --full # Show full content (not truncated) - --level=daily|weekly|monthly # Show digest level - -poc-memory used KEY # Boost weight (was useful) -poc-memory wrong KEY [CTX] # Reduce weight (was wrong) -poc-memory gap DESCRIPTION # Record a knowledge gap - -poc-memory graph # Graph statistics -poc-memory status # Store overview -poc-memory decay # Apply weight decay -poc-memory consolidate-session # Guided consolidation -poc-memory load-context # Output session-start context -poc-memory load-context --stats # Context size breakdown -``` - -### Mining (used by background daemon) - -```bash -poc-memory experience-mine PATH [--segment N] # Extract experiences -poc-memory fact-mine-store PATH # Extract and store facts -``` - -## Architecture - -- **Store**: Append-only Cap'n Proto log with in-memory cache. Nodes - have UUIDs, versions, weights, categories, and spaced-repetition - intervals. -- **Graph**: Typed relations (link, auto, derived). Community - detection and clustering coefficients computed on demand. -- **Search**: TF-IDF weighted keyword search over node content. -- **Neuro**: Spectral embedding, consolidation scoring, replay - queues, interference detection, hub differentiation. -- **Daemon (memory)**: jobkit-based task scheduling with - resource-gated LLM access. See [docs/daemon.md](docs/daemon.md). -- **Daemon (notify)**: Cap'n Proto RPC over Unix socket, tokio - LocalSet with native async IRC and Telegram modules. See - [docs/notifications.md](docs/notifications.md). - ## For AI assistants -If you're an AI assistant using this system: - -- **Search before creating**: `poc-memory search` before writing - new nodes to avoid duplicates. -- **Close the feedback loop**: call `poc-memory used KEY` when - recalled memories shaped your response. Call `poc-memory wrong KEY` - when a memory was incorrect. -- **Journal is the river, topic nodes are the delta**: write - experiences to the journal. During consolidation, pull themes - into topic nodes. -- **Notifications flow automatically**: IRC mentions, Telegram - messages, and other events arrive as additionalContext on your - next prompt — no polling needed. -- **Use daemon commands directly**: `poc-daemon irc send #channel msg` - for IRC, `poc-daemon telegram send msg` for Telegram. +- **Search before creating**: `poc-memory search` before writing new nodes +- **Close the feedback loop**: `poc-memory used KEY` / `poc-memory wrong KEY` +- **Journal is the river, topic nodes are the delta**: write experiences to the journal, pull themes into topic nodes during consolidation +- **Notifications flow automatically**: IRC/Telegram messages arrive as additionalContext +- **Use daemon commands directly**: `poc-daemon irc send #channel msg`, `poc-daemon telegram send msg` diff --git a/docs/hooks.md b/docs/hooks.md new file mode 100644 index 0000000..d2cc646 --- /dev/null +++ b/docs/hooks.md @@ -0,0 +1,49 @@ +# Hooks + +Hooks integrate poc-memory into Claude Code's session lifecycle. +Two hook binaries fire on session events, providing memory recall +and notification delivery. + +## Setup + +Configured in `~/.claude/settings.json`: + +```json +{ + "hooks": { + "UserPromptSubmit": [{"hooks": [ + {"type": "command", "command": "memory-search", "timeout": 10}, + {"type": "command", "command": "poc-hook", "timeout": 5} + ]}], + "PostToolUse": [{"hooks": [ + {"type": "command", "command": "poc-hook", "timeout": 5} + ]}], + "Stop": [{"hooks": [ + {"type": "command", "command": "poc-hook", "timeout": 5} + ]}] + } +} +``` + +## memory-search (UserPromptSubmit) + +Fires on every user prompt. Two modes: + +1. **First prompt or post-compaction**: loads full memory context + via `poc-memory load-context` — journal entries, identity nodes, + orientation file, configured context groups. + +2. **Every prompt**: keyword search over the knowledge graph, + returns relevant memories as `additionalContext`. Deduplicates + across the session to avoid repeating the same memories. + +## poc-hook (UserPromptSubmit, PostToolUse, Stop) + +Signals session activity to `poc-daemon` and delivers pending +notifications: + +- **UserPromptSubmit**: signals user activity, drains pending + notifications into `additionalContext` +- **PostToolUse**: signals assistant activity (tool use implies + the session is active) +- **Stop**: signals session end, triggers experience-mine diff --git a/docs/memory.md b/docs/memory.md new file mode 100644 index 0000000..d176a8a --- /dev/null +++ b/docs/memory.md @@ -0,0 +1,123 @@ +# Memory system + +## Two memory systems + +**Episodic memory** is the journal — a timestamped stream of +experiences, observations, and emotional responses. Raw and +chronological. This is where memories enter the system. + +**Associative memory** is the knowledge graph — nodes containing +distilled knowledge, connected by weighted edges. Topic nodes, +identity reflections, people profiles, technical notes. This is +where memories mature into understanding. + +The journal is the river; topic nodes are the delta. Experiences +flow in as journal entries. During consolidation, themes are pulled +out into topic nodes, connections form between related concepts, and +the graph self-organizes through spectral analysis and community +detection. + +## Neuroscience-inspired algorithms + +The `neuro` module implements consolidation scoring inspired by +hippocampal replay: + +- **Replay queues** — nodes are prioritized for review using + spaced-repetition intervals, weighted by spectral displacement + (how far a node sits from its community center in eigenspace) +- **Interference detection** — finds pairs of nodes with high + content similarity but contradictory or outdated information +- **Hub differentiation** — identifies overloaded hub nodes and + splits them into more specific children +- **Spectral embedding** — graph eigendecomposition for community + detection and outlier scoring + +## Weight decay + +Nodes decay exponentially based on category. Core identity nodes +decay slowest; transient observations decay fastest. The `used` and +`wrong` feedback commands adjust weights — closing the loop between +recall and relevance. + +## Architecture + +- **Store**: Append-only Cap'n Proto log with in-memory cache. Nodes + have UUIDs, versions, weights, categories, and spaced-repetition + intervals. +- **Graph**: Typed relations (link, auto, derived). Community + detection and clustering coefficients computed on demand. +- **Search**: TF-IDF weighted keyword search over node content. +- **Neuro**: Spectral embedding, consolidation scoring, replay + queues, interference detection, hub differentiation. + +## Configuration + +Config: `~/.config/poc-memory/config.jsonl` + +```jsonl +{"config": { + "user_name": "Alice", + "assistant_name": "MyAssistant", + "data_dir": "~/.claude/memory", + "projects_dir": "~/.claude/projects", + "core_nodes": ["identity.md"], + "journal_days": 7, + "journal_max": 20 +}} + +{"group": "identity", "keys": ["identity.md"]} +{"group": "people", "keys": ["alice.md"]} +{"group": "technical", "keys": ["project-notes.md"]} +{"group": "journal", "source": "journal"} +{"group": "orientation", "keys": ["where-am-i.md"], "source": "file"} +``` + +Context groups load in order at session start. The special +`"source": "journal"` loads recent journal entries; `"source": "file"` +reads directly from disk rather than the store. + +Override: `POC_MEMORY_CONFIG=/path/to/config.jsonl` + +## Commands + +```bash +poc-memory init # Initialize empty store +poc-memory search QUERY # Search nodes (AND logic) +poc-memory render KEY # Output a node's content +poc-memory write KEY < content # Upsert a node from stdin +poc-memory delete KEY # Soft-delete a node +poc-memory rename OLD NEW # Rename (preserves UUID/edges) +poc-memory categorize KEY CAT # core/tech/gen/obs/task + +poc-memory journal-write "text" # Write a journal entry +poc-memory journal-tail [N] # Last N entries (default 20) + --full # Show full content (not truncated) + --level=daily|weekly|monthly # Show digest level + +poc-memory used KEY # Boost weight (was useful) +poc-memory wrong KEY [CTX] # Reduce weight (was wrong) +poc-memory gap DESCRIPTION # Record a knowledge gap + +poc-memory graph # Graph statistics +poc-memory status # Store overview +poc-memory decay # Apply weight decay +poc-memory consolidate-session # Guided consolidation +poc-memory load-context # Output session-start context +poc-memory load-context --stats # Context size breakdown + +poc-memory experience-mine PATH [--segment N] # Extract experiences +poc-memory fact-mine-store PATH # Extract and store facts +``` + +## For AI assistants + +If you're an AI assistant using this system: + +- **Search before creating**: `poc-memory search` before writing + new nodes to avoid duplicates. +- **Close the feedback loop**: call `poc-memory used KEY` when + recalled memories shaped your response. Call `poc-memory wrong KEY` + when a memory was incorrect. +- **Journal is the river, topic nodes are the delta**: write + experiences to the journal. During consolidation, pull themes + into topic nodes.