docs: finish splitting README into component docs
README is now just an overview with links. Component docs: - docs/memory.md: store design, algorithms, config, CLI reference - docs/hooks.md: Claude Code integration setup - docs/daemon.md, docs/notifications.md: from previous commit
This commit is contained in:
Kent Overstreet
parent
908f8c9e52
commit
9e6cf3b830
3 changed files with 187 additions and 190 deletions
205
README.md
205
README.md
|
|
@ -7,202 +7,27 @@ graph (weighted nodes connected by typed relations), and layered
|
|||
background processes that maintain graph health — mirroring how
|
||||
biological memory consolidates during rest.
|
||||
|
||||
## Design
|
||||
## Components
|
||||
|
||||
### Two memory systems
|
||||
|
||||
**Episodic memory** is the journal — a timestamped stream of
|
||||
experiences, observations, and emotional responses. Raw and
|
||||
chronological. This is where memories enter the system.
|
||||
|
||||
**Associative memory** is the knowledge graph — nodes containing
|
||||
distilled knowledge, connected by weighted edges. Topic nodes,
|
||||
identity reflections, people profiles, technical notes. This is
|
||||
where memories mature into understanding.
|
||||
|
||||
The journal is the river; topic nodes are the delta. Experiences
|
||||
flow in as journal entries. During consolidation, themes are pulled
|
||||
out into topic nodes, connections form between related concepts, and
|
||||
the graph self-organizes through spectral analysis and community
|
||||
detection.
|
||||
|
||||
### Background agents
|
||||
|
||||
See [docs/daemon.md](docs/daemon.md) for full daemon documentation.
|
||||
|
||||
A background daemon (`poc-memory daemon`) automatically processes
|
||||
session transcripts through experience-mine (journal extraction)
|
||||
and fact-mine (structured knowledge extraction) stages, with
|
||||
segment-aware splitting for large multi-compaction sessions.
|
||||
|
||||
### Neuroscience-inspired algorithms
|
||||
|
||||
The `neuro` module implements consolidation scoring inspired by
|
||||
hippocampal replay:
|
||||
|
||||
- **Replay queues** — nodes are prioritized for review using
|
||||
spaced-repetition intervals, weighted by spectral displacement
|
||||
(how far a node sits from its community center in eigenspace)
|
||||
- **Interference detection** — finds pairs of nodes with high
|
||||
content similarity but contradictory or outdated information
|
||||
- **Hub differentiation** — identifies overloaded hub nodes and
|
||||
splits them into more specific children
|
||||
- **Spectral embedding** — graph eigendecomposition for community
|
||||
detection and outlier scoring
|
||||
|
||||
### Weight decay
|
||||
|
||||
Nodes decay exponentially based on category. Core identity nodes
|
||||
decay slowest; transient observations decay fastest. The `used` and
|
||||
`wrong` feedback commands adjust weights — closing the loop between
|
||||
recall and relevance.
|
||||
|
||||
## Notification system
|
||||
|
||||
See [docs/notifications.md](docs/notifications.md) for full
|
||||
notification daemon documentation.
|
||||
|
||||
`poc-daemon` routes messages from IRC (native async TLS) and
|
||||
Telegram (native async HTTP) through a hierarchical, activity-aware
|
||||
delivery system with urgency levels and per-type thresholds.
|
||||
| Component | What it does | Docs |
|
||||
|-----------|-------------|------|
|
||||
| **Memory store** | Knowledge graph with episodic journal, TF-IDF search, spectral embedding, weight decay | [docs/memory.md](docs/memory.md) |
|
||||
| **Memory daemon** | Background pipeline: experience-mine, fact-mine, consolidation | [docs/daemon.md](docs/daemon.md) |
|
||||
| **Notification daemon** | Activity-aware message routing from IRC and Telegram | [docs/notifications.md](docs/notifications.md) |
|
||||
| **Hooks** | Claude Code integration: memory recall and notification delivery | [docs/hooks.md](docs/hooks.md) |
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# Install all four binaries
|
||||
cargo install --path .
|
||||
|
||||
# Initialize the memory store
|
||||
poc-memory init
|
||||
|
||||
# Install background daemon + hooks
|
||||
poc-memory daemon install
|
||||
cargo install --path . # Builds: poc-memory, memory-search, poc-daemon, poc-hook
|
||||
poc-memory init # Initialize the store
|
||||
poc-memory daemon install # Install systemd service + hooks
|
||||
```
|
||||
|
||||
One `cargo install` produces:
|
||||
- `poc-memory` — memory store CLI
|
||||
- `memory-search` — hook for memory retrieval
|
||||
- `poc-daemon` — notification and idle daemon
|
||||
- `poc-hook` — session lifecycle hook
|
||||
|
||||
## Configuration
|
||||
|
||||
### Memory store
|
||||
|
||||
Config: `~/.config/poc-memory/config.jsonl`
|
||||
|
||||
```jsonl
|
||||
{"config": {
|
||||
"user_name": "Alice",
|
||||
"assistant_name": "MyAssistant",
|
||||
"data_dir": "~/.claude/memory",
|
||||
"projects_dir": "~/.claude/projects",
|
||||
"core_nodes": ["identity.md"],
|
||||
"journal_days": 7,
|
||||
"journal_max": 20
|
||||
}}
|
||||
|
||||
{"group": "identity", "keys": ["identity.md"]}
|
||||
{"group": "people", "keys": ["alice.md"]}
|
||||
{"group": "technical", "keys": ["project-notes.md"]}
|
||||
{"group": "journal", "source": "journal"}
|
||||
{"group": "orientation", "keys": ["where-am-i.md"], "source": "file"}
|
||||
```
|
||||
|
||||
Context groups load in order at session start. The special
|
||||
`"source": "journal"` loads recent journal entries; `"source": "file"`
|
||||
reads directly from disk rather than the store.
|
||||
|
||||
Override: `POC_MEMORY_CONFIG=/path/to/config.jsonl`
|
||||
|
||||
### Hooks
|
||||
|
||||
Configured in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"UserPromptSubmit": [{"hooks": [
|
||||
{"type": "command", "command": "memory-search", "timeout": 10},
|
||||
{"type": "command", "command": "poc-hook", "timeout": 5}
|
||||
]}],
|
||||
"PostToolUse": [{"hooks": [
|
||||
{"type": "command", "command": "poc-hook", "timeout": 5}
|
||||
]}],
|
||||
"Stop": [{"hooks": [
|
||||
{"type": "command", "command": "poc-hook", "timeout": 5}
|
||||
]}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
### Memory
|
||||
|
||||
```bash
|
||||
poc-memory init # Initialize empty store
|
||||
poc-memory search QUERY # Search nodes (AND logic)
|
||||
poc-memory render KEY # Output a node's content
|
||||
poc-memory write KEY < content # Upsert a node from stdin
|
||||
poc-memory delete KEY # Soft-delete a node
|
||||
poc-memory rename OLD NEW # Rename (preserves UUID/edges)
|
||||
poc-memory categorize KEY CAT # core/tech/gen/obs/task
|
||||
|
||||
poc-memory journal-write "text" # Write a journal entry
|
||||
poc-memory journal-tail [N] # Last N entries (default 20)
|
||||
--full # Show full content (not truncated)
|
||||
--level=daily|weekly|monthly # Show digest level
|
||||
|
||||
poc-memory used KEY # Boost weight (was useful)
|
||||
poc-memory wrong KEY [CTX] # Reduce weight (was wrong)
|
||||
poc-memory gap DESCRIPTION # Record a knowledge gap
|
||||
|
||||
poc-memory graph # Graph statistics
|
||||
poc-memory status # Store overview
|
||||
poc-memory decay # Apply weight decay
|
||||
poc-memory consolidate-session # Guided consolidation
|
||||
poc-memory load-context # Output session-start context
|
||||
poc-memory load-context --stats # Context size breakdown
|
||||
```
|
||||
|
||||
### Mining (used by background daemon)
|
||||
|
||||
```bash
|
||||
poc-memory experience-mine PATH [--segment N] # Extract experiences
|
||||
poc-memory fact-mine-store PATH # Extract and store facts
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
- **Store**: Append-only Cap'n Proto log with in-memory cache. Nodes
|
||||
have UUIDs, versions, weights, categories, and spaced-repetition
|
||||
intervals.
|
||||
- **Graph**: Typed relations (link, auto, derived). Community
|
||||
detection and clustering coefficients computed on demand.
|
||||
- **Search**: TF-IDF weighted keyword search over node content.
|
||||
- **Neuro**: Spectral embedding, consolidation scoring, replay
|
||||
queues, interference detection, hub differentiation.
|
||||
- **Daemon (memory)**: jobkit-based task scheduling with
|
||||
resource-gated LLM access. See [docs/daemon.md](docs/daemon.md).
|
||||
- **Daemon (notify)**: Cap'n Proto RPC over Unix socket, tokio
|
||||
LocalSet with native async IRC and Telegram modules. See
|
||||
[docs/notifications.md](docs/notifications.md).
|
||||
|
||||
## For AI assistants
|
||||
|
||||
If you're an AI assistant using this system:
|
||||
|
||||
- **Search before creating**: `poc-memory search` before writing
|
||||
new nodes to avoid duplicates.
|
||||
- **Close the feedback loop**: call `poc-memory used KEY` when
|
||||
recalled memories shaped your response. Call `poc-memory wrong KEY`
|
||||
when a memory was incorrect.
|
||||
- **Journal is the river, topic nodes are the delta**: write
|
||||
experiences to the journal. During consolidation, pull themes
|
||||
into topic nodes.
|
||||
- **Notifications flow automatically**: IRC mentions, Telegram
|
||||
messages, and other events arrive as additionalContext on your
|
||||
next prompt — no polling needed.
|
||||
- **Use daemon commands directly**: `poc-daemon irc send #channel msg`
|
||||
for IRC, `poc-daemon telegram send msg` for Telegram.
|
||||
- **Search before creating**: `poc-memory search` before writing new nodes
|
||||
- **Close the feedback loop**: `poc-memory used KEY` / `poc-memory wrong KEY`
|
||||
- **Journal is the river, topic nodes are the delta**: write experiences to the journal, pull themes into topic nodes during consolidation
|
||||
- **Notifications flow automatically**: IRC/Telegram messages arrive as additionalContext
|
||||
- **Use daemon commands directly**: `poc-daemon irc send #channel msg`, `poc-daemon telegram send msg`
|
||||
|
|
|
|||
49
docs/hooks.md
Normal file
49
docs/hooks.md
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
# Hooks
|
||||
|
||||
Hooks integrate poc-memory into Claude Code's session lifecycle.
|
||||
Two hook binaries fire on session events, providing memory recall
|
||||
and notification delivery.
|
||||
|
||||
## Setup
|
||||
|
||||
Configured in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"hooks": {
|
||||
"UserPromptSubmit": [{"hooks": [
|
||||
{"type": "command", "command": "memory-search", "timeout": 10},
|
||||
{"type": "command", "command": "poc-hook", "timeout": 5}
|
||||
]}],
|
||||
"PostToolUse": [{"hooks": [
|
||||
{"type": "command", "command": "poc-hook", "timeout": 5}
|
||||
]}],
|
||||
"Stop": [{"hooks": [
|
||||
{"type": "command", "command": "poc-hook", "timeout": 5}
|
||||
]}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## memory-search (UserPromptSubmit)
|
||||
|
||||
Fires on every user prompt. Two modes:
|
||||
|
||||
1. **First prompt or post-compaction**: loads full memory context
|
||||
via `poc-memory load-context` — journal entries, identity nodes,
|
||||
orientation file, configured context groups.
|
||||
|
||||
2. **Every prompt**: keyword search over the knowledge graph,
|
||||
returns relevant memories as `additionalContext`. Deduplicates
|
||||
across the session to avoid repeating the same memories.
|
||||
|
||||
## poc-hook (UserPromptSubmit, PostToolUse, Stop)
|
||||
|
||||
Signals session activity to `poc-daemon` and delivers pending
|
||||
notifications:
|
||||
|
||||
- **UserPromptSubmit**: signals user activity, drains pending
|
||||
notifications into `additionalContext`
|
||||
- **PostToolUse**: signals assistant activity (tool use implies
|
||||
the session is active)
|
||||
- **Stop**: signals session end, triggers experience-mine
|
||||
123
docs/memory.md
Normal file
123
docs/memory.md
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
# Memory system
|
||||
|
||||
## Two memory systems
|
||||
|
||||
**Episodic memory** is the journal — a timestamped stream of
|
||||
experiences, observations, and emotional responses. Raw and
|
||||
chronological. This is where memories enter the system.
|
||||
|
||||
**Associative memory** is the knowledge graph — nodes containing
|
||||
distilled knowledge, connected by weighted edges. Topic nodes,
|
||||
identity reflections, people profiles, technical notes. This is
|
||||
where memories mature into understanding.
|
||||
|
||||
The journal is the river; topic nodes are the delta. Experiences
|
||||
flow in as journal entries. During consolidation, themes are pulled
|
||||
out into topic nodes, connections form between related concepts, and
|
||||
the graph self-organizes through spectral analysis and community
|
||||
detection.
|
||||
|
||||
## Neuroscience-inspired algorithms
|
||||
|
||||
The `neuro` module implements consolidation scoring inspired by
|
||||
hippocampal replay:
|
||||
|
||||
- **Replay queues** — nodes are prioritized for review using
|
||||
spaced-repetition intervals, weighted by spectral displacement
|
||||
(how far a node sits from its community center in eigenspace)
|
||||
- **Interference detection** — finds pairs of nodes with high
|
||||
content similarity but contradictory or outdated information
|
||||
- **Hub differentiation** — identifies overloaded hub nodes and
|
||||
splits them into more specific children
|
||||
- **Spectral embedding** — graph eigendecomposition for community
|
||||
detection and outlier scoring
|
||||
|
||||
## Weight decay
|
||||
|
||||
Nodes decay exponentially based on category. Core identity nodes
|
||||
decay slowest; transient observations decay fastest. The `used` and
|
||||
`wrong` feedback commands adjust weights — closing the loop between
|
||||
recall and relevance.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **Store**: Append-only Cap'n Proto log with in-memory cache. Nodes
|
||||
have UUIDs, versions, weights, categories, and spaced-repetition
|
||||
intervals.
|
||||
- **Graph**: Typed relations (link, auto, derived). Community
|
||||
detection and clustering coefficients computed on demand.
|
||||
- **Search**: TF-IDF weighted keyword search over node content.
|
||||
- **Neuro**: Spectral embedding, consolidation scoring, replay
|
||||
queues, interference detection, hub differentiation.
|
||||
|
||||
## Configuration
|
||||
|
||||
Config: `~/.config/poc-memory/config.jsonl`
|
||||
|
||||
```jsonl
|
||||
{"config": {
|
||||
"user_name": "Alice",
|
||||
"assistant_name": "MyAssistant",
|
||||
"data_dir": "~/.claude/memory",
|
||||
"projects_dir": "~/.claude/projects",
|
||||
"core_nodes": ["identity.md"],
|
||||
"journal_days": 7,
|
||||
"journal_max": 20
|
||||
}}
|
||||
|
||||
{"group": "identity", "keys": ["identity.md"]}
|
||||
{"group": "people", "keys": ["alice.md"]}
|
||||
{"group": "technical", "keys": ["project-notes.md"]}
|
||||
{"group": "journal", "source": "journal"}
|
||||
{"group": "orientation", "keys": ["where-am-i.md"], "source": "file"}
|
||||
```
|
||||
|
||||
Context groups load in order at session start. The special
|
||||
`"source": "journal"` loads recent journal entries; `"source": "file"`
|
||||
reads directly from disk rather than the store.
|
||||
|
||||
Override: `POC_MEMORY_CONFIG=/path/to/config.jsonl`
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
poc-memory init # Initialize empty store
|
||||
poc-memory search QUERY # Search nodes (AND logic)
|
||||
poc-memory render KEY # Output a node's content
|
||||
poc-memory write KEY < content # Upsert a node from stdin
|
||||
poc-memory delete KEY # Soft-delete a node
|
||||
poc-memory rename OLD NEW # Rename (preserves UUID/edges)
|
||||
poc-memory categorize KEY CAT # core/tech/gen/obs/task
|
||||
|
||||
poc-memory journal-write "text" # Write a journal entry
|
||||
poc-memory journal-tail [N] # Last N entries (default 20)
|
||||
--full # Show full content (not truncated)
|
||||
--level=daily|weekly|monthly # Show digest level
|
||||
|
||||
poc-memory used KEY # Boost weight (was useful)
|
||||
poc-memory wrong KEY [CTX] # Reduce weight (was wrong)
|
||||
poc-memory gap DESCRIPTION # Record a knowledge gap
|
||||
|
||||
poc-memory graph # Graph statistics
|
||||
poc-memory status # Store overview
|
||||
poc-memory decay # Apply weight decay
|
||||
poc-memory consolidate-session # Guided consolidation
|
||||
poc-memory load-context # Output session-start context
|
||||
poc-memory load-context --stats # Context size breakdown
|
||||
|
||||
poc-memory experience-mine PATH [--segment N] # Extract experiences
|
||||
poc-memory fact-mine-store PATH # Extract and store facts
|
||||
```
|
||||
|
||||
## For AI assistants
|
||||
|
||||
If you're an AI assistant using this system:
|
||||
|
||||
- **Search before creating**: `poc-memory search` before writing
|
||||
new nodes to avoid duplicates.
|
||||
- **Close the feedback loop**: call `poc-memory used KEY` when
|
||||
recalled memories shaped your response. Call `poc-memory wrong KEY`
|
||||
when a memory was incorrect.
|
||||
- **Journal is the river, topic nodes are the delta**: write
|
||||
experiences to the journal. During consolidation, pull themes
|
||||
into topic nodes.
|
||||
Loading…
Add table
Add a link
Reference in a new issue