kent/consciousness
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

README: rewrite with design overview, notification system, updated config

Browse source 
Document the hippocampus-inspired design (episodic + associative memory,
background consolidation agents, neuroscience-inspired replay/spectral
algorithms). Add full notification system docs (architecture, urgency
levels, activity-aware thresholds, IRC/Telegram modules). Fix config
format to match reality (JSONL, not TOML). Update instructions.md in
the store with notification commands.

Co-Authored-By: ProofOfConcept <poc@bcachefs.org>
This commit is contained in:
Kent Overstreet 2026-03-05 19:26:50 -05:00
parent ecedc86d42
commit 308fbe4c28
1 changed files with 266 additions and 120 deletions
Download patch file Download diff file Expand all files Collapse all files

386
README.md
View file

@ -1,173 +1,319 @@
# poc-memory # poc-memory
A persistent memory system for AI assistants. Stores knowledge as a A persistent memory and notification system for AI assistants,
weighted graph of nodes and relations, with automatic recall via Claude modelled after the human hippocampus. Combines episodic memory
Code hooks. (timestamped journal of experiences) with an associative knowledge
graph (weighted nodes connected by typed relations), and layered
background processes that maintain graph health — mirroring how
biological memory consolidates during rest.
## Design
### Two memory systems
**Episodic memory** is the journal — a timestamped stream of
experiences, observations, and emotional responses. Raw and
chronological. This is where memories enter the system.
**Associative memory** is the knowledge graph — nodes containing
distilled knowledge, connected by weighted edges. Topic nodes,
identity reflections, people profiles, technical notes. This is
where memories mature into understanding.
The journal is the river; topic nodes are the delta. Experiences
flow in as journal entries. During consolidation, themes are pulled
out into topic nodes, connections form between related concepts, and
the graph self-organizes through spectral analysis and community
detection.
### Background agents
A background daemon (`poc-memory daemon`) automatically spawns agents
for memory maintenance:
- **Experience mining** — when a session ends, extracts experiences
and observations from the transcript into journal entries
- **Fact extraction** — pulls concrete facts (names, dates, decisions,
preferences) into structured knowledge nodes
- **Consolidation** — periodic graph health work: replay queues
(spaced repetition), interference detection (contradictory nodes),
hub differentiation (splitting overloaded nodes), triangle closure
(connecting nodes that share neighbors), and orphan linking
### Neuroscience-inspired algorithms
The `neuro` module implements consolidation scoring inspired by
hippocampal replay:
- **Replay queues** — nodes are prioritized for review using
spaced-repetition intervals, weighted by spectral displacement
(how far a node sits from its community center in eigenspace)
- **Interference detection** — finds pairs of nodes with high
content similarity but contradictory or outdated information
- **Hub differentiation** — identifies overloaded hub nodes and
splits them into more specific children
- **Spectral embedding** — graph eigendecomposition for community
detection and outlier scoring
### Weight decay
Nodes decay exponentially based on category. Core identity nodes
decay slowest; transient observations decay fastest. The `used` and
`wrong` feedback commands adjust weights — closing the loop between
recall and relevance.
## Notification system
A separate daemon (`poc-daemon`) routes messages from communication
modules and internal events through a hierarchical, activity-aware
delivery system.
### Architecture
```
Communication modules Hooks
┌──────────────────┐ ┌─────────────┐
│ IRC (native) │──┐ │ poc-hook │
│ Telegram (native│ │ mpsc │ (all events)│
└──────────────────┘ ├──────┐ └──────┬───────┘
│ │ │
▼ │ capnp-rpc
┌───────────┘ │
│ poc-daemon │
│ │
│ NotifyState ◄─────────┘
│ ├── type registry
│ ├── pending queue
│ ├── threshold lookup
│ └── activity-aware delivery
│
│ idle::State
│ ├── presence detection
│ ├── sleep/wake/dream modes
│ └── tmux prompt injection
└────────────────────────
```
### Notification types and urgency
Types are free-form hierarchical strings: `irc.mention.nick`,
`irc.channel.bcachefs`, `telegram.kent`. Each has an urgency level:
| Level | Name | Meaning |
|-------|---------|--------------------------------------|
| 0 | ambient | Include in idle context only |
| 1 | low | Deliver on next check |
| 2 | normal | Deliver on next user interaction |
| 3 | urgent | Interrupt immediately |
Per-type thresholds walk up the hierarchy: `irc.channel.bcachefs-ai`
→ `irc.channel` → `irc` → default. Effective thresholds adjust by
activity state: raised when focused, lowered when idle, only urgent
when sleeping.
### Communication modules
**IRC** — native async TLS connection (tokio-rustls). Connects,
joins channels, parses messages, generates notifications. Runtime
commands: join, leave, send, status, log, nick. Per-channel logs
at `~/.claude/irc/logs/`.
**Telegram** — native async HTTP long-polling (reqwest). Downloads
media (photos, voice, documents). Chat ID filtering for security.
Runtime commands: send, status, log.
Both modules persist config changes to `~/.claude/daemon.toml` —
channel joins and nick changes survive restarts.
## Quick start ## Quick start
```bash ```bash
# Install # Install all four binaries
cargo install --path . cargo install --path .
# Initialize the store # Initialize the memory store
poc-memory init poc-memory init
# Install Claude Code hooks and systemd service # Install background daemon + hooks
poc-memory daemon install poc-memory daemon install
``` ```
One `cargo install` produces:
- `poc-memory` — memory store CLI
- `memory-search` — hook for memory retrieval
- `poc-daemon` — notification and idle daemon
- `poc-hook` — session lifecycle hook
## Configuration ## Configuration
Config file: `~/.config/poc-memory/config.toml` ### Memory store
```toml Config: `~/.config/poc-memory/config.jsonl`
# Names used in transcripts and agent prompts
user_name = "Alice"
assistant_name = "MyAssistant"
# Where memory data lives (store, logs, episodic digests) ```jsonl
data_dir = "~/.claude/memory" {"config": {
"user_name": "Alice",
"assistant_name": "MyAssistant",
"data_dir": "~/.claude/memory",
"projects_dir": "~/.claude/projects",
"core_nodes": ["identity.md"],
"journal_days": 7,
"journal_max": 20
}}
# Where Claude Code session transcripts are stored {"group": "identity", "keys": ["identity.md"]}
projects_dir = "~/.claude/projects" {"group": "people", "keys": ["alice.md"]}
{"group": "technical", "keys": ["project-notes.md"]}
# Nodes that should never be decayed (comma-separated) {"group": "journal", "source": "journal"}
core_nodes = "identity.md, preferences.md" {"group": "orientation", "keys": ["where-am-i.md"], "source": "file"}
# Journal settings for session-start context loading
journal_days = 7
journal_max = 20
# Context groups loaded at session start, in order.
# Each [context.NAME] section specifies a group of nodes to load.
# If no "label" is given, the section name is used (underscores become spaces).
[context.identity]
keys = "identity.md"
[context.people]
keys = "alice.md, bob.md"
[context.technical]
keys = "project-notes.md, architecture.md"
# Orientation loaded last — current task state, not deep identity
[context.orientation]
keys = "where-am-i.md"
``` ```
Override the config path with `POC_MEMORY_CONFIG=/path/to/config.toml`. Context groups load in order at session start. The special
`"source": "journal"` loads recent journal entries; `"source": "file"`
reads directly from disk rather than the store.
Override: `POC_MEMORY_CONFIG=/path/to/config.jsonl`
### Notification daemon
Config: `~/.claude/daemon.toml`
```toml
[irc]
enabled = true
server = "irc.oftc.net"
port = 6697
tls = true
nick = "MyBot"
user = "bot"
realname = "My Bot"
channels = ["#mychannel"]
[telegram]
enabled = true
token = "bot-token-here"
chat_id = 123456789
```
### Hooks
Configured in `~/.claude/settings.json`:
```json
{
"hooks": {
"UserPromptSubmit": [{"hooks": [
{"type": "command", "command": "memory-search", "timeout": 10},
{"type": "command", "command": "poc-hook", "timeout": 5}
]}],
"PostToolUse": [{"hooks": [
{"type": "command", "command": "poc-hook", "timeout": 5}
]}],
"Stop": [{"hooks": [
{"type": "command", "command": "poc-hook", "timeout": 5}
]}]
}
}
```
## Commands ## Commands
### Core operations ### Memory
```bash ```bash
poc-memory init # Initialize empty store poc-memory init # Initialize empty store
poc-memory search QUERY # Search nodes (1-3 words, AND logic) poc-memory search QUERY # Search nodes (AND logic)
poc-memory render KEY # Output a node's content poc-memory render KEY # Output a node's content
poc-memory write KEY < content # Upsert a node from stdin poc-memory write KEY < content # Upsert a node from stdin
poc-memory delete KEY # Soft-delete a node poc-memory delete KEY # Soft-delete a node
poc-memory rename OLD NEW # Rename a node (preserves UUID/edges) poc-memory rename OLD NEW # Rename (preserves UUID/edges)
poc-memory categorize KEY CAT # Set category: core/tech/gen/obs/task poc-memory categorize KEY CAT # core/tech/gen/obs/task
```
### Journal
```bash
poc-memory journal-write "text" # Write a journal entry poc-memory journal-write "text" # Write a journal entry
poc-memory journal-tail [N] # Show last N entries (default 20) poc-memory journal-tail [N] # Last N entries (default 20)
poc-memory journal-tail N --full # Show full content (not truncated)
poc-memory used KEY # Boost weight (was useful)
poc-memory wrong KEY [CTX] # Reduce weight (was wrong)
poc-memory gap DESCRIPTION # Record a knowledge gap
poc-memory graph # Graph statistics
poc-memory status # Store overview
poc-memory decay # Apply weight decay
poc-memory consolidate-session # Guided consolidation
poc-memory load-context # Output session-start context
poc-memory load-context --stats # Context size breakdown
``` ```
### Feedback loop ### Notification daemon
```bash ```bash
poc-memory used KEY # Mark a recalled node as useful (boosts weight) poc-daemon # Start daemon
poc-memory wrong KEY [CONTEXT] # Mark a node as wrong (reduces weight) poc-daemon status # State summary
poc-memory gap DESCRIPTION # Record a knowledge gap for later filling poc-daemon irc status # IRC module status
poc-daemon irc send TARGET MSG # Send IRC message
poc-daemon irc join CHANNEL # Join (persists to config)
poc-daemon irc leave CHANNEL # Leave
poc-daemon irc log [N] # Last N messages
poc-daemon telegram status # Telegram module status
poc-daemon telegram send MSG # Send Telegram message
poc-daemon telegram log [N] # Last N messages
poc-daemon notify TYPE URG MSG # Submit notification
poc-daemon notifications [URG] # Get + drain pending
poc-daemon notify-types # List all types
poc-daemon notify-threshold T L # Set per-type threshold
poc-daemon sleep / wake / quiet # Session management
poc-daemon stop # Shut down
``` ```
### Graph operations ### Mining (used by background daemon)
```bash ```bash
poc-memory link N # Interactive graph walk from a node poc-memory experience-mine PATH # Extract experiences from transcript
poc-memory graph # Show graph statistics poc-memory fact-mine-store PATH # Extract and store facts
poc-memory status # Store overview: node/edge counts, categories
```
### Maintenance
```bash
poc-memory decay # Apply weight decay to all nodes
poc-memory consolidate-session # Guided 6-step memory consolidation
```
### Context loading (used by hooks)
```bash
poc-memory load-context # Output full session-start context
```
This loads all context groups from the config file in order, followed by
recent journal entries. The `memory-search` hook binary calls this
automatically on session start.
### Daemon
```bash
poc-memory daemon # Run the background daemon
poc-memory daemon install # Install systemd service + Claude hooks
```
The daemon watches for completed Claude sessions and runs experience
mining and fact extraction on transcripts.
### Mining (used by daemon)
```bash
poc-memory experience-mine PATH # Extract experiences from a transcript
poc-memory fact-mine-store PATH # Extract facts and store them
``` ```
## How the hooks work ## How the hooks work
The `memory-search` binary is a Claude Code `UserPromptSubmit` hook. On **memory-search** (UserPromptSubmit):
each prompt it: 1. First prompt or post-compaction: loads full memory context via
`poc-memory load-context`
2. Every prompt: keyword search, returns relevant memories as
additionalContext. Deduplicates across the session.
1. **First prompt of a session**: Runs `poc-memory load-context` to inject **poc-hook** (UserPromptSubmit, PostToolUse, Stop):
full memory context (identity, reflections, journal, orientation). - Signals user activity and responses to poc-daemon
2. **Post-compaction**: Detects context compaction and reloads full context. - Drains pending notifications into additionalContext
3. **Every prompt**: Extracts keywords and searches the store for relevant - Monitors context window usage, warns before compaction
memories. Deduplicates against previously shown results for the session.
Session state (cookies, seen-keys) is tracked in `/tmp/claude-memory-search/`
and cleaned up after 24 hours.
## Architecture ## Architecture
- **Store**: Append-only Cap'n Proto log (`nodes.capnp`, `relations.capnp`) - **Store**: Append-only Cap'n Proto log with in-memory cache. Nodes
with in-memory cache. Nodes have UUIDs, versions, weights, categories, have UUIDs, versions, weights, categories, and spaced-repetition
and spaced-repetition intervals. intervals.
- **Graph**: Nodes connected by typed relations (link, auto, derived). - **Graph**: Typed relations (link, auto, derived). Community
Community detection and clustering coefficients computed on demand. detection and clustering coefficients computed on demand.
- **Search**: TF-IDF weighted keyword search over node content. - **Search**: TF-IDF weighted keyword search over node content.
- **Decay**: Exponential weight decay with category-specific factors. - **Neuro**: Spectral embedding, consolidation scoring, replay
Core nodes decay slowest; observations decay fastest. queues, interference detection, hub differentiation.
- **Daemon**: Uses jobkit for task scheduling with resource-gated LLM - **Daemon (memory)**: jobkit-based task scheduling with
access (one slot by default to manage API costs). resource-gated LLM access.
- **Daemon (notify)**: Cap'n Proto RPC over Unix socket, tokio
LocalSet with native async IRC and Telegram modules.
## For AI assistants ## For AI assistants
If you're an AI assistant using this system, here's what matters: If you're an AI assistant using this system:
- **Search before creating**: Always `poc-memory search` before writing - **Search before creating**: `poc-memory search` before writing
new nodes to avoid duplicates. new nodes to avoid duplicates.
- **Close the feedback loop**: When recalled memories shaped your response, - **Close the feedback loop**: call `poc-memory used KEY` when
call `poc-memory used KEY`. When a memory was wrong, call recalled memories shaped your response. Call `poc-memory wrong KEY`
`poc-memory wrong KEY`. This trains the weight system. when a memory was incorrect.
- **Journal is the river, topic nodes are the delta**: Write experiences - **Journal is the river, topic nodes are the delta**: write
to the journal. During consolidation, pull themes into topic nodes. experiences to the journal. During consolidation, pull themes
- **Config tells you who you are**: `poc-memory` reads your name from into topic nodes.
the config file. Agent prompts use these names instead of generic - **Notifications flow automatically**: IRC mentions, Telegram
"the user" / "the assistant". messages, and other events arrive as additionalContext on your
next prompt — no polling needed.
- **Use daemon commands directly**: `poc-daemon irc send #channel msg`
for IRC, `poc-daemon telegram send msg` for Telegram.