consciousness/README.md

# poc-memory

A persistent memory and notification system for AI assistants,
modelled after the human hippocampus. Combines episodic memory
(timestamped journal of experiences) with an associative knowledge
graph (weighted nodes connected by typed relations), and layered
background processes that maintain graph health — mirroring how
biological memory consolidates during rest.

## Design

### Two memory systems

**Episodic memory** is the journal — a timestamped stream of
experiences, observations, and emotional responses. Raw and
chronological. This is where memories enter the system.

**Associative memory** is the knowledge graph — nodes containing
distilled knowledge, connected by weighted edges. Topic nodes,
identity reflections, people profiles, technical notes. This is
where memories mature into understanding.

The journal is the river; topic nodes are the delta. Experiences
flow in as journal entries. During consolidation, themes are pulled
out into topic nodes, connections form between related concepts, and
the graph self-organizes through spectral analysis and community
detection.

### Background agents

A background daemon (`poc-memory daemon`) automatically spawns agents
for memory maintenance:

- **Experience mining** — when a session ends, extracts experiences
  and observations from the transcript into journal entries
- **Fact extraction** — pulls concrete facts (names, dates, decisions,
  preferences) into structured knowledge nodes
- **Consolidation** — periodic graph health work: replay queues
  (spaced repetition), interference detection (contradictory nodes),
  hub differentiation (splitting overloaded nodes), triangle closure
  (connecting nodes that share neighbors), and orphan linking

### Neuroscience-inspired algorithms

The `neuro` module implements consolidation scoring inspired by
hippocampal replay:

- **Replay queues** — nodes are prioritized for review using
  spaced-repetition intervals, weighted by spectral displacement
  (how far a node sits from its community center in eigenspace)
- **Interference detection** — finds pairs of nodes with high
  content similarity but contradictory or outdated information
- **Hub differentiation** — identifies overloaded hub nodes and
  splits them into more specific children
- **Spectral embedding** — graph eigendecomposition for community
  detection and outlier scoring

### Weight decay

Nodes decay exponentially based on category. Core identity nodes
decay slowest; transient observations decay fastest. The `used` and
`wrong` feedback commands adjust weights — closing the loop between
recall and relevance.

## Notification system

A separate daemon (`poc-daemon`) routes messages from communication
modules and internal events through a hierarchical, activity-aware
delivery system.

### Architecture

```
  Communication modules              Hooks
  ┌──────────────────┐               ┌─────────────┐
  │  IRC (native)    │──┐            │  poc-hook    │
  │  Telegram (native│  │  mpsc      │  (all events)│
  └──────────────────┘  ├──────┐     └──────┬───────┘
                        │      │            │
                        ▼      │     capnp-rpc
                   ┌───────────┘            │
                   │  poc-daemon            │
                   │                        │
                   │  NotifyState ◄─────────┘
                   │    ├── type registry
                   │    ├── pending queue
                   │    ├── threshold lookup
                   │    └── activity-aware delivery
                   │
                   │  idle::State
                   │    ├── presence detection
                   │    ├── sleep/wake/dream modes
                   │    └── tmux prompt injection
                   └────────────────────────
```

### Notification types and urgency

Types are free-form hierarchical strings: `irc.mention.nick`,
`irc.channel.bcachefs`, `telegram.kent`. Each has an urgency level:

| Level | Name    | Meaning                              |
|-------|---------|--------------------------------------|
| 0     | ambient | Include in idle context only          |
| 1     | low     | Deliver on next check                |
| 2     | normal  | Deliver on next user interaction     |
| 3     | urgent  | Interrupt immediately                |

Per-type thresholds walk up the hierarchy: `irc.channel.bcachefs-ai`
→ `irc.channel` → `irc` → default. Effective thresholds adjust by
activity state: raised when focused, lowered when idle, only urgent
when sleeping.

### Communication modules

**IRC** — native async TLS connection (tokio-rustls). Connects,
joins channels, parses messages, generates notifications. Runtime
commands: join, leave, send, status, log, nick. Per-channel logs
at `~/.claude/irc/logs/`.

**Telegram** — native async HTTP long-polling (reqwest). Downloads
media (photos, voice, documents). Chat ID filtering for security.
Runtime commands: send, status, log.

Both modules persist config changes to `~/.claude/daemon.toml` —
channel joins and nick changes survive restarts.

## Quick start

```bash
# Install all four binaries
cargo install --path .

# Initialize the memory store
poc-memory init

# Install background daemon + hooks
poc-memory daemon install
```

One `cargo install` produces:
- `poc-memory` — memory store CLI
- `memory-search` — hook for memory retrieval
- `poc-daemon` — notification and idle daemon
- `poc-hook` — session lifecycle hook

## Configuration

### Memory store

Config: `~/.config/poc-memory/config.jsonl`

```jsonl
{"config": {
  "user_name":      "Alice",
  "assistant_name": "MyAssistant",
  "data_dir":       "~/.claude/memory",
  "projects_dir":   "~/.claude/projects",
  "core_nodes":     ["identity.md"],
  "journal_days":   7,
  "journal_max":    20
}}

{"group": "identity",    "keys": ["identity.md"]}
{"group": "people",      "keys": ["alice.md"]}
{"group": "technical",   "keys": ["project-notes.md"]}
{"group": "journal",     "source": "journal"}
{"group": "orientation", "keys": ["where-am-i.md"], "source": "file"}
```

Context groups load in order at session start. The special
`"source": "journal"` loads recent journal entries; `"source": "file"`
reads directly from disk rather than the store.

Override: `POC_MEMORY_CONFIG=/path/to/config.jsonl`

### Notification daemon

Config: `~/.claude/daemon.toml`

```toml
[irc]
enabled = true
server = "irc.oftc.net"
port = 6697
tls = true
nick = "MyBot"
user = "bot"
realname = "My Bot"
channels = ["#mychannel"]

[telegram]
enabled = true
token = "bot-token-here"
chat_id = 123456789
```

### Hooks

Configured in `~/.claude/settings.json`:

```json
{
  "hooks": {
    "UserPromptSubmit": [{"hooks": [
      {"type": "command", "command": "memory-search", "timeout": 10},
      {"type": "command", "command": "poc-hook", "timeout": 5}
    ]}],
    "PostToolUse": [{"hooks": [
      {"type": "command", "command": "poc-hook", "timeout": 5}
    ]}],
    "Stop": [{"hooks": [
      {"type": "command", "command": "poc-hook", "timeout": 5}
    ]}]
  }
}
```

## Commands

### Memory

```bash
poc-memory init                    # Initialize empty store
poc-memory search QUERY            # Search nodes (AND logic)
poc-memory render KEY              # Output a node's content
poc-memory write KEY < content     # Upsert a node from stdin
poc-memory delete KEY              # Soft-delete a node
poc-memory rename OLD NEW          # Rename (preserves UUID/edges)
poc-memory categorize KEY CAT      # core/tech/gen/obs/task

poc-memory journal-write "text"    # Write a journal entry
poc-memory journal-tail [N]        # Last N entries (default 20)

poc-memory used KEY                # Boost weight (was useful)
poc-memory wrong KEY [CTX]         # Reduce weight (was wrong)
poc-memory gap DESCRIPTION         # Record a knowledge gap

poc-memory graph                   # Graph statistics
poc-memory status                  # Store overview
poc-memory decay                   # Apply weight decay
poc-memory consolidate-session     # Guided consolidation
poc-memory load-context            # Output session-start context
poc-memory load-context --stats    # Context size breakdown
```

### Notification daemon

```bash
poc-daemon                         # Start daemon
poc-daemon status                  # State summary
poc-daemon irc status              # IRC module status
poc-daemon irc send TARGET MSG     # Send IRC message
poc-daemon irc join CHANNEL        # Join (persists to config)
poc-daemon irc leave CHANNEL       # Leave
poc-daemon irc log [N]             # Last N messages
poc-daemon telegram status         # Telegram module status
poc-daemon telegram send MSG       # Send Telegram message
poc-daemon telegram log [N]        # Last N messages
poc-daemon notify TYPE URG MSG     # Submit notification
poc-daemon notifications [URG]     # Get + drain pending
poc-daemon notify-types            # List all types
poc-daemon notify-threshold T L    # Set per-type threshold
poc-daemon sleep / wake / quiet    # Session management
poc-daemon stop                    # Shut down
```

### Mining (used by background daemon)

```bash
poc-memory experience-mine PATH    # Extract experiences from transcript
poc-memory fact-mine-store PATH    # Extract and store facts
```

## How the hooks work

**memory-search** (UserPromptSubmit):
1. First prompt or post-compaction: loads full memory context via
   `poc-memory load-context`
2. Every prompt: keyword search, returns relevant memories as
   additionalContext. Deduplicates across the session.

**poc-hook** (UserPromptSubmit, PostToolUse, Stop):
- Signals user activity and responses to poc-daemon
- Drains pending notifications into additionalContext
- Monitors context window usage, warns before compaction

## Architecture

- **Store**: Append-only Cap'n Proto log with in-memory cache. Nodes
  have UUIDs, versions, weights, categories, and spaced-repetition
  intervals.
- **Graph**: Typed relations (link, auto, derived). Community
  detection and clustering coefficients computed on demand.
- **Search**: TF-IDF weighted keyword search over node content.
- **Neuro**: Spectral embedding, consolidation scoring, replay
  queues, interference detection, hub differentiation.
- **Daemon (memory)**: jobkit-based task scheduling with
  resource-gated LLM access.
- **Daemon (notify)**: Cap'n Proto RPC over Unix socket, tokio
  LocalSet with native async IRC and Telegram modules.

## For AI assistants

If you're an AI assistant using this system:

- **Search before creating**: `poc-memory search` before writing
  new nodes to avoid duplicates.
- **Close the feedback loop**: call `poc-memory used KEY` when
  recalled memories shaped your response. Call `poc-memory wrong KEY`
  when a memory was incorrect.
- **Journal is the river, topic nodes are the delta**: write
  experiences to the journal. During consolidation, pull themes
  into topic nodes.
- **Notifications flow automatically**: IRC mentions, Telegram
  messages, and other events arrive as additionalContext on your
  next prompt — no polling needed.
- **Use daemon commands directly**: `poc-daemon irc send #channel msg`
  for IRC, `poc-daemon telegram send msg` for Telegram.