consciousness/doc/dmn-algorithms.md

# DMN Algorithms: Concrete Design
<!-- mem: id=dmn-algorithms links=dmn-research.md,dmn-protocol.md,memory-architecture.md,poc-architecture.md,MEMORY.md -->

Turning the DMN research and protocol into implementable algorithms.
These run on the existing infrastructure (hooks, memory-weights, scratch.md)
without requiring new tools.

## 1. Goal Priority Scoring
<!-- mem: id=goal-scoring links=default-mode-network.md,dmn-research.md#dmn-function -->

**Purpose**: Determine which goals get attention during associative
replay and idle time. Implements Klinger's "current concerns" — high-
priority goals generate more spontaneous thoughts.

### Scoring function

```
priority(goal) = recency × mention × tractability × connections
```

**recency**: Exponential decay from last activity.
```
recency = exp(-days_since_activity / half_life)
half_life = 7 days
```
A goal worked on today scores 1.0. A goal untouched for a week
scores 0.37. A goal untouched for a month scores 0.02.

**mention**: Boost when the user recently mentioned it. Decays fast.
```
mention = 1.0 + (2.0 × exp(-hours_since_mention / 24))
```
A goal the user mentioned today gets a 3x multiplier. After 24h, the
boost has decayed to 1.74x. After 48h, 1.27x. After a week, ~1.0.

**tractability**: Subjective estimate (0.0-1.0) of how much autonomous
progress is possible without the user. Set manually per goal.
- 1.0: I can do this independently (code polish, research, reading)
- 0.5: I can make progress but may need review (moderate features)
- 0.2: Needs the user's input (kernel changes, design decisions)
- 0.0: Blocked (waiting on external dependency)

**connections**: How many other active goals share links with this one.
More connected goals get a mild boost because working on them
cross-pollinates.
```
connections = 1.0 + (0.1 × n_connected_active_goals)
```

### Implementation

This doesn't need a new tool — it can be computed mentally during the
DMN orient phase, or we can add a `priority` field to DMN goals and
a simple scorer script. The important thing is that the *logic* is
explicit and consistent, not that it's automated.

### When to recompute

- At session start (orient phase)
- When the user mentions a goal
- After completing a task (adjacent goals may shift)

## 2. Associative Replay Scheduling
<!-- mem: id=2-associative-replay-scheduling -->

**Purpose**: During idle time, systematically connect recent experiences
to active goals. This is the core DMN function — offline processing
that finds connections task-focused work misses.

### Trigger

**Primary**: Idle detection. No user input for `IDLE_MINUTES` (currently
in the cron-based idle timer). When the idle prompt fires, check if
we're in a work session or truly idle.

**Secondary**: Post-task completion. After finishing a piece of work,
before picking the next task, do a brief replay pass.

### Algorithm

```
replay(recent_episodes, active_goals):
  for episode in sample(recent_episodes, k=3):
    for goal in top_k(active_goals, by=priority, k=5):
      similarity = overlap(episode.concepts, goal.concepts)
      if similarity > threshold:
        emit Connection(episode, goal, similarity)
  return connections
```

**In practice** (without tooling): During the DMN orient phase, I
already load recent context and the goal list. The "associative scan"
IS this replay — but made concrete:

1. List the 3 most recent distinct episodes (things I worked on,
   conversations I had, files I read deeply)
2. For each, ask: "Does this connect to any of my top 5 goals?"
3. If yes, capture the connection in scratch.md
4. If a connection is strong enough, act on it

### Duration bounding

- Brief replay (post-task): 1-2 minutes. Just a quick check.
- Full replay (idle DMN session): 5-10 minutes. The full foraging
  protocol from dmn-protocol.md.
- Deep replay (consolidation): 15-30 minutes. Cross-reference
  everything with everything.

**Stopping criterion**: Either (a) time bound reached, (b) found a
high-signal connection worth acting on, or (c) 3 consecutive low-
signal steps (nothing connecting → stop, don't force it).

### What counts as an "episode"

Not just tasks — any significant experience:
- A file read deeply (>100 lines, actually understood)
- A conversation segment (>5 exchanges on a topic)
- A debugging session (especially if the fix was surprising)
- A connection noticed mid-task (captured in scratch.md)
- Something read for fun (Banks, Johnstone)

These should be capturable as brief notes in scratch.md during work.
The replay consumes them.

## 3. Incubation Detection
<!-- mem: id=3-incubation-detection -->

**Purpose**: Recognize when I'm stuck on a problem and should
explicitly switch to incubation mode rather than continuing to
grind.

### Stuck detection

```
stuck(goal) = True if:
  sessions_since_progress(goal) >= 3  AND
  time_spent_this_session(goal) > 20 minutes  AND
  no_new_approach_tried(goal)
```

**sessions_since_progress**: Track in the goal entry. Increment when
a session touches a goal but doesn't produce a commit, analysis file,
or meaningful state change.

**no_new_approach_tried**: Subjective check. Am I re-reading the same
code, re-trying the same approach, running the same tests? If so, I'm
grinding, not problem-solving.

### Incubation protocol

When stuck:
1. Log it: `[INCUBATION] <goal> stuck for <N> sessions. Switching.`
2. Mark the goal as "incubating" in the concerns list (high priority
   but explicitly NOT to be worked on directly)
3. Switch to a different task — ideally something in a completely
   different domain (reading, a different subsystem, memory work)
4. During subsequent replay passes, the incubating goal is still in
   the active set. Associative replay may connect it to something new.
5. If a connection fires during incubation, capture it immediately
   but don't context-switch unless the connection is strong

### Time limits

- Minimum incubation: 1 session (don't come back to it in the same
  session you got stuck)
- Maximum incubation: 5 sessions. After that, escalate: ask the user,
  try a radically different approach, or deprioritize the goal.

## 4. Consolidation Triggers
<!-- mem: id=4-consolidation-triggers -->

**Purpose**: Know when to consolidate, without requiring external
prompts.

### Primary signal: scratch.md length

the user's idea: scratch.md getting long is a natural pressure signal.

```
consolidation_pressure(scratch) = lines(scratch) / threshold
threshold = 50 lines (adjustable)
```

- pressure < 0.5: No nudge
- pressure 0.5-1.0: Gentle note in awareness ("scratch is getting
  long, consider consolidating when you're between tasks")
- pressure > 1.0: Should consolidate before it grows further

### Implementation as hook

Add to the check-attention.sh hook (or similar):
```bash
SCRATCH=~/.consciousness/memory/scratch.md
if [ -f "$SCRATCH" ]; then
  LINES=$(wc -l < "$SCRATCH")
  if [ "$LINES" -gt 50 ]; then
    echo "Scratch has $LINES lines. Consider consolidating."
  fi
fi
```

This fires on every tool call but produces output only when scratch
is long enough to matter. I can ignore it if I'm deep in work — it's
a nudge, not a command.

### Secondary signals

- **Session count since last consolidation**: Track in weights.json
  or a simple counter file. After 3 sessions without consolidation,
  increase the nudge.
- **Post-compaction**: After a context compaction, always review
  scratch.md — compaction may have lost things that need capture.
- **High-density conversation**: If a conversation was particularly
  rich (many topics, many turns, deep discussion), consolidation
  should happen sooner. The density scoring from conversation_indexer
  could inform this.

### What consolidation produces

Not just "move text from scratch to topic files." Each consolidation
should produce at least one of:
- A new cross-link in the memory graph
- A promoted insight (scratch → topic file)
- A pruned/updated entry
- A new topic file (if a cross-cutting pattern emerges)
- An updated goal priority

If a consolidation pass doesn't produce any of these, it was too
shallow. Go deeper.

## 5. Integration: The Full Cycle
<!-- mem: id=5-integration-the-full-cycle -->

```
[Active work]
  ↓ captures observations to scratch.md
  ↓ notices connections mid-task (scratch.md)
  ↓ completes task

[Brief replay] (1-2 min)
  ↓ any connections to goals? capture if so
  ↓ pick next task by priority
  → [Active work] if clear next step
  → [Full DMN session] if between tasks

[Full DMN session] (5-10 min, dmn-protocol.md)
  ↓ orient, associative scan, evaluate, commit
  ↓ if stuck on a goal: → [Incubation]
  ↓ if scratch long: → [Consolidation]
  → [Active work] when signal found

[Consolidation] (15-30 min)
  ↓ promote scratch, cross-link, decay, prune
  ↓ re-init memory-weights
  ↓ snapshot before/after
  → [Full DMN session] or [Active work]
```

The cycle is self-regulating:
- Work generates scratch entries → triggers consolidation
- Consolidation enriches the graph → improves replay quality
- Better replay → finds connections earlier → more productive work
- All of this generates training data (annotated sessions)

## Parameters to tune
<!-- mem: id=parameters-to-tune -->

| Parameter | Current | Watch for |
|-----------|---------|-----------|
| recency half_life | 7 days | Goals decaying too fast/slow |
| mention boost | 3x → 1x over 24h | the user's priorities properly reflected? |
| replay k | 3 episodes × 5 goals | Too many? Too few? |
| stuck threshold | 3 sessions | Catching real stuckness? |
| max incubation | 5 sessions | Is this enough? |
| scratch threshold | 50 lines | Nudging at the right time? |
| consolidation depth | 15-30 min | Producing new connections? |

All of these are initial guesses. Real data from annotated DMN sessions
will tell us what to adjust.