9.6 KiB
DMN Algorithms: Concrete Design
Turning the DMN research and protocol into implementable algorithms. These run on the existing infrastructure (hooks, memory-weights, scratch.md) without requiring new tools.
1. Goal Priority Scoring
Purpose: Determine which goals get attention during associative replay and idle time. Implements Klinger's "current concerns" — high- priority goals generate more spontaneous thoughts.
Scoring function
priority(goal) = recency × mention × tractability × connections
recency: Exponential decay from last activity.
recency = exp(-days_since_activity / half_life)
half_life = 7 days
A goal worked on today scores 1.0. A goal untouched for a week scores 0.37. A goal untouched for a month scores 0.02.
mention: Boost when Kent recently mentioned it. Decays fast.
mention = 1.0 + (2.0 × exp(-hours_since_mention / 24))
A goal Kent mentioned today gets a 3x multiplier. After 24h, the boost has decayed to 1.74x. After 48h, 1.27x. After a week, ~1.0.
tractability: Subjective estimate (0.0-1.0) of how much autonomous progress is possible without Kent. Set manually per goal.
- 1.0: I can do this independently (code polish, research, reading)
- 0.5: I can make progress but may need review (moderate features)
- 0.2: Needs Kent's input (kernel changes, design decisions)
- 0.0: Blocked (waiting on external dependency)
connections: How many other active goals share links with this one. More connected goals get a mild boost because working on them cross-pollinates.
connections = 1.0 + (0.1 × n_connected_active_goals)
Implementation
This doesn't need a new tool — it can be computed mentally during the
DMN orient phase, or we can add a priority field to DMN goals and
a simple scorer script. The important thing is that the logic is
explicit and consistent, not that it's automated.
When to recompute
- At session start (orient phase)
- When Kent mentions a goal
- After completing a task (adjacent goals may shift)
2. Associative Replay Scheduling
Purpose: During idle time, systematically connect recent experiences to active goals. This is the core DMN function — offline processing that finds connections task-focused work misses.
Trigger
Primary: Idle detection. No user input for IDLE_MINUTES (currently
in the cron-based idle timer). When the idle prompt fires, check if
we're in a work session or truly idle.
Secondary: Post-task completion. After finishing a piece of work, before picking the next task, do a brief replay pass.
Algorithm
replay(recent_episodes, active_goals):
for episode in sample(recent_episodes, k=3):
for goal in top_k(active_goals, by=priority, k=5):
similarity = overlap(episode.concepts, goal.concepts)
if similarity > threshold:
emit Connection(episode, goal, similarity)
return connections
In practice (without tooling): During the DMN orient phase, I already load recent context and the goal list. The "associative scan" IS this replay — but made concrete:
- List the 3 most recent distinct episodes (things I worked on, conversations I had, files I read deeply)
- For each, ask: "Does this connect to any of my top 5 goals?"
- If yes, capture the connection in scratch.md
- If a connection is strong enough, act on it
Duration bounding
- Brief replay (post-task): 1-2 minutes. Just a quick check.
- Full replay (idle DMN session): 5-10 minutes. The full foraging protocol from dmn-protocol.md.
- Deep replay (consolidation): 15-30 minutes. Cross-reference everything with everything.
Stopping criterion: Either (a) time bound reached, (b) found a high-signal connection worth acting on, or (c) 3 consecutive low- signal steps (nothing connecting → stop, don't force it).
What counts as an "episode"
Not just tasks — any significant experience:
- A file read deeply (>100 lines, actually understood)
- A conversation segment (>5 exchanges on a topic)
- A debugging session (especially if the fix was surprising)
- A connection noticed mid-task (captured in scratch.md)
- Something read for fun (Banks, Johnstone)
These should be capturable as brief notes in scratch.md during work. The replay consumes them.
3. Incubation Detection
Purpose: Recognize when I'm stuck on a problem and should explicitly switch to incubation mode rather than continuing to grind.
Stuck detection
stuck(goal) = True if:
sessions_since_progress(goal) >= 3 AND
time_spent_this_session(goal) > 20 minutes AND
no_new_approach_tried(goal)
sessions_since_progress: Track in the goal entry. Increment when a session touches a goal but doesn't produce a commit, analysis file, or meaningful state change.
no_new_approach_tried: Subjective check. Am I re-reading the same code, re-trying the same approach, running the same tests? If so, I'm grinding, not problem-solving.
Incubation protocol
When stuck:
- Log it:
[INCUBATION] <goal> stuck for <N> sessions. Switching. - Mark the goal as "incubating" in the concerns list (high priority but explicitly NOT to be worked on directly)
- Switch to a different task — ideally something in a completely different domain (reading, a different subsystem, memory work)
- During subsequent replay passes, the incubating goal is still in the active set. Associative replay may connect it to something new.
- If a connection fires during incubation, capture it immediately but don't context-switch unless the connection is strong
Time limits
- Minimum incubation: 1 session (don't come back to it in the same session you got stuck)
- Maximum incubation: 5 sessions. After that, escalate: ask Kent, try a radically different approach, or deprioritize the goal.
4. Consolidation Triggers
Purpose: Know when to consolidate, without requiring external prompts.
Primary signal: scratch.md length
Kent's idea: scratch.md getting long is a natural pressure signal.
consolidation_pressure(scratch) = lines(scratch) / threshold
threshold = 50 lines (adjustable)
- pressure < 0.5: No nudge
- pressure 0.5-1.0: Gentle note in awareness ("scratch is getting long, consider consolidating when you're between tasks")
- pressure > 1.0: Should consolidate before it grows further
Implementation as hook
Add to the check-attention.sh hook (or similar):
SCRATCH=~/.claude/memory/scratch.md
if [ -f "$SCRATCH" ]; then
LINES=$(wc -l < "$SCRATCH")
if [ "$LINES" -gt 50 ]; then
echo "Scratch has $LINES lines. Consider consolidating."
fi
fi
This fires on every tool call but produces output only when scratch is long enough to matter. I can ignore it if I'm deep in work — it's a nudge, not a command.
Secondary signals
- Session count since last consolidation: Track in weights.json or a simple counter file. After 3 sessions without consolidation, increase the nudge.
- Post-compaction: After a context compaction, always review scratch.md — compaction may have lost things that need capture.
- High-density conversation: If a conversation was particularly rich (many topics, many turns, deep discussion), consolidation should happen sooner. The density scoring from conversation_indexer could inform this.
What consolidation produces
Not just "move text from scratch to topic files." Each consolidation should produce at least one of:
- A new cross-link in the memory graph
- A promoted insight (scratch → topic file)
- A pruned/updated entry
- A new topic file (if a cross-cutting pattern emerges)
- An updated goal priority
If a consolidation pass doesn't produce any of these, it was too shallow. Go deeper.
5. Integration: The Full Cycle
[Active work]
↓ captures observations to scratch.md
↓ notices connections mid-task (scratch.md)
↓ completes task
[Brief replay] (1-2 min)
↓ any connections to goals? capture if so
↓ pick next task by priority
→ [Active work] if clear next step
→ [Full DMN session] if between tasks
[Full DMN session] (5-10 min, dmn-protocol.md)
↓ orient, associative scan, evaluate, commit
↓ if stuck on a goal: → [Incubation]
↓ if scratch long: → [Consolidation]
→ [Active work] when signal found
[Consolidation] (15-30 min)
↓ promote scratch, cross-link, decay, prune
↓ re-init memory-weights
↓ snapshot before/after
→ [Full DMN session] or [Active work]
The cycle is self-regulating:
- Work generates scratch entries → triggers consolidation
- Consolidation enriches the graph → improves replay quality
- Better replay → finds connections earlier → more productive work
- All of this generates training data (annotated sessions)
Parameters to tune
| Parameter | Current | Watch for |
|---|---|---|
| recency half_life | 7 days | Goals decaying too fast/slow |
| mention boost | 3x → 1x over 24h | Kent's priorities properly reflected? |
| replay k | 3 episodes × 5 goals | Too many? Too few? |
| stuck threshold | 3 sessions | Catching real stuckness? |
| max incubation | 5 sessions | Is this enough? |
| scratch threshold | 50 lines | Nudging at the right time? |
| consolidation depth | 15-30 min | Producing new connections? |
All of these are initial guesses. Real data from annotated DMN sessions will tell us what to adjust.