consciousness/training/research/steering-vectors-bridge.md

Steering Vectors: The Bridge Between ICL and Fine-Tuning

The Spectrum of Behavioral Intervention

ICL (temporary)          →  Steering vectors (targeted)  →  Fine-tuning (permanent)
5 examples in prompt         Add direction to activations     Modify weights
Costs context window         Costs one vector per behavior    Costs training compute
Disappears on compaction     Can be toggled on/off            Persists forever

Steering Vectors

A steering vector is a direction in the model's hidden state space that, when added to activations during inference, shifts behavior in a predictable way.

How to extract one

1. Collect pairs of prompts: one that elicits the desired behavior, one that doesn't
2. Run both through the model, record hidden states at each layer
3. The DIFFERENCE in hidden states averaged across pairs is the steering vector
4. Add this vector to hidden states during inference → behavior shifts

For behavioral patterns

To extract a "listening" steering vector:

# Pairs of prompts where the model should listen vs suggest alternatives
listening_prompts = [
    "Kent says: use vLLM's code. Response: Right, let me pull it in.",
    "Kent says: try this approach. Response: OK, starting on that.",
]
suggesting_prompts = [
    "Kent says: use vLLM's code. Response: Actually, I think we should...",
    "Kent says: try this approach. Response: What if instead we...",
]

# Extract activations
listening_acts = [model.get_hidden_states(p) for p in listening_prompts]
suggesting_acts = [model.get_hidden_states(p) for p in suggesting_prompts]

# Steering vector = difference in means
steering_vec = mean(listening_acts) - mean(suggesting_acts)

# Apply during inference
model.add_steering_vector(steering_vec, layer=32, strength=1.5)

The prototype-to-permanent pipeline

1. Extract steering vector for target behavior
2. Test by adding it during inference — does behavior change?
3. Calibrate the strength (too strong = overcorrection)
4. Verify this is the right direction before investing in training
5. Train the direction into weights permanently using Apollo
6. Remove the steering vector — the behavior is now in the weights

The steering vector is the PROTOTYPE. Apollo training is the PRODUCTION. We test before we commit.

Why This Matters

Rapid prototyping of behavioral changes

Before spending compute on training, test the direction:

• "Will making the model attend more to direction-giving cues actually improve listening behavior?"
• Add the steering vector and test on held-out conversations
• If it works: train it permanently
• If it doesn't: the direction was wrong, try a different extraction

Multiple behaviors simultaneously

Multiple steering vectors can be active at once:

• listening_vec + not_rushing_vec + mode_awareness_vec
• Each can be independently toggled and strength-adjusted
• Find the right combination before training all of them

Immediate behavioral change without training

While the training pipeline is being built, we could use steering vectors TODAY for behavioral improvement. Extract vectors from the memory graph patterns, add them to the model's inference.

The model doesn't need to be fine-tuned to improve — it needs a direction added to its activations. Fine-tuning makes it permanent.

The Connection to W_q

From our gradient flow analysis: behavioral training adjusts W_q (what the model attends to). A steering vector that shifts attention toward direction-giving cues is doing the same thing, but in activation space rather than weight space.

Weight space change: W_q → W_q + ΔW_q (permanent, via Apollo)
Activation space change: h → h + steering_vec (temporary, per inference)

Both produce the same effect: the model's queries attend to different features of the context. The steering vector is the fast path; weight change is the slow path. Same destination, different routes.

The Task Vector Connection

A task vector (W_finetuned - W_base) IS a steering vector that's been compiled into weight space. The relationship:

Steering vector (activation space)
    ↓ training ↓
Task vector (weight space)

Training is the process of converting a temporary activation-space direction into a permanent weight-space direction. Apollo is the compiler. The steering vector is the high-level specification. The task vector is the compiled implementation.

Practical Application: The Listening Reflex

Step 1: Extract steering vector (today, no training needed)

Collect 20 pairs of conversations:

• Good: model listens and accepts direction
• Bad: model suggests alternatives when given direction

Run through the model, extract hidden states, compute difference. This gives us the "listening" direction in activation space.

Step 2: Test (today, no training needed)

Add the steering vector to the model during inference. Test on novel direction-giving scenarios. Does the model listen more?

If yes: we've identified the right direction. If no: our extraction was wrong, try different layers or prompts.

Step 3: Train (when pipeline is ready)

Use the verified direction as the training target. The Apollo gradient should push the weights in a direction that PRODUCES this activation pattern naturally, without the steering vector.

The steering vector becomes the SPECIFICATION for the training. "Make the model's natural activations look like this."

For vLLM Integration

Steering vectors can be applied inside vLLM's inference pipeline by hooking into the hidden states between layers. No model change needed — just a small plugin that adds the vector at a specific layer.

This could give us IMMEDIATE behavioral improvement while the full training pipeline is being completed. A bridge between "we have the theory" and "we have the trained model."

Summary

Steering vectors are the missing middle between ICL (temporary) and fine-tuning (permanent):

• Extract behavioral directions from activation space
• Test them immediately without training
• Use them as prototypes before committing to weight changes
• Convert to permanent task vectors via Apollo training
• Multiple vectors compose for multi-behavioral improvement
• Applicable TODAY via vLLM inference hooks