Facts are localized (ROME). Behaviors are hierarchically distributed: core circuit (small set of mid-late layer attention heads) + supporting circuits (distributed context encoding). Apollo's flat minima are right for distributed change. Rank-256 captures the full hierarchy. Includes measurement plan for validating which heads change during training. |
||
|---|---|---|
| .. | ||
| checkpoint | ||
| research | ||
| apollo_mini.py | ||
| apollo_worker.py | ||
| DESIGN.md | ||
| export_weights.py | ||
| start_vllm_with_apollo.sh | ||
| train.py | ||
| training_example.py | ||
| vllm_export_hook.py | ||
| weight_mapping.py | ||