amygdala lib: move_to_cpu=True to avoid bf16 SVD on CUDA
torch.svd doesn't support bf16 on CUDA; moving activations to CPU first makes pca_aggregator work. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
This commit is contained in:
parent
2ea89b1cb0
commit
7f6d94417e
1 changed files with 1 additions and 0 deletions
|
|
@ -185,6 +185,7 @@ def main() -> None:
|
|||
aggregator=aggregator,
|
||||
batch_size=args.batch_size,
|
||||
show_progress=False,
|
||||
move_to_cpu=True,
|
||||
)
|
||||
# sv.layer_activations is a dict {layer_idx: tensor[hidden]}
|
||||
for l_idx, layer in enumerate(target_layers):
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue