forked from kent/consciousness
bb80225942
render_into(&mut String) and token_ids_into(&mut Vec<u32>) recurse the tree extending the output in place. Branches emit their wrapping (im_start/role/im_end) and recurse into children — same structure in both methods. token_ids() now composes from cached leaf tokens instead of re-encoding the full rendered string. Killed the AstEvent/AstIter iterator experiment — explicit recursion is cleaner for a tree walk that isn't truly flattening. Co-Authored-By: Proof of Concept <poc@bcachefs.org> |
||
|---|---|---|
| .. | ||
| api | ||
| tools | ||
| context.rs | ||
| context_new.rs | ||
| mod.rs | ||
| oneshot.rs | ||
| tokenizer.rs | ||