AstNode fields are now private with read-only accessors. All mutation
goes through ContextState methods (push, set_message, set_score, del)
which guarantee token_ids stays in sync with text on every leaf.
Also fix ResponseParser to use AstNode::tool_call() constructor,
widen parsing module visibility to pub(crate).
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
New context_new.rs with the AST-based context window design:
- AstNode: role + NodeBody (Leaf with text+token_ids, or Branch with children)
- Tokens only on leaves, branches walk children
- render() produces UTF-8, tokenize produces token IDs, same path
- ResponseParser state machine for streaming assistant responses
- Role enum covers all node types including sections
Still needs: fix remaining pattern match issues, add ContextState wrapper,
wire into mod.rs, replace old context.rs.
Does not compile yet — this is a design checkpoint.
Co-Authored-By: Proof of Concept <poc@bcachefs.org>
