ContextState + private AstNode fields: enforce token_ids invariant

AstNode fields are now private with read-only accessors. All mutation
goes through ContextState methods (push, set_message, set_score, del)
which guarantee token_ids stays in sync with text on every leaf.

Also fix ResponseParser to use AstNode::tool_call() constructor,
widen parsing module visibility to pub(crate).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
This commit is contained in:
Kent Overstreet 2026-04-08 12:58:59 -04:00
parent 29dc339f54
commit 6730d136d4
3 changed files with 124 additions and 42 deletions

View file

@ -7,7 +7,7 @@
// Set POC_DEBUG=1 for verbose per-turn logging.
pub mod http;
mod parsing;
pub(crate) mod parsing;
mod types;
mod openai;

View file

@ -17,7 +17,7 @@ use super::types::{ToolCall, FunctionCall};
/// Looks for `<tool_call>...</tool_call>` blocks and tries both
/// XML and JSON formats for the body.
/// Parse a single tool call body (content between `<tool_call>` and `</tool_call>`).
pub(super) fn parse_tool_call_body(body: &str) -> Option<ToolCall> {
pub(crate) fn parse_tool_call_body(body: &str) -> Option<ToolCall> {
let normalized = normalize_xml_tags(body);
let body = normalized.trim();
let mut counter = 0u32;