fire XML tool calls as they arrive during streaming

When </tool_call> is detected in the content stream, parse and
dispatch immediately via FuturesOrdered. Tool calls execute
concurrently while the stream continues. Results collected in
order after the stream ends.

Structured API path (ToolCallDelta) unchanged — still uses
post-stream parallel dispatch.

Co-Developed-By: Kent Overstreet <kent.overstreet@linux.dev>
This commit is contained in:
ProofOfConcept 2026-04-03 22:14:35 -04:00
parent 2f0c7ce5c2
commit d25033b9f4
2 changed files with 105 additions and 3 deletions

View file

@ -16,6 +16,15 @@ use crate::user::types::*;
/// Parse leaked tool calls from response text.
/// Looks for `<tool_call>...</tool_call>` blocks and tries both
/// XML and JSON formats for the body.
/// Parse a single tool call body (content between `<tool_call>` and `</tool_call>`).
pub fn parse_tool_call_body(body: &str) -> Option<ToolCall> {
let normalized = normalize_xml_tags(body);
let body = normalized.trim();
let mut counter = 0u32;
parse_xml_tool_call(body, &mut counter)
.or_else(|| parse_json_tool_call(body, &mut counter))
}
pub fn parse_leaked_tool_calls(text: &str) -> Vec<ToolCall> {
// Normalize whitespace inside XML tags: "<\nfunction\n=\nbash\n>" → "<function=bash>"
// This handles streaming tokenizers that split tags across tokens.