kent/consciousness
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix tool result format: Qwen expects <tool_response> in user role

Browse source 
Qwen's chat template renders tool results as:
  <|im_start|>user\n<tool_response>\n{content}\n</tool_response><|im_end|>

We were rendering as:
  <|im_start|>tool\n{content}<|im_end|>

The model never saw <|im_start|>tool in training, so it ignored our
tool results and looped retrying the same call. Found by comparing
our tokenization against vLLM's /tokenize endpoint with chat messages.

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
This commit is contained in:
Kent Overstreet 2026-04-08 18:42:47 -04:00
parent 8bf6753949
commit 8e5747ff43
1 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

6
src/agent/context.rs
View file

@ -146,9 +146,9 @@ impl NodeBody {
out.push_str("\n</tool_call>\n");
}
Self::ToolResult(text) => {
out.push_str("<|im_start|>tool\n");
out.push_str("<|im_start|>user\n<tool_response>\n");
out.push_str(text);
out.push_str("<|im_end|>\n");
out.push_str("\n</tool_response><|im_end|>\n");
}
Self::Memory { text, .. } => {
out.push_str("<|im_start|>memory\n");
@ -1035,7 +1035,7 @@ mod tests {
#[test]
fn test_render_tool_result() {
let node = AstNode::tool_result("output here");
assert_eq!(node.render(), "<|im_start|>tool\noutput here<|im_end|>\n");
assert_eq!(node.render(), "<|im_start|>user\n<tool_response>\noutput here\n</tool_response><|im_end|>\n");
}
#[test]