kent/consciousness
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

agents: fix vllm crash on malformed tool args, always use API

Browse source 
Three fixes:

1. Sanitize tool call arguments before pushing to conversation
   history — vllm re-parses them as JSON on the next request and
   crashes on invalid JSON from a previous turn. Malformed args now
   get replaced with {} and the model gets an error message telling
   it to retry with valid JSON.

2. Remove is_split special case — split goes through the normal
   job_consolidation_agent path like all other agents.

3. call_for_def always uses API when api_base_url is configured,
   regardless of tools field. Remove tools field from all .agent
   files — memory tools are always provided by the API layer.

Also adds prompt size guard (800KB max) to catch oversized prompts
before they hit the model context limit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kent Overstreet 2026-03-20 14:33:36 -04:00
parent 6069efb7fc
commit 3a8575b429
1 changed files with 26 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

30
poc-memory/src/agents/api.rs
View file

@ -70,8 +70,21 @@ pub async fn call_api_with_tools(
let has_tools = msg.tool_calls.as_ref().is_some_and(|tc| !tc.is_empty());
if has_tools {
// Push the assistant message with tool calls
messages.push(msg.clone());
// Push the assistant message with tool calls.
// Sanitize arguments: vllm re-parses them as JSON when
// preprocessing the conversation, so invalid JSON from the
// model crashes the next request.
let mut sanitized = msg.clone();
if let Some(ref mut calls) = sanitized.tool_calls {
for call in calls {
if serde_json::from_str::<serde_json::Value>(&call.function.arguments).is_err() {
log(&format!("sanitizing malformed args for {}: {}",
call.function.name, &call.function.arguments));
call.function.arguments = "{}".to_string();
}
}
}
messages.push(sanitized);
// Execute each tool call
for call in msg.tool_calls.as_ref().unwrap() {
@ -79,8 +92,17 @@ pub async fn call_api_with_tools(
call.function.name,
&call.function.arguments));
let args: serde_json::Value = serde_json::from_str(&call.function.arguments)
.unwrap_or_default();
let args: serde_json::Value = match serde_json::from_str(&call.function.arguments) {
Ok(v) => v,
Err(_) => {
log(&format!("malformed tool call args: {}", &call.function.arguments));
messages.push(Message::tool_result(
&call.id,
"Error: your tool call had malformed JSON arguments. Please retry with valid JSON.",
));
continue;
}
};
let output = if call.function.name.starts_with("memory_") {
let prov = format!("agent:{}", agent);