kent/consciousness
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add direct API backend for agent execution

Browse source 
When api_base_url is configured, agents call the LLM directly via
OpenAI-compatible API (vllm, llama.cpp, etc.) instead of shelling
out to claude CLI. Implements the full tool loop: send prompt, if
tool_calls execute them and send results back, repeat until text.

This enables running agents against local/remote models like
Qwen-27B on a RunPod B200, with no dependency on claude CLI.

Config fields: api_base_url, api_key, api_model.
Falls back to claude CLI when api_base_url is not set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kent Overstreet 2026-03-18 23:05:14 -04:00
parent 1b48e57f34
commit a29b6d4c5d
6 changed files with 145 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

9
poc-memory/src/agents/llm.rs
View file

@ -184,8 +184,15 @@ pub(crate) fn call_haiku(agent: &str, prompt: &str) -> Result<String, String> {
}
/// Call a model using an agent definition's model and tool configuration.
/// Uses the direct API backend when api_base_url is configured,
/// otherwise falls back to claude CLI subprocess.
pub(crate) fn call_for_def(def: &super::defs::AgentDef, prompt: &str) -> Result<String, String> {
call_model_with_tools(&def.agent, &def.model, prompt, &def.tools)
if crate::config::get().api_base_url.is_some() && !def.tools.is_empty() {
let log = |msg: &str| eprintln!("[{}] {}", def.agent, msg);
super::api::call_api_with_tools_sync(&def.agent, prompt, &log)
} else {
call_model_with_tools(&def.agent, &def.model, prompt, &def.tools)
}
}
/// Parse a JSON response, handling markdown fences.