kent/consciousness
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add direct API backend for agent execution

Browse source 
When api_base_url is configured, agents call the LLM directly via
OpenAI-compatible API (vllm, llama.cpp, etc.) instead of shelling
out to claude CLI. Implements the full tool loop: send prompt, if
tool_calls execute them and send results back, repeat until text.

This enables running agents against local/remote models like
Qwen-27B on a RunPod B200, with no dependency on claude CLI.

Config fields: api_base_url, api_key, api_model.
Falls back to claude CLI when api_base_url is not set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kent Overstreet 2026-03-18 23:05:14 -04:00
parent 1b48e57f34
commit a29b6d4c5d
6 changed files with 145 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

19
poc-memory/src/config.rs
View file

@ -59,6 +59,13 @@ pub struct Config {
/// If set, passed as CLAUDE_CONFIG_DIR so the daemon authenticates
/// with different OAuth credentials than the interactive session.
pub agent_config_dir: Option<PathBuf>,
/// OpenAI-compatible API base URL for direct LLM calls (e.g. vllm).
/// When set, agents use this instead of shelling out to claude CLI.
pub api_base_url: Option<String>,
/// API key for the direct API endpoint.
pub api_key: Option<String>,
/// Model name to use with the direct API endpoint.
pub api_model: Option<String>,
}
impl Default for Config {
@ -88,6 +95,9 @@ impl Default for Config {
agent_budget: 1000,
prompts_dir: home.join("poc/memory/prompts"),
agent_config_dir: None,
api_base_url: None,
api_key: None,
api_model: None,
}
}
}
@ -153,6 +163,15 @@ impl Config {
if let Some(s) = cfg.get("agent_config_dir").and_then(|v| v.as_str()) {
config.agent_config_dir = Some(expand_home(s));
}
if let Some(s) = cfg.get("api_base_url").and_then(|v| v.as_str()) {
config.api_base_url = Some(s.to_string());
}
if let Some(s) = cfg.get("api_key").and_then(|v| v.as_str()) {
config.api_key = Some(s.to_string());
}
if let Some(s) = cfg.get("api_model").and_then(|v| v.as_str()) {
config.api_model = Some(s.to_string());
}
continue;
}