consciousness/src/agent
Kent Overstreet f458af6dec Add /v1/completions streaming path with raw token IDs
New stream_completions() in openai.rs sends prompt as token IDs to
the completions endpoint instead of JSON messages to chat/completions.
Handles <think> tags in the response (split into Reasoning events)
and stops on <|im_end|> token.

start_stream_completions() on ApiClient provides the same interface
as start_stream() but takes token IDs instead of Messages.

The turn loop in Agent::turn() uses completions when the tokenizer
is initialized, falling back to the chat API otherwise. This allows
gradual migration — consciousness uses completions (Qwen tokenizer),
Claude Code hook still uses chat API (Anthropic).

Co-Authored-By: Proof of Concept <poc@bcachefs.org>
2026-04-08 11:42:22 -04:00
..
api Add /v1/completions streaming path with raw token IDs 2026-04-08 11:42:22 -04:00
tools Agent-aware provenance for memory tools 2026-04-07 17:46:40 -04:00
context.rs Kill tiktoken — all token counting now uses Qwen 3.5 tokenizer 2026-04-08 11:25:28 -04:00
mod.rs Add /v1/completions streaming path with raw token IDs 2026-04-08 11:42:22 -04:00
oneshot.rs Fix bail script: pass own pid file so it can exclude itself 2026-04-08 09:35:52 -04:00
tokenizer.rs Replace token counting with token generation via HuggingFace tokenizer 2026-04-08 11:20:03 -04:00