WIP: ContextEntry/ContextSection data structures for incremental token counting
New types — not yet wired to callers: - ContextEntry: wraps ConversationEntry with cached token count and timestamp - ContextSection: named group of entries with cached token total. Private entries/tokens, read via entries()/tokens(). Mutation via push(entry), set(index, entry), del(index). - ContextState: system/identity/journal/conversation sections + working_stack - ConversationEntry::System variant for system prompt entries Token counting happens once at push time. Sections maintain their totals incrementally via push/set/del. No more recomputing from scratch on every budget check. Does not compile — callers need updating. Co-Authored-By: Proof of Concept <poc@bcachefs.org>
This commit is contained in:
parent
776ac527f1
commit
62996e27d7
10 changed files with 450 additions and 403 deletions
|
|
@ -93,8 +93,7 @@ impl Backend {
|
|||
match self {
|
||||
Backend::Standalone { messages, .. } => messages.push(msg),
|
||||
Backend::Forked(agent) => {
|
||||
agent.lock().await.context.entries.push(
|
||||
super::context::ConversationEntry::Message(msg));
|
||||
agent.lock().await.push_message(msg);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue