Problem
Right now, every LLM call Codebuff makes goes through its backend → OpenRouter, regardless of what model string is set in an AgentDefinition. There is no way to point an agent at a locally-running model.
This blocks three real use cases:
- Privacy — code never leaves the machine
- Offline / air-gapped — no internet, no Codebuff
- Cost splitting — cheap tasks (file picking) locally, complex reasoning on cloud
Related: #288 requested LiteLLM integration. This is a narrower, zero-dependency alternative — no new packages, no Python proxy.
Proposed Solution
Add an optional localProvider field to AgentDefinition. When present, the CLI calls the local endpoint directly, bypassing the Codebuff backend entirely.
Type change
// .agents/types/agent-definition.ts
export interface AgentDefinition {
// ...existing fields unchanged
localProvider?: {
baseUrl: string // e.g. "http://localhost:11434/v1"
apiKey?: string // optional — Ollama ignores it
}
}Dispatch logic
// cli/src/<agent-dispatch>
if (agent.localProvider) {
const client = new OpenAI({
baseURL: agent.localProvider.baseUrl,
apiKey: agent.localProvider.apiKey ?? 'ollama',
})
return client.chat.completions.create({ model: agent.model, messages, stream: true })
}
// Default: existing Codebuff backend → OpenRouter path
openai is already in the dependency tree via OpenRouter — zero new packages needed.
Global fallback (env var)
CODEBUFF_LOCAL_BASE_URL=http://localhost:11434/v1
If set, applies as a fallback localProvider for any agent that doesn't define one explicitly.
Scope
- Files touched: ~3 (
agent-definition.ts, CLI dispatch, env schema)
- Lines of code: ~80
- New dependencies: None
- Breaking changes: None — fully backward-compatible
Happy to Submit a PR for this if the approach looks good.
Problem
Right now, every LLM call Codebuff makes goes through its backend → OpenRouter, regardless of what
modelstring is set in anAgentDefinition. There is no way to point an agent at a locally-running model.This blocks three real use cases:
Proposed Solution
Add an optional
localProviderfield toAgentDefinition. When present, the CLI calls the local endpoint directly, bypassing the Codebuff backend entirely.Type change
Dispatch logic
Global fallback (env var)
If set, applies as a fallback
localProviderfor any agent that doesn't define one explicitly.Scope
agent-definition.ts, CLI dispatch, env schema)Happy to Submit a PR for this if the approach looks good.