Skip to content

feat(providers): per-model context windows and a global auto-compact percentage - #184

Merged
GoDiao merged 1 commit into
mainfrom
feat/provider-context-windows
Oct 25, 2026
Merged

GoDiao merged 1 commit into
mainfrom
feat/provider-context-windows

Conversation

@GoDiao

@GoDiao GoDiao commented Oct 25, 2026

Copy link
Copy Markdown
Owner

Closes #105

What changed

Context window per model

  • Each model row in Settings → Providers has its own context window field next to the display name. The per-model inputs in the collapsed "Context and auto-compact" section are gone.
  • Fetching models fills a row's field only when it is empty, so typed and saved values are never replaced. Presets that define modelContextWindows keep pre-filling as before. Storage (modelContextWindows) and resolution (getContextWindowForModel) are unchanged.
  • Discovery returns an optional contextWindow per model. Lookup lives in src/server/upstream/contextWindows/, one source per provider, run in order after the model list:
    • listed: reads max_input_tokens, context_length or context_window from the model list itself. Covers OpenRouter, Anthropic and Kimi with no extra request.
    • ollama: POST /api/show per model (concurrency 4), reading model_info["<arch>.context_length"]. Used for the Ollama preset or when the list has owned_by: library.
    • lmstudio: GET /api/v1/models, max_context_length by key. LM Studio preset only.
    • deepseek: GET https://api.deepseek.com/models, context_window.
    • qwen: DashScope native GET /api/v1/models (paged), model_info.max_input_tokens.
    • Values outside 16,000–10,000,000 are ignored. Lookups share a 10 s timeout, send the key only to the provider's own host, and a failed lookup is logged with the source, model and error and leaves that model empty without failing discovery. Adding a provider is one file plus a line in index.ts.
  • The form can be saved only after the current settings have loaded; if loading fails it says so instead of saving over them.

Auto-compact

  • The CLI compacts at a percentage of the context window left after the summary reserve, 90% by default, instead of about 33K tokens before it is full. A 1M model compacts around 882K tokens (was ~967K), a 200K model around 162K (was ~167K). /context uses the same threshold function instead of its own copy of the formula.
  • Settings → General → Auto-compact sets the percentage (50–95%), saved as autoCompactPercent in user settings and read by the CLI.
  • "Unknown Model Fallback Window" is removed from the provider form and from the saved provider. Despite its name it replaced the window of every model of the provider when deciding when to compact. A stale CLAUDE_CODE_AUTO_COMPACT_WINDOW in settings.json is cleared on activation. CLAUDE_AUTOCOMPACT_PCT_OVERRIDE is removed.

Other

  • Dialogs no longer light up their whole border while a field inside has focus; the outer focus ring is now limited to the composer and the command palette (glass-panel-focus).

How it was tested

  • bun test: 1410 pass, 0 fail. desktop: bun run lint clean, bun run test -- --run 1292 pass, 4 skipped. Touched server and CLI test files also pass with --randomize.
  • New tests: each context window source with mocked fetch (field precedence, range, failures, concurrency, paging, no extra requests for other providers), the 90% and configured thresholds against a temporary config dir, stale window cleanup on activation, the General setting and store, the form rows and fill-only-empty behavior, and the composer focus ring (fails without the change).
  • Manually in the dev app with an OpenRouter custom provider: fetched models, rows filled, typed values kept on refetch, saved and reopened; row alignment, dialog focus ring and the General setting.

Not verified

  • DeepSeek, Qwen and Kimi Code are implemented from their public docs and tested with mocked responses only, not with real keys. In particular, discovery for the DeepSeek preset first requests /anthropic/v1/models; if that endpoint does not exist, discovery fails before the DeepSeek source runs.
  • Ollama and LM Studio against a real local server.
  • Windows and Linux.

…percentage

Each model row in the provider form has its own context window field, filled
from model discovery when the provider reports one. Context window lookup goes
through one source per provider (listed fields, Ollama, LM Studio, DeepSeek,
Qwen). Auto-compact now triggers at a percentage of the context window, 90% by
default and set in General, replacing the per-provider fallback window.

Closes #105
@GoDiao
GoDiao merged commit 544ffac into main Oct 25, 2026
4 checks passed
@GoDiao
GoDiao deleted the feat/provider-context-windows branch October 25, 2026 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fetch model context windows from the provider instead of guessing

1 participant