Skip to content

Accept every reasoning effort ReqLLM accepts, including max - #254

Merged
deepfates merged 4 commits into
mainfrom
claude/imp-max-effort
Sep 28, 2026
Merged

deepfates merged 4 commits into
mainfrom
claude/imp-max-effort

Conversation

@deepfates

@deepfates deepfates commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

What changed

  • Imp.Clients.ReqLLM reads its accepted :reasoning_effort values from ReqLLM's own generation schema (ReqLLM.Provider.Options.generation_schema(), the {:in, [...]} type of reasoning_effort) instead of keeping a second list, so Imp accepts max, which ReqLLM's list already had (since at least 1.18) and Imp's hand-written list lacked. The match fails the build if ReqLLM changes the option's shape.
  • The saved-program loader (Imp.Saving) checks against the same list, so a saved "max" loads.
  • A string effort is turned into its atom on its way into ReqLLM (the client's stored options, which feed the cache key, are unchanged). ReqLLM checks reasoning_effort against its atom list before most providers see it; only OpenAI, xAI and Meta convert the string form first. So in 0.6.0 a string effort, including every effort loaded from a saved program (always a string), failed every call on OpenRouter's default top-level wire, Anthropic, Google and Groq, with a NimbleOptions error. OpenRouter's nested wire, OpenAI and xAI were not affected.
  • CHANGELOG: Unreleased entry.
  • Dialyzer: the new code shifted the two .dialyzer_ignore.exs entries pinned by line in this file. The catch-all after ReqLLM.model/1's {:ok, _}/{:error, _} clauses in resolve_model/1 could never match (ReqLLM's contract); an unexpected result now raises a CaseClauseError that the function's own rescue turns into {:error, _}, so the clause and its ignore entry are removed. The response.provider_meta || %{} guard stays defensive (the struct does not enforce its map type); its entry is re-pinned to line 1902 with its existing reason. mix dialyzer.check passes locally.

Why

Some model residents are configured for the highest thinking level. ReqLLM accepts :max; Imp 0.6.0 refused it at build and per call.

Decision: accept every effort ReqLLM accepts, on every provider, and leave the mapping to ReqLLM. I read how ReqLLM 1.24 handles :max per provider: OpenRouter, Zenmux, Groq, xAI, Mistral and other pass-through providers send "max"; OpenAI's param profiles send "max"; Anthropic maps to "max" where the model supports it, otherwise "xhigh" or a fixed budget; Google maps to :high / 32768 tokens; LM Studio clamps to xhigh with a warning; Ollama and Fireworks send "max"; Moonshot uses :max. DeepSeek sends "max" as a string pass-through. None rejects it inside ReqLLM; whether a given model accepts "max" is the provider API's call, as it already is for xhigh.

Tests

  • New: ReqLLMClientTest "reasoning_effort max reaches OpenRouter as "max" on either wire": a Req adapter (no network) captures the request body. "max" configured on the client arrives as top-level "reasoning_effort": "max"; :max on a call over a client with no effort does the same; openrouter_reasoning_wire: :nested sends "reasoning": {"effort": "max"}.
  • New: ReqLLMClientTest "a string reasoning_effort reaches OpenRouter and Anthropic requests": the existing string "high" on OpenRouter's top-level wire arrives as "reasoning_effort": "high", and on Anthropic arrives as a thinking block.
  • Extended: Imp.SavingReqLLMTransportTest "saved reasoning effort round-trips and remains narrowly allowlisted" round-trips :max through dump/load.
  • Changed: the native-reasoning test ("ReqLLM registry capability drives the explicit typed reasoning contract") now sees :low rather than "low" reaching the ReqLLM module, which is the new conversion.

Falsification: with the old hard-coded list restored, the new OpenRouter test and the saving test fail (2 failures). With the string-to-atom conversion removed, the new OpenRouter test fails with ReqLLM's invalid value for :reasoning_effort ... got: "max", and the native-reasoning test fails. With lib/ taken from main, the string-effort test fails on its OpenRouter half and, with that half removed, on its Anthropic half, both with ReqLLM's got: "high" validation error. All restored: pass.

mix check after merging main (with #253) and the dialyzer fix: exit 0; 59 doctests, 9 properties, 3545 tests, 0 failures, 13 skipped (221 excluded).

Unsure / left alone

Imp's accepted efforts are read from ReqLLM's own reasoning_effort
option instead of a second list, so max is accepted at build, per call
and in a saved program. A string effort reaches ReqLLM as its atom:
ReqLLM's OpenRouter provider refused the string form.
…guard

The catch-all after ReqLLM.model/1's ok/error clauses could never match;
an unexpected result is a CaseClauseError the function's rescue already
turns into an error, so the clause and its ignore entry go. The
provider_meta guard stays deliberately defensive and its ignore entry
moves to the line it now covers.
@deepfates
deepfates merged commit dcf20f2 into main Sep 28, 2026
10 checks passed
@deepfates
deepfates deleted the claude/imp-max-effort branch September 28, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant