From 34f59904d5deba006a42ad346e770aa1f1e87088 Mon Sep 17 00:00:00 2001
From: neubig <neubig@users.noreply.github.com>
Date: Wed, 22 Jul 2026 20:10:46 +0000
Subject: [PATCH] docs: explain model capability overrides

---
 sdk/guides/llm-reasoning.mdx | 43 ++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

diff --git a/sdk/guides/llm-reasoning.mdx b/sdk/guides/llm-reasoning.mdx
index c46500b3c..eb3cf6280 100644
--- a/sdk/guides/llm-reasoning.mdx
+++ b/sdk/guides/llm-reasoning.mdx
@@ -11,6 +11,45 @@ This guide demonstrates two provider-specific approaches:
 1. **Anthropic Extended Thinking** - Claude's thinking blocks for complex reasoning
 2. **OpenAI Reasoning via Responses API** - GPT's reasoning effort parameter
 
+## Configure New Or Proxied Models
+
+Use `reasoning_effort` as the provider-neutral reasoning control. Common values include `"none"`, `"minimal"`,
+`"low"`, `"medium"`, `"high"`, `"xhigh"`, and `"max"`. Supported values depend on the model and provider. The SDK
+passes the value to LiteLLM, which translates it to the provider's request format.
+
+By default, the SDK resolves reasoning, sampling, prompt caching, and endpoint support from LiteLLM model metadata.
+For a gateway alias, provide the underlying model name so capability discovery does not depend on the alias:
+
+```python icon="python" wrap
+llm = LLM(
+    model="litellm_proxy/customer-sonnet",
+    model_canonical_name="anthropic/claude-sonnet-5",
+    reasoning_effort="high",
+)
+```
+
+Adaptive-thinking models use `reasoning_effort`; the SDK does not send a legacy manual `thinking` block for them.
+`extended_thinking_budget` remains available only for models confirmed to use manual extended thinking.
+
+If a custom gateway cannot expose model metadata, override only the missing capabilities:
+
+```python icon="python" wrap
+llm = LLM(
+    model="litellm_proxy/customer-sonnet",
+    reasoning_effort="high",
+    api_mode="chat",
+    capability_overrides={
+        "supports_reasoning_effort": True,
+        "thinking_mode": "adaptive",
+        "supports_sampling_params": False,
+        "supports_prompt_cache": True,
+    },
+)
+```
+
+`api_mode` accepts `"auto"` (the default), `"chat"`, or `"responses"`. Explicit capability overrides take precedence
+over gateway metadata, local LiteLLM metadata, and SDK fallbacks.
+
 ## Anthropic Extended Thinking
 
 > A ready-to-run example is available [here](#ready-to-run-example-antrophic)!
@@ -148,8 +187,8 @@ llm = LLM(
 )
 ```
 
-The `reasoning_effort` parameter can be set to `"none"`, `"low"`, `"medium"`, or `"high"` to control the amount of
-reasoning performed by the model.
+The values supported by `reasoning_effort` depend on the selected model. For example, many models accept `"low"`,
+`"medium"`, or `"high"`, while newer models may also accept values such as `"minimal"`, `"xhigh"`, or `"max"`.
 
 Then capture reasoning traces in your callback: