BYOK Azure Foundry: no quota warning and /compact cannot recover a long session #5277

Description of the bug

With a BYOK model deployed in Microsoft Foundry, a long Copilot CLI session becomes unusable because requests exceed the deployment's token rate limit, even though the displayed model context window is far from full. Ordinary messages fail, and /compact also fails, leaving no working in-session recovery through compaction.

The CLI did not give me an actionable warning about the deployment's TPM limit before the session reached this state. Its context display still shows substantial free space.

Affected version

GitHub Copilot CLI 1.0.104.

Environment

• Operating system: Windows.
• Provider: BYOK using a Microsoft Foundry deployment.
• Model: gpt-6.1-sol.
• Region: eastus.
• Deployment token rate limit: 250,000 TPM (user-reported configuration).
• CLI-displayed context capacity: 922k tokens.

Steps to reproduce the behavior

1. Configure Copilot CLI to use a Foundry BYOK deployment with a large model context window but a substantially lower TPM allocation.
2. Continue a long conversation until /context shows approximately 257k/922k tokens.
3. Send an ordinary message. It fails to obtain a model response.
4. Run /compact to reduce the history and continue the session.
5. Compaction also fails with the deployment token-rate-limit error below.

Actual behavior

Exact user-visible compaction error:

Compaction Failed: Error: Failed to get response from the AI model; retried 5 times (total retry wait time: 5.00 seconds) Last error: Your requests to gpt-6.1-sol for gpt-6.1-sol in eastus have exceeded token rate limit.

Relevant /context output:

gpt-6.1-sol: 257k/922k tokens (28%)
Messages:   239.5k (26%)
Free Space: 618.7k (67%)
Buffer:      46.1k (5%)

The error identifies a token-rate-limit failure, not exhaustion of the advertised 922k model context window. The displayed accumulated context is already greater than the deployment's reported 250k TPM allocation.

Expected behavior

• Distinguish model context capacity from deployment throughput limits in BYOK diagnostics. Explain that a large context window does not guarantee the deployment can accept the corresponding request under its current rate limit.
• Provide proactive, actionable warnings using provider rate-limit headers when available, and/or support a user-configured effective request budget for BYOK deployments. Account for prompt overhead and the provider's output-token reservation/estimation where applicable.
• Trigger compaction early enough to remain within that effective budget.
• Provide a recovery path when the session is already too large for a normal compaction request, such as bounded/chunked compaction or export of a handover that can be used in a fresh session. Preserve the original conversation rather than silently discarding history.
• Respect provider retry timing for temporary throttling, but do not rely only on resubmitting an unchanged request if its estimated token cost exceeds the deployment's entire available rate budget.

Additional context

This report distinguishes observed behavior from the suspected mechanism: the exact error confirms token-rate-limit rejection, but I have not captured the HTTP status, rate-limit headers, actual compaction request size, or concurrent deployment traffic. The context display alone does not prove the exact provider-side token estimate or that the compaction request resubmits the entire history.

Related reports:

• BYOK Azure OpenAI: 429 throttling exhausts all retries in ~0.15s with no effective backoff; retry count/wait not configurable #3679: BYOK Azure OpenAI throttling retries/backoff. This report additionally concerns missing quota-aware warning and inability to recover using /compact; the observed retry wait here is 5.00 seconds.
• BYOK custom providers: CLI forces greedy sampling (temperature=0) causing reasoning-model degeneration and silent hangs on context overflow #4950: BYOK context-overflow/compaction hangs with another provider. This report concerns a Foundry token rate limit while model context usage is only 28%, not the sampling regression described there.

No API keys, endpoint URLs, client data, conversation contents, or private logs are included.