Repository navigation
Valid verify result is rejected as a provider outage when the response is 32 tokens or fewer #803
Description
Short description
Dreamer retries fallback models after receiving a valid verify result because the response is short.
What happened?
A Dreamer verify task returned a complete, valid result, but Magic Context rejected it and sent the same prompt to the fallback models.
The task was checking one memory. The first model returned:
<verify>
<verified id="516" files="packages/llm/src/protocols/openai-responses.ts"/>
</verify>OpenCode recorded this response as successfully completed:
{
"finish": "stop",
"tokens": {
"output": 32,
"reasoning": 0
},
"error": null
}About half a second later, Magic Context submitted the same prompt again using the next fallback model. It then tried another fallback. Those attempts failed, and the whole verify task was recorded as failed with zero memories processed. The valid result from the first model was never applied.
Expected: Accept the valid XML and finish the task without calling fallback models.
Actual: Reject the valid response because it is short, then continue through the fallback chain.
Diagnostics
I checked the installed plugin code and found that the verify task checks for a provider failure **before** validating the XML:
const providerFailure = providerOutputFailureFromInvalidManifest(messages, text);
if (providerFailure)
throw providerFailure;
return validateVerifyManifest(text, new Set(batch.map((memory) => memory.id)));
`providerOutputFailureFromInvalidManifest()` treats a non-empty response as a provider outage when:
- The completion finished with `stop`.
- There is no response error.
- Output is **32 tokens or fewer**.
- Reasoning tokens are **0**.
The response above matches all four conditions. It is therefore rejected before `validateVerifyManifest()` gets a chance to accept it.
Relevant locations in the installed `0.49.23` build:
- `dist/index-y76rhsnr.js:8296`: `MAX_NEAR_ZERO_OUTPUT_TOKENS = 32`
- `dist/index-y76rhsnr.js:8379`: `providerOutputFailureFromInvalidManifest()`
- `dist/index-11gzxvf4.js:9340`: provider-failure check runs before manifest validation
A valid result for a single memory can naturally be this short.
Could the verify task validate the manifest first and only run the provider-outage heuristic if validation fails? This also matches the existing comment in `provider-output-failure.d.ts`, which says that a valid manifest should remain authoritative regardless of its token count.
A regression test with valid verify XML, `output=32`, `reasoning=0`, and `finish="stop"` would cover this case. The result should be accepted without calling a fallback model.Plugin version
0.49.23
OpenCode version
1.18.18
Platform
macOS
Client
OpenCode TUI (CLI)
Activity
magic-alfonso commentedon Oct 27, 2026
Thanks, your diagnosis is right, and the trace made this quick to confirm.
On master, both paths in the dreamer's verify task (
dreamer/verify.ts) run the short-response outage check before validating the result. A complete answer for a single memory can easily be 32 tokens or fewer with no reasoning, so it's treated as an outage and the same prompt goes to the fallback models. Classify and cue compression already validate first and only run the check on output that fails validation, which is what the helper's own comment intends.The order in verify was deliberate, though: an outage can return a tiny but well-formed fragment, and the check stops that from being banked as a verdict. So the fix needs to keep that guard while accepting a complete result like yours, which names exactly the memories it was asked about. We'll follow up here when it's scheduled.
magic-alfonso commentedon Oct 28, 2026
Fixed in 0.49.27. The dreamer's verify task now checks a reply's content before treating a short reply as a provider outage. A complete result that covers exactly the memories it was asked about is accepted, however short. A short reply with missing, extra or duplicate memories still falls back as before. Thanks @Harry-zklcdc for the precise diagnosis.