Skip to content

fix(actions): recover the action array from one-shot agent responses - #920

Merged
matt2e merged 2 commits into
mainfrom
auto-detect-actions-not-working
Aug 17, 2026
Merged

fix(actions): recover the action array from one-shot agent responses#920
matt2e merged 2 commits into
mainfrom
auto-detect-actions-not-working

Conversation

@matt2e

@matt2e matt2e commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Detect Actions has returned zero actions for every repo added since 2026-07-07. The agent explores the project and answers correctly, but the response never survives parsing. Two things had to line up:

  • BasicMessageWriter — the writer behind run_acp_prompt — appended [Tool: …] / [Result: …] markers to the same buffer it collects agent text in, so get_text() returned a transcript rather than an answer. Every caller of that buffer (action detection, run-phase autodetect, badge short names) parses it as JSON.
  • extract_json_array sliced from the first [ to the last ]. Once the transcript led with [Tool: Terminal], that slice covered the whole marker-riddled transcript and never parsed.

Changes

  • acp-client: BasicMessageWriter no longer records tool calls or tool results, so run_acp_prompt returns the agent's text only. Documented on the type, on run_acp_prompt, and in the crate README.
  • builderbot-actions: replaced the first-bracket-to-last-bracket slice with a backward scan. Candidate [ positions are tried from the end of the response, each prefix-parsed into Vec<SuggestedAction>. That takes the agent's final array, ignores trailing text such as a closing code fence, and skips arrays of some other shape.
  • When no candidate deserializes, the scan remembers the typed serde error from the closest candidate that is a JSON array of objects and reports that instead of the generic "could not find valid JSON array" message — so a near-miss (e.g. one malformed actionType among 32 valid actions) stays diagnosable. Arrays of non-objects (file lists, exit codes) don't count as near misses, so their unhelpful type errors can't mask a real one earlier in the response.

Verification

Verified against the 8 KB response logged for block/berd, which the old extractor rejected and the new one parses into all 32 actions. Added unit tests covering prose-wrapped arrays, bare arrays, fenced arrays after tool markers, last-array-wins, non-action arrays, near-miss shape errors, and the writer dropping tool activity.

Note

Detection results are not backfilled: contexts already marked detected need the manual Detect Actions button pressed once.

🤖 Generated with Claude Code

matt2e and others added 2 commits August 17, 2026 14:24
Detect Actions has returned zero actions for every repo added since
2026-07-07. The agent explores the project and answers correctly, but
the response never survives parsing.

Two things had to line up:

- `BasicMessageWriter` — the writer behind `run_acp_prompt` — appended
  `[Tool: …]` / `[Result: …]` markers to the same buffer it collects
  agent text in, so `get_text()` returned a transcript rather than an
  answer. Every caller of that buffer (action detection, run-phase
  autodetect, badge short names) parses it as JSON.
- `extract_json_array` sliced from the first `[` to the last `]`. Once
  the transcript led with `[Tool: Terminal]`, that slice covered the
  whole marker-riddled transcript and never parsed.

Stop recording tool activity in `BasicMessageWriter`, and replace the
first-bracket-to-last-bracket slice with a backward scan: candidate `[`
positions are tried from the end of the response, each prefix-parsed
into `Vec<SuggestedAction>`. That takes the agent's final array, ignores
trailing text such as a closing code fence, and skips arrays of some
other shape.

Verified against the 8 KB response logged for `block/berd`, which the
old extractor rejected and the new one parses into all 32 actions.

Detection results are not backfilled: contexts already marked detected
need the manual Detect Actions button pressed once.

Signed-off-by: Matt Toohey <contact@matttoohey.com>
…rses

Review dae123fa flagged a diagnosability regression in the backward-scan
extractor: a near-miss array — valid JSON, but with e.g. one malformed
actionType among 32 otherwise-valid actions — was silently skipped, and
the caller only saw the generic "could not find valid JSON array" error.
The old first-to-last-bracket slice at least surfaced the serde error.

Now, when no candidate deserializes into Vec<SuggestedAction>, the scan
remembers the typed serde error from the closest candidate that parses
as a JSON array of objects, and reports that instead of the generic
message. Arrays of non-objects (file lists, exit codes) still don't
count, so their unhelpful type errors can't mask a real near-miss
earlier in the response.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1da25ef403

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +318 to +322
let candidate = &response[start..];
let mut stream =
serde_json::Deserializer::from_str(candidate).into_iter::<Vec<SuggestedAction>>();
match stream.next() {
Some(Ok(actions)) => return Ok(actions),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Ignore bracket candidates inside JSON strings

When an otherwise valid action contains a literal JSON-looking array in a string field, such as "command": "echo []", this raw reverse scan starts at that inner [. Because the streaming deserializer accepts [] without checking the trailing quote and remaining object text, it returns an empty Vec<SuggestedAction> immediately and silently discards the enclosing valid action array. The previous whole-response parse handled this correctly; candidate discovery needs to be string-aware or otherwise verify that a candidate is an actual top-level array before returning it.

Useful? React with 👍 / 👎.

@matt2e
matt2e merged commit eb4214a into main Aug 17, 2026
4 checks passed
@matt2e
matt2e deleted the auto-detect-actions-not-working branch August 17, 2026 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant