[feat]: auto-pause agent during Browserbase captcha solving

[derekmeegan]

why

Browserbase can solve captchas asynchronously, but agents were still trying to interact with the page while the solver was active. That led to CUA and DOM/hybrid flows clicking solved captcha widgets again, pausing on confirmation questions, or resuming with stale assumptions instead of continuing the original task cleanly.

This PR pauses agent execution while Browserbase's captcha solver is active and hardens the post-solve resume path so the agent keeps working on the original task after Browserbase finishes.

what changed

• added a shared CaptchaSolver utility that listens for Browserbase browserbase-solving-started/finished/errored console events, supports concurrent waiters, and disposes listeners cleanly
• paused DOM/hybrid prepareStep execution and CUA prepareStep / action execution while Browserbase is solving a captcha
• enabled Browserbase captcha solving by default unless browserSettings.solveCaptchas: false
• updated agent prompts and follow-up messages to tell the model that captchas are handled automatically and should not be clicked again after they are solved
• added OpenAI CUA recovery behavior so it can:
  • carry one-shot context notes into the next model turn
  • auto-continue when the model asks for confirmation instead of acting
  • guard post-solve clicks that target the solved captcha widget and restate the original instruction so the model re-anchors on the task
• added focused unit coverage for solver state, Browserbase session accessors, CUA/regular agent hooks, and OpenAI CUA confirmation handling

test plan

• pnpm --filter @browserbasehq/stagehand lint
• pnpm --filter @browserbasehq/stagehand build:esm
• cd packages/core && pnpm exec vitest run --config vitest.esm.config.mjs dist/esm/tests/unit/openai-cua-client.test.js dist/esm/tests/unit/captcha-solver.test.js dist/esm/tests/unit/agent-captcha-hooks.test.js dist/esm/tests/unit/browserbase-session-accessors.test.js

Browserbase smoke:

• OpenAI CUA + reCAPTCHA demo: strict pass on Verification Success
• Anthropic CUA + reCAPTCHA demo: strict pass on Verification Success
• Hybrid Gemini + reCAPTCHA demo: strict pass on Verification Success
• OpenAI CUA + solveCaptchas: false: solver stays disabled, no wait/resume path is triggered, and the agent stops at the captcha instead of bypassing it

[changeset-bot]

🦋 Changeset detected

Latest commit: 54d9bf4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/browse-cli	Patch
@browserbasehq/stagehand-evals	Patch
@browserbasehq/stagehand-server-v3	Patch
@browserbasehq/stagehand-server-v4	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[greptile-apps]

Greptile Summary

This PR adds Browserbase captcha solver awareness to all Stagehand agent execution paths (DOM/hybrid, Anthropic CUA, Google CUA, Microsoft CUA, OpenAI CUA). A new shared CaptchaSolver utility listens for browserbase-solving-started/finished/errored console events and exposes a waitIfSolving() barrier that all agent step and action handlers call before proceeding.

Key changes:

• CaptchaSolver utility: event-driven state machine with shared waitPromise for concurrent waiters, page-provider callback for cross-navigation re-attachment, and 90 s timeout fallback
• DOM/hybrid (V3AgentHandler): prepareStep blocks on waitIfSolving() and injects a solved/errored notification into the message stream
• CUA (V3CuaAgentHandler): both the prepareStepHandler and actionHandler block; a 3-attempt click guard (captchaClickGuardRemaining) additionally intercepts post-solve clicks on captcha widgets by evaluating their bounding boxes against the click coordinates
• OpenAI CUA specific: addContextNote() carries solve notifications into the next model turn; a new isFollowUpQuestion heuristic auto-continues the loop when the model asks for confirmation instead of acting
• isCaptchaSolverEnabled getter defaults to true on Browserbase unless browserSettings.solveCaptchas is explicitly false; system prompts updated to inform the model captchas are handled automatically

Issues found:

• shouldContinueWithoutConfirmation in OpenAICUAClient is evaluated on every loop iteration, not just steps following a captcha solve — legitimate model confirmation requests (e.g., for risky actions) will be auto-continued without caller knowledge
• isFollowUpQuestion includes "go ahead" as a bare substring, which is broader than the other question-word patterns and could produce false positives on model responses that contain the string incidentally

Confidence Score: 3/5

• Safe to merge with awareness of the unconstrained auto-continue behaviour in OpenAI CUA, which could silently bypass model-generated confirmation requests on non-captcha steps.
• The core CaptchaSolver implementation is solid, tests are focused and comprehensive, and the DOM/hybrid path is clean. The main concern is in OpenAICUAClient: the shouldContinueWithoutConfirmation guard is not scoped to post-captcha recovery steps, and the "go ahead" substring in isFollowUpQuestion is overly broad — both of which could affect runtime behaviour in tasks unrelated to captchas.
• packages/core/lib/v3/agent/OpenAICUAClient.ts — the auto-continue and follow-up-question detection logic need tighter scoping.

Important Files Changed

Filename	Overview
packages/core/lib/v3/agent/utils/captchaSolver.ts	New utility class that tracks Browserbase captcha solver state via console events. Concurrent waiter sharing pattern is sound; page-change re-attachment logic is well-handled. Minor readability concern around waitPromise being cleared before resolve() in settle().
packages/core/lib/v3/agent/OpenAICUAClient.ts	Two logic issues: (1) shouldContinueWithoutConfirmation applies to ALL steps, not just post-captcha recovery — this can silently bypass legitimate model confirmations; (2) isFollowUpQuestion includes "go ahead" as a bare substring which is broader than the other question-word patterns and can produce false positives.
packages/core/lib/v3/handlers/v3CuaAgentHandler.ts	Captcha solver lifecycle (init/dispose) is handled correctly via try/finally. captchaClickGuardRemaining and shouldSkipSolvedCaptchaInteraction provide a solid post-solve click guard. handleCaptchaSolveResult is called from both action and step hooks without double-notification due to destructive consumeSolveResult().
packages/core/lib/v3/handlers/v3AgentHandler.ts	Captcha solver integrated cleanly into both the blocking execute() and streaming executeStream() paths; dispose is covered in finally blocks and all callbacks (onFinish/onError/onAbort). Injected messages for solved/errored state are well-worded.
packages/core/lib/v3/v3.ts	New isCaptchaSolverEnabled getter correctly defaults to true on Browserbase unless solveCaptchas: false is explicitly set. CUA system prompt augmentation is wired correctly in the handler construction path.
packages/core/lib/v3/agent/AgentClient.ts	Clean base-class additions: prepareStepHandler hook and no-op addContextNote(). The void note idiom correctly suppresses the unused-parameter lint warning.
packages/core/lib/v3/agent/prompts/agentSystemPrompt.ts	Renamed isBrowserbase → solveCaptchas is more semantically accurate; captcha prompt text updated to clearly instruct the model not to re-click solved widgets.
packages/core/tests/unit/captcha-solver.test.ts	New unit tests cover solver state transitions, timeout behaviour, concurrent waiter sharing, and dispose clean-up — good coverage of the core logic paths.
packages/core/tests/unit/agent-captcha-hooks.test.ts	Integration-style unit tests verify blocking in both DOM/hybrid prepareStep and CUA action/step hooks; click guard tests are thorough.
packages/core/tests/unit/openai-cua-client.test.ts	Tests for confirmation auto-continue and context note draining; however, there are no tests that verify the guard is NOT triggered for non-captcha confirmation requests, which would catch the scoping issue.

Sequence Diagram

sequenceDiagram
    participant Agent as Agent Loop
    participant PSH as prepareStepHandler
    participant CS as CaptchaSolver
    participant BB as Browserbase (console)
    participant AH as actionHandler
    participant Guard as CaptchaClickGuard

    Agent->>PSH: call prepareStepHandler()
    PSH->>CS: waitIfSolving()
    BB-->>CS: browserbase-solving-started
    Note over CS: solving = true
    CS-->>PSH: (blocks — waitPromise pending)
    BB-->>CS: browserbase-solving-finished
    Note over CS: solving = false, settled
    CS-->>PSH: (resolves)
    PSH->>CS: consumeSolveResult()
    CS-->>PSH: {solved: true}
    PSH->>Agent: addContextNote("captcha solved…")
    Agent->>Agent: executeStep (model call)
    Agent->>AH: action {type:"click", x, y}
    AH->>CS: waitIfSolving() — returns immediately
    AH->>Guard: shouldSkipSolvedCaptchaInteraction?
    Guard->>Guard: page.evaluate(captcha bounding boxes)
    alt click inside captcha widget
        Guard-->>AH: true — skip click, inject note
    else click outside captcha
        Guard-->>AH: false — execute action
    end
    Note over Agent: next loop iteration includes captcha note in inputItems

Loading

_{Last reviewed commit: 8b70cd3}

[greptile-apps]

_{5 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[cubic-dev-ai]

2 issues found across 5 files

Confidence score: 3/5

• Concurrent waitIfSolving() calls in packages/core/lib/v3/agent/utils/captchaSolver.ts can orphan earlier waiters due to a single resolveWait slot, causing hangs until timeout for some callers.
• packages/core/lib/v3/handlers/v3AgentHandler.ts may leak captchaSolver if streamText() throws before callbacks run, since disposal only happens inside stream handlers in stream().
• These are concrete runtime risks (waiting hangs and resource leaks), so there is some user-impacting risk despite the rest of the change looking contained.
• Pay close attention to packages/core/lib/v3/agent/utils/captchaSolver.ts and packages/core/lib/v3/handlers/v3AgentHandler.ts - concurrency waiting and disposal paths.

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/utils/captchaSolver.ts">

<violation number="1" location="packages/core/lib/v3/agent/utils/captchaSolver.ts:72">
P1: Bug: `resolveWait` is a single slot, so concurrent `waitIfSolving()` calls orphan earlier waiters. The second call overwrites the resolver, leaving the first caller's promise unresolved until its 90s timeout. Consider sharing a single deferred promise across all waiters, e.g.:

```ts
private waitPromise: Promise<void> | null = null;

waitIfSolving(): Promise<void> {
  if (!this.solving) return Promise.resolve();
  if (!this.waitPromise) {
    this.waitPromise = new Promise<void>((resolve) => {
      const timer = setTimeout(() => { /* ... */ }, SOLVE_TIMEOUT_MS);
      this.resolveWait = () => { clearTimeout(timer); resolve(); this.waitPromise = null; };
    });
  }
  return this.waitPromise;
}
```</violation>
</file>

<file name="packages/core/lib/v3/handlers/v3AgentHandler.ts">

<violation number="1" location="packages/core/lib/v3/handlers/v3AgentHandler.ts:293">
P2: Resource leak: if `streamText()` throws synchronously, `captchaSolver` is never disposed. The `execute()` method properly wraps this in a `try/finally`, but `stream()` only disposes inside stream callbacks (`onError`/`onFinish`/`onAbort`), which won't fire if `streamText` itself throws. Wrap the `streamText` call and subsequent setup in a try/catch that calls `captchaSolver?.dispose()` on failure.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant User as Client/V3
    participant Handler as Agent Handler (DOM/CUA)
    participant Solver as NEW: CaptchaSolver
    participant Page as Browser Page (Browserbase)
    participant LLM as LLM Provider

    User->>Handler: execute(instruction)
    
    opt NEW: isBrowserbase && captchaSolverEnabled
        Handler->>Solver: attach(page)
        Solver->>Page: on("console", listener)
    end

    Note over Handler, LLM: CHANGED: System prompt tells LLM to ignore captchas

    loop Agent Step Loop
        Page-->>Solver: NEW: console("browserbase-solving-started")
        Note right of Solver: solving = true

        Handler->>Handler: prepareStep() / actionHandler()
        
        Handler->>Solver: NEW: waitIfSolving()
        
        alt Solver is active
            Note over Solver: NEW: Block execution (max 90s timeout)
            
            alt Solve Success
                Page-->>Solver: NEW: console("browserbase-solving-finished")
                Solver-->>Handler: Resolve (Resume)
            else Solve Error
                Page-->>Solver: NEW: console("browserbase-solving-errored")
                Solver-->>Handler: Resolve (Resume with error flag)
                Handler->>Handler: Log solver error & resetError()
            else Timeout (90s)
                Solver->>Solver: Internal timeout
                Solver-->>Handler: Resolve (Resume)
            end
        else Solver is idle
            Solver-->>Handler: Resolve immediately
        end

        Handler->>LLM: Request next action
        LLM-->>Handler: Action response
        Handler->>Page: Execute action
    end

    Handler->>Solver: NEW: dispose()
    Solver->>Page: off("console", listener)
    Handler-->>User: Result

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[derekmeegan]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dc13b5a25a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[cubic-dev-ai]

1 issue found across 4 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/utils/captchaSolver.ts">

<violation number="1" location="packages/core/lib/v3/agent/utils/captchaSolver.ts:54">
P1: Stale `solving` state after page change: when `ensureAttached()` detects a page switch and detaches the old listener, `this.solving` is not reset. If a solve was in progress on the old page, the agent will block for up to 90 seconds waiting for a finish/error event that can never arrive (since the old page's listener was removed). Reset `solving` and settle pending waiters when detaching from a changed page.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/utils/captchaSolver.ts">

<violation number="1" location="packages/core/lib/v3/agent/utils/captchaSolver.ts:148">
P2: When `detachListener()` resolves waiting callers via `settle()`, it should also set `_erroredSinceLastConsume = true` to signal the solve was interrupted. Without this, consumers of `consumeSolveResult()` get `{solved: false, errored: false}` and silently skip the error-handling/notification path. The timeout codepath correctly sets `_erroredSinceLastConsume = true` before `settle()` — this should be consistent.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/prompts/agentSystemPrompt.ts">

<violation number="1" location="packages/core/lib/v3/agent/prompts/agentSystemPrompt.ts:200">
P1: Existing tests in `agent-hybrid-mode.spec.ts` pass `isBrowserbase: true` (without `solveCaptchas`) and expect captcha content in the prompt. Since the condition now depends on `solveCaptchas`, these tests will break. The tests need to be updated to pass `solveCaptchas: true` and the expected substring should match the new prompt text (e.g., `"automatically detected and solved"` instead of `"automatically be solved"`).</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[shrey150]

@greptileai

[greptile-apps]

waitPromise cleared before resolve() — add a comment explaining the ordering

this.waitPromise = null is set before resolve() is called. This is intentional (allows new callers to create a fresh waitPromise once resolution is signalled) but can confuse readers into thinking it might orphan existing awaiters. A brief comment would clarify the intent:

Suggested change

	private settle(): void {
	if (this.waitTimer) {
	clearTimeout(this.waitTimer);
	this.waitTimer = null;
	}
	if (this.resolveWait) {
	const resolve = this.resolveWait;
	this.resolveWait = null;
	this.waitPromise = null;
	resolve();
	}
	}
	/** Resolve the shared wait promise and clear the timeout. */
	private settle(): void {
	if (this.waitTimer) {
	clearTimeout(this.waitTimer);
	this.waitTimer = null;
	}
	if (this.resolveWait) {
	const resolve = this.resolveWait;
	this.resolveWait = null;
	// Clear waitPromise first so new callers can create a fresh promise
	// once this one is resolved; existing awaiters still hold a reference
	// to the old promise and will be unblocked by resolve().
	this.waitPromise = null;
	resolve();
	}
	}

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

[pirate]

Good feature, agents definitely waste a lot of tokens and race with auto-handling logic implemented by extensions / browser providers currently.

Just a few comments:

• naming of the fields and helpers like solveCaptchas and setPrepareStepHandler, what about captchasWillAutoSolve and attachPreStagehandStepHook or something?

• duplication of logic / system prompts / LLM steering code. would it be possible to centralize the system prompt text / hook logic for captchas in one place and then import it in agentSystemPrompt.ts, OpenAICUAClient.ts, v3AgentHandler.ts, v3CuaAgentHandler.ts, and v3.ts

• can you add a live integration test that uses a real url with a captcha like https://2captcha.com/demo/recaptcha-v2 (or ask the stealth team for some good test urls) and assert it gets auto-solved by bb prod browser and that the agent doesn't race/conflict with it