Skip to content

feat(accessibility): add a fix-accessibility-issues prompt - #567

Open
Aaryan430 wants to merge 3 commits into
mainfrom
feat/a11y-fix-workflow-prompt
Open

Aaryan430 wants to merge 3 commits into
mainfrom
feat/a11y-fix-workflow-prompt

Conversation

@Aaryan430

@Aaryan430 Aaryan430 commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Merge #566 first. The prompt tells the agent to re-run a scan by scanId, to use localIdentifier, and to compare runs with compareWith: "previous". Those come from #541 and #553, which #566 lands on main. This PR builds and passes tests on its own, and the two merge cleanly in either order.

What changed

  • A new MCP prompt, fix-accessibility-issues ("Fix accessibility issues"), with optional target and scope arguments. The user starts it on purpose:
    • Claude Code: /mcp__browserstack__fix-accessibility-issues <target> <scope>
    • VS Code Copilot: /mcp.browserstack.fix-accessibility-issues
    • Claude Desktop: the "+" menu
  • It returns one user message with the workflow in four stages, detect → analyze → fix → verify, then a report. Each stage ends in a gate the agent must meet before it moves on. Along the way it tells the agent to:
    • sort issues into needs review, confirmed WCAG failure and best practice, and mark possible false positives without fixing them;
    • name the kind of change each fix is (code only, needs content, changes look or behaviour, shared component, third-party), and ask for alt text and labels instead of inventing them;
    • show a plan and wait, even when told to fix everything;
    • for questions about the report that the issue data can't answer, such as why a rule fired or what to fix first to raise the score, call askBrowserStackAI with product: "a11y": a few focused calls, not one per issue;
    • leave third-party code unchanged and report the package;
    • never hide an issue with tricks such as aria-hidden or tabindex="-1";
    • give an issue still open after the re-scan one more fix, then revert it and mark it not fixed;
    • never call a fix verified without a BrowserStack compare;
    • stop when the approved plan is done, report (including the questions asked of BrowserStack AI), and ask before the next set.
  • Where it lives:
    • The text is in src/tools/accessiblity-utils/fix-workflow.ts.
    • addAccessibilityFixWorkflow registers the prompt, called from server-factory.ts. It isn't registered inside addAccessibilityTools, because 12 test files call that with mock servers that only have tool().
  • Nothing else changes: no resources, no change to the server instructions, and tool results are unchanged.

Why a prompt and not a resource

We tried both: first with a stub server (21 sessions), then with this server against real scans (8 sessions).

  • Prompt: it reaches the agent only when the user starts it, but then it arrives every time, filled in with the user's target and scope.
  • Resource: the agent never read it on its own. With a pointer in the server instructions it was read in 6 of 6 sessions. But only clients whose model reads resources by itself can use that (Claude Code, Gemini CLI, Codex, Cline); in Copilot, Kiro and Claude Desktop the user has to attach it.
  • Server instructions can't hold the text: Claude Code cuts them at 4,096 characters.

Testing

  • Unit: the new tests/tools/accessibilityFixWorkflow.test.ts checks that the prompt is listed, its arguments are filled in, the message role is user, and the server advertises no resources.
  • Live: Claude Code 2.1.319 (Sonnet), isolated --strict-mcp-config sessions, against a locally built server, a local test app and real scans. These sessions ran an earlier version of the text (six steps, before the wording changes below and before the rework into four stages). The current text has not been run live yet.
    • The prompt arrived in 2 of 2 sessions, with the URL and scope filled in.
    • In 7 sessions with the workflow:
      • the agent showed a plan and waited in 6;
      • the third-party carousel was never edited (7 of 7);
      • it asked before changing the shared icon button in 7 of 7;
      • every session that edited code (6) re-ran the same scanId;
      • 5 of those 6 honestly reported "fixed, not verified".
    • The baseline without the workflow showed no plan, batched its fixes, fixed needs-review issues, never re-scanned, and patched the third-party carousel at runtime.
  • Wording tightened from those sessions:
    • stop after the plan even when told to fix everything;
    • a "go" covers the shared components the plan named;
    • one line before each edit;
    • open an image before writing its alt text (the current text goes further: ask the user for alt text and labels);
    • a re-run that comes back as a new scan means "fixed, not verified";
    • use the user's full words when the client splits arguments on spaces.

Known limits (follow-ups)

  • Verify doesn't work for local pages yet. The MCP opens its tunnel under a random identifier, so re-running a local scan starts a new scan, with nothing to compare. The prompt has the agent report "fixed, not verified" instead. The fix is to reuse one identifier per scan, or hand it back to the agent.
  • The MCP's tunnel start runs pkill -f BrowserStackLocal. That kills a tunnel the user is running, and the MCP's own tunnel outlives the MCP process.
  • Claude Code splits prompt arguments on spaces (critical issues only becomes critical). The agent still sees the full command.
  • From the a11y team: agent fix steps for the top 10 rules.

🤖 Generated with Claude Code

A prompt the user starts on purpose (/mcp__browserstack__fix-accessibility-issues
in Claude Code, /mcp.browserstack.fix-accessibility-issues in Copilot). It hands
their coding agent BrowserStack's fix workflow: agree the scope, get the issues,
triage and show a plan, fix one issue at a time, verify with a re-run and a
compare, and report. Optional target and scope arguments fill in what to check
and what to fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto (fallback from Default branch)
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Central YAML (base), Organization UI (inherited), Workspace UI (inherited); Enforced overrides: Organization
  • Review profile: ASSERTIVE
  • Plan: Enterprise
  • Run ID: 04245c29-0280-4a25-89a1-66c8f84ae778

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Aaryan430 and others added 2 commits November 12, 2026 11:02
…detect, analyze, fix and verify stages

Each stage now ends in a gate the agent must meet before the next. Analyze sorts
issues into needs review, confirmed WCAG failure and best practice, names the kind
of change each fix is, and sends questions the issue data can't answer to
askBrowserStackAI. Verify gives a still-open issue one more fix before it is
reverted, and the report lists the questions asked of BrowserStack AI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tyExpert

#542 deprecates accessibilityExpert, so the fix-accessibility-issues
prompt no longer points general WCAG or ARIA questions at it. The agent
answers those itself and names the success criterion the answer rests
on. A test checks the prompt doesn't name the tool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant