feat(seer): Experimenting around by jshchnz · Pull Request #119794 · getsentry/sentry

jshchnz · 2026-07-16T00:37:25Z

What

A new demo route /issues/autofix/overview/ — a card-based "mission control" for issues Autofix has worked on. Builds on the runs/issues demo plumbing from #118828 (this PR is based on that branch so the diff shows only the overview work).

Each card answers, at a glance: what broke, what Seer did, why, and what a human should do next.

Card anatomy

▍<LLM headline, replaces raw JSON-blob issue titles>   [1 file +27 −3] [Open PR]
<one-sentence summary: failure mechanism; concrete run outcome>
┌ PROPOSED FIX ────────────────────────────────────────────────┐
│ what the drafted change is and why it fixes the root cause   │  ← only when code was drafted
└──────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────
▸ Full analysis         148K events · updated a day ago · ⬡ sentry

Action button encodes run state (Review PR / Open PR / Generate code / View run / Retry / Running) and deep-links to the PR or the issue's Seer drawer (?seerDrawer=true). Stage chips were deliberately removed — the verb carries the state.
Exact diff stats (1 file +27 −3) come from the run's merged_file_patches, not LLM estimates.
Clickable stat cards (Awaiting input / Code changes ready / Awaiting review / Merged PRs) double as quick filters; Outcome/Trigger/Attention/Activity dropdowns filter client-side over the loaded page.
Full analysis (collapsed): issue id + raw title, fixability tag, root cause, needs-you action (with parsed DECIDE/VERIFY/REVIEW/PROVIDE tag), reviewer notes.

How the LLM content works

Five one-shot questions are asked per run via the runs endpoint's repeatable question param (seer-run-questions feature; answered by Seer's agent_question one-shot over the run's history; cached in Redis per (run, prompt-hash) for 24h). All prompts live in static/app/views/seerWorkflows/overview/runQuestions.tsx — editing a prompt string invalidates only that prompt's cache, so iteration needs no backend change. Two prompts return parseable pseudo-structure (headline|root cause, CATEGORY|sentence) with plain-text fallbacks when parsing fails.

Merged-PR state (forward-compatible)

The UI consumes an optional pullRequests: [{key, state, mergedAt}] field on the runs endpoint: merged runs get a green Merged tag instead of a review action, and the Merged PRs stat card + quick filter go live. The deployed API doesn't return this field yet, so the card renders a disabled "—" with an explanatory tooltip until the companion serializer PR ships (linked below). The overview also queries without is:unresolved so issues resolved by a merged fix still appear.

Notes for review

useAutofixIssues was extended backward-compatibly (optional runsQuery / questions params + wider issue/run types); both existing demo pages are byte-for-byte unaffected in behavior.
The overview components originated in the jc-SeerWorkflowE2E mock prototype and were rewired onto real data.
Page size stays at 10 (the runs endpoint caps outputs-enabled pages); stat counts reflect the loaded page and say so in a caption.
Cold loads are slow (~10 runs × 5 one-shots); warm loads are instant. A fast-cards/async-answers split was considered and deferred.

Testing

static/app/views/seerWorkflows/overview/index.spec.tsx — 9 tests covering card rendering from mocked issues/runs/autofix-state, headline fallback to raw titles, face summary + proposed-fix block, disclosure contents, quick-filter URL round-trips, merged-state rendering, and the disabled-Merged-card fallback. Plus typecheck/lint/oxfmt clean.

🤖 Generated with Claude Code

Add two demo views under issues/autofix/: - /issues/autofix/runs/ lists the organization's recent Seer runs from the seer/runs/ endpoint (with ?expand=questions) and renders each run's one-shot question answers via SeerMarkdown, one collapsible section per run. Sends a hardcoded DEMO_QUESTIONS list via the repeatable `question` param instead of the server-side set, rendering each echoed question text as the answer heading so prompts can be iterated on without a backend change. Requests a small page since expansion can trigger one-shot calls per run on a cold cache. - /issues/autofix/issues/ lists issues Seer has run autofix on (has:issue.seer_last_run) and enriches each with its most recent explorer run and one-shot outputs. Data-fetching lives behind a single paginated useAutofixIssues hook: it queries the issue stream, then makes one group-scoped runs request so we only fetch runs for the issues on the page. Each issue also shows its autofix phase as a colored Status tag (Root cause -> Planning -> Coding -> PR open), derived by reusing getOrderedAutofixSections. Reuses the standard IssueListSearchBar with the required has:issue.seer_last_run filter always applied by the hook. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Send a fixed list of demo questions as repeatable `question` params to the seer runs endpoint instead of the generic `outputs` flag, and render each answer under its echoed-back question text rather than the raw output key. Refs the seer runs demo UI. Co-Authored-By: Claude <noreply@anthropic.com> Agent transcript: https://claudescope.sentry.dev/share/3OId6GAKdIxHX-BkJJm4ZZFHtF_4YouJOyG0ewb-xEQ

A new /issues/autofix/overview/ route rendering one card per issue with a recent Seer autofix run, built for triage: what broke, what Seer did, why, and what the human should do next. Each card is driven by real data plus five one-shot questions asked per run (runQuestions.tsx — edit a prompt string to iterate; answers are cached per (run, prompt-hash) for 24h): - headline|root cause: a plain-language title replacing raw issue titles (raw title stays in the hover tooltip and expanded state) - summary: one <=25-word sentence — failure mechanism, then the run's concrete outcome - a structured "Proposed fix" block, rendered only when the run actually drafted code - needs-you (parsed DECIDE|/VERIFY|/REVIEW|/PROVIDE prefix -> action tag) and reviewer notes in the collapsed Full analysis Structured data fills the rest: derived pipeline outcomes, exact file/ line diff stats from merged_file_patches, attention-driven action buttons (Review PR / Open PR / Generate code / View run / Retry) deep- linking to the PR or the issue's Seer drawer, abbreviated impact counts, and quick-filter stat cards over the loaded page. Merged-PR state: the UI consumes an optional pullRequests field on the runs endpoint (merged tag, live Merged PRs stat card + quick filter) and degrades to a disabled card while the API doesn't return it — that serializer change ships separately. The overview components were ported from the jc-SeerWorkflowE2E mock prototype and rewired onto the runs/issues/autofix-state endpoints via useAutofixIssues (extended backward-compatibly with runsQuery/questions params; both existing demo pages are unchanged). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

github-actions · 2026-07-16T00:40:47Z

📊 Type Coverage Diff

Metric	Before	After	Delta
Coverage	93.91%	93.92%	🟢 +0.01%
Typed	134,088	134,290	🟢 +202
Untyped	8,691	8,697	🔴 +6

🔍 6 new type safety issues introduced

Type assertions (as) (6 new)

File	Line	Detail
`static/app/views/seerWorkflows/overview/index.tsx`	47	`as AutofixTrigger[]` — `Object.keys(TRIGGER_META) as AutofixTrigger[]`
`static/app/views/seerWorkflows/overview/index.tsx`	90	`as AutofixOutcome[]` — `decodeList(location.query.outcome) as AutofixOutcome[]`
`static/app/views/seerWorkflows/overview/index.tsx`	91	`as AutofixTrigger[]` — `decodeList(location.query.trigger) as AutofixTrigger[]`
`static/app/views/seerWorkflows/overview/index.tsx`	92	`as AttentionReason[]` — `decodeList(location.query.attention) as AttentionReason[]`
`static/app/views/seerWorkflows/overview/index.tsx`	93	`as QuickFilterValue
`static/app/views/seerWorkflows/overview/index.tsx`	95	`as SortValue

This is informational only and does not block the PR.

Recency ordering buried the actionable under the busy: a week-old PR awaiting review sat below a still-running run that needs nothing. Sort cards by what needs a human first — awaiting input, review PR, open PR, generate code, retry — then loading-state rows parked in the middle, then running, diagnosed-only, and merged wins sinking to the bottom as an archive shelf. Within a tier: event count desc (same action, highest impact first), then run recency as a stable tiebreak. Impact stays inside tiers on purpose: the queue's promise is "work top to bottom", and a huge diagnosed-only issue shouldn't outrank a small PR that's actionable right now. The issues request now passes sort=date explicitly (last-seen desc picks the actively-occurring candidate pool; the client orders the loaded page), and the caption declares the ordering. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The triage sort was invisible — its only trace was a caption line, so the page read as unsorted. Add a Sort select to the filter bar (defaulting to "Needs you first", the triage-queue order) with two plain alternatives: Recent activity and Most events. URL-driven via ?sort=, omitted at the default; not part of Clear all since sort isn't a filter. The caption drops back to counts-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

When a run drafted code, the summary sentence and the Proposed-fix block described the same change twice. The body is now exactly one block with a shared anatomy (contained panel, icon + uppercase label, markdown): "Proposed fix" (commit icon, success label) when code was drafted, otherwise "Diagnosis" (search icon, muted label). The symptom stays in the headline and the mechanism detail in Full analysis. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Hovering "1 file +27 −3" now shows which files the drafted diff touches: per-file path and churn from merged_file_patches (data we already walked and then discarded after summing), biggest files first, capped at 5 with a "+N more files" line. Paths are repo-prefixed only when the diff spans repositories. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The stat grid keeps its spacing (marginBottom xs -> md). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Hovering a headline showed a bare, cut-off JSON blob with no hint of what it was. The tooltip now carries an "Original issue title" header, truncates the raw title at 200 chars (the full text remains in the expanded analysis), and gets a 480px max width. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The needs_you one-shot duplicated what the action button and summary already carry, and live answers frequently ignored its CATEGORY| format. Remove the question and all supporting code (prefix parsing, actionType, the Decide/Verify/Review/Provide tag meta) — the question set drops to four. The attention/action-button system is untouched. The notes question replaces "What to double-check" with an adaptive section: for runs that drafted code, a Review checklist (3–5 reasoned bullets on risks, assumptions, and untested paths); for diagnosis-only runs, Next steps (2–4 concrete bullets on how to take the issue forward). The label is derived client-side from whether a fix exists. Full-analysis section headings now use the same uppercase label voice as the body blocks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The single batched group:[...] runs request collided with the endpoint's 10-run cap on outputs-enabled pages: when the page's groups collectively had more explorer runs than that, the oldest groups' runs fell off the page and their cards rendered bare (raw title, no headline/body/analysis). Widening the overview to type:explorer made this easy to hit. Fetch each group's latest run individually (per_page=1, same question params) — guaranteed coverage with identical total one-shot work, mirroring the existing per-group autofix-state fan-out. A failed per-group request now degrades that row to run-less instead of erroring the page. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The expanded area dumped the raw issue title (redundant with the headline tooltip) into a grey junk line, used bare-text section labels while the body blocks carry icon+label headers, and stacked everything in one narrow column leaving half the card empty. Now: a compact id + fixability strip up top; sections wear the same icon+label voice as the body blocks (Root cause -> IconFocus, Review checklist -> IconCircleCheckmark, Next steps -> IconArrow); and sections sit side by side on wide screens via a responsive grid, stacking on narrow ones. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

deriveAutofixPhase, deriveAutofixOutcomes, buildOverviewRow, RunPullRequest, SeerRun, and PatchFile are only consumed within their own modules. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Icons were wrapped in Text spans for color, which put the SVG on a text baseline inside a box sized by the default line-height — so flex centering aligned the box, not the glyph. Icons natively take a variant prop with the same vocabulary, so render them directly as flex children (body blocks, analysis sections, trigger badges, filter bar, stat cards) and type icon slots as typeof IconUser so variant passes through. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The model sometimes emits inline "•" bullets run together in one paragraph, which markdown renders as prose. Normalize such answers into "- " lines before rendering, and tighten the notes prompt to demand one hyphen-bullet per line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 4 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 54e647b. Configure here.}

cursor · 2026-07-16T11:33:43Z

+    query: '',
+    cursor,
+    runsQuery: OVERVIEW_RUNS_QUERY,
+    questions: RUN_QUESTION_PROMPTS,


Wrong explorer run paired

High Severity

The overview loads the latest type:explorer run per issue for one-shot answers, while card actions, diff stats, and outcomes come from the issue autofix state. If a newer non-autofix explorer run exists on the same issue, headlines and summaries can describe the wrong session while PR and patch UI still reflect autofix.

Additional Locations (2)

static/app/views/autofixIssuesDemo/useAutofixIssues.tsx#L214-L262

static/app/views/seerWorkflows/overview/buildOverviewRows.ts#L202-L238

^{Reviewed by Cursor Bugbot for commit 54e647b. Configure here.}

cursor · 2026-07-16T11:33:43Z

+      return 'night_shift';
+    default:
+      return null;
+  }


Trigger filters never match

Medium Severity

The “Triggered by” filter offers Issue summary, Alert, and Post-process, but mapRunSourceToTrigger only maps autofix, slack_thread, chat, and night_shift. Rows with other SeerRun.source values get trigger: null, so those filter selections always hide every card.

Additional Locations (1)

static/app/views/seerWorkflows/overview/index.tsx#L136-L138

^{Reviewed by Cursor Bugbot for commit 54e647b. Configure here.}

cursor · 2026-07-16T11:33:43Z

-    isError: issuesQuery.isError || runsQuery.isError,
+    isPending:
+      issuesQuery.isPending ||
+      (runsEnabled && runResults.some(result => result.isPending)),


Loading ends before autofix

Medium Severity

useAutofixIssues treats the page as ready once issues and per-group run queries finish, but not while per-group autofix state requests are still pending. The overview then renders cards with LLM copy while actions, outcomes, and diff pills can be wrong until autofix state arrives.

Additional Locations (1)

static/app/views/seerWorkflows/overview/index.tsx#L395-L412

^{Reviewed by Cursor Bugbot for commit 54e647b. Configure here.}

cursor · 2026-07-16T11:33:43Z

+    if (quickFilter === 'merged') {
+      if (!row.prMerged) {
+        return false;
+      }


Merged filter hides everything

Low Severity

The quick=merged filter keeps only rows where prMerged is truthy. Until the runs API exposes pullRequests, prMerged stays undefined for every row, so a bookmarked or shared merged quick-filter URL shows no cards even though the Merged stat control is disabled.

^{Reviewed by Cursor Bugbot for commit 54e647b. Configure here.}

sentry · 2026-07-17T00:01:12Z

+  for (const key of ['question', 'text', 'message']) {
+    const value = data[key];
+    if (typeof value === 'string' && value.trim()) {
+      return value;
+    }
+  }


Bug: The extractPendingQuestion function uses an incorrect data structure to find the question text for ask_user_question inputs, causing it to never be displayed.
_{Severity: MEDIUM}

Suggested Fix

Modify extractPendingQuestion to correctly handle the nested data structure for ask_user_question inputs. The function should check for data.questions?.[0]?.question and return that value if it's a non-empty string, in addition to the existing checks for other input types.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: static/app/views/seerWorkflows/overview/buildOverviewRows.ts#L117-L122 Potential issue: The `extractPendingQuestion` function incorrectly assumes the structure of the `pending_user_input.data` object for `ask_user_question` type inputs. It attempts to access flat keys like `question`, `text`, or `message` directly on the `data` object. However, the actual data structure is `{questions: [{question: string, ...}]}`. As a result, the function always returns `undefined` for this input type, and the pending question is never displayed to the user on the overview card, silently failing to surface the blocking question.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

sentry · 2026-07-17T00:01:12Z

+
+/**
+ * Join the run's answered questions back to their question configs.
+ *


Bug: The regex in normalizeBulletList fails to handle bullet points that are not followed by a space, leading to incorrect markdown rendering.
_{Severity: LOW}

Suggested Fix

Update the regular expression in the normalizeBulletList function to make the trailing whitespace optional. Changing \s*•\s+ to \s*•\s* will correctly handle cases where a bullet point is immediately followed by text without a space.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: static/app/views/seerWorkflows/overview/buildOverviewRows.ts#L128 Potential issue: The `normalizeBulletList` function uses the regex `\s*•\s+` to convert bullet points from a string into a markdown list. This regex requires at least one whitespace character after the `•` symbol. If a language model response contains a bullet point without a trailing space (e.g., `"Diagnosis: •Item 1"`), the regex will fail to match, and the bullet will not be converted into a proper markdown list item. This results in the `•` character being rendered as literal text instead of a list item.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

chromy and others added 3 commits July 7, 2026 20:07

github-actions Bot added the Scope: Frontend Automatically applied to PRs that change frontend components label Jul 16, 2026

jshchnz changed the title ~~feat(seer): Card-based Autofix mission-control overview~~ feat(seer): Experimenting around Jul 16, 2026

vercel Bot deployed to Preview July 16, 2026 00:39 View deployment

jshchnz mentioned this pull request Jul 16, 2026

feat(seer): Serialize linked pull requests with state on Seer runs #119795

Draft

vercel Bot deployed to Preview July 16, 2026 00:50 View deployment

vercel Bot deployed to Preview July 16, 2026 00:59 View deployment

vercel Bot deployed to Preview July 16, 2026 01:05 View deployment

vercel Bot deployed to Preview July 16, 2026 02:23 View deployment

ref(seer): Drop the loaded-counts caption from the overview

eaf12ce

The stat grid keeps its spacing (marginBottom xs -> md). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vercel Bot deployed to Preview July 16, 2026 02:29 View deployment

vercel Bot deployed to Preview July 16, 2026 02:32 View deployment

ref(seer): Rename headline tooltip label to "Raw issue title"

2c03806

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vercel Bot deployed to Preview July 16, 2026 02:35 View deployment

vercel Bot deployed to Preview July 16, 2026 02:51 View deployment

vercel Bot deployed to Preview July 16, 2026 02:58 View deployment

vercel Bot deployed to Preview July 16, 2026 03:12 View deployment

ref(seer): Un-export module-internal symbols flagged by knip

b243c58

deriveAutofixPhase, deriveAutofixOutcomes, buildOverviewRow, RunPullRequest, SeerRun, and PatchFile are only consumed within their own modules. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vercel Bot deployed to Preview July 16, 2026 03:36 View deployment

vercel Bot deployed to Preview July 16, 2026 03:53 View deployment

vercel Bot deployed to Preview July 16, 2026 03:58 View deployment

chromy marked this pull request as ready for review July 16, 2026 11:31

cursor Bot reviewed Jul 16, 2026

View reviewed changes

chromy force-pushed the chromy/2026-07-01-seer-runs-demo-ui branch 3 times, most recently from c2b28fe to eb3f45f Compare July 16, 2026 14:08

Base automatically changed from chromy/2026-07-01-seer-runs-demo-ui to master July 16, 2026 14:56

sentry Bot reviewed Jul 17, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(seer): Experimenting around#119794

feat(seer): Experimenting around#119794
jshchnz wants to merge 16 commits into
masterfrom
jc-autofix-mission-control

jshchnz commented Jul 16, 2026

Uh oh!

github-actions Bot commented Jul 16, 2026 •

edited

Loading

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Jul 16, 2026

Uh oh!

cursor Bot Jul 16, 2026

Uh oh!

cursor Bot Jul 16, 2026

Uh oh!

cursor Bot Jul 16, 2026

Uh oh!

sentry Bot Jul 17, 2026

Uh oh!

sentry Bot Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

jshchnz commented Jul 16, 2026

What

Card anatomy

How the LLM content works

Merged-PR state (forward-compatible)

Notes for review

Testing

Uh oh!

github-actions Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Type Coverage Diff

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 16, 2026

Choose a reason for hiding this comment

Wrong explorer run paired

Uh oh!

cursor Bot Jul 16, 2026

Choose a reason for hiding this comment

Trigger filters never match

Uh oh!

cursor Bot Jul 16, 2026

Choose a reason for hiding this comment

Loading ends before autofix

Uh oh!

cursor Bot Jul 16, 2026

Choose a reason for hiding this comment

Merged filter hides everything

Uh oh!

sentry Bot Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

sentry Bot Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jul 16, 2026 •

edited

Loading