Skip to content

fix(self-driving): drop replay from the step 4 native sources - #1174

Open
skoob13 wants to merge 1 commit into
mainfrom
fix/self-driving-retired-replay-source
Open

fix(self-driving): drop replay from the step 4 native sources#1174
skoob13 wants to merge 1 commit into
mainfrom
fix/self-driving-retired-replay-source

Conversation

@skoob13

@skoob13 skoob13 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Problem

The self-driving run prompt and its architecture doc tell the agent that step 4 enables a replay source. It no longer does.

Nobody sees this directly. It matters because the agent reads the prompt, and the next person to change step 4 reads the doc.

Changes

  • The step 4 summary and the step table in ARCHITECTURE.md list error tracking and support as the step-3b-derived sources. Replay is out, with a line saying step 6c's scanners are the replay path.
  • The step 6 scout exclusion in ARCHITECTURE.md no longer justifies itself with step 4's native source for replay. It names step 6c's scanners.
  • The step 3b prompt text now says the products it enables feed the sources in step 4 and the Replay Vision scanners in step 6c. Step 3b still turns Session Replay on, because scanners need recordings.

No user-visible behavior changes. The prompt edit changes one sentence the agent reads; every step and its order stay as they are.

How did you test this code?

vitest run src/lib/programs/__tests__/self-driving-prompt.test.ts src/lib/programs/self-driving/__tests__/step-keys.test.ts passes, and prettier is clean on the touched file. Those cover step order, step labels, and the 6c section the prompt edit sits beside.

No wizard run was executed against a live project.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Written by Claude Opus 5 in Claude Code, directed by @skoob13, who asked whether a planned move to Replay Vision had shipped and then to fix it.

Skills invoked: /writing-pr-descriptions.

This is the smallest of three PRs. PostHog/context-mill#370 changes what a run actually does; a PostHog/posthog PR removes the same retired source from the desktop app's sources grid. This one only keeps the wizard's own copy of step 4 true.

Step 4's skill no longer enables the retired `session_replay` /
`session_analysis_cluster` source, so the prompt and the architecture doc
that describe step 4 have to say the same thing.

Replay reaches the inbox through step 6c's Replay Vision scanners. Step 3b
still turns the Session Replay product on, because scanners need recordings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RGaVkph7kqgkSgbTxaonbC
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/groq
  • /wizard-ci ai-observability/manual-capture
Show more apps
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@skoob13 skoob13 self-assigned this Aug 31, 2026
@skoob13
skoob13 marked this pull request as ready for review August 31, 2026 17:25
@skoob13
skoob13 requested a review from a team as a code owner August 31, 2026 17:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-31T17:28:39.729209Z ef601d1 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@skoob13
skoob13 requested a review from sortafreel August 31, 2026 17:26

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ef601d1a41

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +311 to +312
enabled because each surface already has its own pipeline (step 4's native
source for error tracking, step 6c's scanners for replay) that a scout would

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the remaining replay scout rationale

Apply this scanner-based rationale to every STEP 6 description: the backbone at line 109 and the table at line 138 still say that session-replay is excluded because it is consumed or covered by native sources, directly contradicting the new statement that replay has no native source. The completed-work note around lines 534–543 repeats the obsolete claim as well. Leaving these active instructions inconsistent can lead the next context-mill change to restore the retired replay source.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant