Skip to content

feat: add semantic browser recording via bsk record#28

Open
shnpd wants to merge 4 commits into
Tencent:mainfrom
shnpd:feat/semantic-record-v3
Open

feat: add semantic browser recording via bsk record#28
shnpd wants to merge 4 commits into
Tencent:mainfrom
shnpd:feat/semantic-record-v3

Conversation

@shnpd

@shnpd shnpd commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Capture user flows as LLM-readable trace.json (v1) with intent, page context, and semantic targets. Wire record_start/stop/await through CLI, daemon, and extension; bump CLI to 0.1.8 and extension to 0.1.4.

“haonan” and others added 3 commits July 16, 2026 16:09
Capture user flows as LLM-readable trace.json (v1) with intent, page
context, and semantic targets. Wire record_start/stop/await through
CLI, daemon, and extension; bump CLI to 0.1.8 and extension to 0.1.4.

Co-authored-by: Cursor <cursoragent@cursor.com>
Apply import ordering and line wrapping only; no behavioral changes.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@BB-fat

BB-fat commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Suggestion: Add a recording entry point in the Popup UI (generate a prompt for the Agent)

This PR’s recording capability (bsk record) is primarily meant to be invoked by an Agent via the CLI. There is currently no user-facing entry point in the Extension UI. BrowserSkill itself cannot call a large language model, and end users will not type CLI commands themselves.

Proposal: add a recording-related entry in the Popup, e.g. “Record a flow / Teach the Agent how to do this”:

  1. The user clicks the entry in the Popup
  2. The extension generates a Prompt that can be pasted directly to an Agent (with one-click copy)
  3. The Prompt must include the current BrowserID (instance_id), so the Agent can target the correct browser instance and run bsk record start …
  4. The user pastes the Prompt to the Agent → the Agent starts recording → the user demonstrates in the Agent Window → clicks Finish → the Agent receives trace.json

This would turn the collaboration model—“humans only demonstrate; the Agent starts recording and saves the textbook”—from CLI-only documentation into an in-product discovery path, lowering first-time discovery and onboarding cost.

Could this be a follow-up to this PR, or a minimal Popup entry included in this PR?

@BB-fat

BB-fat commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

RecordOverlay doesn't match our existing overlay style (ControlOverlay):

  • Dark card vs white pill + logo
  • Red rectangular button vs orange pill CTA with icon
  • No fade-in / breathe treatment

Please align with the existing bottom pill pattern before merge.

Drop LLM-era semantic fields so recording stores only objectively
observable facts; regenerate schemas and document the architecture
for maintainers.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants