Repository navigation
perf(annotations): apply batched notes from source without re-checks - #125
Open
lauridskern wants to merge 2 commits into
Open
lauridskern wants to merge 2 commits into
lauridskern wants to merge 2 commits into
Conversation
Annotation edits slowed down when the agent reconciled several notes on one source, rewrote components for unique targets, and inspected or verified the device around every note. - Group notes from one capture and source: say when they cover every matching element (edit the source once) or only some of them. - Mark a target as the only match on screen, and give a search order when a note has no source location. - Guidance: go straight to Source, do not re-inspect the device before editing, make every edit, then check once. - Skill: a Screen annotations section covering Fast Refresh (no rebuilds) and a single screenshot check. - Telemetry: ui.annotations.send_batch counts sends by note count bucket (1, 2-3, 4+).
tooling/annotation-eval runs Codex on a fixture app with the annotation formatter and skill at a git ref against the working tree, grades each diff, and summarizes time, tool calls and correctness. Device mode drives an Expo app on a simulator with the plugin's MCP server.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Users reported that sending several annotations at once (for example three small color changes) took much longer than one. This PR makes the panel's message and the Mobile Dev skill lead the agent straight to the edit, and adds a benchmark to measure it.
What was slow
With the old guidance, Codex:
Changes
formatAnnotationGroups): notes from one capture on one source say whether they cover every matching element, in which case the agent makes the change once at the source, or only some of them. Sent as model context and in the standard-host draft.device_open→device_screenshot→device_close.ui.annotations.send_batchcounts sends with anotesbucket (1,2-3,4+), documented indocs/telemetry.md.tooling/annotation-eval: a Codex benchmark that compares the formatter and skill at a ref with the working tree, on a fixture app or on a live simulator. See its README.Results
Codex
gpt-6.1-sol, medium effort. Every one of the 108 runs made a correct edit.Live simulator (Expo app, plugin MCP server, 12 runs per version)
Code only (21 runs per version, 7 scenarios): median 21.7 s → 19.0 s, mean 33.4 s → 20.9 s, runs over 60 s 3 → 0. With the old prompt, three notes without a source location (scenario G) had a median of 97 s; with the new one, 29 s.
Caveats
Testing
node --test packages/ui/test/*.test.ts packages/core/test/mobile-dev-skill.test.ts: 460 passingnode --test packages/shared/test/*.test.ts tooling/test/*.test.ts: 190 passingnpm run typecheck:telemetry, plustsc --stricton the changed sources and the harnesssend_batchattribute.