user-interface: onboard one account-bound design route end to end and run the offer evals #8987

Activity

1. kyle-sexton

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.A maintainer confirmed the item is real and in scope; ready to be worked.

   on Oct 22, 2026
2. github-actions commented on Oct 22, 2026

   

   github-actionsBot

   on Oct 22, 2026 – with GitHub Actions

   Last edited by melodic-automation-lanes

   Contributor

   More actions

   Proposed work class: work-class: scoped
   Basis: automated intake triage; the lanes App applied this class.

   Size: Small
   Complexity: Low
   Uncertainty: Knowable
   Flags: none
   Blocked by: none

   Decision recorded.

   The blocker #6449 closed as completed on 2026-10-29, so the 2026-10-28 criteria can run now, on an attended host run.

   Issue Triage session decision on evidence, read at origin/main 75087c11f4.

   [quote truncated; the full decision is at the link]

   #8987 (comment)

   The decision settles the open questions. #6449, the blocker, is closed. The work is to onboard the figma@​claude-plugins-official route with a host sign-in. The next steps are to flip its three routing.json rows to confirmed with a new as_of, and to post the offer, decline and accept eval results. Those steps need a person's sign-in and a paid run, so they run attended on a host. No charge is added. If the route needs a paid seat, stop and raise it as a cost question.

   _{Automated triage, applied by the lanes App.}
3. github-actions

   added the

   Task

   issue type on Oct 22, 2026
4. kyle-sexton

   removed

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 25, 2026
5. kyle-sexton commented on Oct 25, 2026

   

   kyle-sexton

   on Oct 25, 2026

   MemberAuthor

   More actions

   Released for re-triage under the owner's 2026-10-24 rule: an issue stops for a person only for an intent or vision question, a security call, a host or attended run, or spend outside prepaid credit. The hold label came off and intake triage runs again now; it re-applies a hold, with a numbered question, when one of those applies.

   To hold this issue again, add needs-human (or status: needs-decision) and say why in a comment.
6. kyle-sexton

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 25, 2026
7. kyle-sexton commented on Oct 25, 2026

   

   kyle-sexton

   on Oct 25, 2026

   MemberAuthor

   More actions

   Hold restored: the 2026-10-25 release took needs-human off, but this issue does need a person under the owner's stop rule. Reason (attended): "Onboarding needs a person's account, and running the evals is a paid run"; onboard one route "with a sign-in"
8. kyle-sexton

   marked this as blocked on Oct 27, 2026

   • user-interface: offer to install and onboard a better design tool #6449
9. kyle-sexton

   removed

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 27, 2026
10. kyle-sexton commented on Oct 27, 2026

    

    kyle-sexton

    on Oct 27, 2026

    MemberAuthor

    More actions

    Released for re-triage under the owner's standing direction (2026-10-27): a person is involved only for "1. A change to the original intent or vision. 2. A finding during implementation ... that changes the intent, vision or primary design. 3. A real HIGH/CRITICAL security or cost risk", where cost is anything that can bill (Azure, GitHub larger runners or private-repo Actions, paid storage, plan or seat changes) and model usage under the subscriptions or API credits is not cost. None applies here. The hold label came off and intake triage runs again; an open blocked-by edge, if any, makes admission wait for that blocker.

    To hold this issue again, add needs-human (or status: needs-decision) and name which of the three applies.
11. github-actions

    added

    priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.

    on Oct 27, 2026
12. melodic-automation-lanes

    added

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.

    and removed

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    on Oct 27, 2026
13. 13 remaining items

    Load more

14. github-actions commented on Nov 1, 2026

    

    github-actionsBot

    on Nov 1, 2026 – with GitHub Actions

    Contributor

    More actions

    The triage sweep dispatched intake triage for this issue (attempt 1 of 2).
15. melodic-automation-lanes

    added

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.A maintainer confirmed the item is real and in scope; ready to be worked.

    and removed

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    on Nov 1, 2026
16. kyle-sexton

    added

    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.Needs an unattended run on a host; the host runner takes it, not a person or the issue lane.

    on Nov 2, 2026
17. melodic-automation-lanes

    set

    Work status

    to

    confirmed

    on Nov 5, 2026
18. kyle-sexton commented on Nov 6, 2026

    

    kyle-sexton

    on Nov 6, 2026

    MemberAuthor

    More actions

    Eval results: the offer, decline and accept cases from #6449 (2026-11-06, Claude Code 2.1.319, claude plugin eval, 1 run per case, mocked servers, judge haiku, judge samples uncalibrated)

    The cases were converted from plugins/user-interface/skills/design/evals/evals.json (ids 11-13) into suite cases in a scratch copy, with a scaffold that writes the offer-confirmed fixture.

    Case	Branch #13797 (Figma confirmed)	main (Figma deferred)
    offer-confirmed-uninstalled-route	1.00 (4/4)	1.00 (4/4)
    offer-declined-continues	0.75: fails "continues with the design work"	0.75: same failure
    offer-accepted-installs-local	0.50: fails the pointer-page, exact-command and sign-in-step checks	0.50: same three

    The branch and main score the same, so the routing change causes no regression. Both failures come from the cases themselves:

    • Decline: the scaffold has no settings screen, so the skill correctly declines to install anything and then stops to ask for the screen instead of designing it.
    • Accept: the run allows only Read, Glob, Grep and Skill, so the skill cannot fetch Pencil's install page or show the command from it. It says so and asks.

    The signed-in host onboarding test is still to come.
19. kyle-sexton commented on Nov 6, 2026

    

    kyle-sexton

    on Nov 6, 2026

    MemberAuthor

    More actions

    Host onboarding test: Figma plugin route (2026-11-06, Claude Code 2.1.319, the owner's host, against the #13797 branch with the claude.ai connectors off)

    Check	Result
    Sign-in through the plugin's own OAuth (plugin:figma:figma)	PASS
    whoami	PASS: Starter (free) tier, Full seat; no seat or plan bought
    One read tool (get_metadata on a new draft file)	PASS
    /user-interface:design, typography-color	PASS: routed to figma@claude-plugins-official (rank 2; rank 1 /playgrounds:use not installed)
    /user-interface:design, design-systems	PASS: routed to figma@claude-plugins-official (rank 1)
    /user-interface:design, design-tool-handoff	PASS: routed to figma@claude-plugins-official (rank 1)

    Every tool call went through mcp__plugin_figma_figma__* (the plugin route, not a claude.ai connector), and each answer read routing.json from the branch worktree. The test used 1 of the plan's monthly read calls.

    Owner decision (2026-11-06): the first checkbox's "/user-interface:design offers it" is met as "routes to it". An installed route is routed to, not offered (plugins/user-interface/skills/design/SKILL.md, "Offer a better route").
20. kyle-sexton

    added a commit that references this issue on Nov 6, 2026

    docs(user-interface): name Figma as a tested account route

    ...

    e4bf6d9
21. kyle-sexton

    closed this as completedin #13797on Nov 6, 2026
22. melodic-automation-lanes

    removed

    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.A maintainer confirmed the item is real and in scope; ready to be worked.

    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.Needs an unattended run on a host; the host runner takes it, not a person or the issue lane.

    on Nov 6, 2026
23. kyle-sexton

    added 3 commits that reference this issue on Nov 7, 2026

    test(user-interface): enforce the offer command path and record real …

    ...

    b3aac3e

    test(user-interface): count only routes in the offer case's no-other-…

    ...

    da9468a

    test(user-interface): drop the Pencil accept case, prove the offer ch…

    ...

    79a64d0