[ronniegeraghty]

What changed

Use the existing pinned github/evals-action to run a changed existing plugin skill's evals against its base and proposed versions, with the shared marketplace environment preserved.

• Select each changed skill's existing eval.yaml; explicitly warn and skip new, deleted or eval-less skills rather than inventing a passing result.
• Keep both runs at the PR merge measurement revision. Restore only the selected skill from the exact base-parent SHA and then the exact PR-head SHA, recording both sources and rejecting other measurement-file drift.
• Run the two Action invocations sequentially in one job so they reuse the Action's verified dependency installation. Use one trial, one worker and zero trial retries. A failed before verdict still permits the after run.
• Require approval through an existing protected evals environment. Exclude fork PRs, use pull_request rather than pull_request_target, and grant model-billing permission only to the gated job. Do not upload raw transcripts or grant PR-write permissions.
• Introduce the workflow and its boundaries in the README and existing eval documentation.

Required setup and execution limits

The protected evals environment has not been configured. A repository owner must create it with required maintainer reviewers, and the organization must permit Copilot workflow-token billing. Preflight deliberately fails closed if approval protection cannot be confirmed. This PR changes no repository settings or permissions.

Maintainers must review the PR's workflow, eval/setup/grader code and cost before approving. Same-repository code with a billing token is trusted code, not an OS/network sandbox. Native Docker sandboxing and the local standard-model experiment are separate work; this Action remains single-model for now.

Focused model-free tests exercise selection, source switching, measurement-drift rejection and workflow policy. Skill/eval lint, catalog matching and diff checks pass. No Actions workflow or paid eval was dispatched, so live billing/environment integration is not claimed.

Source contracts

Action pinned to 6e4c077e (v0.2.0), with Vally 0.19.0:

• PR/merge-SHA policy
• Same-job verified tool acquisition

No custom eval runner, regression-only gate, dashboard or scheduled updates. Results-directory changes and #13's skill/grader fixes are separate PRs.

[Copilot]

🟡 Changes recommended

Missing working-directory inputs prevent both eval invocations from running, and the selection job lacks the actions: read permission needed for private-repository environment checks.

2 open issues

• Add required working directories to both eval action calls · New
• Grant the selection job actions:read for private environments · New

What changed in this PR

Introduces approval-gated before/after evaluations for changed existing plugin skills, preserving the marketplace’s shared evaluation setup while switching only the selected skill’s source.

Changes:

• Adds sequential, pinned evals-action runs against exact base and PR-head skill versions.
• Requires protected-environment approval for same-repository PRs and skips skills without a valid comparison pair.
• Adds model-free workflow checks and documents setup, billing, and execution boundaries.

File	Description
README.md	Introduces the approval-gated PR evaluation workflow.
docs/​checks-and-evals.md	Documents comparison inputs, approval requirements, and execution limits.
.github/​workflows/​skill-evals.yml	Adds skill selection, source switching, and sequential evaluation jobs.
.github/​workflows/​evals.yml	Updates comments to distinguish manual smoke evaluations from skill comparisons.
.github/​scripts/​validation.test.mjs	Adds checks for workflow policy, selection, source restoration, and measurement drift.

_{Review effort: Max (auto)}