Repository navigation
chore: update eval remote pin to copilot-evals-v0.24.0 - #10
Conversation
There was a problem hiding this comment.
🔵 Needs a closer look
The externally hosted plugin adds substantial MCP/extension code, automations, packaged distribution, and native binaries whose runtime behavior and supply-chain contents cannot be fully validated in this local diff review.
No open issues 🎉
What changed in this PR
Updates the optional remote eval plugin from copilot-evals-v0.23.3 to the verified copilot-evals-v0.24.0 release and synchronizes the catalog and documentation with the new upstream commit.
Changes:
- Updates the plugin reference, commit SHA, and version in
marketplace.json. - Refreshes Evals release, setup, and license links in
README.md.
| File | Description |
|---|---|
| README.md | Updates Evals documentation, release, setup, and license links to reference version 0.24.0. |
| marketplace.json | Pins the remote eval plugin to the v0.24.0 tag, matching commit, and manifest version. |
Review effort: Lite (auto)
Updates the remote �val plugin pin in marketplace.json and README.md from copilot-evals-v0.23.3 (�e7b4ee0d58b7bea143e74fd64275b2c47091cab) to copilot-evals-v0.24.0 (34ddb5803aeaee8330ab70c19ad1a3f37fb77ded). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
b8cf989 to
325f131
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
Human review is warranted because the pinned third-party release adds native helpers, an MCP server, automations, and default telemetry that cannot be fully runtime-tested from the local marketplace snapshot.
No open issues 🎉
Review effort: Lite (auto)
There was a problem hiding this comment.
🔵 Needs a closer look
The pin is consistent and resolves to the upstream manifest, but the release adds native Windows binaries, an MCP server, extensions, and automations whose runtime behavior, permissions, and supply-chain implications cannot be verified from this marketplace-only diff.
No open issues 🎉
Review effort: Lite (auto)
Summary
Updates the remote
evalplugin pin inmarketplace.jsonandREADME.mdto the newly publishedcopilot-evals-v0.24.0release.Upstream Review (
update-remote-pins)copilot-evals-v0.23.3(be7b4ee0d58b7bea143e74fd64275b2c47091cab, version0.23.3)copilot-evals-v0.24.0(34ddb5803aeaee8330ab70c19ad1a3f37fb77ded, version0.24.0)packages/plugin/plugin.json(name: "eval",version: "0.24.0")/eval improve <skill>(asset-hillclimbmode), and improved eval report readability across CLI, Canvas, and VS Code.Verification
node .github/scripts/check-remote-pins.mjs— passed (eval: github/evals@34ddb5803aeaee8330ab70c19ad1a3f37fb77ded (packages/plugin/plugin.json) ok)node .github/scripts/check-local-catalog.mjs— passednpm run lint— passed (6 skills linted, 7 eval suites valid)node --test .github/scripts/validation.test.mjs .github/scripts/eval-report.test.mjs .github/scripts/release-plugins.test.mjs— 36/36 passed