[ronniegeraghty]

Summary

Add the supported Vally 0.21 native Docker eval path without changing execution defaults or deciding whether Docker should become mandatory.

Changes

• Pin @microsoft/vally-cli to stable 0.21.0; keep the explicit host Copilot 1.0.98 pin and marketplace remote pins unchanged.
• Document explicit Docker invocation, the locally built version-matched appliance, Linux Docker/Compose and authentication prerequisites, task/grade separation, networking, credentials, and cleanup limits.
• Keep project backend defaults, Evals plugin Run behavior, repository settings, and paid CI triggers unchanged.
• Generate a registry-neutral lockfile with npm's documented omit-lockfile-registry-resolved option. Removing 218 registry-specific endpoints preserved every version, integrity hash, and other lock entry field from the initial upgrade.
• Add no custom runner, repository Dockerfile, company registry setting, or default-cache image workaround.

Verification

• Clean npm ci, npm run lint (6 skills and 7 eval definitions), local catalog consistency, git diff --check, and 51 retained model-free script tests passed.
• npm ci used the existing configured development registry. This does not establish independent public-registry availability.
• Corrected synthetic native-Docker probe: 1/1 passed with the same immutable validation image, zero retries, both role networks offline, UID 1000, zero effective capabilities, no-new-privileges, hidden grading-file separation, role-only environment values, and separate task/grade home state. The earlier invalid diagnostic result was retained separately.
• One approved real copilot-sdk run executed both unchanged pr-description cases and their native deterministic graders: one worker, one attempt per case, zero retries, normal task network, offline grading.

Case	Recorded result
Positive PR-description request	FAIL: required skill was not activated; output missed the required heading contract. Completed and empty-diff checks passed.
Negative off-by-one question	PASS: no forbidden skill activation; all four checks passed.

Both task executions and both grading executions completed without execution or grading errors. The eval verdict still failed (1/2 cases, 79.17% score against the unchanged 100% threshold); no skill, grader, model, or threshold was changed to obtain a pass.

Runtime observations: core 0.21.0, SDK 1.0.31, actual Copilot runtime 1.0.109, selected model claude-haiku-5.5 and response model claude-haiku-5-5. The runtime listed 15 available skills, including the target, but neither case recorded a skill activation. Recorded usage was 68,909 tokens across three model calls. The SDK's managed runtime uses --no-auto-update; this container runtime is distinct from the preserved host Copilot pin.

Run IDs: native-docker-21-custom-pr-description-20261111 and native-docker-21-corrected-mock-20261111. Post-run container and volume inventories were empty.

Risks / Rollback

• The unmodified default-image bootstrap was not successfully validated here. Public npm TLS handshakes fail in this environment; the upstream cause is not established. Validation used a separately reviewed local image built from the released official Vally recipe with only an npm registry build argument added. Its immutable ID was sha256:654c029a370cb5cb0bce6cea326fc47cdeda3eaa67e89f20ff3f02566f98723d; it was never tagged as Vally's canonical default cache image and is not published by this PR.
• Successful native execution with that image is not a passing skill evaluation or proof of default-bootstrap/public-network reproducibility. The positive behavioral failure remains visible for separate investigation.
• Task credentials and allowed network effects remain real risks; Docker is not protection against every kernel escape. The docs distinguish offline deterministic grading from default role networking.
• Revert this commit to restore the previous tooling and documentation. No machine-wide setting or execution-policy migration is required.