diff --git a/docs/CODEX_SANDBOX_RUNBOOK.md b/docs/CODEX_SANDBOX_RUNBOOK.md index 9a975a0..9334ddc 100644 --- a/docs/CODEX_SANDBOX_RUNBOOK.md +++ b/docs/CODEX_SANDBOX_RUNBOOK.md @@ -1,6 +1,6 @@ # Codex sandbox qualification runbook -Full run #25 is archived and passed baseline 2.3 on Codex CLI 0.153.4. The next remaining publication check is the finite C08 stop/repair follow-up. `QUALIFICATION_STATUS.md` is the current status; this runbook describes both full requalification and targeted checks. +Full run #25 is archived and passed baseline 2.3 on Codex CLI 0.153.4. The finite C08 stop/repair replacement is merged. Publication now requires a successful new full run and import of its complete evidence. `QUALIFICATION_STATUS.md` is the current status; this runbook describes both full requalification and targeted checks. ## Sandbox prerequisites @@ -27,9 +27,11 @@ If the probe fails with `setting up uid map: Operation not permitted`, fix the P The preferred qualification path is `.github/workflows/plananvil-codex-qualification.yml`. The workflow is intentionally `workflow_dispatch`-only, accepts execution only from `main`, uses Environment `plananvil-codex`, and targets `[self-hosted, linux, x64, plananvil, codex]`. -Use `mode=full` for the release-gating C01-C16 sequence. Use `mode=c08` now to verify the finite stop/repair replacement without rerunning already-qualified paths. `mode=recovery` retains C09/C10/C13; `mode=c13` selects C13 only. `mode=smoke` verifies basic runtime/authentication. The precision/variant matrices are diagnostics, not release evidence. All targeted modes keep `release_gate_passed=false`, even when their selected checks pass. +Use **mode=full** for the next publication-closing C01-C16 sequence. The production validator requires one complete source-bound full-run archive with the finite C08 result, not a combination of partial archives. `mode=c08` is available to diagnose only the finite stop/repair replacement. `mode=recovery` retains C09/C10/C13; `mode=c13` selects C13 only. `mode=smoke` verifies basic runtime/authentication. The precision/variant matrices are diagnostics, not release evidence. All targeted modes keep `release_gate_passed=false`, even when their selected checks pass. -For the C08 follow-up choose **PlanAnvil Codex qualification → Run workflow → main → c08**. Start a new run, not a rerun of a historical SHA. It uses the same active v7 C08 runtime as full, with real live model inference. The outer harness deliberately starts without a checkpoint, observes the actual PreCompact stop and termination, creates/validates a real checkpoint, then requires one pressure → compaction → SessionStart(compact) → finish sequence and a completed positive result. The disposable fixture excludes ordinary startup recovery only; product files and C10 startup coverage remain unchanged. +Start a NEW **PlanAnvil Codex qualification -> Run workflow -> main -> full** from an account allowed by the local runner's initiating-actor policy. Automated run #26 (`34109662176`) was rejected by that policy before any Codex trial; do not rerun it or relax the allowlist. The repair is on main, but that rejected launch provides no model-backed result. + +Both `full` and `c08` use the same active v7 C08 runtime with real live model inference. The outer harness deliberately starts without a checkpoint, observes the actual PreCompact stop and termination, creates/validates a real checkpoint, then requires one pressure -> compaction -> SessionStart(compact) -> finish sequence and a completed positive result. The disposable fixture excludes ordinary startup recovery only; product files and C10 startup coverage remain unchanged. The controlled runner must provide `plananvil-qualification-workspace`. The workflow creates a disposable workspace with that helper, fetches only the exact dispatched `main` SHA, materializes the C01-C16 evidence templates, and runs `tools/live_codex_qualification_harness_v7.py`. Model `gpt-5.6-sol` is pinned, approval policy remains `never`, model-tool network access is disabled, and `workspace-write` is granted only to disposable fixture roots when a trial requires it. Vetted project hooks may bypass only the interactive hook-trust prompt; approval and filesystem sandboxing remain enabled. @@ -115,9 +117,9 @@ C04 is informational/non-gating in baseline 2.3 but should still be observed if ## Final gate -Full capability qualification and publication closure are separate. Preserve the original full run, then commit the finite C08 follow-up archive/provenance as described in `RELEASE.md`. Never replace the full-run index with a partial-mode index. Production checks require the same qualified product inputs and real, complete C08 protocol evidence, not only a status label. +Full capability qualification and publication closure are separate. Preserve the original #25 archive unchanged. For publication, review and commit a new successful full-run archive whose C08 has finite completion evidence, along with its exact C01-C16 packages and actual executed source SHA. Update `qualifications/index.json` to that full run as described in `RELEASE.md`. Never replace the full-run index with a partial-mode index or splice a targeted result into an older archive. -When all required capabilities are `REPRODUCED` and the supplementary completion evidence is committed: +When the complete current full-run evidence is committed: ```text python tools/validate_capabilities.py diff --git a/docs/QUALIFICATION_STATUS.md b/docs/QUALIFICATION_STATUS.md index 177bdef..4d04c04 100644 --- a/docs/QUALIFICATION_STATUS.md +++ b/docs/QUALIFICATION_STATUS.md @@ -1,30 +1,71 @@ # Qualification status -## Recorded result +## Recorded live result Full run #25 (`34060321283`) passed baseline 2.3 C01-C16 on product source `d0384f76bc4150d33bb8f51ef5981f3243b3cfb3`, Codex CLI 0.153.4, `gpt-5.6-sol`, Debian 13. C04 is informational. The immutable original archive, summary, checksums and caveats are in `qualifications/34060321283/`. -## Closure in progress +## Implemented closure + +PR #34 merged the finite C08 repair and the reviewed evidence into main at +`ba3f56a644a385c8aa2e8e5c9c934999184cc282`. PR CI #118 and post-merge CI #119 +passed all eight jobs, including actual Codex CLI loopback conformance. These +are implementation/integration checks, not a new live-model result. The historical positive C08 trial timed out after meeting its narrow unblock -assertion. It remains unchanged in the archive. The current finite replacement -requires the same canonical run to stop at PreCompact without a checkpoint, -then, after outer checkpoint repair, complete pressure -> compact recovery -> -finish with exit zero. Timeout or incomplete telemetry cannot pass. -The replacement is not yet claimed as live evidence. Production release checks -require its actual committed finite evidence and qualified product identity. +assertion. It remains unchanged in the archive. The finite replacement requires +the same canonical run to stop at PreCompact without a checkpoint, then, after +outer checkpoint repair, complete pressure -> compact recovery -> finish with +exit zero. Timeout or incomplete telemetry cannot pass. C13 passed in the explicitly allowed project-scoped non-ephemeral fallback. -Ephemeral spawning is not claimed as working. The current product .agents/.codex -payload and the C09/C10/C13 runtime paths are unchanged by this closure. +Ephemeral spawning is not claimed as working. The product .agents/.codex payload +and the C09/C10/C13 runtime paths are unchanged by this closure. + +## Required operator action + +The automated full launch #26 (`34109662176`) used the repaired main commit, +but the self-hosted runner's job-start policy rejected its initiating actor: + +```text +PlanAnvil runner policy denied this job: initiating actor is not allowed +``` + +The hook exited 77 before runner preflight, fixture preparation or any Codex +trial. This run has no new capability evidence and is not a C08 test failure. +Do not change the runner allowlist, spoof the actor or relax the trusted-workflow +policy to make an automated launch pass. + +An operator whose account is allowed by the runner policy must start a NEW run: + +**Actions -> PlanAnvil Codex qualification -> Run workflow -> main -> full**. + +Use Run workflow, not Re-run of the denied automated run. GitHub Actions +orchestrates the job; all model-backed trials run on the existing authenticated +self-hosted runner, not on a GitHub-hosted machine. + +## Evidence required for publication + +The current production validator requires ONE complete, source-bound full-run +archive, exact matching current capability packages, unchanged qualified product +bytes, and finite C08 stop/repair proof in that same archive. Therefore the next +publication proof must be a successful new `full` run. The `c08` mode remains a +useful targeted diagnostic, but its partial archive alone cannot close the +production gate. Do not splice it into or relabel the immutable #25 archive. + +After success, review and verify the new sanitized archive, preserve it under +`qualifications//`, import its exact C01-C16 packages, and update +`qualifications/index.json` through a protected PR. Keep #25 unchanged as history. +Record the actually executed source SHA, not the later import commit. Run the +strict release check on a clean tree after import. No production tag or release +has been created by these preparation steps. ## Verification layers -Unit/process tests and the pinned real-CLI loopback tests check implementation -and protocol behavior. Only the authenticated self-hosted qualification workflow +Unit/process tests and pinned real-CLI loopback tests check implementation and +protocol behavior. Only the authenticated self-hosted qualification workflow provides live-model evidence. Newly materialized templates always start BLOCKED; archived evidence cannot silently qualify a fresh fixture or changed product. @@ -34,7 +75,7 @@ archived evidence cannot silently qualify a fresh fixture or changed product. - `CODEX_QUALIFICATION_EXECUTION_AUDIT_2026-09-05.md`: C13 argv and C10 worktree discovery. - `CODEX_RECOVERY_DELIVERY_AUDIT_2026-09-05.md`: supported context delivery and diagnostics. - `CODEX_C09_FINITE_RECOVERY_AUDIT_2026-09-06.md`: completed finite C09 repair. -- `CODEX_C08_CLOSURE_AUDIT_2026-09-07.md`: remaining C08 and evidence closure. +- `CODEX_C08_CLOSURE_AUDIT_2026-09-07.md`: finite C08 and evidence closure. Older harness modules are retained because v7 still imports them. They are not independent supported entry points. Do not delete imported layers as cosmetic cleanup. diff --git a/docs/RELEASE.md b/docs/RELEASE.md index ad5f221..a750ae4 100644 --- a/docs/RELEASE.md +++ b/docs/RELEASE.md @@ -25,6 +25,8 @@ python tools/build_release.py --output dist Before a production tag, execute `docs/CODEX_SANDBOX_RUNBOOK.md`. Required entries in `capabilities/index.json` must be changed from `BLOCKED` to `REPRODUCED` only after their complete sanitized evidence package is committed and `python tools/validate_capabilities.py` passes. +For the current C08 closure, use a NEW `mode=full` run on main, initiated by an account allowed by the self-hosted runner. The production validator requires a single complete full-run archive containing the finite C08 proof. A successful `c08` or `recovery` diagnostic is useful but its partial archive cannot replace that full run. See `docs/QUALIFICATION_STATUS.md` for the merged repair and the automated launch rejected before testing. + ## Publish `VERSION` is the canonical distribution/release version. The deterministic artifact generator version is recorded independently in generated artifacts for compatibility. A production tag must be exactly `v`. @@ -60,14 +62,23 @@ actually executed commit, never the later evidence-import commit. `release_check.py` also validates the original archive digest/manifest, exact current capability packages, qualified .agents/.codex bytes, and finite C08 negative-stop/positive-completion evidence. The old C08 timeout keeps production -blocked until the new live result is committed. Candidate checks intentionally -remain usable before that result exists. +blocked until the new complete full-run result is committed. Candidate checks +intentionally remain usable before that result exists. + +To import a successful new full result, preserve its exact archive, summary and +verified provenance under `qualifications//`, import its exact C01-C16 +packages, and point `qualifications/index.json.current_run` to that run. Keep the +original #25 archive and provenance unchanged. Never merge a partial index into +the full index, edit old actual observations, or label the denied automated launch +as tested. Validate archive integrity and production readiness on the resulting +clean tree through a protected PR before any signed publication. The active main ruleset was verified on 2026-09-07: PR-only squash changes, seven required status checks, strict up-to-date branch, conversation resolution, no deletion or force push, no bypass actors. The protected distribution job now also requires real-CLI conformance to succeed. No policy is loosened for closure. -After finite live evidence is imported and CI is green, preparation is complete; -the signed annotated production tag remains a separate authorized publication. -No unsigned or lightweight tag may substitute for the required verified signature. +After the complete new full-run evidence is imported and CI is green, preparation +is complete; the signed annotated production tag remains a separate authorized +publication. No unsigned or lightweight tag may substitute for the required +verified signature.