Skip to content

Complete live Codex capability evidence for baseline 2.3 #7

Description

@KeyffMS

Completed live qualification and evidence import

Full run #27 completed C01-C16 successfully on the authenticated self-hosted runner: Codex CLI 0.153.4, gpt-5.6-sol, Debian 13. Actual tested source: a9cdcdc1e0cad70e88b60869e75e4166046dd306.

PR #36 merged the exact sanitized packages, original archive and provenance into main at c7c28cd46a07a353fdd88356c3fe6f8ff1fd6f21. The source SHA in evidence remains the tested SHA, not the import commit. The full archive is preserved under qualifications/34140846679/; the original #25 archive remains unchanged. Strict production evidence validation passes for the imported tree. The original issue tracked baseline 2.2; the completed qualification uses the subsequently adopted baseline 2.3.

Required

  • C01 — repository skill discovery
  • C02 — explicit vs implicit skill activation
  • C03 — PlanAnvil flat direct-child contract
  • C05 — explicit immutable reviewer handoff
  • C06 — PreToolUse supported paths and deterministic postconditions
  • C07 — unsafe Git guard decisions and postconditions
  • C08 — actual missing-checkpoint stop and finite repaired completion without timeout
  • C09 — checkpoint → two automatic compactions → recovery and completed turn
  • C10 — startup and SessionStart(source=compact) recovery context; PostCompact advisory
  • C11 — project instruction scope and precedence
  • C12 — project_doc_max_bytes truncation fixture
  • C13 — SubagentStart context/startup semantics in the explicitly allowed project-scoped non-ephemeral fallback
  • C14 — planning isolation in an actual Codex run
  • C15 — fresh blind reviewer with seeded defects
  • C16 — Git capability matrix across supported Codex permission modes

C04 is informational/non-gating in baseline 2.3 and was also REPRODUCED.

Evidence and retained limits

Each capabilities/CXX contains the exact fixture, prompt, configuration, run command, expectations, sanitized actual results, evaluation and SHA-256 hashes from the same complete full run. Archive integrity covers 166 listed files plus the manifest, including three hidden fixtures. No timeout=true is recorded in the current actual evidence. C13's ephemeral parent-thread limitation remains explicitly documented; this closes the baseline semantic capability requirement, not that upstream transport limitation.

No product/runtime/security or release-validator change was made during import. Qualification in this tested configuration does not guarantee all future model behavior or untested versions. Signed production publication remains a separate action; no tag or GitHub Release was created by this closure.

Historical blocker — 2026-08-28

The original qualification environment had no authenticated Codex runtime or codex executable. Deterministic CI was not accepted as live evidence. That blocker was subsequently resolved through the controlled self-hosted runner. Historical run #25 met its narrow C08 assertion but timed out later; its unedited record is retained. Full #27 supplies the required finite C08 completion.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions