Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 24 additions & 31 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,30 @@ All notable changes to PlanAnvil are documented here.

## [Unreleased]

### Qualification closure — 2026-09-07

- full self-hosted run #27 (`34140846679`) reproduced all C01-C16 on executed source `a9cdcdc1e0cad70e88b60869e75e4166046dd306`, Codex CLI 0.153.4, `gpt-5.6-sol`, Debian 13;
- finite C08 now proves the intended missing-checkpoint stop and completed repaired recovery without timeout; C09/C10 regressions passed;
- preserve the exact full archive and provenance, update current capability evidence, and retain historical #25 unchanged;
- C13 retains the documented project-scoped, known-error-gated non-ephemeral fallback; no new transport claim or product/runtime/security change;
- qualification is recorded; signed production publication remains a separate action.
## [0.2.0] - Pending publication

### Added

- standard-library repository distribution manager with install, verify, upgrade, status and uninstall operations;
- transactional rollback, ownership/hash state, conservative Codex config merging, and structural hook merging;
- distribution tests covering clean repositories, existing `AGENTS.md`, existing `.codex/config.toml`, unrelated hooks, upgrade conflicts and uninstall conflicts;
- deterministic release archive builder, candidate/production release gate, and tag-driven GitHub Release workflow;
- deterministic C01-C16 qualification template archive containing fixture, prompt, config, expected result, current sanitized BLOCKED result, evaluation and SHA-256 manifests;
- capability evidence materializer/validator/rehash tools and a live Codex sandbox runbook;
- installation, troubleshooting, and release documentation;
- add a controlled `workflow_dispatch`-only Codex qualification workflow for the trusted `plananvil-codex` self-hosted runner;
- add a sequential C01-C16 live qualification controller that creates disposable fixture repositories, runs fresh ephemeral Codex trials, sanitizes structural evidence, rehashes/validates each package, and exports evidence only as a GitHub Actions artifact;
- add regression tests ensuring the live qualification controller redacts private-looking data and never disables the Codex sandbox/approval boundary.

### Changed

- refresh the Codex capability baseline through 2.2 (2026-08-28) to the qualified baseline 2.3 described below;
- replace legacy agent concurrency/depth configuration with `agents.enabled` and `agents.max_concurrent_threads_per_session`;
- enforce flat direct-child execution topology in the generated contract instead of relying on undocumented `agents.max_depth` behavior;
- record the 2026-08-28 C01–C16 qualification attempt and its live Codex runtime blocker;
- pin GitHub Actions to immutable SHAs and Node-24-based checkout/setup-python releases;
- expand CI across Python 3.11 and the current upper supported interpreter on Ubuntu, macOS and Windows;
- split distribution/release-candidate validation into a stable named CI check;
- update pinned `actions/checkout` and `actions/setup-python` workflow SHAs to the current v7 releases while retaining immutable action pinning and Node 24 compatibility;
- require the full Linux Codex qualification job to pass a system-`bubblewrap` user-namespace probe before C01-C16, so incompatible Podman runners fail fast instead of timing out capability-by-capability;
- update the qualification evidence uploader to the Node-24-native `actions/upload-artifact` v6 immutable SHA;
Expand All @@ -33,38 +41,23 @@ All notable changes to PlanAnvil are documented here.
- fail the production release gate closed when the release worktree is dirty or Git cleanliness cannot be verified;
- document the controlled self-hosted Codex qualification path and keep the previous sandbox procedure as a manual fallback.

## [0.2.0] - 2026-08-28

### Added

- standard-library repository distribution manager with install, verify, upgrade, status and uninstall operations;
- transactional rollback, ownership/hash state, conservative Codex config merging, and structural hook merging;
- distribution tests covering clean repositories, existing `AGENTS.md`, existing `.codex/config.toml`, unrelated hooks, upgrade conflicts and uninstall conflicts;
- deterministic release archive builder, candidate/production release gate, and tag-driven GitHub Release workflow;
- deterministic C01-C16 qualification template archive containing fixture, prompt, config, expected result, current sanitized BLOCKED result, evaluation and SHA-256 manifests;
- capability evidence materializer/validator/rehash tools and a live Codex sandbox runbook;
- installation, troubleshooting, and release documentation.

### Changed

- refresh the Codex capability baseline to 2.2 against current 2026-08-28 official documentation;
- replace legacy agent concurrency/depth configuration with `agents.enabled` and `agents.max_concurrent_threads_per_session`;
- enforce flat direct-child execution topology in the generated contract instead of relying on undocumented `agents.max_depth` behavior;
- record the 2026-08-28 C01–C16 qualification attempt and its live Codex runtime blocker;
- pin GitHub Actions to immutable SHAs and Node-24-based checkout/setup-python releases;
- expand CI across Python 3.11 and the current upper supported interpreter on Ubuntu, macOS and Windows;
- split distribution/release-candidate validation into a stable named CI check.

### Fixed

- canonicalize event repository paths before active-run routing so source-worktree matching is stable across macOS symlink aliases and Windows path aliases;
- keep checkpoint recovery assertions platform-neutral by comparing canonical paths;
- update the artifact-sealing lock regression test to observe the current `validate_plan_contract` gate;
- make the Git-hook probe fixture emit explicit hook diagnostics while preserving fail-closed classification for unrelated commit failures;
- synchronize golden blind-review fixtures and dependent comparison hashes with the required independent `plan-anvil-reviewer` author role.
- synchronize golden blind-review fixtures and dependent comparison hashes with the required independent `plan-anvil-reviewer` author role;
- reconcile the compliance checklist and current qualification record with full run #27, retaining the historical #25 timeout and current C13 limitation;
- check compliance metadata and finite C08 checklist status against the committed qualification index and summary in CI.

### Qualification closure — 2026-09-07

- full self-hosted run #27 (`34140846679`) reproduced all C01-C16 on executed source `a9cdcdc1e0cad70e88b60869e75e4166046dd306`, Codex CLI 0.153.4, `gpt-5.6-sol`, Debian 13;
- finite C08 now proves the intended missing-checkpoint stop and completed repaired recovery without timeout; C09/C10 regressions passed;
- preserve the exact full archive and provenance, update current capability evidence, and retain historical #25 unchanged;
- C13 retains the documented project-scoped, known-error-gated non-ephemeral fallback; no new transport claim or product/runtime/security change;
- qualification is recorded; signed production publication remains a separate action;
- preserve complete baseline 2.3 live evidence from full run #25, `34060321283`, tested at `d0384f76bc4150d33bb8f51ef5981f3243b3cfb3` with Codex CLI 0.153.4, `gpt-5.6-sol`, Debian 13; all C01–C16 were reproduced;
- retain the exact source-bound archive, hashes and original limitations; C13 passed via the permitted project-native non-ephemeral fallback;
- replace the old C08 repaired-path workload with a finite pressure/finish scenario and strict termination checks; the old positive timeout remains in historical evidence; full run #27 subsequently confirmed the replacement live;
Expand All @@ -74,7 +67,7 @@ All notable changes to PlanAnvil are documented here.

### Release status

0.2.0 is qualified for the recorded configuration. Full run #27 (`34140846679`) confirmed C01-C16 and finite C08 stop/repair on source `a9cdcdc1e0cad70e88b60869e75e4166046dd306`; its complete source-bound evidence is committed and #25 remains unchanged. Production publication remains a separate verified signed annotated-tag action after strict release validation. No tag or release has been published by this evidence import.
0.2.0 is qualified for the recorded configuration. Full run #27 (`34140846679`) confirmed C01-C16 and finite C08 stop/repair on source `a9cdcdc1e0cad70e88b60869e75e4166046dd306`; its complete source-bound evidence is committed and #25 remains unchanged. Production publication remains pending a verified signed annotated tag after strict release validation and green protected CI. No production tag or release is created by this documentation update.

## [0.1.0] - 2026-07-12

Expand Down
79 changes: 68 additions & 11 deletions docs/OPENAI_COMPLIANCE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PlanAnvil — OpenAI Codex Compliance Record

> **Original contract review:** 2026-07-12; scoped recovery/source review: 2026-09-05; live qualification: 2026-09-06
> **Original contract review:** 2026-07-12; scoped recovery/source review: 2026-09-05; live qualification: 2026-09-07; qualification-record reconciliation: 2026-09-08
> **Scope:** contract 2.1; baseline 2.3 qualification of the 0.2.0 release candidate
> **Rule:** current official OpenAI documentation is authoritative for Codex behavior.

Expand Down Expand Up @@ -199,13 +199,70 @@ A relevant documentation change blocks release until the affected record and tes
- [x] Local profile and local state are ignored
- [x] Machine-state formats defined
- [x] Unsupported behaviors excluded from the active contract
- [x] Complete source-scoped live C01–C16 evidence committed for run #25
- [ ] Finite C08 repaired-path completion verified live and committed for production closure

## 7. Source-scoped qualification and release closure — 2026-09-07

Run `34060321283` reproduced all baseline 2.3 capabilities with Codex CLI `0.153.4`, model `gpt-5.6-sol`, Debian 13, at PlanAnvil `d0384f76bc4150d33bb8f51ef5981f3243b3cfb3`. The archived result retains its original source identifier and hashes. Current status and remaining constraints are in `QUALIFICATION_STATUS.md`; earlier audits are historical, not current blockers by default.

C10 delivers recovery through `SessionStart(source=compact)`; `PostCompact` remains advisory. C13's project-native agent and hook pass via the allowed non-ephemeral retry, not an unrestricted home-scoped replacement. C08's historical positive trial timed out after proving unblocking. The stricter finite C08 follow-up has not yet produced live evidence and remains a production-publication prerequisite.

The C08 fixture now isolates the deliberate invalid-checkpoint trigger from ordinary startup recovery by narrowing SessionStart to `^compact$` in the disposable root checkout before bootstrap. It does not change product hook behavior. Repaired recovery still runs through the real compact source. A correct stop is an expected terminal failure of the negative CLI invocation, whereas the positive invocation must complete normally. No timeout is accepted as completed repair.
- [x] Complete source-scoped live C01–C16 evidence committed for current full run #27
- [x] Finite C08 repaired-path completion verified live and committed for production closure

## 7. Source-scoped qualification and release closure — reconciled 2026-09-08

### Current qualification

The current reviewed full result is selected by `qualifications/index.json.current_run`.
These fields mirror its committed `qualification-summary.json`; the source is
the actually executed commit, not the later evidence-import commit.

| Field | Recorded value |
|---|---|
| Run | `34140846679` |
| Source commit | `a9cdcdc1e0cad70e88b60869e75e4166046dd306` |
| Live date | `2026-09-07` |
| Codex | `codex-cli 0.153.4` |
| Model | `gpt-5.6-sol` |
| OS | `Debian GNU/Linux 13 (trixie)` |
| Full release gate | `true` |
| Finite C08 closure | `REPRODUCED` |

Full run #27 reproduced all C01–C16, including the finite C08 stop/repair
protocol without timeout. Its exact archive, summary and provenance are committed
under `qualifications/34140846679/`; current capability packages are exact
archive copies. C04 remains informational and fifteen capabilities gate release.

C08's missing-checkpoint invocation stopped as intended with exit 1. After
checkpoint repair, the positive invocation completed with exit 0; all nine
positive protocol checks passed. C09 completed three reconciliations and two
ordered automatic recovery cycles. Both independent C10 context-delivery probes
completed with exact recovery values. See `QUALIFICATION_STATUS.md` for details.

C10 delivers recovery through `SessionStart(source=compact)`; `PostCompact`
remains advisory. C13's real project-scoped agent and hook passed via the
explicitly allowed, known-error-gated non-ephemeral retry. Ephemeral custom-agent
spawning is not claimed as working. Qualification applies only to the recorded
Codex/model/OS configuration and does not guarantee future model behavior.

### Preserved history

Run #25 (`34060321283`, source
`d0384f76bc4150d33bb8f51ef5981f3243b3cfb3`) passed the older narrow assertions,
but its positive C08 trial timed out after proving unblocking. Its original
archive, source identifier, hashes and limitation remain unchanged. It is
historical evidence, not the current finite C08 publication proof.

The replacement C08 fixture isolates the deliberate invalid-checkpoint trigger
from ordinary startup recovery by narrowing SessionStart to `^compact$` in the
disposable root checkout before bootstrap. It does not change product hook
behavior. Full #27 verified this finite protocol live; no timeout is accepted
as completed repair.

### Publication and follow-up

The finite C08 evidence prerequisite is satisfied. Production publication still
requires strict validation on a clean tree, green protected CI, and a
GitHub-verified signed annotated tag under `RELEASE.md`. A documentation
reconciliation does not itself publish a release or require another live run
when the qualified product/runtime inputs remain unchanged.

[Issue #37](https://github.com/KeyffMS/PlanAnvil/issues/37) tracks intermittent
receipt observation in hosted C09 loopback conformance. It remains open; a
passing rerun does not establish the cause or fix the observer. The independent
full #27 evidence remains valid and immutable. C13's retained transport
limitation and its baseline 2.3 resolution are tracked in
[issue #17](https://github.com/KeyffMS/PlanAnvil/issues/17).
24 changes: 23 additions & 1 deletion docs/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ the full index, edit old actual observations, or label the denied automated laun
as tested. Validate archive integrity and production readiness on the resulting
clean tree through a protected PR before any signed publication.

The active main ruleset was verified on 2026-09-07: PR-only squash changes,
The active main ruleset was reverified on 2026-09-08: PR-only squash changes,
seven required status checks, strict up-to-date branch, conversation resolution,
no deletion or force push, no bypass actors. The protected distribution job now
also requires real-CLI conformance to succeed. No policy is loosened for closure.
Expand All @@ -84,3 +84,25 @@ the evidence-import commit, qualification preparation is complete; a signed
annotated production tag remains a separate authorized publication. This import
does not create a tag or release. No unsigned or lightweight tag may substitute
for the required verified signature.

## Remaining publication steps and follow-up

All changes intended for the first 0.2.0 publication are grouped under its
pending-publication section in `CHANGELOG.md`. `VERSION` remains 0.2.0;
qualification reconciliation alone does not change the distribution version.

Before tagging, merge the documentation reconciliation through protected CI,
run strict release validation on the resulting clean checkout, and use the
verified signed annotated-tag procedure above. The current qualification
metadata and C08 compliance checkbox are compared with the archived summary
and index by `tests/test_qualification_documentation.py` in PR and release CI.

The baseline 2.3 contract and full #27 evidence resolve the C13 qualification
question in [issue #17](https://github.com/KeyffMS/PlanAnvil/issues/17).
Ephemeral custom-agent spawning remains a documented compatibility limitation;
requalify it when the upstream runtime changes.

[Issue #37](https://github.com/KeyffMS/PlanAnvil/issues/37) remains open for
hosted C09 receipt-observer reliability. It does not invalidate the separate
full #27 proof. Future Codex/model/OS support needs scoped requalification.
Plugin packaging remains outside v1 under `IMPLEMENTATION_SPEC.md`.
54 changes: 54 additions & 0 deletions tests/test_qualification_documentation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
from __future__ import annotations

import json
import re
import unittest
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]


class QualificationDocumentationTests(unittest.TestCase):
def setUp(self):
self.index = json.loads((ROOT / 'qualifications/index.json').read_text(encoding='utf-8'))
folder = ROOT / 'qualifications' / self.index['current_run']
self.summary = json.loads((folder / 'qualification-summary.json').read_text(encoding='utf-8'))
self.compliance = (ROOT / 'docs/OPENAI_COMPLIANCE.md').read_text(encoding='utf-8')

def test_current_compliance_record_matches_archived_qualification(self):
# Scope to the current record so preserved historical runs cannot satisfy it.
section = self.compliance.split('### Current qualification\n', 1)[1]
section = section.split('\n### ', 1)[0]
rows = re.findall(r'^\| ([^|]+?) \| `([^`]+)` \|$', section, re.MULTILINE)
self.assertEqual(len(rows), len(dict(rows)), 'duplicate qualification fields')
self.assertEqual(dict(rows), {
'Run': self.index['current_run'],
'Source commit': self.summary['source_commit'],
'Live date': self.summary['date'],
'Codex': self.summary['codex_version'],
'Model': self.summary['model'],
'OS': self.summary['os'],
'Full release gate': json.dumps(self.summary['release_gate_passed']),
'Finite C08 closure': self.index['c08_closure'],
})
self.assertEqual(self.summary['github_actions_run'], self.index['current_run'])

def test_finite_c08_checklist_matches_committed_closure(self):
matches = re.findall(
r'^- \[([ x])\] Finite C08 repaired-path completion verified live and committed for production closure$',
self.compliance, re.MULTILINE,
)
self.assertEqual(len(matches), 1, 'finite C08 checklist must have one current entry')
self.assertEqual(matches[0] == 'x', self.index['c08_closure'] == 'REPRODUCED')

def test_current_version_notes_include_current_source_bound_result(self):
version = (ROOT / 'VERSION').read_text(encoding='utf-8').strip()
changelog = (ROOT / 'CHANGELOG.md').read_text(encoding='utf-8')
section = changelog.split(f'## [{version}]', 1)[1].split('\n## [', 1)[0]
for value in (self.index['current_run'], self.summary['source_commit']):
with self.subTest(value=value):
self.assertIn(f'`{value}`', section)


if __name__ == '__main__':
unittest.main()