Skip to content

Compare code readiness against the catalog checkpoint - #13

Merged
exploitintel merged 1 commit into
mainfrom
fix/catalog-checkpoint-readiness
Sep 26, 2026
Merged

exploitintel merged 1 commit into
mainfrom
fix/catalog-checkpoint-readiness

Conversation

@exploitintel

Copy link
Copy Markdown
Owner

Readiness incorrectly compared the code index with the whole-corpus checkpoint, reporting possible missing code after enrichment alone advanced that checkpoint. Compare the index with the API's code-search catalog checkpoint, show both identities and preserve the API's subsystem status. Missing comparison inputs do not imply either a match or a mismatch.

Tests cover matching catalogs with differing corpus revisions, actual catalog mismatches, missing fields and comparisons before display truncation. Prepare package version 3.0.2 for the required release artifact; this PR does not tag or publish a release.

Validation: Ruff, formatting and mypy passed; 1,973 tests passed, 232 skipped, 97.34% coverage. Wheel and sdist build and twine validation passed. Three independent review lenses found no significant issues. The deployed formatter will be checked against live API checkpoint identities after release and deployment.

@claude

claude Bot commented Sep 26, 2026

Copy link
Copy Markdown

Reviewed this PR against the corpus-injection / derived-judgment / fail-open checklist.

Scope: format_system.py/format_common.py checkpoint-comparison rewrite, docs update, version bump, and tests. No corpus text, PoC tokens, HTTP transport, or write paths are touched.

Checked in detail:

  • Comparison correctness: the new gate (format_system.py format_readiness) only renders INDEX_CATALOG_MISMATCH when both code_search_checkpoint_sha256 and code_search_catalog_checkpoint_sha256 are non-empty, and compares the raw dict values rather than the inline()-truncated display strings — so two checkpoints that only differ past the 88-char display ceiling still get flagged (covered by test_readiness_compares_full_checkpoint_values_before_display_truncation). Missing either field skips the comparison entirely rather than defaulting to match or mismatch, per the stated requirement.
  • No derived judgment: the new message ("differs from the API's code-search catalog checkpoint... not aligned with the current catalog") is a direct restatement of an API-returned inequality, not a synthesized score/ranking, and drops the old "makes no claim about which is newer" line only because the new message no longer implies a direction either.
  • Subsystem status preserved: code_search_status is rendered verbatim, unaffected by the checkpoint comparison change; the existing fail-closed notice for a non-ready status is untouched.
  • No injection surface added: both checkpoint values still go through inline() for display.

One non-blocking note: tests/fixtures/readiness.json (recorded from a live response) predates the code_search_catalog_checkpoint_sha256 field, so the readiness fixture used by test_format_poc.py/test_tools.py never exercises the new comparison path. Not a defect in this PR — those tests target other fields — but worth refreshing that fixture from a live capture in a follow-up so it stops silently skipping the new code path.

No findings that meet the bar for inline comments.

@exploitintel

Copy link
Copy Markdown
Owner Author

Review disposition: no significant findings from the independent reviews or CI review. The historical readiness fixture does omit the catalog checkpoint, as noted; refreshing that general fixture is real but outside this targeted correction. The new tests explicitly exercise catalog/index equality, inequality, absent fields, API failure status and full-value comparison before display truncation. The unchanged fixture additionally retains missing-field coverage. All quality and package checks passed.

@exploitintel
exploitintel merged commit 7d8ec2d into main Sep 26, 2026
4 checks passed
@exploitintel
exploitintel deleted the fix/catalog-checkpoint-readiness branch September 26, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant