Skip to content

cics-crucible: mutate the four PC* ports; strengthened scenarios for pc-wizard and pc-aid-menu (Part of #4443) - #5392

Merged
squid-protocol merged 2 commits into
mainfrom
churn/4443-20261017T2331Z
Oct 18, 2026
Merged

squid-protocol merged 2 commits into
mainfrom
churn/4443-20261017T2331Z

Conversation

@squid-protocol

@squid-protocol squid-protocol commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Part of #4443 (item 2). Item 1 merged as #5318.

What changed

  • Mutation run on PCDETL, PCMENU (pc-aid-menu) and PCCONF, PCWIZ (pc-wizard): seed 0, 24 mutants each, strengthened scenarios on, java-ported and java-facade sides. Killed/survived: PCDETL 15/9, PCMENU 16/8, PCCONF 15/8, PCWIZ 16/7 (PCCONF and PCWIZ one stillborn each).
  • 36 first-run survivors, 6 of them case gaps. Four new strengthened scenarios (expected logs derived from the COBOL stub with --derive-expected, never by hand) kill 4: enter-sends-no-data (PCMENU), amount-not-numeric, name-erased, amount-erased (PCCONF/PCWIZ). back-xctl-fails drives the failing XCTL; its target mutant is equivalent. The other two case gaps were re-triaged (below).
  • Result: 0 case gaps, 0 harness gaps; 23 equivalent, 9 unreachable; adjusted score 100% for all four.
  • test_proof_reach.py: the four PC* names leave the stale set (the four PCIMM* stay), flagged survivors 5 to 6 (one more unreachable), exact counts in a new test.
  • Regenerated: mutation_scores.json, the table in mutation_testing.md, the four evidence records (evidence.py prove).
  • Hand-written files (4, within the 5-file limit): the two case.json, test_proof_reach.py, mutation_testing.md prose.

Class table: crucible ports with a recorded mutation run

Class: every CICS-crucible port must have a current mutation run, all survivors triaged, no case gap.

Member Status Evidence
PCDETL handled (this PR) 15/9, 9 equivalent
PCMENU handled (this PR) 16/8, 6 equivalent, 2 unreachable; enter-sends-no-data kills the MAPFAIL mutant
PCCONF handled (this PR) 15/8, 4 equivalent, 4 unreachable; 2 case gaps killed
PCWIZ handled (this PR) 16/7, 4 equivalent, 3 unreachable; 1 case gap killed
CALINK, CASUB, CAXA, GTSTART, GTWORK, GTSHOW, GTTERM, HCMAIN, HCSUB, HCQREAD, HXATTR, HXEXT handled (#5318 and earlier) test_the_re_run_crucible_ports_keep_their_measured_floor
CAXB partial 4 case gaps remain
PCIMMA, PCIMMB, PCIMML, PCIMMS missing recorded run is stale (still in the proof_reach stale set)

Finding

A cold start of PCCONF (PC03 typed at the terminal, EIBCALEN 0) diverges: the COBOL stub records no events, the port posts to PCLEDGER. DFHCOMMAREA is unaddressable on IBM there, so it is not a valid proof input; I left the scenario out and triaged the mutant it would have judged (79fe0ecaad) unreachable with that reason. Not fixed here.

Sibling gaps

Member Status Why not in this PR Source
PCIMMA/B/L/S stale mutation run needs its own re-run and triage (different mechanism: ports changed with #4706) test_proof_reach.py stale set
CAXB 4 case gaps needs new scenarios for another case (ca-xctl-versions) mutation_scores.json
PCCONF cold start (EIBCALEN 0) port and COBOL stub disagree needs an IBM-semantics decision, not a proof input /tmp repro: scenario PC03 at terminal; baseline report.json

Remaining slices (#4443)

Item 3 (spot-check of the 35 equivalent verdicts); carddemo-acctview and carddemo-menu re-run; READ INTO LENGTH to LENGERR scenario (waits on the next cics-crucible release).

Gates

pr_gates.py --fast 6/6, --ratchets 8/8; pytest on proof_reach, test_evidence, tests/cics_crucible and the pre-PR set passes. After merging origin/main the four PC* records are stale on the generator only, like every other record (the scheduled job re-proves).

Cross-repo

None.

Lessons for agents

  • Crucible triage keys carry the crucible: prefix: crucible:<case>/<PROG>:<id>.
  • A strengthened file changes the case hash: re-run evidence.py prove for that case's ports.
  • A new scenario that fails the baseline proof (the committed port is not proven) means port and COBOL disagree on that input: read baseline/report.json before dropping it.
  • A throw inserted before a mutated line, then re-proved, tells reachable from unreachable.
  • A Monitor using pgrep -f can match itself; watch log-line counts instead.

🤖 Generated with Claude Code

squid-protocol and others added 2 commits October 17, 2026 19:57
…pc-wizard and pc-aid-menu (Part of #4443)

Mutation run (seed 0, 24 each, strengthened on): PCDETL 15/9, PCMENU 16/8, PCCONF 15/8, PCWIZ 16/7
(killed/survived). 36 first-run survivors, 6 case gaps; four new scenarios (expected logs derived from the
COBOL) kill 4, the other two are equivalent / unreachable. 0 case gaps, 0 harness gaps remain.
The four PC* names leave the proof_reach stale set; exact counts added. Evidence re-proven.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@squid-protocol squid-protocol added the shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md label Oct 18, 2026
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

1 similar comment
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

@squid-protocol
squid-protocol marked this pull request as ready for review October 18, 2026 00:04
@squid-protocol
squid-protocol enabled auto-merge (squash) October 18, 2026 00:05
@squid-protocol squid-protocol added shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md and removed shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md labels Oct 18, 2026
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

@squid-protocol
squid-protocol merged commit 7470bef into main Oct 18, 2026
53 checks passed
@squid-protocol
squid-protocol deleted the churn/4443-20261017T2331Z branch October 18, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant