Skip to content

feat: wire diagnostic costs into profile triage pairs - #309

Merged
acidkill merged 2 commits into
mainfrom
codex/vcr408-pilot-diagnostic-costs
Sep 30, 2026
Merged

acidkill merged 2 commits into
mainfrom
codex/vcr408-pilot-diagnostic-costs

Conversation

@acidkill

@acidkill acidkill commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

Completes the in-progress wiring of caller-verified diagnostic costs (VCR405, #307) into the paired-trial harness. Builds on an uncommitted work-in-progress edit that previously crashed ProfileTriageSpec construction and failed the local suite (8 errors). Also fixes a pre-existing argv-order divergence so rank-plus-observation triage commands are intercepted by the bridge instead of silently falling back to the real interpreter.

Change

  • Case profiles may declare optional triage.diagnostic_costs for supplied hypothesis IDs only (plain-string enum tokens, loader-validated).
  • Both arms receive the same cost disclosure; the treatment arm is told costs may guide next-check ordering only, are never causal-likelihood evidence, and never waive required checks.
  • ProfileTriageSpec gains a validated diagnostic_costs field (plain strings only; they are embedded verbatim in the generated shim).
  • command_for, the intercepted python shim, the bridge request contract, and _decide forward the costs end to end, so cost-bearing triage requests stay bridge-counted instead of silently falling back to the real CLI, and the tokens reach the production triage_failure decision state.
  • _triage_argv now places observation flags before --rank-hypotheses, matching command_for and the shim; previously the exact command for a rank-plus-observation profile was rejected by the shim's parser and fell back to the real interpreter, bypassing bridge call counting and receipts.
  • Profiles without costs keep identical prompts, commands, requests, and decision state.

Evidence (local, offline)

  • 10 new tests in tests/test_pilot_profile_diagnostic_costs.py: loader accept/reject, equal-arm disclosure, supervisor argv construction, spec validation, bridge request matching/forwarding, shim interception without fallback, and rank-plus-observation command order (verified to fail against the pre-fix ordering).
  • Updated existing doubles/request bodies for the new wire-contract key.
  • Full suite: 858 tests OK (43.1 s); git diff --check clean.
  • Ledger entries: VCR408 in TASKS.md and PILOT.md.

Local results only — the required hosted CI check must be green before merge. No trial was run; no efficacy, delivery, or release claim. No historical trial is rescored or rerun.

@acidkill
acidkill merged commit ed4a34b into main Sep 30, 2026
1 check passed
@acidkill
acidkill deleted the codex/vcr408-pilot-diagnostic-costs branch September 30, 2026 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant