feat: gate, packs, records, and offline development (v0.1.2)

[bearmug]

What

Grows jev-cli from a transport wrapper into a decision tool. Inference stays exit 0 — policy moves to jev gate, which is offline.

jev ask  --pack verify --state-file claim.json --state-format json --record decisions/claim-1.json
jev gate --input decisions/claim-1.json --pack verify; echo "exit $?"   # 0 2 3 4
jev replay --record decisions/claim-1.json

exit	decision	caller action
0	accept	proceed
2	review	escalate
3	deny	refuse
4	abstain	not enough information — never acceptance

deny and abstain are separate codes because remediation differs.

Why

Two external reviews of the 0.1.0 surface agreed on the same gap: the CLI returns probabilities, and every caller then hand-rolls thresholds. gate is that code, once, with documented semantics — fail-closed, and reproducible because it reads a saved answer rather than calling the model.

Changes

• jev gate — rules over noul/choice/score answers with accept_at/review_at bands, mode: all|any, optional rules, and per-rule reasons on stdout. A choice label outside accept never accepts regardless of confidence; what can soften a denial to review is probability mass on an accepted label. A missing or wrong-typed answer abstains. Policies are validated on load, so review_at > accept_at fails loudly instead of never firing.
• packs/ — verify (claim vs. cited evidence), screen (injection, harmful content, severity, plus substance/relevance for the caller; advisory, not a security boundary), route (deterministic/specialist/human/none + complexity). Each carries a state contract, an embedded policy, and a content hash that records and gate output name.
• Records and replay — --record writes model resolved, latency, pack hash, and hashes of state and questions. The state is never stored, so a record can sit next to a log without carrying confidential text. replay re-emits stored answers marked "replayed": true; it is not a rerun.
• jev doctor [--live] — engine, key presence (never the value), base URL, model, packs; --live adds auth, model list, probe latency, and cost.
• Version single-source — --version reads package.json instead of const VERSION = "0.1.0".
• Bug fix (cancellation) — SIGINT/SIGTERM were dropped: the signal was passed in client options, which ignore it, instead of request options. Affected every inference command.
• Bug fix (batch cost) — JSON batch rows bypassed formatOutput, so they lacked the cost block the README promises for every JSON response.
• Offline development — tools/stub-server.mjs answers deterministically and non-committally (last choice option, middle score, noul 0.5) so a stub run cannot look like an approval; tools/evaluate.mjs scores a policy against labeled records (accuracy, outcome mix, per-bucket acceptance rate). examples/ ships inputs and labels only — no committed model output.

Release unblock

Run 35279646306 signed provenance, then failed: 403 Forbidden — OIDC permission denied. Trusted-publisher connections created after 2026-09-03 are staging-only (npm stage publish is always allowed; direct npm publish is opt-in), which matches the error exactly. The workflow now:

• requires the release tag to equal the committed version instead of silently rewriting it from the tag,
• installs a staging-capable npm CLI (npm stage publish needs >= 11.15.0; 11.6.x reports "Unknown command: stage"),
• runs npm stage publish --access public and prints the stage id for 2FA approval.

v0.1.1 pointed at the same commit as v0.1.0 and never reached the registry; v0.1.2 supersedes it.

Verification

• 82 tests pass (npm test), up from 42: gate.test.ts covers every decision, both precedence modes, optional/missing/mismatched answers, lint rejections, record round-trips, and pack loading; contract.test.ts spawns the real CLI for exit codes 0/2/3/4, --version against package.json, batch cost blocks, and a doctor run that proves the key value never reaches stdout.
• Live runs against the real API: doctor --live (model jev-1.13.0, 856 ms model list, 923 ms probe) and the verify/screen packs — supports → exit 0, contradicts → 3, says_nothing → 3, and the sample injected page → injection 0.98 → exit 3 with severity in review.
• tools/evaluate.mjs over those live records: 3/3 label accuracy, 1 accept / 2 deny, and it skips records with no label.
• Offline stub run end to end: ask → record → gate → replay → evaluate, no key.

Notes

• Node >= 22 unchanged.
• No MCP surface — deliberately out of scope.
• Question design still belongs in the official skill; lint is structural only.

[bearmug]

Review response — Astra (slow role), every finding triaged. All blocking and important items are fixed in 61343f9; the two unverifiable CI claims are addressed below.

BLOCKING

1. Invalid choice probabilities could accept. Confirmed, and worse than stated: confidence describes the answer as a whole, not the chosen label, so substituting it was a category error. The fallback is gone. A choice answer must now carry a probability map whose entries are numbers in [0, 1] summing to 1 within 0.05; a missing, off-label, unnormalized, or invalid map abstains (exit 4). The test that asserted the old fallback now asserts abstention, and {"supports": 2}, an off-label map, and {0.7, 0.7} are each covered.

2. Out-of-range scores could authorize. Confirmed. A score is now checked against its scale — the rule's optional range: [min, max], else the level indices the answer reports in legend/probabilities (real responses carry both). -100 and 99 abstain; an answer with no derivable scale abstains rather than being compared to the bands.

3. Ambiguous record envelopes. Confirmed. extractResponse no longer falls back: anything carrying record_version or response is treated as a record and must validate as one. An unrecognized record_version, a non-object response, or missing created_at/model_resolved/latency_ms exits 1. replay uses the same validator, so a truncated record fails with a message instead of a TypeError.

IMPORTANT

4. Pack mismatch did not affect authorization. Fixed, split by what changed, because the two cases mean different things:

change since the record	behavior
questions changed	exit 1 — stored answers no longer mean what the current rules assume; re-run, or gate with --policy <file> to re-decide deliberately
thresholds changed only	current policy applies, with a stderr warning
identical	no note

questions_sha256 is now compared against the pack's question hash, so the two are distinguishable. Both the pack hash and the comparison print in table output as well as JSON, which also fixes the claim in finding 9.

5. State hash collisions. Fixed: the hash is now a versioned, type-tagged envelope, so text '{"a":1}' and JSON {"a":1}, "123" and 123, and JSON null and absent state all hash distinctly (test covers all six). README narrowed accordingly — it is an input commitment, not proof of what the model read.

6. Choice lint gaps. Both confirmed: review_at > accept_at on a choice rule now fails lint (including via the default review_at), repeated accepted labels are rejected, and evaluation dedupes the accept list regardless of provenance. Malformed range is rejected too.

7. Calibration mixed opposite outcomes. Confirmed — bucketing the chosen label's probability against "is the ground-truth label acceptable" mixed correct supports and correct contradicts into one bucket. The tool now buckets by the probability the policy treats as permission for the primary rule (accepted-label mass for choice, supportive probability for noul), which is the quantity accept_at thresholds. Score rules are excluded from the curve and say so. On the three live records this now reads 0.0 → 0% and 0.9 → 100% instead of a single mixed bucket.

8. Stub could claim a real model. Fixed: the stub never echoes the requested model — model_resolved is stub:<requested>, so a stub record is identifiable by the field that matters. The non-acceptance claim is now narrower and tested: a contract test starts the stub, runs ask --pack + gate --pack for all three packs through the real transport (no fetch fixture), and asserts exit 3 each time and a stub: model. I did not add a mechanism to stop custom policies from accepting stub answers; the docs now say the guarantee is about bundled policies, and that a stub must not be wired into a production path at all.

MINOR

All four fixed: --record creates missing parent directories before the request is sent, so a bad path fails before you pay; the route example writes out/route.json before gating it; the stub's accuracy against the six labels is stated as roughly 33% (two of six), not 17%; and table output now prints the pack hash and its comparison.

Publishing verification

You are right that the diff alone cannot establish the staging claims. Evidence, from the npm docs rather than the diff:

• Staged publishing: "Staged publishing requires npm CLI version 11.15.0 or later"; npm stage publish submits for review, npm stage approve <stage-id> releases with 2FA; the package must already exist on the registry (it does — 0.1.0 is published).
• Trusted publishers: "Configurations created after Sep 03, 2026 are automatically set to allow npm stage publish, and you can choose whether to also permit direct publishing with npm publish." That matches run 35279646306 exactly: provenance signed, then 403 — OIDC permission denied on direct publish.
• Generating provenance: "If you use trusted publishing, provenance attestations are automatically generated for your packages without requiring the --provenance flag."

Local check: the npm CLI here is 11.6.2 and reports Unknown command: "stage", which is why the workflow installs one. Per your note it now pins npm@^11.15.0 instead of @latest, so a release cannot jump a major. id-token: write and contents: read are unchanged from the 0.1.0 workflow and remain in the job. Approval and provenance on the approved artifact are the maintainer's step and will be reported on the release.

Verification

91 tests pass (up from 82): new coverage for malformed probability maps, out-of-scale scores, record envelope rejection, state-hash distinctness, pack question drift, threshold drift, and the stub-denial claim. Re-ran the live API afterwards — supports → 0, contradicts → 3, says_nothing → 3, injected page → 3 with injection 0.98 — unchanged decisions, now with pack provenance printed and a support curve that actually separates.