Skip to content

Refuse US releases that store model inputs the certified engine does not define (#1026) - #1031

Merged
MaxGhenis merged 21 commits into
mainfrom
us-release/stored-input-contract
Sep 26, 2026
Merged

MaxGhenis merged 21 commits into
mainfrom
us-release/stored-input-contract

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Closes the recurrence half of #1026 (decision d271). It does not re-cut any published release.

What this adds

policyengine-core loads a stored column as an input only when the engine defines a variable of that name. Before that loop it reads the structural columns that build the entities: person_id, each {group}_id and person_{group}_id (all variables of policyengine-us 2.2.1) and, when stored, the role column person_{group}_role or role (not variables). Every column that is not a variable is named in one logged warning and otherwise ignored (Simulation.build_from_dataset, read in policyengine-core 3.32.5). populace-us-2024-spm-20260915 and its reported-receipt child were both certified against policyengine-us 2.2.1. Both still store the WIC take-up draw as would_claim_wic, a name 2.2.1 does not define, so the draw was ignored.

microcosm.data.stored_inputs refuses a US release if a stored table has a column that meets all three conditions:

  1. it is model-named: [a-z][a-z0-9_]*, the engine's naming convention (measured below);
  2. it is not a variable of the policyengine-us version the release is certified against;
  3. it is not in the reviewed register US_STORED_NON_VARIABLE_COLUMNS ({column: reason}, 49 entries).

Each refusal is one line. The line quotes the column, names every table that stores it and the engine version, and gives both remedies: rename it to its live input (or stop storing it), or add a reviewed register entry with its reason.

Where it runs, and why there

The check needs the variable list of the engine the release is certified against. It runs at the three seams where that engine is provably the one loaded in the process, because the same process records it as build.built_with_model_package:

  • tools/build_us_fiscal_refresh_release.py: a batched pre-export gate. It grades the export frame's stored tables. A refusal joins the single batched pre-export raise, so every other failing gate is reported with it and no H5 is written. The exact-k ladder lane (tools/build_us_exact_k_ladder_release.py) runs this tool, so it passes the same gate. The gate works from a model of what the writer stores (_export_stored_tables): the frame's non-empty tables plus the materialized household_weight. Right after the write, _written_stored_input_verdict_mismatch grades the written file's HDF metadata, whatever the gate reached. If the gate evaluated, the written file must refuse exactly the columns the gate refused. If the gate could not evaluate (reachable only when evidence mode owns that failure line), the written file must refuse nothing, because no refusal of any column was reviewed. A written file it cannot grade (unlistable tables, or no engine) is reported too. Any of these aborts the run; this is a premise failure that evidence mode cannot convert. Evidence mode (--evidence-release) can record a stored-input refusal as an owned known failure, as it can for every batched gate.
  • run_native_loader_compatibility, the source-enrichment probe. The check runs before either loader. The probe runs in the tested runtime at --certify, and again whenever the contract replays it: validation, publisher --preflight-only, and publication. _check_compatibility already refuses a bundle whose built_with_model_package is not the runtime the probe tested. When the check refuses inside the probe, _check_compatibility now also reports that built-with mismatch, first, so a refusal graded against an engine the bundle was never certified with is not read as a verdict on it. Both affected releases went through this lane. The receipt gains a stored_inputs block, holding the register's sha256 and the registered non-variable columns the H5 stores, plus a country:defines_every_stored_model_input check label.
  • tools/build_us_acs_local_release.py, the ACS local-area package stage. It grades the calibrated H5's metadata before the release directory exists, and records the verdict as the stored_inputs gate, bound to the packaged bytes, in gate_summary.json and build_manifest.json. A refusal or an engine that cannot be imported stops packaging with nothing written.

Two other places were considered and not used:

  • contract.py's validate_release_dir checks calibration releases from JSON manifests and hashes only. It has no engine, and nothing ties the publishing environment to the built-with version.
  • tools/preflight_us_release_gates.py is advisory. It grades the base pool before the solve, not the export that ships.

Annual static-aging projections (microcosm.build.us_annual_static_aging) are built from a certified base release and are not re-checked here.

python -m microcosm.data.stored_inputs <h5>... prints a JSON verdict per file from HDF metadata alone. The five files below took 9.4 s together in one process, with a 1.0 GB peak RSS; most of that is the engine import.

Naming convention, measured (policyengine-us 2.2.1)

  • The engine defines 6,167 variables, and 6,114 match [a-z][a-z0-9_]*.
  • The other 53 are the postal codes of the 50 states, DC, PR and VI. Each is a household Boolean formula from in_state.py.
  • No variable name starts with _ or a digit.
  • The fiscal-refresh writer refuses to store formula-owned columns (a requires_us test confirms it refuses CA), and the ACS local-area lane holds every formula-owned column back (project_input_only, classified by PolicyEngineUSEngine.formula_owned_outputs).
  • So uppercase raw Census fields (A_AGE, H_TENURE, ...) pass by rule. The examined files store 163 to 189 uppercase columns each, 191 distinct; none is an engine variable and none is a state code. The only two-letter ones are the ACS PUMS fields ST and NP.
  • Tests pin the convention, the exact 53 codes and the formula ownership against the locked engine.

The H5 reader

h5_stored_tables reads each top-level pandas frame's column labels (table and fixed format). Two top-level keys are metadata, not tables: _time_period and the _populace_staging_metadata series that Microcosm's nullable writer (us_runtime.h5_io) adds, which the ACS local-area release and the multispine pools carry. USSingleYearDataset reads only the six entity tables and _time_period (read in policyengine-us 2.2.1). Each metadata key must be a pandas series, which has no columns. Anything else is refused rather than guessed: a values_block_* field, a missing or displaced index field, a non-frame object, a series under an unknown key, or a metadata key stored as a frame.

Only top-level objects are read. A frame pandas nests under an entity frame's group (person/extra) is not listed, and a test pins that. It suffices because policyengine-us 2.2.1 loads an H5 path through USSingleYearDataset, which reads only the six top-level entity frames and _time_period. USMultiYearDataset reads year-keyed frames (person/2024), but only when a caller builds it from a file explicitly; that layout leaves the top-level entity groups untyped, and a test pins that the reader refuses it.

Core's role columns. person_{group}_role and role are model-named, not variables, and read by core. The rule does not exempt them, so a file storing one would be refused by name: a fail-closed false refusal, never a silent pass. No examined file stores one; the only examined column ending in _role is the engine variable is_spm_independent_minor_role. test_the_check_refuses_core_role_columns_and_no_examined_file_stores_one pins both, and a requires_us test pins that none of the six names is a 2.2.1 variable.

The register and its evidence

Every entry is a model-named, non-variable column stored by at least one examined file. test_every_register_entry_is_stored_by_a_file_examined_for_1026 enforces this against tests/fixtures/stored_input_inventories.json, the stored column names of the five files below. test_only_the_acs_lane_spine_tags_rest_on_refused_files_alone requires every entry to be stored by one of the two files that pass (the rehearsal export and the stacked pool), except the six *_spine tags. Those are stored only by the ACS local-area release, which is itself refused for the two retired inputs; they stay registered on the strength of their live producer, base_pool.spine_column, which test_us_stored_input_register.py binds. test_every_register_entry_is_a_named_producer_column requires the register to equal exactly the union of the producer definitions below.

Entries Producer (bound by test) Why the engine may ignore it
{entity}_source_id, {entity}_support_channel, {entity}_support_clone_index, {entity}_spine_source_id for the 6 US entities (24) us_runtime.support_provenance.support_*_column and spine_source_id_column Support provenance, the four columns operator_boundary's support_provenance family declares. The source ids can key seeded draws (spec_engine.seeds, us_runtime.take_up). The spine source id is the raw id before assembly remaps collisions; puf_support requires it on a preassembled frame and the stacked-spine lineage receipts bind it.
{entity}_spine for the 6 US entities (6) us_runtime.base_pool.spine_column The base-spine tag (asec_puf or acs_2024_1yr) written when acs_multispine pools the ACS spine for the ACS local-area lane; spm_universe_source and acs_local_hours read it.
source_year, source_household_id, source_person_id, source_row_id (4) outer_stage_runtime._POOLED_SOURCE_PROVENANCE_COLUMNS, assigned by asec_pool and acs_pums Pooled source provenance. The first three enter the seed key grammar; source_row_id links a record back to its source row.
tax_unit_role_input, filing_status_input (2) microcosm.frame.units (microunit output) Build-internal predictors and filters (late-producer registry, imputation spec). Neither 1.764.6 nor 2.2.1 defines or reads them. There is no live input to rename them to: the engine derives filing_status and is_tax_unit_head / _spouse / _dependent by formula, and the writer refuses to store those.
puf_capital_gains_tail_* (6) us_runtime.puf_capital_gains_tail *_COLUMN constants Provenance of the PUF own-tail transfer, read by assert_puf_capital_gains_tail_survives_selection.
acs_social_security_income, acs_retirement_income, acs_interest_dividend_rental_income (3) operator_boundary._ACS_NATIVE_INPUT_CONTRACTS (the acs_ keys), mapped by acs_inputs ACS SSP, RETP and INTP times ADJINC. Each combines what the engine models as several inputs, so the ACS transfer reads it as a recipient-side predictor of those inputs (acs_transfer._RECIPIENT_COMBINED_SOURCES). A test binds each reason's input list to _DONOR_COMBINED_COMPONENTS, and a requires_us test checks each is an engine input.
acs_monthly_contract_rent, acs_monthly_gross_rent, acs_annual_property_tax (3) the same contract ACS RNTP, GRNTP and TAXAMT times ADJHSG: source lineage in the native-input receipt. The engine's housing inputs are filled elsewhere: pre_subsidy_rent by the ACS transfer and real_estate_taxes from TAXAMT on the reference person.
puma_geoid (1) us_runtime.acs_pums (source-bound by test) An explicit alias of the puma input with the same value; the engine reads puma, a household input in 2.2.1.

Two columns are deliberately not registered. Each is a retired engine input, not metadata:

  • would_claim_wic. policyengine-us 1.777.0 renamed it (its changelog entry for 1.777.0, 2026-07-21; the cached versions read agree: 1.775.8 defines only the old name, 1.779.4 only the new); 1.764.6 defines it and 2.2.1 does not. The live input is takes_up_wic_if_eligible, and the builder moved to it in 05d254a (the policyengine-us 1.819.0 bump).
  • medicare_part_b_premiums. The ASEC PEMCPREM transfer target, stored under a retired input name: the same defect class as would_claim_wic. Every policyengine-us version read from 1.452.0 to 1.670.2 (nine) defines it as a Person, YEAR, float input with no formula. In 1.670.2 health_insurance_premiums adds it, and spm_unit_medical_out_of_pocket_expenses subtracts it and adds the rules-based income_adjusted_part_b_premium in its place. By 1.690.7 it had been replaced by medicare_part_b_premiums_reported, the same Person, YEAR, float input under a new name. Every version read from 1.690.7 to 2.15.1 (91, 1.764.6 and 2.2.1 included) defines only the new name, and nothing in 1.690.7, 1.764.6, 2.2.1 or 2.15.1 reads it; the engine computes medicare_part_b_premium itself. Transfer targets must be engine input leaves: drop formula-owned weeks_worked, fix medicare premiums leaf, add ownership guard (#578 inc 2) #590 (f53032f) dropped the transfer from the build, and the rehearsal export no longer stores it, so the remedy is to stop storing it. Registering it would allow a model-named column that the engine ignores.

Evidence for these two, and for the two tax-unit columns, comes from reading source. For policyengine-us 1.764.6 that is the cached wheel, searched with grep and not imported; for 2.2.1 it is the installed package and its live system.variables. For the Part B history: the variables trees of all 100 policyengine-us versions unpacked in the local uv cache (1.452.0 to 2.15.1) were grepped for both class definitions. The 9 from 1.452.0 to 1.670.2 define only class medicare_part_b_premiums(Variable) (in household/expense/health/medicare_part_b_premiums.py), each as value_type = float, entity = Person, definition_period = YEAR with no formula, adds or subtracts; the 91 from 1.690.7 to 2.15.1 define only class medicare_part_b_premiums_reported(Variable), each with the same three attributes and no formula, adds or subtracts. Archives holding fewer than 500 variable files (partial unpacks) were excluded. The whole package of 1.452.0, 1.670.2, 1.690.7, 1.764.6, 2.2.1 and 2.15.1 was grepped for both names, which is how the consumers above were found. The cache holds no version between 1.670.2 and 1.690.7, so when inside that gap the rename landed is not known here, and no version before 1.452.0 was read.

Results on the examined files

File sha256 Verdict
Published default populace-us-2024-spm-20260915 (same bytes as -20260909), read from the local HF cache 6496cc43…aee84 Refused: medicare_part_b_premiums, would_claim_wic (person table). 377 stored columns; 24 registered non-variables.
Its reported-receipt child populace-us-2024-spm-receipts-20260923, the local certification candidate 00946972…89221c, equal to the published manifest's artifact hash Refused: the same two. 380 stored columns; 24 registered.
ACS local-area release populace-us-2024-buildo-acs-local-767312d60-20260923T074941Z, the local package artifact (built with 2.2.1, donor: the receipt child) 769756c3…b6a7ec, equal to the published manifest's artifact hash Refused: the same two. 418 stored columns; 37 registered.
Route A rehearsal R4 export, rebuilt by the R4 rehearsal script from run 8f63bf0's vaulted checkpoint and written with the fiscal-refresh tool's writer at 8f63bf0 (route-a-integration-20260923, per its rebuild receipt) 58304078…07f7c3f Passes. 385 stored columns; 30 registered.
Build Q stacked multispine pool us_input_pool_f010_s578.h5 (pipeline us-stacked-pool, built 2026-09-16), the kind of pool --base-h5 takes 7b626086…e55754, equal to its manifest Passes. 397 stored columns; 43 registered. Before this revision it refused 13 columns: the six *_spine_source_id, the six acs_* and puma_geoid.
BuildP parent populace-us-2024-buildp-sparse-rmloss100-cae8640-20260728T011454Z (extra check) 48b9d479…94c7e Refused: the same two.

The fixture's column listings match pandas' HDFStore storer metadata exactly, in names and order, on every file.

Consequence for the live lanes. Every pinned parent stores the two refused columns: BuildP for the role lane, the national default for the receipt lane, and the receipt child for the ACS local-area lane. So none of the three can certify or package a new release under 2.2.1 until its parent or donor is re-cut with the live WIC name and without the retired Part B column. Replaying the contract on the published default or the receipt bundle now refuses them, which is what d271 asked for. The runbooks say so. The stacked-pool lanes (fiscal refresh with a pool --base-h5, exact-k ladder) pass on the Build Q pool's columns.

Receipt shape. The probe's receipt gains a stored_inputs block and one check label, and _check_compatibility requires the stored receipt to equal the replayed one. So a source-enrichment receipt certified before this change no longer matches a replay: republishing such a bundle needs re-certification. Both existing source-enrichment bundles are refused by the check itself anyway. Consumers are unaffected: only the publisher paths (prepare_release, validate_release_dir) replay the probe.

Invariants, and how each is tested

Property tests use Hypothesis. They import it inside each test, as the repo already does, because the wheels job has no Hypothesis.

  1. Refusal, if and only if. For any stored tables, engine variables and register, the check refuses exactly when some column is model-named, not an engine variable and not registered. Its lines quote exactly those columns, one per line, sorted. Tested by test_property_refusal_if_and_only_if_and_the_error_names_exactly_those, 300 examples, checked against an independently written model-name oracle, plus example tests.
  2. Monotonicity. Adding an engine variable or a register entry only shrinks the refused set, so a pass stays a pass. Storing one more column only grows it. Tested by test_property_adding_a_variable_or_register_entry_never_refuses_more, which draws the engine and the register from the stored columns (as the refusal property does), so they overlap them. It usually kills a rule whose verdict on a registered column depends on the register's size (a reviewer measured 29 of 30 fresh Hypothesis databases); the refusal property always does.
  3. Register consistency. An entry is sound exactly when it is model-named, not an engine variable, and has a non-blank reason. An entry that is an engine variable is dead: dropping it changes no verdict. Tested by test_property_register_consistency_flags_exactly_the_dead_or_bare_entries. The shipped register is checked against the installed engine (requires_us) and engine-free.
  4. Determinism. The verdict does not depend on table order, column order or repeats. Tested by test_property_the_verdict_ignores_table_and_column_order_and_repeats.
  5. The metadata reader round-trips stored columns, and refuses what it cannot see. Round trip: test_property_h5_metadata_round_trips_the_stored_columns over random table and fixed layouts with either metadata series, plus a real pandas HDFStore example with both series and a test through the ACS lane's own writer. Refusal: test_property_h5_layouts_the_reader_cannot_see_are_refused takes any such layout and adds one defect anywhere (a values_block_* field at any position, a missing or displaced index field, a stray dataset, an untyped group, a series under an unknown key, a metadata key stored as a frame, a fixed frame with no column axis).
  6. The writer model matches the writer (differential, requires_us). A frame written through the real PolicyEngineUSEngine().write_dataset, including register columns and would_claim_wic, stores exactly the tables _export_stored_tables predicts, and the gate and the written bytes refuse the same column. In production the post-write check enforces the weaker property that matters for certification: the written file's refused columns equal the gate's, or are empty when the gate could not evaluate. A writer that adds or drops a registered or variable column does not trip it. Unit tests cover each direction: the written file refusing more than the gate, refusing less (the gate refused would_claim_wic and the written file stores none), refusing different columns, an unevaluated gate with a clean and with a stale file, and a file or engine the check cannot read.
  7. Regression. A synthetic stored table carrying would_claim_wic is refused by an engine that defines only takes_up_wic_if_eligible, and passes once the column is renamed. This holds for in-memory tables, a synthetic H5, the probe, the release tool's gate and the ACS local-area package stage (an H5 written by the lane's writer, refused with no release directory, then packaged once renamed).
  8. Wiring. Behaviour, through _main: in an otherwise green run, a stored-input refusal is the batched raise's only line and no H5 is written; a post-write premise failure aborts after the write, before any post-export scorer opens and before any manifest (the stored_input_refused and stored_input_premise modes of test_main_writes_diagnostics_before_post_calibration_gate_failure). Structure, by AST: the gate assignment and the join of its failures are unconditional statements of _main's own body before the batched if terminal_gate_failures: block; the write, the post-write check and its raise RuntimeError are consecutive statements before the scorer. The probe calls the check before USSingleYearDataset is constructed, observed with fake country and wrapper modules and pinned by AST together with the receipt block.

Mutation checks, run locally. Each mutant below was caught by the named tests and the file was restored.

  • Wiring: joining the gate's failures commented out, or wrapped in if False:; the post-write raise replaced by pass. Each fails both the AST test and the matching _main mode.
  • Reader: values_block_* searched only in the first two fields; non-frame top-level objects skipped instead of refused; metadata keys skipped without the series check. Each fails the refusal property on its own.
  • Rule (from the first revision): ignoring the register, accepting mixed case, accepting blank reasons, quoting table names. Each fails a property test.
  • Round 2: and column not in register replaced by and (column not in register or len(register) > 3) fails the refusal property deterministically and the monotonicity property in most seeds (a reviewer measured 29 of 30 fresh Hypothesis databases); the refusal property is the one that always catches it. The post-write equality weakened to set(refused) <= set(expected) fails test_the_post_write_check_reports_a_verdict_the_gate_did_not_reach (the gate refused would_claim_wic, the written file refuses nothing). An early return None when the gate did not evaluate fails test_the_post_write_check_grades_the_file_when_the_gate_could_not and the unevaluated case of test_the_post_write_check_reports_a_file_it_cannot_grade.

Round-2 review findings

  • P2, medicare_part_b_premiums was called a name no engine version defines. False: every version read from 1.452.0 to 1.670.2 defines it as an input, and every version read from 1.690.7 to 2.15.1 defines medicare_part_b_premiums_reported instead. Reclassified as a retired engine input, with the version range stated as the versions read (above), in the register comment in stored_inputs.py, the changelog, the README, both runbooks, two test comments and this body. The refusal message does not describe the column, so it is unchanged. A requires_us test now pins the replacement's shape in 2.2.1 (Person, YEAR, float, no formula).
  • P3, the monotonicity property rarely overlapped the register with the stored columns. It now draws both from them; the register-size mutant usually fails it, and the refusal property always does (above).
  • P3, the post-write equality was tested in one direction. Added the missing direction and the other cases (invariant 6); the subset mutant fails.
  • P3, the reader's docstring said every frame is listed "whatever its key". It now says only top-level objects are read and why that suffices; two tests pin the nested-frame and year-keyed behaviour.
  • P3, the module docstring said core ignores every other column. It now names the structural columns core reads, including the role columns, and says the check refuses a role column fail-closed and that no examined file stores one; a test pins that.
  • P3, evidence mode could ship an ungraded H5 when the gate could not evaluate. The written file is now graded whatever the gate reached (invariant 6).
  • P3, test_no_register_entry_rests_only_on_the_stale_releases counted the refused ACS local release as live. Replaced by test_only_the_acs_lane_spine_tags_rest_on_refused_files_alone (above).
  • Same class as the P2, found while fixing it. A comment in test_us_stored_input_register.py said no engine version defines the acs_ keys. It now cites the requires_us test that checks the installed engine.

Split into a separate follow-up, not in this PR (tracked in #1034): recording the fiscal-refresh verdict and register digest in the release manifest or gate_evidence (today it is in telemetry only); requiring the stored_inputs gate in _check_local_area_gates; and re-grading publisher compatibility-claim version ranges. On the last: the check grades only the policyengine-us version a release is certified against, so a compatibility claim that widens a source-enrichment release's range (up to the next major) is not re-graded for the other versions it admits.

Round-1 review findings

  • P2, register missing {entity}_spine_source_id; acs_* and puma_geoid unadjudicated. Registered all six spine source ids, the six ACS-native acs_* amounts and puma_geoid, each bound to its producer, with the Build Q pool added to the evidence. The pool lanes now pass (table above).
  • P2 / P3, the docs claimed every US release; the ACS local-area lane was unguarded. Wired the check into that lane's package stage, taught the reader the _populace_staging_metadata series, registered the lane's *_spine tags, and narrowed the changelog, README and agent guide to name the three guarded seams. The lane's 2026-09-23 release is refused for the same two columns as its donor.
  • P3, a refusal inside the probe hid the built-with runtime mismatch. _check_compatibility now reports the mismatch first.
  • P3, no test covered the post-write abort; the PR overstated what production enforces. Added the two _main modes above and corrected invariant 6.
  • P3, the wiring test only ordered substrings. Replaced by the AST statement test above.
  • P3, the reader's refusal half had example tests only. Added the refusal property.

Verification

  • Final wording commits 71934da65 and d35c43b0d change comments, the changelog and a fixture's role text only. An independent claims review approved 71934da65, and its P3s are fixed in d35c43b0d. The stored-input tests (92) pass at 71934da65; CI results on the latest head are on this PR.

Round 2 (the round-2 fixes, at eea3aabfd; engine env: policyengine-us 2.2.1, policyengine-core 3.32.5):

  • ruff check .: All checks passed!. ruff format --check on the 4 Python files this round touched: 4 files already formatted. python tools/ci_test_groups.py --verify: verification=ok (no test file added).
  • test_stored_inputs.py: 69 passed. test_us_stored_input_register.py: 23 passed. The requires_us subset of the two is 11 passed (81 deselected); peak RSS 2.8 GB for the two files together.
  • The engine-free (fast) lane was not rebuilt locally, because the machine is short on disk. Instead the same two files ran in the engine env with a pytest plugin that makes policyengine_us and policyengine_core unimportable and hides their distribution metadata: 81 passed, 11 skipped, every skip requires policyengine-us extra. CI ran both files at 0a460ced9 in the fast rest group and in the engine-us us-am and us-qs groups (all 24 checks green); eea3aabfd changes only comments and the changelog.
  • test_us_fiscal_refresh_builder.py: 315 passed, 2 skipped (both need MICROCOSM_US_CHRONICLE_FACTS), peak RSS 3.1 GB.
  • The three round-2 mutants under the mutation checks, each applied in place, run against the named tests, and restored with git checkout: the register-size rule fails the refusal property in every run and the monotonicity property in most; the subset form of the post-write equality fails the (clean file, gate refused would_claim_wic) case; the early return None fails test_the_post_write_check_grades_the_file_when_the_gate_could_not and the unevaluated case of test_the_post_write_check_reports_a_file_it_cannot_grade.
  • The Part B version history, read from engine source as described under the register. The mechanism claims in the two docstrings were read in source: policyengine-core 3.32.5's build_from_dataset reads person_{group}_role, else role, before its variable loop; policyengine-us 2.2.1 routes an HDFStore path through USSingleYearDataset, which reads the top-level person, household, tax_unit, spm_unit, family and marital_unit frames, and passes a USMultiYearDataset instance through unchanged.

Round 1 (before the round-2 fixes):

  • ruff check . (the workspace's ruff 0.15.16): All checks passed!
  • ruff format --check on the 8 touched Python files: 8 files already formatted
  • python tools/ci_test_groups.py --verify: verification=ok. No test file was added in this revision.
  • Engine env (uv sync --all-packages --extra us --locked; policyengine-us 2.2.1, policyengine-core 3.32.5): 30 test files, one pytest process per file, 2,097 passed, 9 skipped, 0 failed; peak RSS 5.2 GB (test_us_post_export_scoring.py). The 30 are every test file that names build_us_fiscal_refresh_release, source_enrichment, stored_inputs or build_us_acs_local_release (27, found with git grep), plus test_contract.py, test_publish_guard.py and test_us_release_gate_preflight.py. Among them: test_stored_inputs.py 66 passed; test_us_stored_input_register.py 17 passed; test_us_fiscal_refresh_builder.py 315 passed, 2 skipped; test_us_acs_local_release_tool.py 49 passed, 1 skipped (the pinned-feed test, which needs MICROCOSM_US_CHRONICLE_FACTS); test_source_enrichment.py 140 passed; test_contract.py 254 passed; test_us_multispine_pool_tool.py 188 passed.
  • Engine-free env (uv sync --all-packages --locked, the fast lane's environment): the same 30 files, 1,979 passed, 87 skipped (engine-gated), 0 failed; test_us_medicaid_take_up.py collects nothing without the engine. Peak RSS 1.5 GB. test_stored_inputs.py 58 passed, 8 skipped; test_us_stored_input_register.py 14 passed, 3 skipped; test_us_fiscal_refresh_builder.py 309 passed, 8 skipped.
  • The CLI on all five examined files at this revision: verdicts as in the table; 9.4 s and 1.0 GB peak RSS for the five together. The fixture equals each file's live metadata listing and pandas' storer metadata. The default, the R4 export, the receipt child, the ACS local-area artifact and the Build Q pool were rehashed locally to the sha256 values in the table.
  • policyengine-us 1.764.6, read from the cached wheel with grep: it defines would_claim_wic and medicare_part_b_premiums_reported, and none of medicare_part_b_premiums, tax_unit_role_input, filing_status_input, puma_geoid or the acs_* names. In 2.2.1, medicare_part_b_premiums_reported is defined and referenced nowhere else outside tests.

Not verified here

  • The real probe end to end. The policyengine wrapper package is not in the us extra, so neither this environment nor CI's engine lane can run run_native_loader_compatibility for real. Its use of the check is tested through fake modules and AST ordering. The check itself runs against the real engine.
  • Full release-tool, exact-k and ACS local-area runs. On this shared machine, the gates are tested on small frames, the real fiscal-refresh writer and the real ACS lane writer, not on full-population builds.
  • A stacked pool built at current main. The Build Q pool was built on 2026-09-16. A pool built since could store a new model-named column; the gate would refuse it by name.
  • Re-cutting the published default. Out of scope (Published US releases store would_claim_wic, which policyengine-us 2.x ignores, so WIC take-up is 100% under policyengine.py 6.1.1 #1026 option 1).
  • The source-enrichment and ACS local-area test files in round 2. This round changed only docstrings and comments in stored_inputs.py and did not touch those seams, so test_source_enrichment.py and test_us_acs_local_release_tool.py were not rerun; their round-1 results stand.

🤖 Generated with Claude Code

MaxGhenis and others added 20 commits September 25, 2026 17:40
…1026)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ster digest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y the written H5's verdict

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ories

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and writer model

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontract

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nt-guide note

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ut check test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e staging metadata series

Registers {entity}_spine_source_id, {entity}_spine, the six ACS-native acs_*
amounts and puma_geoid, each bound to its producer by test, and adds the
receipts child, the Build Q stacked pool and the 9/23 ACS local-area release
to the examined-file inventories. The metadata reader accepts
_populace_staging_metadata and _time_period only as pandas series, and a
Hypothesis property now covers the reader's refusal half.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The package stage records the installed policyengine-us as built-with, so it
now refuses a calibrated H5 that stores a model input that engine does not
define, before any release directory exists, and records the verdict as the
stored_inputs gate bound to the packaged bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the three seams in the docs

When the stored-input contract refuses inside the native-loader probe, the
compatibility check now also reports a built-with engine that differs from
the installed one, which a passing probe would have reported. The agent
guide, README and changelog name the three guarded seams instead of claiming
every US release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… structure

Two harness modes drive _main to each stored-input abort: a refusal is the
batched pre-export raise's only line and no H5 is written; a premise failure
aborts after the write and before the post-export scorer opens. The wiring
test now locates each statement in _main's own body, so a join that is
commented out, made conditional, or an abort that no longer raises fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…put module docstring

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ite direction; draw monotonicity from the stored pool; correct the Part B classification and the reader and core docstrings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…changelog, README and runbooks

policyengine-us 1.452.0 through at least 1.670.2 define it as a Person,
YEAR, float input; 1.690.7 had replaced it with
medicare_part_b_premiums_reported. The old wording said no engine version
defines it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ry engine version

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ous range

The changelog, the register comment and two test comments now say the old
name is an input in every policyengine-us version read from 1.452.0 to
1.670.2 (the nine in the local uv cache) and that every version read from
1.690.7 to 2.15.1 (91) defines only medicare_part_b_premiums_reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round-3 review P2/P3s on #1031, each verified against source:
- In policyengine-us 1.670.2, health_insurance_premiums adds
  medicare_part_b_premiums, but spm_unit_medical_out_of_pocket_expenses
  subtracts it and adds income_adjusted_part_b_premium in its place (it did
  not add it).
- policyengine-us 1.777.0 renamed would_claim_wic, not 2.x: the cached
  versions read bracket it (1.775.8 old name only, 1.779.4 new name only), and
  release_input_coverage.py records 1.777.0.
- The Route A rehearsal export was written by the Route A integration tool
  (8f63bf0 on route-a-integration-20260923, per its rebuild receipt), not by
  main's tools.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Final-delta review P3s on #1031:
- The Route A rehearsal export was rebuilt by the R4 rehearsal script from run
  8f63bf0's vaulted checkpoint and written with the fiscal-refresh tool's
  writer at 8f63bf0, not by a "Route A integration tool".
- The builder moved to takes_up_wic_if_eligible in 05d254a (the
  policyengine-us 1.819.0 bump); #746 was closed as not planned.
- The 1.777.0 rename is cited to the policyengine-us changelog entry
  (2026-07-21) as well as the cached-version bracket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit 2bce661 into main Sep 26, 2026
24 checks passed
@MaxGhenis
MaxGhenis deleted the us-release/stored-input-contract branch September 26, 2026 08:51
MaxGhenis added a commit that referenced this pull request Sep 26, 2026
Brings in the 19 main merges since 6442030: the Route A remediation stack
(#1016, #1018, #1017, #1025, #1024, #1028), #1029, #1005, #1008, #1004, #992,
#1031, #1015, #1033, #994, #954, #966, #1006 and #1010.

Eight conflicted paths, each recorded in docs/native-integration-20260923.md:
- test.yml keeps the native sharded matrix (main's --durations=25 is already
  in every native pytest call);
- the us_runtime facade stays lazy and gains main's three fiscal-target
  exclusion exports (the parent union, 920 names; the facade-union test is
  re-pinned to that union's digest);
- reform_validation.py keeps main's _released_engine_state and the native
  explicit-constructor default_simulate_factory;
- build_us_fiscal_refresh_release.py carries the native explicit consumer
  seams (formula metadata, dataset and microsimulation constructors, SPM
  selection) onto main's household-batched post-export scorer and batched
  base materialization; omitted seams keep main's exact calls, and _main
  supplies none. Three native test files that addressed the removed
  frame-based factory now address the scorer with the same assertions;
- identity pins observed on the merged tree: five EXPECTED_HASHES entries,
  the regenerated F0 coverage report (42,239/42,239 fields, 41/41 checks),
  the US spec identity 2dfa51b8... and the loader golden f2047cb9...

tools/generate_us_bundle_from_constants.py --check, tools/ci_test_groups.py
--verify, the CI matrix contract and ruff pass. The fiscal consumer,
formula-metadata, shared target/solve and calibration-attachment files pass;
the rest of the fiscal battery and the known pre-existing failures are
follow-up work on this branch.

No actual-data native run, release certification or publication.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant