Add IRS SOI Historic Table 2 TY2023 state AGI-band facts

[MaxGhenis]

Summary

Adds IRS SOI Historic Table 2 for tax year 2023 at state level: for the 50 states and DC, the number of returns (N1) and adjusted gross income (A00100) in each of the ten AGI classes. The package emits 1,020 facts, and every one is a single published cell.

The existing TY2022 package sums the $500,000-$1,000,000 and $1,000,000-or-more classes into one 500k_plus row. In this package they are separate facts (500k_to_1m, 1m_plus), so a consumer can see a state's returns and AGI at $1M+. For example, Colorado TY2023 has 15,610 returns with $47,266,883,000 of AGI at $1M+.

Consumer context: PolicyEngine/microcosm#940. State totals match SOI while a state's income above $1M does not. Microcosm needs these rows to bind state top-tail targets. Facts are inert until Microcosm re-pins its feed.

What changes

• db/data/irs_soi/historic_table_2/23in55cmcsv.csv: the publisher file from https://www.irs.gov/pub/irs-soi/23in55cmcsv.csv (Historic Table 2 page, updated 21 August 2026). It is 757,078 bytes with SHA-256 d1f7c8901fcefb2c46f5dd14f22715b7c8513794ce7b9ea6a6a947f1c548668f.
  • Registered with chronicle fetch-artifact --upload-r2, which added only the 2023: entry to the shared manifest.
  • Read back from ledger-raw at the recorded key and hash-verified. The same object is also in chronicle-raw at the same key, also read back and verified, matching how the TY2022 file is mirrored there.
  • The file uses CRLF line endings and is listed with -diff in .gitattributes. The committed blob hashes to the publisher sha.
• packages/irs_soi/historic_table_2_state_agi_2023/source_package.yaml: one record set per state, irs_soi.ty2023.historic_table_2.state_agi.<st>.
  • The record set layout, source_table and record_set_spec_id are the same as TY2022, so the two vintages are one series.
  • It has ten rows (AGI stubs 1-10) and two measures, return_count and adjusted_gross_income. The AGI measure keeps the TY2022 concept alignment (IRC §62, exact, ledger-us).
  • Labels are literal TY2023 throughout: period, ids, vintage and legal_vintage. There is no {year}, so building at any year yields the same TY2023 facts, and the package is already clean under the pending artifact-year restamp guard.
  • Both measures guard their column header (N1, A00100), because the TY2023 file inserts two columns after column DP (N07262 ... A07265 in place of N07260, A07260). The columns this package reads (C, T) did not move.
  • I generated the YAML by a deterministic transform of the TY2022 package. It keeps the same selected_rows, row numbers and guards, splits the summed top row into two single-cell rows, and uses literal-year labels. The same PyYAML dump settings reproduce the TY2022 file's body byte for byte; the header label maps are written as text, as in TY2022.
• soi-historic-table-2-state-agi-2023 is registered in SOURCE_PACKAGE_ALIASES, in the irs_soi_filer_income_tax_credits family (chronicle/targets/us_poverty.py), and in docs/pe-calibration-targets.md.

The TY2022 package is unchanged. Its taxable-interest rows are what Microcosm binds today. An unpushed restamp-guard branch edits that file, so I did not touch it. This package has no taxable-interest measures, because a consumer that binds state interest needs the matching TY2023 national Historic Table 2 total, which is not packaged yet. Those measures should arrive together with that package.

Chronicle governance

• Approved agent role: ledger-source-ingestor.
• Deterministic checks:
  • chronicle validate-package ... --year 2023: valid. It reports 51 record sets, 510 rows, 102 measures, 1,020 source records and 51 regions.
  • chronicle build-bundle --year 2023 --source packages/irs_soi/historic_table_2_state_agi_2023: valid. It reports 1,020 facts, lineage coverage 1.0, 84,315 source cells, 594 source rows and 0 agent-acceptance errors.
  • Independent oracle: I parsed every exported consumer fact against the CSV with the standard library. 1,020 of 1,020 values equal cell x scale.
  • Full default bundle, build-bundle --year 2023: valid. It has 351,913 facts (+1,020), 228 packages (+1), and irs_soi 41,083 (+1,020). Semantic duplicate keys (467), warnings (76) and source tables (222) are unchanged, so no new fact shares a semantic key with an existing one.
• LLM judge verdicts:
  • ledger-source-fidelity: PASS (independent Opus 5.5 review via subfleet, read-only). It re-read all 510 selectors, the row headers and state guards, the 51 identical measure blocks, the band constraints (51 of each), the CSV header columns and the Colorado and California cells. That reviewer had no shell, so the sha, the build and the full per-fact comparison come from my runs listed under the deterministic checks.
  • ledger-contract: no schema or consumer-contract change.
  • ledger-boundary: PASS (same review). The change is a pinned publisher artifact, a declarative one-cell-per-fact package and alias/coverage/doc entries. It contains no reconciliation, aging, imputation, activation or solver logic, and the TY2022 package is untouched.

Review findings and responses

• The per-fact oracle did not tie geography, filters and bounds to the cell. Fixed in a6cc399. Each fact now also checks its geography id against an independent FIPS table, its income_range filter, its AGI bounds and its source column.
• Paths outside ledger-source-ingestor's allowed_paths (db/data, .gitattributes, the alias maps and the docs row). Every package PR touches these; Add HMRC CGT Table 4 BADR and Investors’ Relief facts #288, Add HMRC CGT Tables 7–9 source packages #273 and Add TY2020 IRS SOI Table 1.4 facts (aging-chain bridge for TY2020 dollar targets) #113 did the same. Widening the role file belongs in a separate governance change.
• Same record_set_spec_id as TY2022 with a different band set. Kept on purpose. The spec id names the Historic Table 2 state AGI table, so TY2022 and TY2023 facts stay one series (source_series_key), and Add SOI state AGI taxable interest facts #62 changed these record sets' measures without a bump. A vintage's band set is what its package publishes. Consumers should select bands by their constraints, as Microcosm does.
• -diff does not stop line-ending conversion. This follows the file's existing convention for pinned CRLF artifacts. No text attribute or autocrlf applies here (git check-attr), and every build re-verifies the sha, so a converted copy would fail loudly.
• TY2022 legal_vintage: tax_year_{year} relabels at other build years. This predates the PR and the unpushed restamp-guard branch fixes it. This package is literal-year.
• No taxable-interest measures in TY2023. On purpose; see "What changes".

Invariants (tested in tests/test_chronicle_soi_state_agi_2023.py)

1. One cell per fact. Every fact equals its named CSV cell x value_scale, read with Python's csv module rather than Chronicle's parser, and has exactly one source row.
2. Partition. For every state and measure, the ten bands are ordered, contiguous and non-overlapping, and cover (-inf, inf).
3. Additivity against the publisher's total. AGI bands sum exactly to the state's stub-0 AGI. Return counts, which the publisher rounds to tens, sum to stub 0 within ±55 (eleven rounded cells) and are all multiples of 10.
4. Pinned vintage. Building at 2022 or 2024 yields the same facts as 2023, with every fact at TY2023 and on the registered sha.
5. Parallel ids. With the tax year removed and 500k_to_1m mapped to 500k_plus, the TY2023 ids of rows below $1M equal the TY2022 package's return-count and AGI ids. Consumers match vintages on these ids.

Tests

• New tests/test_chronicle_soi_state_agi_2023.py (9 tests), plus the TY2023 counts in test_national_soi_source_package_aliases_validate_fixture_counts.
• tests/test_chronicle_bundle.py default-bundle pins re-measured with the build above.
• uv run pytest -q tests/test_chronicle_source_package.py tests/test_us_poverty_target_coverage.py tests/test_source_package_alias_drift.py tests/test_chronicle_pe_source_plan.py tests/test_chronicle_soi_state_agi_2023.py: 156 passed. The default-bundle test is covered by the full build-bundle --year 2023 run above, whose counts it pins; CI runs it.
• ruff check chronicle policyengine_chronicle db scripts tests passes.

🤖 Generated with Claude Code