Skip to content

Model capital rules for income-based JSA and income-related ESA#1580

Merged
MaxGhenis merged 8 commits intomainfrom
codex/fix-1575-esa-jsa-capital-rules
Apr 12, 2026
Merged

Model capital rules for income-based JSA and income-related ESA#1580
MaxGhenis merged 8 commits intomainfrom
codex/fix-1575-esa-jsa-capital-rules

Conversation

@MaxGhenis
Copy link
Copy Markdown
Collaborator

@MaxGhenis MaxGhenis commented Apr 11, 2026

Summary

  • add legislation-backed capital parameter trees for income-based JSA and income-related ESA
  • screen reported JSA/ESA awards through assessable-capital, tariff-income, and upper-threshold logic
  • add focused YAML regressions for thresholds, tariff rounding, source inclusion, claimant allocation, the public esa aggregate, and the existing ESA-to-IS interaction

Scope

This is a bounded consistency pass on reported income-based JSA / income-related ESA awards. It does not attempt to build a full first-principles entitlement model for either legacy benefit.

2026 impact on current calibrated weights

Compared with current main on enhanced_frs_2023_24:

  • jsa_income: £125.2m -> £93.8m (-25.0%), recipients 26.1k -> 22.2k (-15.1%)
  • esa_income: £3.13bn -> £2.25bn (-28.0%), recipients 372.9k -> 266.4k (-28.5%)
  • income_support: unchanged in aggregate on current weights
  • housing_benefit: +£18.1m (+0.12%), +0.8k recipients (+0.05%)

Closes #1575.

@MaxGhenis
Copy link
Copy Markdown
Collaborator Author

Update on calibration impact after a fairer admin-only rerun.

The earlier optimistic ~18% national mean absolute relative error result was not a fair apples-to-apples comparison. I re-ran this as a matched admin-only benchmark using the same expanded target set, the same constituency calibration harness, and 5 matched random seeds for:

  • current main
  • all_assets = UC + HB + IS + PC + ESA/JSA asset-rule bundle

Post-constituency-recalibration summary across 5 seeds:

Model National mean abs rel error National median abs rel error National within 10% National within 20% Constituency-local median abs rel error Constituency-local within 10%
current_main 69.0% median (68.4% to 69.2%) 18.4% 36.4% 52.2% 5.65% 89.9%
all_assets 61.6% median (61.3% to 61.9%) 18.7% 36.6% 50.9% 5.71% 89.4%

So the stable result is not “clear across-the-board improvement”. The full asset-rule bundle does consistently improve national mean error by about 7.1pp, but it is roughly flat to slightly worse on the other national metrics and on constituency-local fit.

This makes the next step pretty clear: target-family audit before making strong claims from aggregate loss. We already found one real mapping issue on JSA, and the current objective seems able to trade off target families in ways that are not obvious from one top-line metric.

@MaxGhenis
Copy link
Copy Markdown
Collaborator Author

Corrected calibration note after rerunning this fairly against current main on the same expanded target set and over matched random seeds.

Using 5 fresh constituency-recalibration runs per model:

  • current_main fresh constituency recalibration: national mean absolute relative error median 69.0% (range 68.4% to 69.2%)
  • all_assets fresh constituency recalibration: national mean absolute relative error median 61.6% (range 61.3% to 61.9%)
  • stable delta on national mean error: about -7.1pp in favour of the full asset-rule package

But this is not a clean across-the-board win:

  • national median absolute relative error: 18.4% -> 18.7% (slightly worse)
  • national share of targets within 10%: 36.4% -> 36.6% (slightly better)
  • national share of targets within 20%: 52.2% -> 50.9% (worse)
  • constituency-local median absolute relative error: 5.65% -> 5.71% (slightly worse)
  • constituency-local share within 10%: 89.9% -> 89.4% (slightly worse)

So the corrected read is: this PR bundle improves national mean admin-target error, but the rest of the calibration picture is mixed rather than uniformly better.

Also, the calibrator is stochastic. The earlier single-run ~18% national-error result was not reproducible and should be treated as invalid.

@MaxGhenis MaxGhenis merged commit a0e92cb into main Apr 12, 2026
2 checks passed
@MaxGhenis MaxGhenis deleted the codex/fix-1575-esa-jsa-capital-rules branch April 12, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Properly model capital rules for income-based JSA and income-related ESA

1 participant