Skip to content

Populate CPS inputs for SPM childcare formula#705

Draft
MaxGhenis wants to merge 9 commits intomainfrom
codex/childcare-spm-cap-formula
Draft

Populate CPS inputs for SPM childcare formula#705
MaxGhenis wants to merge 9 commits intomainfrom
codex/childcare-spm-cap-formula

Conversation

@MaxGhenis
Copy link
Copy Markdown
Contributor

@MaxGhenis MaxGhenis commented Apr 9, 2026

Summary

  • carry CPS WKSWORK through the microdata build as weeks_worked
  • stop rebuilding spm_unit_capped_work_childcare_expenses inside policyengine-us-data
  • leave capped childcare to the model formula in policyengine-us and add regression coverage for the new input plumbing

Validation

  • uv run pytest -q tests/unit/test_extended_cps.py tests/unit/test_weeks_worked.py
  • uv run ruff check policyengine_us_data/datasets/cps/extended_cps.py tests/unit/test_extended_cps.py tests/unit/test_weeks_worked.py
  • git diff --check

Notes

  • On raw CPS 2024, SPM_WKXPNS is highly reproducible from other CPS inputs: MAE is about $12, 97.9% of units are within $1, and 99.1% are within $5.
  • SPM_CAPWKCCXPNS is not reproducible nearly as cleanly from current public CPS inputs, so this PR intentionally stops short of reconstructing the capped value in us-data.
  • Paired model PR: Add Census SPM work expense formulas policyengine-us#7960.

@MaxGhenis MaxGhenis changed the title Use Census childcare capping formula Populate CPS inputs for SPM childcare formula Apr 9, 2026
Copy link
Copy Markdown
Collaborator

@baogorek baogorek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize this is an early review and we're waiting on policyengine-us#7960, but I wanted to get some thoughts in here. Interesting that the tests are failing on state-level calibration of aca. (I need to add that to my scorecard.)

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 10, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
pipeline-diagrams Error Error Apr 11, 2026 0:57am

Request Review

@MaxGhenis
Copy link
Copy Markdown
Contributor Author

Addressed the review comments in 27169f8:

  • Replaced the brittle source-text tests with behavioral tests that call the CPS extraction functions.
  • Added a named PERRP unmarried-partner code mapping with Census CPS ASEC 2024 technical documentation cited in code.
  • Kept the WKSWORK test focused on generated weeks_worked output and clipping behavior.

Local checks:

  • uv run pytest -q tests/unit/test_reference_partner.py tests/unit/test_weeks_worked.py
  • uv run pytest -q tests/unit/test_extended_cps.py
  • ruff check tests/unit/test_reference_partner.py tests/unit/test_weeks_worked.py

Note: ruff check policyengine_us_data/datasets/cps/cps.py still reports pre-existing lint debt in that module (star import/unused locals), not introduced by this patch.

@MaxGhenis
Copy link
Copy Markdown
Contributor Author

Follow-up for the unit-test failure on the new run: 89b58ac extracts derive_weeks_worked and tests that helper directly, while production still assigns cps["weeks_worked"] = derive_weeks_worked(person.WKSWORK).

The CI failure came from the test calling the full add_personal_income_variables path after package-spec state had been changed elsewhere in the full unit suite. Local full unit verification now passes with CI-style env:

  • HUGGING_FACE_TOKEN=dummy uv run pytest -q tests/unit -> 589 passed, 9 skipped

@MaxGhenis
Copy link
Copy Markdown
Contributor Author

Integration follow-up: the Modal run failed in tests/integration/test_cps.py::test_add_personal_variables_maps_current_health_coverage_flags because that synthetic fixture does not include PERRP.

I pushed a narrow fix in this commit:

  • default missing PERRP to all-false when building is_unmarried_partner_of_household_head
  • added a unit test covering the missing-column fallback

Local verification:

  • uv run pytest -q tests/unit/test_reference_partner.py tests/integration/test_cps.py -k 'missing_perrp_defaults_to_false or add_personal_variables_maps_current_health_coverage_flags'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants