Skip to content

Add Stage 2 package payload reader#1075

Draft
anth-volk wants to merge 1 commit into
mainfrom
agent/stage-2/pr-2c-payload-reader-writer
Draft

Add Stage 2 package payload reader#1075
anth-volk wants to merge 1 commit into
mainfrom
agent/stage-2/pr-2c-payload-reader-writer

Conversation

@anth-volk
Copy link
Copy Markdown
Collaborator

@anth-volk anth-volk commented May 20, 2026

Fixes #1073

Summary

  • Add CalibrationPackagePayload, reader, and writer helpers for typed access to calibration_package.pkl while preserving legacy dictionary pickle compatibility.
  • Route Stage 2 package summaries, geography summaries, persisted contract validation, and remote metadata sidecar writing through the typed payload boundary.
  • Expose package payload read/write/checksum/sidecar failures as canonical ValidationReport records through CalibrationPackagePayloadError.
  • Document the payload reader/writer nodes in the pipeline map and regenerated pipeline docs, with focused unit coverage for round trips, required keys, structured failures, legacy missing geography, summary parity, and remote sidecar generation.

Validation

  • uv run --no-sync --with pyyaml python scripts/extract_pipeline_docs.py
  • ruff check policyengine_us_data/calibration_package/__init__.py policyengine_us_data/calibration_package/payload.py tests/unit/calibration_package/test_payload.py tests/unit/test_calibration_package_stage_contract.py tests/unit/test_pipeline_docs_extractor.py
  • ruff format --check policyengine_us_data/calibration_package/__init__.py policyengine_us_data/calibration_package/payload.py tests/unit/calibration_package/test_payload.py tests/unit/test_calibration_package_stage_contract.py tests/unit/test_pipeline_docs_extractor.py
  • uv run --no-sync --with sqlmodel python -m pytest tests/unit/calibration_package/test_payload.py tests/unit/test_calibration_package_stage_contract.py tests/unit/test_pipeline_doc_guards.py tests/unit/test_pipeline_docs_extractor.py tests/unit/test_remote_calibration_runner.py (58 passed)
  • uv run --no-sync --with pyyaml python scripts/run_quality_guards.py
  • make lint
  • git diff --check origin/main...HEAD
  • git diff --check

Notes

  • Compatibility with legacy dictionary pickles is intentional in PR 2c. The Stage 2 plan removes or isolates the legacy subprocess path later in PR 2j, after parity is proven.

@anth-volk anth-volk force-pushed the agent/stage-2/pr-2b-input-artifact-bundles branch 3 times, most recently from 5d9cde6 to 7c1c3e5 Compare May 22, 2026 16:54
@anth-volk anth-volk force-pushed the agent/stage-2/pr-2c-payload-reader-writer branch from 905ca62 to 712354b Compare May 22, 2026 19:44
@anth-volk anth-volk changed the base branch from agent/stage-2/pr-2b-input-artifact-bundles to main May 22, 2026 19:45
@anth-volk anth-volk force-pushed the agent/stage-2/pr-2c-payload-reader-writer branch from 712354b to ae20bab Compare May 22, 2026 20:53
@anth-volk anth-volk force-pushed the agent/stage-2/pr-2c-payload-reader-writer branch from ae20bab to 2b97bdb Compare May 22, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Stage 2 package payload reader and writer

1 participant