Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,11 @@ See the [shared PolicyEngine contribution guide](https://github.com/PolicyEngine
```bash
make install # install deps (uv)
make format # format (required)
ruff check . # lint
make lint # format check and lint
uv run mypy src/policyengine # type check
make test # test suite
make docs # build documentation
make push-pr-branch # push to origin with correct tracking before opening PRs

uv run pytest tests/test_household_impact.py::TestUKHouseholdImpact -v
```
Expand All @@ -29,6 +30,21 @@ policyengine.py is the user-facing analysis package. It wraps `policyengine-uk`

Country pins live in `pyproject.toml` under the `[uk]` / `[us]` / `[dev]` extras. Bumping a pin is a patch-level change most of the time; include the motivation in the PR body.

## Opening PRs

Read `docs/engineering/skills/github-prs.md` before opening, replacing, or
sharing a PR. In short:

1. Open or identify an issue.
2. Put `Fixes #ISSUE_NUMBER` as the first line of the PR body.
3. Add a Towncrier fragment such as `changelog.d/ISSUE_NUMBER.fixed.md`.
4. Run `make format` and `make lint`.
5. Run `make push-pr-branch`.
6. Open a same-repository draft PR:
`gh pr create --draft --repo PolicyEngine/policyengine.py --head "$(git branch --show-current)" --base main`.
7. Verify it is draft and from `PolicyEngine/policyengine.py`:
`gh pr view <PR> --repo PolicyEngine/policyengine.py --json isDraft,headRepositoryOwner,headRepository`.

## Repo-specific anti-patterns

- **Don't** bypass the country-model APIs with direct `policyengine-core` calls from user-facing code. The wrapper exists so analyses survive core API changes.
Expand Down
10 changes: 3 additions & 7 deletions .github/check-changelog.sh
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail

FRAGMENTS=$(find changelog.d -type f ! -name '.gitkeep' | wc -l)
if [ "$FRAGMENTS" -eq 0 ]; then
echo "::error::No changelog fragment found in changelog.d/"
echo "Add one with: echo 'Description.' > changelog.d/\$(git branch --show-current).<type>.md"
echo "Types: added, changed, fixed, removed, breaking"
exit 1
fi
BASE_REF="${1:-origin/main}"

towncrier check --compare-with "$BASE_REF"
11 changes: 11 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Copilot Instructions

Follow the repository's canonical engineering skills under
`docs/engineering/skills/`.

For pull requests, read `docs/engineering/skills/github-prs.md` before opening,
replacing, or sharing a PR. This repository only accepts same-repository PRs
from branches in `PolicyEngine/policyengine.py`; never create fork PRs.

For repository-wide API, testing, documentation, release, or package-boundary
changes, read `docs/engineering/skills/repository-guidance.md`.
4 changes: 4 additions & 0 deletions .github/workflows/pr_code_changes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Install towncrier
run: pip install towncrier
- name: Check for changelog fragment
run: .github/check-changelog.sh
Lint:
Expand Down
32 changes: 32 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Agent Instructions

These instructions apply repository-wide.

## Skills System

Canonical AI-facing engineering skills live under `docs/engineering/skills/`.
Use those files as the source of truth across Codex, Claude, Copilot, and other
AI tools.

Before opening, replacing, or sharing any pull request, read
`docs/engineering/skills/github-prs.md`.

Before making or reviewing repository-wide API, testing, documentation, release,
or package-boundary changes, read
`docs/engineering/skills/repository-guidance.md`.

## Repository Boundaries

`policyengine.py` is the user-facing analysis package. It wraps
`policyengine-uk` and `policyengine-us` with a common `Simulation` object,
dataset loaders, and result models.

Do not bypass the country-model APIs with direct `policyengine-core` calls from
user-facing code unless the change explicitly needs a lower-level primitive.
The wrapper exists so analyses survive country-package and core API changes.

Do not add public input or output classes without Pydantic models. JSON
round-trip is a documented property of the public surface.

Do not cache arbitrary Python objects in public result structures. The
`core.Simulation` output must stay serialisable.
21 changes: 21 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Claude Instructions

These instructions apply repository-wide.

## Canonical Guidance

Repository-wide AI-facing engineering guidance lives in `AGENTS.md`.
Canonical skills live under `docs/engineering/skills/`.

Use those files as the source of truth. This file is a Claude adapter and should
stay thin; do not duplicate detailed testing, CI, formatting, release, or
architecture rules here.

## Required Skill Lookup

Before opening, replacing, or sharing a PR, read
`docs/engineering/skills/github-prs.md`.

Before making or reviewing repository-wide API, testing, documentation, release,
or package-boundary changes, read
`docs/engineering/skills/repository-guidance.md`.
25 changes: 24 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: docs docs-serve docs-generate-reference docs-reference-smoke
.PHONY: all build-package changelog clean docs docs-serve docs-generate-reference docs-reference-smoke format install lint push-pr-branch test

all: build-package

Expand All @@ -25,6 +25,10 @@ install:
format:
ruff format .

lint:
ruff format --check .
ruff check .

clean:
find . -not -path "./.venv/*" -type d -name "__pycache__" -exec rm -rf {} +
find . -not -path "./.venv/*" -type d -name "_build" -exec rm -rf {} +
Expand All @@ -42,3 +46,22 @@ build-package:

test:
pytest tests --cov=policyengine --cov-report=term-missing

BRANCH := $(shell git branch --show-current)

push-pr-branch:
@if [ "$(BRANCH)" = "main" ]; then \
echo "Refusing to push main as a PR branch."; \
exit 1; \
fi
@REMOTE_URL=$$(git remote get-url origin 2>/dev/null || true); \
if [ -z "$$REMOTE_URL" ]; then \
echo "Missing origin remote. Add PolicyEngine/policyengine.py as origin before opening PRs."; \
exit 1; \
fi; \
case "$$REMOTE_URL" in \
*PolicyEngine/policyengine.py*) ;; \
*) echo "Refusing to push: origin ($$REMOTE_URL) is not PolicyEngine/policyengine.py."; exit 1 ;; \
esac
@git push -u origin HEAD:$(BRANCH)
@echo "Create the PR with: gh pr create --draft --repo PolicyEngine/policyengine.py --head $(BRANCH) --base main"
File renamed without changes.
1 change: 1 addition & 0 deletions changelog.d/374.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Align the pull-request changelog check with Towncrier's post-merge fragment discovery and add model-neutral AI PR guidance.
15 changes: 15 additions & 0 deletions docs/engineering/skills/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Engineering Skills

This directory is the canonical source for AI-facing engineering rules.

Tool-specific instruction files such as `AGENTS.md`, `CLAUDE.md`, and
`.github/copilot-instructions.md` should point here instead of duplicating
implementation-specific guidance. When a rule changes, update the skill here
first, then keep adapters thin.

Current skills:

- `github-prs.md`: same-repository PR workflow, required pre-PR checks,
changelog-fragment requirements, PR head verification, and title conventions.
- `repository-guidance.md`: policyengine.py structure, commands, package
boundaries, test expectations, and repo-specific anti-patterns.
45 changes: 45 additions & 0 deletions docs/engineering/skills/github-prs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# GitHub PRs

These rules apply to every developer and AI agent opening pull requests in this
repository.

## Same-Repository PRs Only

Open PRs from branches in `PolicyEngine/policyengine.py`, not from personal
forks. Fork PRs are more likely to miss repository secrets and can produce
different CI behavior from branches in the canonical repository.

Before creating or sharing a PR:

1. Confirm the canonical repository is reachable:
`gh repo view PolicyEngine/policyengine.py --json nameWithOwner`.
2. Open a GitHub issue for the work, or verify that an appropriate issue already
exists.
3. Put `Fixes #ISSUE_NUMBER` as the first line of the PR description, using the
issue number from the issue created or found in the previous step.
4. Add a Towncrier changelog fragment under `changelog.d/` using the issue
number and the appropriate configured type, for example
`changelog.d/ISSUE_NUMBER.fixed.md`.
5. Run the repository format target:
`make format`.
6. Run the repository lint target:
`make lint`.
7. Push the current branch to the canonical repository:
`make push-pr-branch`.
8. Create the PR as a draft from that same repository:
`gh pr create --draft --repo PolicyEngine/policyengine.py --head "$(git branch --show-current)" --base main`.
9. Verify the PR is draft and the head repository is canonical:
`gh pr view <PR> --repo PolicyEngine/policyengine.py --json isDraft,headRepositoryOwner,headRepository`.
10. Leave the PR as draft unless a maintainer explicitly asks for it to be
marked ready for review.

The PR is valid only if `isDraft` is `true` and the head repository is
`PolicyEngine/policyengine.py`. If you cannot push to the canonical repository,
stop and ask for access. Do not create a fork PR as a fallback. If you
accidentally create one, close it immediately and replace it with a
same-repository draft PR.

## PR Title

Do not add `[codex]`, `[claude]`, `[copilot]`, or other agent labels to PR
titles. Use a plain descriptive title.
61 changes: 61 additions & 0 deletions docs/engineering/skills/repository-guidance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Repository Guidance

Use this skill when making or reviewing repository-wide API, testing,
documentation, release, or package-boundary changes.

## Commands

```bash
make install # install deps (uv)
make format # format files
make lint # format check and ruff lint
uv run mypy src/policyengine # type check
make test # test suite
make docs # build documentation

uv run pytest tests/test_household_impact.py::TestUKHouseholdImpact -v
```

Python 3.13+. Default branch: `main`. Tests that download representative
datasets need `HUGGING_FACE_TOKEN` set.

## What Lives Here

- `src/policyengine/core/` contains the shared simulation, dataset, and policy
model.
- `src/policyengine/tax_benefit_models/uk/` contains UK-specific loaders and
analysis.
- `src/policyengine/tax_benefit_models/us/` contains US-specific loaders and
analysis.
- `src/policyengine/outputs/` contains decile, inequality, poverty, program
statistics, and impact calculators.
- `src/policyengine/utils/` contains parametric-reform helpers, entity utilities,
and shared support code.

Country pins live in `pyproject.toml` under the `[uk]`, `[us]`, and `[dev]`
extras. Bumping a pin is usually a patch-level change; include the motivation in
the PR body.

## Public Surface

- Prefer the country-model APIs over direct `policyengine-core` calls from
user-facing code.
- Public input and output classes should be Pydantic models.
- Public result structures must be JSON-serialisable.
- Examples under `examples/` double as documentation. When changing a public
API, update or add an example when useful.

## Testing

Keep tests focused on the changed behavior and avoid adding long-running
representative-data runs unless the change specifically needs that coverage.
When a test needs country package data, make the dependency explicit and skip
cleanly if credentials or local artifacts are unavailable.

## Anti-Patterns

- Do not bypass the wrapper layer without a clear reason.
- Do not add untyped or non-serialisable public result containers.
- Do not hide country-package version changes; call them out in the PR body.
- Do not add `[codex]`, `[claude]`, `[copilot]`, or other agent labels to PR
titles.
31 changes: 31 additions & 0 deletions tests/test_bump_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,37 @@
spec.loader.exec_module(bump_version)


def test_infer_bump_uses_flat_towncrier_fragment_types(tmp_path):
changelog_dir = tmp_path / "changelog.d"
changelog_dir.mkdir()
(changelog_dir / "feature.added.md").write_text("Add a feature.\n")

bump = bump_version.infer_bump(changelog_dir)

assert bump == "minor"


def test_infer_bump_uses_breaking_fragment_precedence(tmp_path):
changelog_dir = tmp_path / "changelog.d"
changelog_dir.mkdir()
(changelog_dir / "bugfix.fixed.md").write_text("Fix a bug.\n")
(changelog_dir / "api.breaking.md").write_text("Break an API.\n")

bump = bump_version.infer_bump(changelog_dir)

assert bump == "major"


def test_infer_bump_rejects_fragments_towncrier_would_ignore(tmp_path):
changelog_dir = tmp_path / "changelog.d"
nested_dir = changelog_dir / "added"
nested_dir.mkdir(parents=True)
(nested_dir / "feature.md").write_text("Add a feature.\n")

with pytest.raises(SystemExit):
bump_version.infer_bump(changelog_dir)


def test_get_current_version_prefers_highest_seen_version(tmp_path):
pyproject = tmp_path / "pyproject.toml"
pyproject.write_text('[project]\nversion = "3.4.1"\n')
Expand Down