feat(governance): add distributional drift detection (PSI, KS)

[Hopelynconsult]

Summary

• Adds governance/drift_detector.py — distributional drift checks that compare recent prediction/input windows against a reference baseline.
• Two non-parametric methods: Population Stability Index (PSI) and two-sample Kolmogorov-Smirnov.
• Per-feature DriftResult rolled up into a DriftReport. Designed to plug into the prediction-history JSONL written by the anomaly detector (feat(governance): add anomaly detection for inference outputs #35).

Why this is distinct from the anomaly detector

The anomaly detector (#35) flags individual predictions whose features sit outside historical norms — point anomalies. A model can have zero point anomalies and still be silently drifting if the distribution of its predictions has shifted (e.g., post-monsoon Sentinel-2 statistics on a deforestation model). PSI/KS catch that distributional shift over a window.

What's in the PR

• DriftResult (per-feature) and DriftReport (multi-feature) dataclasses with JSON serialisation, mirroring governance.calibration.CalibrationReport style.
• population_stability_index() — bins on the reference's quantiles (canonical PSI), industry-standard severity bands (< 0.1 stable, 0.1–0.25 moderate, >= 0.25 severe). Constant-reference fallback to a single bin.
• kolmogorov_smirnov() — supremum CDF gap with an asymptotic p-value computed from the standard Kolmogorov series. Avoids pulling scipy into evaluation-time deps.
• detect_drift() — one-shot entrypoint that takes feature-name → values dicts for both windows and runs PSI or KS per feature.
• write_drift_report() — persistence alongside model cards / calibration reports.
• 13 unit tests: identical vs shifted distributions, both methods, per-feature severity isolation, constant-reference edge case, validation (non-finite, empty, feature mismatch, unknown method), JSON round-trip.

Plugging into the existing pipeline

The anomaly detector already writes a JSONL of per-prediction features (mean_confidence, std_confidence, positive_fraction, entropy). Wiring PSI on those four features over a rolling window is a 30-line script — left as a follow-up so this PR stays focused.

Follow-ups (out of scope here)

• A scheduled CI job that reads the last N days of outputs/anomalies/history.jsonl, picks a baseline window, and emits a drift report.
• A drift_score threshold added to scripts/governance_ci_gate.py so a release fails if any monitored feature is in the severe band.

Test plan

• pytest tests/test_drift_detector.py -q → 13 passed
• Reviewer: confirm the severity bands (PSI_STABLE=0.10, PSI_MODERATE=0.25, KS significance 0.05) are the right defaults for our use case before we wire them into the CI gate.

🤖 Generated with Claude Code