Skip to content

feat(dml): DMLDiD panel=False repeated cross sections - Chang (2020) Case 2 (DML PR-B2) - #794

Open
igerber wants to merge 2 commits into
mainfrom
dml-b2
Open

feat(dml): DMLDiD panel=False repeated cross sections - Chang (2020) Case 2 (DML PR-B2)#794
igerber wants to merge 2 commits into
mainfrom
dml-b2

Conversation

@igerber

@igerber igerber commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • Ships Chang (2020) Case 2 (repeated cross sections) as an RCS lane inside DMLDiD, selected by a new panel: bool = True constructor parameter (mirrors CallawaySantAnna). Declared RCS only: one row per unit, unique row IDs required; no allow_unbalanced_panel RC-routing.
  • diff_diff/_dr_scores.py: Case 2 score family - chang_rcs_score (Eq 3.2 summand with the single control-only (T - lam) * Y regression, global within-cell p_hat/lam_hat), chang_rcs_lambda_slope (sample-analogue G2_lambda from the closed-form lambda derivative; the paper prints no estimator - documented REGISTRY Note), chang_rcs_score_augmented (Theorem 2 variance with BOTH finite-dimensional corrections; SE = sqrt(mean(psi_bar^2)/n_cell)), plus the shared fail-closed validator.
  • diff_diff/dml_did.py: _precompute_rcs + _compute_dml_rcs_gt - pooled two-period cells on level outcomes, four-group guard (zero_treated_control), D x T 4-class stratified folds (singleton stratum -> cross_fit_degenerate), clip-never-drop propensities, lam_hat extremeness warning, per-observation influence-function payload, lam_hat/g2_lambda diagnostics. Stationary-sampling warning (Assumption 2.3) emitted only after the declared-RCS structure validates.
  • RCS aggregation weights are the fixed cohort row masses via per-cell agg_weight (CS-RCS convention, WIF-consistent SEs); agg_cohort_masses deliberately unset (float-keyed lookups collide on >2^53 int64 labels; the bincount fallback is numerically identical under unique row IDs). aggregate('total') fails closed on RCS fits.
  • Design-aware results/report surfaces: summary Design: line + obs labels, BusinessReport cites Assumptions 2.3 + 3.2(h) on RCS, practitioner snippet carries panel=False, target-parameter text names the cohort-mass weighting, DiagnosticReport skips Goodman-Bacon on RCS fits (auto-refit path only; precomputed passthrough honored; covers CallawaySantAnna(panel=False) too).
  • benchmarks/doubleml/chang_rcs_characterization.py: CHARACTERIZATION spike (not parity - DoubleMLDIDCSBinary implements the Sant'Anna-Zhao 4-regression score and omits the lambda-correction) with pasted transcript; Part 2 golden literals pin the public estimator against the hand Eq 3.2/Thm 2 pipeline at 0.0.
  • Documentation surfaces: REGISTRY DMLDiD section covers both cases (Case 2 equations, new Notes, joint Case 1/Case 2 assumption citations), docs/api/dml_did.rst rework with a loud no-survey-weights note, chang-2020-review checklist flips, llms guides, README/index one-liners, choosing_estimator, migration-4.0, survey-roadmap, practitioner_decision_tree, REPORTING.md, variance-conventions, references, doc-deps, CHANGELOG; ROADMAP DML section removed (Case 3 stays in DEFERRED.md).

Methodology references (required if estimator / math changes)

  • Method name(s): DMLDiD repeated cross sections - Chang (2020) Case 2 score (Equation 3.2) with the Theorem 2 lambda-corrected variance, per Callaway-Sant'Anna (g, t) cell.
  • Paper / source link(s): Chang, N.-C. (2020). Double/debiased machine learning for difference-in-differences models. The Econometrics Journal 23(2), 177-191. https://doi.org/10.1093/ectj/utaa001 - full transcription in docs/methodology/papers/chang-2020-review.md.
  • Any intentional deviations from the source (and why): all recorded as labeled Notes in docs/methodology/REGISTRY.md (DMLDiD section): staggered per-cell extension; global within-cell lam_hat = mean(T); sample-analogue G2_lambda estimator (the paper prints none); D x T 4-class fold stratification (guarantees control rows in both periods per training complement; mirrors DoubleML's d + 2t encoding); zero_treated_control reused for any-empty-of-four-groups; propensity clipping (the paper gives no trimming rule); fixed cohort-row-mass RCS aggregation weights; aggregate('total') fails closed on RCS fits; Case-2-only level-outcome moment conditions not imposed on the panel path.

Validation

  • Tests added/updated: tests/test_dr_scores.py (Case 2 validator matrix), tests/test_methodology_dr_scores.py (hand fixtures at 1e-15/1e-12, finite-difference checks of G2_lambda and d/dp psi_2, DR both directions, lambda-term variance), tests/test_methodology_dml_did.py (oracle-nuisance equivalence at 1e-12, hand-pipeline replay at rtol 1e-14, lambda-correction regression guard, ATT recovery, golden characterization with live-fit gap pins, slow-lane MC coverage), tests/test_dml_did.py (TestRCS* construction/validation/estimation/payload/degenerate/aggregation/results/semantics/complete-cases/agg-weights incl. >2^53 int64 labels), tests/test_variance_conventions.py (dml_did_rcs row; table regenerated).
  • Backtest / simulation / notebook evidence (if applicable): benchmarks/doubleml/chang_rcs_characterization.py executed in the .venv-doubleml side venv (doubleml 0.11.4, sklearn 1.9.0); transcript pasted in the header - self-parity at 0.0, documented nonzero Chang-vs-DoubleML gaps on shared folds.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

…Case 2 (DML PR-B2)

Ships Chang (2020) Case 2 (repeated cross sections) as an RCS lane inside
DMLDiD, selected by a new `panel: bool = True` constructor parameter
(mirrors CallawaySantAnna). Declared RCS only: one row per unit, unique
row IDs required; no allow_unbalanced_panel RC-routing.

Score family (diff_diff/_dr_scores.py):
- chang_rcs_score: Eq 3.2 uncentered summand
  (D - g)/(p lam (1-lam)(1-g)) * ((T - lam) y - l2_hat), with the single
  control-only (T - lam) Y regression (Chang's I_kz^c) as the outcome
  nuisance and global within-cell p_hat = mean(D), lam_hat = mean(T).
- chang_rcs_lambda_slope: sample-analogue G2_lambda from the closed-form
  d/d-lambda of psi_2 (the paper prints no estimator - documented
  implementation decision, REGISTRY Note).
- chang_rcs_score_augmented: Theorem 2 psi_bar = summand - D theta/p
  + G2_lambda (T - lam); SE = sqrt(mean(psi_bar^2)/n_cell). The
  lambda-correction is mandatory (review warning quoted in docstring)
  and pinned by a regression test that recomputes the no-lambda SE.
- Shared fail-closed validator (strict-binary D AND T, strictly interior
  p_hat/lam_hat, ps in [0,1)).

Estimator lane (diff_diff/dml_did.py):
- panel=False validation: unique unit IDs, covariates required,
  stationary-sampling UserWarning (Assumption 2.3) emitted only after
  the declared-RCS structure validates.
- _precompute_rcs: per-row cohorts, is_panel False, canonical_size =
  n_obs; agg_cohort_masses deliberately unset (float-keyed lookups
  collide on >2^53 int64 labels; the bincount fallback is numerically
  identical under unique row IDs).
- _compute_dml_rcs_gt: pooled two-period cells on level outcomes,
  FOUR-group guard (any empty treated/control x period group ->
  zero_treated_control), D x T 4-class stratified folds (singleton
  stratum -> cross_fit_degenerate), propensity clip-never-drop,
  lam_hat extremeness warning, per-observation IF payload
  psi_bar/n_cell, diagnostics gain lam_hat and g2_lambda.
- RCS aggregation weights are the FIXED cohort row masses via per-cell
  agg_weight (CS-RCS convention, WIF-consistent SEs);
  aggregate('total') fails closed on RCS fits.
- Results/reports design-aware: summary Design line + obs labels,
  BusinessReport cites Assumption 2.3 + 3.2(h) on RCS, practitioner
  snippet carries panel=False, target-parameter text names the
  cohort-mass weighting, DiagnosticReport skips the Goodman-Bacon
  check on RCS fits (auto-refit path only; precomputed passthrough
  honored).

Validation (no Case 2 parity oracle exists - DoubleMLDIDCSBinary
implements the Sant'Anna-Zhao 4-regression score and omits the
lambda-correction):
- benchmarks/doubleml/chang_rcs_characterization.py: CHARACTERIZATION
  spike (not parity) with pasted transcript; Part 1 documents nonzero
  Chang-vs-DoubleML gaps on shared folds, Part 2 golden literals pin
  public DMLDiD(panel=False) == hand Eq 3.2/Thm 2 pipeline at 0.0.
- Equation-level fixtures at 1e-15/1e-12, finite-difference checks of
  G2_lambda and d/dp psi_2, oracle-nuisance recovery (true propensity
  + true l20), hand-pipeline replay at rtol 1e-14, DR in BOTH nuisance
  directions, ATT recovery on a fixed RCS DGP (theta_0 = 3),
  lambda-correction regression guard, native golden reproduction with
  live-fit gap pins, slow-lane Monte Carlo coverage, payload/idx
  contracts, degenerate handling with assert_nan_inference, >2^53
  int64 cohort-label aggregation + bootstrap replay, variance-
  conventions dml_did_rcs row (table regenerated).

Docs: REGISTRY DMLDiD section covers both cases (Case 2 equations,
8 new Notes, Case 2 assumption citations 3.2(a)/3.2(h) + Assumption
2.3 bullet, characterization reference block); dml_did.rst full
rework (Case 2 methodology, declared-RCS restrictions, loud
no-survey-weights note); chang-2020-review.md checklist flips;
llms*.txt guides; README/index one-liners; choosing_estimator,
migration-4.0, survey-roadmap, practitioner_decision_tree,
REPORTING.md, variance-conventions prose, references.rst,
doc-deps.yaml, CHANGELOG. ROADMAP DML section removed (Case 3 stays
in DEFERRED.md); new TODO row tracks Chang section-4 RCS DGP
replication.
@github-actions

Copy link
Copy Markdown

Overall assessment

Looks good — no unmitigated P0 or P1 findings.

Executive summary

  • Case 2 score and λ-corrected variance match Chang (2020) Eq. 3.2/Theorem 2.
  • Control selection, four-group guards, and NaN inference handling are correct.
  • panel propagates through parameters, results, aggregation, bootstrap, and reporting.
  • Intentional deviations are documented in the Methodology Registry.
  • Only a minor redundant computation and properly tracked deferred work remain.

Methodology

  • P3 — Documented deviations. Global /λ̂, D×T-stratified folds, clipping, staggered-cell extension, and fixed cohort-mass aggregation are explicitly registered.
    Impact: Informational; not defects.
    Concrete fix: None.
    docs/methodology/REGISTRY.md:L2792-L3009, diff_diff/_dr_scores.py:L316-L426, diff_diff/dml_did.py:L1227-L1492

No incorrect score, control composition, variance, or assumption handling found.

Code Quality

No findings.

Performance

  • P3 — λ-slope computed twice. chang_rcs_score_augmented() computes G₂λ, then the caller recomputes it for diagnostics.
    Impact: One unnecessary O(n_cell) pass per cell; ML fitting still dominates.
    Concrete fix: Optionally compute g2_lambda once and pass or return it alongside the augmented score.
    diff_diff/_dr_scores.py:L425-L426, diff_diff/dml_did.py:L1419-L1429

Maintainability

No findings.

Tech Debt

  • P3 — Properly tracked. The tutorial, Chang §4 RCS simulation replication, and survey/cluster support are recorded in TODO.md/DEFERRED.md.
    Impact: No approval blocker.
    Concrete fix: None required for this PR.
    TODO.md:L85-L86, DEFERRED.md:L159

Security

No findings; no secrets or unsafe diagnostic persistence identified.

Documentation/Tests

No untracked gaps. Tests cover equations, derivatives, double robustness, λ-correction, degeneracy, aggregation, bootstrap, and parameter interactions.

Validation note: changed Python files parsed successfully and git diff --check passed; runtime tests could not be executed because the supplied environment lacks pytest and NumPy.

@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 88cc8b75ba49fc14551870a0418a109e24d67dc2


Overall assessment

Looks good — no unmitigated P0 or P1 findings.

Executive summary

  • Chang Case 2 score and λ-corrected variance match Eq. 3.2/Theorem 2.
  • Control composition, degeneracy guards, inference, aggregation, and bootstrap propagation are correct.
  • Registered deviations are informational, not defects.
  • The prior duplicate λ-slope computation is now tracked in TODO.md.

Methodology

  • P3 — Documented deviations. Global λ̂, sample-analogue Ĝ₂λ, D×T folds, clipping, staggered extension, and fixed cohort-mass weighting are explicitly registered.
    Impact: Informational only.
    Concrete fix: None.
    docs/methodology/REGISTRY.md:L2790-L2819, docs/methodology/REGISTRY.md:L2965-L3009, diff_diff/_dr_scores.py:L314-L426, diff_diff/dml_did.py:L1227-L1492

Code Quality

No findings.

Performance

  • P3 — Tracked. Ĝ₂λ is computed twice per successful cell.
    Impact: Minor O(n_cell) redundant work; nuisance fitting dominates.
    Concrete fix: Already tracked in TODO.md; no PR blocker.
    diff_diff/dml_did.py:L1423-L1428, TODO.md:L77

Maintainability

No findings.

Tech Debt

  • P3 — Properly tracked. Survey/cluster support, tutorial work, and Chang §4 RCS replication are recorded in TODO.md/DEFERRED.md.
    Impact: No approval blocker.
    Concrete fix: None required here.
    TODO.md:L86-L87, DEFERRED.md:L159

Security

No findings.

Documentation/Tests

No untracked gaps. Tests cover formulas, derivatives, double robustness, λ correction, degenerate cells, aggregation, bootstrap, and parameter interactions.

Validation: git diff --check passed and all 15 changed Python files parsed successfully. Runtime tests were unavailable because pytest and numerical dependencies are not installed.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant