release: 4.0.0 - close the 4.0 cut window (item 10)

.claude/commands/bump-version.md

@@ -20,7 +20,7 @@ Files that need updating:
 
 | File | Format | Line |
 |------|--------|------|
-| `diff_diff/__init__.py` | `__version__ = "X.Y.Z"` | ~134 |
+| `diff_diff/__init__.py` | `__version__ = "X.Y.Z"` | ~332 |
 | `pyproject.toml` | `version = "X.Y.Z"` | ~7 |
 | `rust/Cargo.toml` | `version = "X.Y.Z"` | ~3 |
 | `CHANGELOG.md` | Section header + comparison link | Top + bottom |

.claude/memory.md

@@ -81,18 +81,16 @@ results = estimator.fit(df, outcome='y', treatment='treated', ...)
 print(results.summary())
 ```
 
-### Results Objects — CURRENT (3.x)
-The canonical inference quintet is `att` / `se` / `t_stat` / `p_value` /
-`conf_int` (never `.pvalue` / `.ci`). Every results class exposes all five,
-but native STORAGE varies in 3.x (`overall_att` on the staggered family,
-`avg_att` on the MultiPeriodDiD class removed at the 4.0 cut, with canonical
-names as properties). `summary()`
-everywhere; `to_dict()` / `to_dataframe()` on most classes. See
-`.claude/../docs/methodology/REGISTRY.md` per estimator.
-
-### Results Objects — 4.0 TARGET (do not write against this pre-4.0)
-Target contract: `docs/v4-design.md` section 5; per-surface lifecycle:
-`docs/v4-deprecations.yaml` (CI-enforced by `tests/test_v4_matrix.py`).
+### Results Objects — CURRENT (4.0)
+The canonical inference quintet `att` / `se` / `t_stat` / `p_value` /
+`conf_int` is the NATIVE storage on every results class since 4.0 (the 3.x
+`overall_*` names, `EventStudyResults.period_effects` and LWDiD's `.pvalue` /
+`.ci` were removed at 4.0 - `AttributeError`; 3.x pickles still load via
+`__setstate__`). `summary()` everywhere; `to_dict()` / `to_dataframe()` on
+most classes; `to_dataframe(level="event_study")` emits the unified
+`EVENT_STUDY_SCHEMA`. Contract: `docs/v4-design.md` section 5; per-surface
+lifecycle: `docs/v4-deprecations.yaml` (CI-enforced by
+`tests/test_v4_matrix.py`). See `docs/methodology/REGISTRY.md` per estimator.
 
 ### Column Naming — CURRENT (4.0)
 - `unit` unit id everywhere (the 3.x `unit_col` on HAD and `group` on dCDH

.claude/scripts/changelog_compile.py

@@ -36,6 +36,8 @@
 # order. changelog.d/README.md mirrors this list; the mirror is pinned by
 # tests/test_changelog_fragments.py (README<->compiler parity test).
 CATEGORIES = (
+    "Breaking Changes",
+    "Behavioral Changes",
     "Added",
     "Changed",
     "Deprecated",
@@ -45,8 +47,6 @@
     "Performance",
     "Documentation",
     "Testing",
-    "Breaking Changes",
-    "Behavioral Changes",
     "Internal",
 )
 

CITATION.cff

@@ -7,8 +7,8 @@ authors:
     family-names: Gerber
     orcid: "https://orcid.org/0009-0009-3275-5591"
 license: MIT
-version: "3.13.0"
-date-released: "2026-10-17"
+version: "4.0.0"
+date-released: "2026-11-01"
 doi: "10.5281/zenodo.19646175"
 url: "https://github.com/igerber/diff-diff"
 repository-code: "https://github.com/igerber/diff-diff"

DEFERRED.md

@@ -123,7 +123,7 @@ For survey-specific limitations (`NotImplementedError` paths), see the
 
 ## Version-gated (v4)
 
-Gated on the next major version, not on demand or derivation. Lifecycle state and
+Gated on a future major version (the 4.0 cut closed 2026-11-01; no open rows), not on demand or derivation. Lifecycle state and
 removal/flip targets live ONLY in the CI-enforced `docs/v4-deprecations.yaml`; rows
 here carry the remaining implementation work and cross-link their `M-xxx` ids.
 
@@ -143,7 +143,7 @@ decisions (refactor waivers, perf trade-offs, test-infrastructure calls) are rec
 |----------|----------|----------|
 | **`utils.validate_pscore_trim` ships without the `allow_zero` flag the retired TODO row proposed.** The row (retired in the M-145 PR) suggested `validate_pscore_trim(value, *, allow_zero)`; after ContinuousDiD's alignment to `0 < x < 0.5`, no consumer needs `allow_zero=True` — every migrated copy (TripleDifference, DMLDiD, ContinuousDiD, LWDiD, CallawaySantAnna) wants the strict interval, and the former `StaggeredTripleDifference` class's bare check went with the class (PR 4a) rather than becoming a flagged call. A dead-on-arrival parameter on a shared validator is a drift magnet; re-add the flag only when a real caller needs a `[0, 0.5)` domain, with a distinct range message | `diff_diff/utils.py` | M-145 / 2026-08-27 |
 | **Fixed-B Stata SE-golden comparisons in the LWDiD suite run unmarked in default CI** (no `ci_params.bootstrap()` scaling, no `slow` marker): the SE tolerance derives from BOTH fixed rep counts (ours B=999, Stata's R from the golden meta) so scaling either side would invalidate the committed-golden comparison — the ci_params convention governs convergence-style tests, not fixed-B golden comparisons — and these tests are the PR #588 acceptance bar, which must run in default CI; module-scoped fit memoization bounds the cost (~1-2 s per fit locally) | `tests/test_methodology_lwdid.py` | 2026-08-16 |
-| **The 4.0 migration guide's code blocks are not snippet-executed.** `tests/test_doc_snippets.py` discovers a hardcoded list of `.rst` files and only `.. code-block:: python` / RST `::` bodies, so `docs/migration-4.0.md` gets no coverage. Deliberate: the guide is a MIXED document - most "after" examples (the renames, `results.att`) run on the current release, but the `field-flip` and `df-convention-flip` examples describe 4.0 behaviour that cannot run until 4.0, so a blanket execution lane would fail by construction. Closing the gap fully means a markdown-fence extractor plus a skip-marker convention for the future-API blocks - a harness change, out of scope for a docs PR. What IS gated: the appendix's ledger parity (`test_migration_guide_*`), which pins the row set and every mechanically checkable cell; and, since the first local review found all three merge examples carrying invalid keywords, `test_migration_guide_examples_bind_to_real_signatures`, which ast-parses the guide's python blocks and asserts every constructor/`fit()` keyword exists on the target signature. That is signature binding, NOT execution - it deliberately skips calls whose owner it cannot resolve (e.g. `results.aggregate(...)`), and it cannot catch a wrong VALUE or a wrong sequence of calls. The hand-written `Fix` prose remains unverifiable by any available means. | `docs/migration-4.0.md`, `tests/test_doc_snippets.py` | Phase 4 / 2026-08-09 |
+| **The 4.0 migration guide's code blocks are not snippet-executed.** `tests/test_doc_snippets.py` discovers a hardcoded list of `.rst` files and only `.. code-block:: python` / RST `::` bodies, so `docs/migration-4.0.md` gets no coverage. Deliberate: the guide is a MIXED document - most "after" examples (the renames, `results.att`) run on the current release, but the python fences are MIXED in both directions: through 3.x the `# 4.0` halves could not run, and since 4.0.0 (2026-11-01) the `# 3.x` halves raise `ImportError` (`MultiPeriodDiD`, `StaggeredTripleDifference`, `QDiD`) and every block uses an undefined `data`, so a blanket execution lane fails by construction at every version. Closing the gap fully means a markdown-fence extractor plus a skip-marker convention for the future-API blocks - a harness change, out of scope for a docs PR. What IS gated: the appendix's ledger parity (`test_migration_guide_*`), which pins the row set and every mechanically checkable cell; and, since the first local review found all three merge examples carrying invalid keywords, `test_migration_guide_examples_bind_to_real_signatures`, which ast-parses the guide's python blocks and asserts every constructor/`fit()` keyword exists on the target signature. That is signature binding, NOT execution - it deliberately skips calls whose owner it cannot resolve (e.g. `results.aggregate(...)`), and it cannot catch a wrong VALUE or a wrong sequence of calls. The hand-written `Fix` prose remains unverifiable by any available means. | `docs/migration-4.0.md`, `tests/test_doc_snippets.py` | Phase 4 / 2026-08-09 |
 | **MMM launch carousel carries scoped marketing claims, not exporter-contract documentation.** The deck (a LinkedIn marketing artifact, not a docs surface) states capabilities in scoped-but-punchy form: guardrail copy says "the easy mistakes fail loudly" / "you own the design, it owns the math" (the caller-owned estimand/population/window/outcome-scale alignment lives in REGISTRY.md's MMM section and the exporter docstrings, not on slides), and slide 8 shows tutorial 29's worked staggered-boost lift row WITHOUT an on-slide linearity qualifier - the compression's linear-channel scoping is the tutorial's job, which the CTA points to. Absolutes ("no silent mis-calibration", "any estimate exports", "anything with an estimate + SE") are ban-listed by `tests/test_mmm_carousel_claims.py`; local review R1/R2 pressed for compliance-style scoping language on the slides and the marketing-appropriate scoped copy was chosen instead. | `carousel/generate_mmm_carousel.py`, `tests/test_mmm_carousel_claims.py` | mmm-carousel / 2026-08-20 |
 | **DCDH `sklearn.base.clone` param-identity failure won't-fix.** `ChaisemartinDHaultfoeuille._validate_paths_of_interest` unconditionally canonicalizes `paths_of_interest` into a fresh `List[Tuple[int, ...]]`, so sklearn `clone()`'s post-construction `param1 is param2` identity check fails for configured instances - a pre-existing normalization the BaseEstimator mixin PR documented rather than changed (get_params/set_params signatures are clone-compatible; the dependency-free `cls(**est.get_params())` config-equality contract is the enforced one, `tests/test_base_estimator.py`). Fixing would mean returning the caller's raw object from a validator whose job is canonicalization. | `chaisemartin_dhaultfoeuille.py` | mixin PR / 2026-08-01 |
 | **scikit-learn is a required dev/CI dependency, optional at runtime; clone-identity tests stay `importorskip`-guarded.** Supersedes the 2026-08-01 decision that kept scikit-learn out of the dev extra: #816 added it (unpinned) to the `dev` extra and to both `rust-test.yml` install steps, so the sklearn `clone()` round-trip tests (`test_base_estimator.py`, had/rdd/cic suites) and the DML cross-fit learner tests always run in CI. The library's runtime dependency posture is unchanged (numpy/pandas/scipy only) and the `importorskip` guards remain, so the suite still passes in an environment without scikit-learn; the always-running contract is still the dependency-free re-instantiation config-equality test. | `pyproject.toml`, `tests/test_base_estimator.py` | #816 |

METHODOLOGY_REVIEW.md

@@ -408,7 +408,8 @@ and covariate-adjusted specifications.)
 
 3. **Deprecation warnings for unused parameters** (`sun_abraham.py`, `fit()`):
    Added `FutureWarning` for `min_pre_periods` and `min_post_periods` parameters that
-   are accepted but never used (no-op). These will be removed in a future version.
+   are accepted but never used (no-op). These were subsequently removed (neither name
+   is accepted by `SunAbraham.fit` any more).
 
 4. **Removed event-time truncation at [-20, 20]** (`sun_abraham.py`):
    Removed the hardcoded cap `max(min(...), -20)` / `min(max(...), 20)` to match
@@ -1087,8 +1088,8 @@ CI and extending covariate-adjusted R parity are tracked follow-ups in `DEFERRED
 6. **Deprecated `lambda_reg` and `zeta` params; new params are `zeta_omega` and
    `zeta_lambda`**. The old parameters had unclear semantics and did not correspond to
    the paper's notation. The new parameters directly match the paper and R package
-   naming conventions. `lambda_reg` and `zeta` are deprecated with warnings and will
-   be removed in a future release.
+   naming conventions. `lambda_reg` and `zeta` were deprecated with warnings through
+   3.x and removed at 4.0 (ledger M-001/M-002).
 
 **Outstanding Concerns:**
 - Cross-language parity anchor against R's default `synthdid::vcov(method="bootstrap")`
@@ -1105,7 +1106,7 @@ CI and extending covariate-adjusted R parity are tracked follow-ups in `DEFERRED
    in REGISTRY.md §SyntheticDiD `Note (default variance_method deviation from R)`.
 2. **Parameter names**: `zeta_omega` / `zeta_lambda` (matching the paper's notation);
    R uses `eta.omega` / `eta.lambda`. The deprecated Python aliases `lambda_reg` / `zeta`
-   from prior releases emit `DeprecationWarning` and will be removed in a future release.
+   from prior releases were removed at 4.0 (ledger M-001/M-002).
 
 ---
 

ROADMAP.md

@@ -8,9 +8,9 @@ Forward-looking plan for diff-diff, organized as queued work, candidates under c
 
 Queued work, ordered by expected leverage. Each item is its own PR. Ordering is priority-sequenced, not time-committed.
 
-### 4.0 API unification
+### 4.1 `event_study()` comparison front door
 
-- **The 4.0 program: estimator consolidation + one API contract.** Three merges (TwoWayFixedEffects absorbs MultiPeriodDiD, TripleDifference absorbs StaggeredTripleDifference, ChangesInChanges absorbs QDiD), post-fit `results.aggregate()`, a canonical results quintet, and library-wide column/param unification - sequenced as a 3.9 shim release (new surface + FutureWarnings), 4.0 enforcement, and a 4.1 `event_study()` comparison front door. Normative spec: `docs/v4-design.md`; every queued deprecation is tracked in `docs/v4-deprecations.yaml` and enforced by CI (`tests/test_v4_matrix.py`).
+- **One entry point over the staggered family.** `event_study(data, outcome, unit, time, first_treat, estimator=...)` runs the chosen staggered estimator and returns the unified event-study surface, so the estimators can be compared on one call signature. Design sketch: `docs/v4-design.md` phase 6 (sketch only; specified in its own plan).
 
 ### Practitioner-ready output
 

TODO.md

@@ -86,7 +86,6 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 | Issue | Location | Origin | Effort | Priority |
 |-------|----------|--------|--------|----------|
 | Relay `to_dataframe(level='group')` to `aggregate('group').to_dataframe()` on CallawaySantAnna/DMLDiD, EfficientDiD and TripleDifference (staggered) - the level still raises with the post-fit pointer while `level='event_study'` was rewired to the aggregate route at the 4.0 cut (item 8b, [M-093]); ImputationDiD/TwoStageDiD keep their native `group` frames and need the same decision | `diff_diff/staggered_results.py`, `diff_diff/efficient_did_results.py`, `diff_diff/triple_diff_results.py` | #883 | Quick | Low |
-| Re-execute the eight tutorials whose committed outputs still show the 3.x residual-df p-values/CIs after the `df_convention` default moved to `"cluster"` (t(G-1)): `01_basic_did`, `02_staggered_did`, `04_parallel_trends`, `05_honest_did`, `07_pretrends_power`, `09_real_world_examples`, `11_imputation_did`, `13_stacked_did` (not `16_survey_did`: the survey df wins; `16_wooldridge_etwfe` was re-executed and reconciled in the 8b PR). Run under `DIFF_DIFF_BACKEND=rust` with the per-worktree kernel, reconcile any prose that restates a moved number, and `--nbmake` each; scheduled for the 4.0 release PR (`docs/v4-design.md` section 9 item 10) together with any item-8/9 movers (24 was refreshed in #879; 23 in 7b, re-validated in 7c - it prints no p-values or CIs). 7b's label changes also drift any committed analytical DiD/TWFE summary (", G=" on the variance line) and every SunAbraham summary (the new "Std. errors:" line) - covered by the same re-execution. | `docs/tutorials/*.ipynb` | #879 | Mid | Medium |
 | TWFE event-study one-way HC2-BM absorbed-FE R parity against the `mpd_absorbed_fe_did` golden (`clubSandwich::vcovCR(cluster=1:n, type="CR2")`): since 4.0 (item 7b) `cluster=False` + hc2_bm reaches the one-way lane directly (no singleton-cluster column needed; `tests/test_v4_cluster_policy.py` pins finiteness and the static-surface LinearRegression parity only), so the open item is the R pin itself; the hc2 lane is pinned by `TestTWFEEventStudyAbsorbedFERParity` and the clustered hc2_bm per-period lane by `test_twfe_event_study_cluster_hc2_bm_per_period_df_matches_clubsandwich`; the MultiPeriodDiD lanes died at the 4.0 cut (PR 4c-ii) | `tests/test_estimators_vcov_type.py` | #v4-4c-i | Mid | Medium |
 | TWFE event-study power profile: `simulate_power`/`simulate_mde`/`simulate_sample_size` have no `_EstimatorProfile` for `TwoWayFixedEffects.fit(event_study=True)` since the MultiPeriodDiD registry entry went with the class (PR 4c-ii); `EventStudyResults` carries no scalar headline, so the profile needs a design (post-period average via the `tests/helpers/event_study_average.py` arithmetic, or a named period) plus its `_SURVEY_FIT_BUILDERS` twin | `diff_diff/power.py` | #v4-4c-ii | Mid | Low |
 | Clustered hc2_bm Bell-McCaffrey DOF guard is batch-relative (`linalg._cr2_bm_dof_inner`: a contrast is NaN'd when its scaled max\|B\| falls below 1e-10 of the batch maximum), so the event-study core keeps the otherwise-unused post-period-average contrast column as the anchor to hold the 3.x per-period NaN masks bit-identical (PR 4c-ii, user decision 2026-10-25). Reproduction: `x_nc = treated * 1[period >= 4] + 3e-6 * default_rng(seed).normal(size=n)` as a covariate on the oracle panel flips the mask between `[eye(k), avg]` and `eye(k)` on 20/20 seeds (cond(X) ~ 9e5; 1e-7 never flips, <= 3e-8 rank-drops) - too BLAS-sensitive for a committed oracle, hence the structural pin `test_hc2_bm_contrast_batch_keeps_post_average_anchor`. Decide whether the guard should use a batch-independent anchor (the treatment contrast, or the absolute floor only) and then drop the column | `diff_diff/estimators.py`, `diff_diff/linalg.py` | #v4-4c-ii | Mid | Medium |