Skip to content

[mypyc] Fix cross-group call to inherited __mypyc_defaults_setup#21481

Merged
p-sawicki merged 5 commits into
python:masterfrom
VaggelisD:upstream-fix-cross-group-defaults-setup-call
May 19, 2026
Merged

[mypyc] Fix cross-group call to inherited __mypyc_defaults_setup#21481
p-sawicki merged 5 commits into
python:masterfrom
VaggelisD:upstream-fix-cross-group-defaults-setup-call

Conversation

@georgesittas
Copy link
Copy Markdown
Contributor

@georgesittas georgesittas commented May 13, 2026

With separate=True and cross-module inheritance, when only the subclass module is recompiled incrementally and the parent is loaded from mypy's incremental cache, find_attr_initializers gathers no defaults from the parent. The subclass therefore has no __mypyc_defaults_setup of its own, and ClassIR.get_method walks the MRO and returns the parent's.

Without this fix, emit_attr_defaults_func_call emits a raw CPyDef_<parent>___...(...) call. The parent's header only declares that function as a pointer inside struct export_table_<group>, so the symbol isn't reachable as a free function from the subclass's compilation unit and clang/gcc fail with:

error: call to undeclared function 'CPyDef_<parent_module>___<Parent>_____mypyc_defaults_setup';
ISO C99 and later do not support implicit function declarations

A cold build doesn't hit this because the parent's defs.body is populated (everything is freshly parsed), so the subclass gets its own __mypyc_defaults_setup and the call is intra-group. Likewise, an incremental change that propagates through interface-hash deps to the parent makes it get reparsed too, avoiding the trigger. The bug requires an invalidation pattern that touches the subclass but not the parent.

Fix

emit_attr_defaults_func_call in mypyc/codegen/emitclass.py now applies emitter.get_group_prefix(defaults_fn.decl) when emitting the call, matching the pattern already used by the other cross-group call sites in this file (emit_setup_or_dunder_new_call, generate_constructor_for_class, etc.).

get_group_prefix returns "" for same-group calls (so intra-group behaviour is unchanged) and "exports_<group>." when the target lives in a different group. It also registers the target group in context.group_deps so the right header gets #included.

Tests

Added testIncrementalCrossModuleInheritedAttrDefaults in mypyc/test-data/run-multimodule.test, a two-step test that reproduces the bug under TestRunSeparate: parent in other_b.py with attribute defaults, empty subclass in other_a.py, step 2 modifies other_a.py to trigger a recompile without touching the parent. Verified to fail under TestRunSeparate (with the implicit-declaration error) on the unpatched tree and to pass under all three modes (TestRun, TestRunMultiFile, TestRunSeparate) with the fix.

Local checks:

  • pre-commit run --all-files — pass
  • python runtests.py self — pass
  • mypyc unit tests (1077 + 1 skipped) — pass
  • TestRunSeparate + TestRunMultiFile run tests (110) — pass

Real-world impact

Surfaced first against sqlglot-mypy==1.20.0.post6 (a downstream of mypy used to compile sqlglot) in CI when build/ and .mypy_cache/ were preserved across GitHub Actions runs and a PR happened to edit only subclass modules. The incremental compile produced malformed C and failed with the implicit-declaration error.

Stripped of sqlglot specifics, the bug requires only separate=True + cross-module inheritance + parent with attribute defaults + subclass without its own defaults + an invalidation pattern where the subclass is rechecked but the parent is not. These conditions are common in any non-trivial codebase using mypyc with separate=True, so this should be considered a latent issue affecting incremental builds of such codebases.

georgesittas and others added 2 commits May 13, 2026 17:58
With `separate=True` and cross-module inheritance, when a subclass module
is recompiled incrementally without its parent (parent loaded from
mypy's cache, so `ClassDef.defs.body` is empty), `find_attr_initializers`
gathers no defaults from the parent. The subclass therefore has no
`__mypyc_defaults_setup` of its own, and `ClassIR.get_method` returns
the parent's. `emit_attr_defaults_func_call` then emitted a raw
`CPyDef_<parent>___...` call with no cross-group export-table prefix,
producing C that fails to compile:

    error: call to undeclared function
    'CPyDef_<parent_module>___<Parent>_____mypyc_defaults_setup'

The parent's header only declares the function as a pointer inside
`struct export_table_<group>`, so the symbol isn't reachable as a free
function from the subclass's compilation unit.

Apply `emitter.get_group_prefix(defaults_fn.decl)` at this call site,
matching the pattern already used by `emit_setup_or_dunder_new_call`,
`generate_constructor_for_class`, and the other cross-group call sites
in `emitclass.py`. `get_group_prefix` returns `""` for same-group calls
(intra-group behaviour unchanged) and `"exports_<group>."` when the
target lives in a different group; it also registers the target group
in `context.group_deps` so the right header gets `#include`d.

Reproducer (`base.py` with attribute defaults, `child.py` empty subclass,
`mypycify([...], separate=True)`): cold build succeeds, then touching
only `child.py` and rebuilding previously failed with the
implicit-declaration error. Generated C now correctly emits
`exports_base.CPyDef_base___Parent_____mypyc_defaults_setup(...)` and
`Child().x` returns the inherited default.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reproduces the bug fixed in the parent commit: under TestRunSeparate,
the subclass module gets recompiled while the parent module is loaded
from mypy's cache (so `ClassDef.defs.body` is empty and the subclass
inherits no own `__mypyc_defaults_setup`). The emitted call to the
parent's setup function must use the cross-group `exports_<group>.`
prefix or the generated C fails to compile.

The test passes under TestRun and TestRunMultiFile (which don't
exercise cross-group calls) and fails under TestRunSeparate without
the fix. Verified by temporarily reverting `emit_attr_defaults_func_call`
to the pre-fix form and observing the implicit-declaration error in
`__native_other_a.c`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread mypyc/codegen/emitclass.py Outdated
Copy link
Copy Markdown
Collaborator

@p-sawicki p-sawicki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@p-sawicki p-sawicki merged commit 5e0c274 into python:master May 19, 2026
18 checks passed
@georgesittas georgesittas deleted the upstream-fix-cross-group-defaults-setup-call branch May 19, 2026 14:33
alicederyn pushed a commit to alicederyn/mypy that referenced this pull request May 20, 2026
…hon#21481)

With `separate=True` and cross-module inheritance, when only the
subclass module is recompiled incrementally and the parent is loaded
from mypy's incremental cache, `find_attr_initializers` gathers no
defaults from the parent. The subclass therefore has no
`__mypyc_defaults_setup` of its own, and `ClassIR.get_method` walks the
MRO and returns the parent's.

Without this fix, `emit_attr_defaults_func_call` emits a raw
`CPyDef_<parent>___...(...)` call. The parent's header only declares
that function as a pointer inside `struct export_table_<group>`, so the
symbol isn't reachable as a free function from the subclass's
compilation unit and clang/gcc fail with:

```
error: call to undeclared function 'CPyDef_<parent_module>___<Parent>_____mypyc_defaults_setup';
ISO C99 and later do not support implicit function declarations
```

A cold build doesn't hit this because the parent's `defs.body` is
populated (everything is freshly parsed), so the subclass gets its own
`__mypyc_defaults_setup` and the call is intra-group. Likewise, an
incremental change that propagates through interface-hash deps to the
parent makes it get reparsed too, avoiding the trigger. The bug requires
an invalidation pattern that touches the subclass but **not** the
parent.

## Fix

`emit_attr_defaults_func_call` in `mypyc/codegen/emitclass.py` now
applies `emitter.get_group_prefix(defaults_fn.decl)` when emitting the
call, matching the pattern already used by the other cross-group call
sites in this file (`emit_setup_or_dunder_new_call`,
`generate_constructor_for_class`, etc.).

`get_group_prefix` returns `""` for same-group calls (so intra-group
behaviour is unchanged) and `"exports_<group>."` when the target lives
in a different group. It also registers the target group in
`context.group_deps` so the right header gets `#include`d.

## Tests

Added `testIncrementalCrossModuleInheritedAttrDefaults` in
`mypyc/test-data/run-multimodule.test`, a two-step test that reproduces
the bug under `TestRunSeparate`: parent in `other_b.py` with attribute
defaults, empty subclass in `other_a.py`, step 2 modifies `other_a.py`
to trigger a recompile without touching the parent. Verified to fail
under `TestRunSeparate` (with the implicit-declaration error) on the
unpatched tree and to pass under all three modes (`TestRun`,
`TestRunMultiFile`, `TestRunSeparate`) with the fix.

Local checks:
- `pre-commit run --all-files` — pass
- `python runtests.py self` — pass
- mypyc unit tests (1077 + 1 skipped) — pass
- `TestRunSeparate` + `TestRunMultiFile` run tests (110) — pass

## Real-world impact

Surfaced first against `sqlglot-mypy==1.20.0.post6` (a downstream of
mypy used to compile sqlglot) in CI when `build/` and `.mypy_cache/`
were preserved across GitHub Actions runs and a PR happened to edit only
subclass modules. The incremental compile produced malformed C and
failed with the implicit-declaration error.

Stripped of sqlglot specifics, the bug requires only `separate=True` +
cross-module inheritance + parent with attribute defaults + subclass
without its own defaults + an invalidation pattern where the subclass is
rechecked but the parent is not. These conditions are common in any
non-trivial codebase using mypyc with `separate=True`, so this should be
considered a latent issue affecting incremental builds of such
codebases.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants