fix(python): avoid false edges for pytest decorators - #2739
Conversation
There was a problem hiding this comment.
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
This PR adds pytest-decorator filtering to the Python extractor to prevent false decorator edges. It introduces an ordered module-level binding tracker (_python_pytest_bindings / _PythonBindingEvent) and an _is_pytest_decorator_noise check that classifies decorators like @pytest.fixture, @pytest.mark.*, and rebound @fixture names by qualified path or import binding in force at the decorator's location, rather than by bare tail name. It wires this check into the generic decorator extraction branch, adds supporting constants and a changelog entry, and includes a corresponding test.
No blocking issues surfaced. 4 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 818 functions depend on the 423 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
_extract_generic()— 18 callers, 24 callees - worse:
walk()— 1 callers, 56 callees
Verification — 818 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 759 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify \_extract\_generic.
The verifier did not have enough to check \_extract\_generic, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_python\_decorator\_name.
The verifier did not have enough to check \_python\_decorator\_name, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 182 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly AttributeError — names the real obstacle, not a sampling gap)
· 2 more finding(s) on lines outside this diff (see the check run).
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 2 advisory finding(s) below merit a look before merge.
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
Suppresses false decorator edges for pytest decorators (@pytest.fixture, @pytest.mark.*, @fixture imported from pytest) that the unique-function rewire would otherwise collapse onto same-named local definitions in the corpus. Matches these by qualified path and import-scope binding rather than bare tail name — via _python_module_pytest_bindings, which records ordered module-level binding events replayed per decorator site through _python_binding_at, so a def fixture/del pytest before a decorator (or a rebinding after an earlier genuine @fixture) resolves to the binding actually in force. A wildcard from pytest import * records no events and is conservatively not treated as pytest vocabulary.
Worth a look
- Decorator tail extraction preserves whitespace after dotted attributes —
graphify/extractors/engine.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- pytest_symbol binding for
marktreated as decorator, but@markbare use returns True incorrectly —graphify/extractors/engine.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 907 functions depend on the 466 functions this change touches.
Health — this change adds coupling hotspots:
- new:
_extract_generic()— 18 callers, 27 callees - new:
extract_js()— 85 callers, 4 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_objc()— 27 callers, 9 callees - new:
extract_julia()— 17 callers, 7 callees - new:
extract_cpp()— 27 callers, 3 callees - new:
extract_vue()— 10 callers, 7 callees - new:
walk()— 1 callers, 61 callees - …and 8 more — each is listed as a finding
Verification — 907 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 847 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
268 of 268 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_affected_cli.py— full-run-safetytests/test_affected_member_seed.py— full-run-safetytests/test_agents_platform.py— full-run-safetytests/test_analyze.py— full-run-safetytests/test_anthropic_custom_endpoint.py— full-run-safetytests/test_antigravity_install.py— full-run-safetytests/test_apm_fallback_version.py— full-run-safetytests/test_architecture_doc.py— full-run-safetytests/test_astro_extraction.py— impact, full-run-safetytests/test_astro_import_ids.py— full-run-safetytests/test_atomic_canvas_export.py— full-run-safetytests/test_atomic_version_stamp.py— full-run-safetytests/test_atomic_writes.py— full-run-safetytests/test_backend_env_isolation.py— full-run-safetytests/test_backend_extras.py— full-run-safetytests/test_benchmark.py— full-run-safetytests/test_benchmark_raw_graph.py— full-run-safetytests/test_build.py— impact, full-run-safetytests/test_build_merge_hyperedges_and_prune.py— full-run-safetytests/test_build_merge_shrink_guard.py— full-run-safetytests/test_builtin_global_type_refs.py— full-run-safetytests/test_cache.py— full-run-safetytests/test_callflow_html.py— full-run-safetytests/test_cargo_introspect.py— full-run-safetytests/test_carried_hyperedge_remap.py— full-run-safetytests/test_case_sensitive_resolution.py— full-run-safetytests/test_charmap_encoding.py— full-run-safetytests/test_chunking.py— full-run-safetytests/test_cjs_module_extension.py— impact, full-run-safetytests/test_claude_cli_backend.py— full-run-safetytests/test_claude_md.py— full-run-safetytests/test_cli_broken_pipe.py— full-run-safetytests/test_cli_export.py— full-run-safetytests/test_cli_help.py— full-run-safetytests/test_cluster.py— full-run-safetytests/test_codebuddy.py— full-run-safetytests/test_community_hub_labels.py— full-run-safetytests/test_community_labels_skill.py— full-run-safetytests/test_confidence.py— full-run-safetytests/test_corrupt_graph_json.py— full-run-safetytests/test_cpp_nested_and_cli.py— impact, full-run-safetytests/test_cpp_objc_cross_file_calls.py— full-run-safetytests/test_cpp_preprocess.py— full-run-safetytests/test_cross_extension_reexport_self_cycle.py— full-run-safetytests/test_cross_language_call_resolution.py— full-run-safetytests/test_cross_repo_member_calls.py— full-run-safetytests/test_cross_repo_shared_types.py— full-run-safetytests/test_csharp_call_site_generic_args.py— full-run-safetytests/test_csharp_enum_members.py— full-run-safetytests/test_csharp_field_generic_args.py— full-run-safety- … and 218 more
non-code file(s) changed (
CHANGELOG.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
CHANGELOG.md) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
Could not verify: Could not verify \_extract\_generic.
The verifier did not have enough to check \_extract\_generic, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_python\_decorator\_name.
The verifier did not have enough to check \_python\_decorator\_name, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 28 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly AttributeError — names the real obstacle, not a sampling gap)
· 16 more finding(s) on lines outside this diff (see the check run).
) Python decorator extraction resolves decorators by their bare tail symbol, so third-party pytest decorators (@pytest.fixture, @pytest.mark.parametrize) collapse onto same-named functions in the corpus when one exists, minting spurious reference edges and misleading high-degree nodes. Track pytest bindings at module scope in source order and classify a decorator as pytest vocabulary only when the binding in force at its source location proves it: via the pytest module root (import pytest, import pytest as pt), a decorator-capable direct import (from pytest import fixture), or the mark root (from pytest import mark) used qualified (@mark.x / @m.x). Rebindings (def/class/assignment/for/walrus/with) and del statements shadow the name and restore normal edges. A wildcard from pytest import * records nothing and is conservatively treated as non-pytest. Corpus-owned decorators (bare @fixture, @mark) keep their edges. Co-authored-by: graphify-labs bot review feedback (Graphify-Labs#2739 findings).
39bef83 to
71bb212
Compare
|
Thanks for the review. Both "Worth a look" findings are addressed; the branch was also rebased onto current 1. "Decorator tail extraction preserves whitespace after dotted attributes" — not reproducible; no code change. 2. " Now Regression tests added:
Prior behavior is unchanged: 31/31 decorator tests pass and the full suite shows no new failures (the profile of pre-existing environmental failures is identical). |
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 2 advisory finding(s) below merit a look before merge.
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
Suppresses false decorator edges from pytest's ambient decorators (@pytest.fixture, @pytest.mark.*, @pytest.hookimpl/hookspec, and from pytest import-rebound forms) so the unique-function rewire no longer collapses them onto same-named local definitions in a corpus. Matches pytest decorators by qualified path and by an ordered, source-position–aware replay of module-level bindings (_python_module_pytest_bindings / _python_binding_at) rather than by bare tail symbol, so a corpus's own @fixture keeps its edge, and a def/class/assignment/del that rebinds or unbinds pytest before a decorator site is honored. Only module-level bindings are tracked and a wildcard from pytest import * records nothing, so those cases are conservatively left un-suppressed.
Worth a look
- Non-pytest imports do not shadow prior pytest decorator bindings —
graphify/extractors/engine.py:6468· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Decorator names include whitespace for valid spaced attribute syntax —
graphify/extractors/engine.py:6595· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 909 functions depend on the 468 functions this change touches.
Health — this change adds coupling hotspots:
- new:
_extract_generic()— 18 callers, 27 callees - new:
extract_js()— 85 callers, 4 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_objc()— 27 callers, 9 callees - new:
extract_julia()— 17 callers, 7 callees - new:
extract_cpp()— 27 callers, 3 callees - new:
extract_vue()— 10 callers, 7 callees - new:
walk()— 1 callers, 61 callees - …and 8 more — each is listed as a finding
Verification — 909 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 849 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
268 of 268 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_affected_cli.py— full-run-safetytests/test_affected_member_seed.py— full-run-safetytests/test_agents_platform.py— full-run-safetytests/test_analyze.py— full-run-safetytests/test_anthropic_custom_endpoint.py— full-run-safetytests/test_antigravity_install.py— full-run-safetytests/test_apm_fallback_version.py— full-run-safetytests/test_architecture_doc.py— full-run-safetytests/test_astro_extraction.py— impact, full-run-safetytests/test_astro_import_ids.py— full-run-safetytests/test_atomic_canvas_export.py— full-run-safetytests/test_atomic_version_stamp.py— full-run-safetytests/test_atomic_writes.py— full-run-safetytests/test_backend_env_isolation.py— full-run-safetytests/test_backend_extras.py— full-run-safetytests/test_benchmark.py— full-run-safetytests/test_benchmark_raw_graph.py— full-run-safetytests/test_build.py— impact, full-run-safetytests/test_build_merge_hyperedges_and_prune.py— full-run-safetytests/test_build_merge_shrink_guard.py— full-run-safetytests/test_builtin_global_type_refs.py— full-run-safetytests/test_cache.py— full-run-safetytests/test_callflow_html.py— full-run-safetytests/test_cargo_introspect.py— full-run-safetytests/test_carried_hyperedge_remap.py— full-run-safetytests/test_case_sensitive_resolution.py— full-run-safetytests/test_charmap_encoding.py— full-run-safetytests/test_chunking.py— full-run-safetytests/test_cjs_module_extension.py— impact, full-run-safetytests/test_claude_cli_backend.py— full-run-safetytests/test_claude_md.py— full-run-safetytests/test_cli_broken_pipe.py— full-run-safetytests/test_cli_export.py— full-run-safetytests/test_cli_help.py— full-run-safetytests/test_cluster.py— full-run-safetytests/test_codebuddy.py— full-run-safetytests/test_community_hub_labels.py— full-run-safetytests/test_community_labels_skill.py— full-run-safetytests/test_confidence.py— full-run-safetytests/test_corrupt_graph_json.py— full-run-safetytests/test_cpp_nested_and_cli.py— impact, full-run-safetytests/test_cpp_objc_cross_file_calls.py— full-run-safetytests/test_cpp_preprocess.py— full-run-safetytests/test_cross_extension_reexport_self_cycle.py— full-run-safetytests/test_cross_language_call_resolution.py— full-run-safetytests/test_cross_repo_member_calls.py— full-run-safetytests/test_cross_repo_shared_types.py— full-run-safetytests/test_csharp_call_site_generic_args.py— full-run-safetytests/test_csharp_enum_members.py— full-run-safetytests/test_csharp_field_generic_args.py— full-run-safety- … and 218 more
non-code file(s) changed (
CHANGELOG.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
CHANGELOG.md) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
Could not verify: Could not verify \_extract\_generic.
The verifier did not have enough to check \_extract\_generic, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_python\_decorator\_name.
The verifier did not have enough to check \_python\_decorator\_name, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 28 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly AttributeError — names the real obstacle, not a sampling gap)
· 16 more finding(s) on lines outside this diff (see the check run).
Problem
Python decorator extraction currently resolves decorator names using the bare symbol name. This causes third-party pytest decorators such as
@pytest.fixtureand@pytest.mark.parametrizeto be treated as references to same-named functions in the corpus.When a project contains a local
fixture()orparametrize()function, repeated pytest decorators can therefore collapse onto that local node and create spuriousreferencesedges, producing misleading high-degree/"God Node" results.Fixes #2732.
Fix
import pytestand aliased imports such asimport pytest as pt.from pytest import fixtureandfrom pytest import mark.@fixture.delstatements so shadowed pytest names are not incorrectly suppressed.pytest.mark.*decorators without fabricating nodes for marker names.Tests
Adds regression coverage for:
@pytest.fixture@pytest.mark.parametrizewith/for/walrus rebindingdelunbinding@pytest.fixtureremaining a normal corpus referenceAlso updates the changelog for the fix.
Result
Pytest decorators are treated as external test-framework vocabulary when their binding proves they refer to pytest, preventing false cross-file decorator edges and spurious God Nodes while preserving legitimate project-owned decorators.