Python test-selection recall: -m spawns, getattr f-string dispatch, typing gaps, builtin protocol (stacked on #1873, #1878)

graph/bundle/SCHEMA.md

@@ -442,7 +442,7 @@ One row per place a call is written (or, for a synthesised edge, the construct t
 - **typescript** — end_line / end_column come from the expression row; the call-site row itself records only the start.
 - **javascript** — caller_id is the parser's enclosing method, or the module initializer for top-level code. end_line / end_column come from the expression row. `require()` is a module edge, not a call site.
 - **typescript** — PROPERTY_READ and PROPERTY_WRITE rows are accessor invocations with no written call: the site is the property-access expression that runs the getter or setter, positioned from the expressions table, and callee_name is NULL because nothing was written; the accessor's name is on the callee's methods row. Filter them out with kind NOT IN (…) when counting calls.
-- **python** — PROPERTY_READ, CONTEXT_MANAGER, ITERATION_PROTOCOL, METACLASS_CREATION and DYNAMIC_CALL rows are protocol or indirect edges with no written call: their site is the expression that triggers them, and callee_name is always NULL because nothing was written. SUBSCRIPT_CALL is NULL only when the subscript is not a written name (measured 206 of 337 rows on a Python subject). Filter them out with kind NOT IN (…) when counting calls.
+- **python** — PROPERTY_READ, PROPERTY_WRITE, CONTEXT_MANAGER, ITERATION_PROTOCOL, BUILTIN_PROTOCOL, METACLASS_CREATION and DYNAMIC_CALL rows are protocol or indirect edges with no written call: their site is the expression that triggers them, and callee_name is always NULL because nothing was written. SUBSCRIPT_CALL is NULL only when the subscript is not a written name (measured 206 of 337 rows on a Python subject). Filter them out with kind NOT IN (…) when counting calls.
 - **python** — The id is an EXPRESSION hash for a written call; a DECORATOR hash (PY_DECORATOR_…) for DECORATOR_APPLICATION and DECORATOR_* sites, positioned at the decorator line; and the class's TYPE hash for METACLASS_CREATION, positioned at the class declaration.
 
 ### `call_edges`
@@ -531,9 +531,11 @@ THE GRAPH. One row per (site, resolved target). A site with N possible targets h
 | `DECORATOR_*` | python | Applying an unparenthesised decorator; the suffix is the parser's decorator kind: BARE, ATTRIBUTE, SUBSCRIPT, EXPRESSION (and CALL/ATTRIBUTE_CALL when the factory expression is not itself a call site). The site is the decorator hash. |
 | `METACLASS_CREATION` | python | A class statement invokes its metaclass's `__new__` / `__init__` at import time, whether the metaclass is written on the statement (`class X(metaclass=M)`) or inherited from a base, and the nearest base's `__init_subclass__`. No written call; the site is the class's type hash. |
 | `PROPERTY_READ` | python | Reading `obj.attr` where `attr` is a `@property` runs the getter; reading `Cls.attr` where the METACLASS defines `attr` as a property runs that getter. No written call; the site is the attribute-access expression. |
+| `PROPERTY_WRITE` | python | Assigning `obj.attr = v` where `attr` is a `@property` with a setter runs the setter; `del obj.attr` runs its deleter. No written call; the site is the attribute-access expression. |
 | `CONTEXT_MANAGER` | python | `with expr:` runs `__enter__` / `__exit__` (or the async pair). No written call; the site is the context-manager expression. |
 | `ITERATION_PROTOCOL` | python | `for x in expr:` (and comprehensions) runs `__iter__` / `__next__` (or the async pair). No written call; the site is the iterated expression. |
 | `SUBSCRIPT_PROTOCOL` | python | `x[k]` runs `__getitem__` (and `x[k] = v` / `del x[k]` the setter and deleter) of the receiver's class. No written call; the site is the subscript expression. Its own kind so it is never counted as a written call. |
+| `BUILTIN_PROTOCOL` | python | `repr(x)`, `str(x)`, `len(x)`, `hash(x)`, `bool(x)`, `iter(x)`, `next(x)`, `abs(x)`, `format(x)` and `reversed(x)` run the matching dunder of the argument's class (str falls back to `__repr__`, bool to `__len__`). The written call is to the builtin; this edge is the dunder it runs. The site is the argument expression. Its own kind so it is never counted as a written call. |
 
 **`call_edges.tier` values**
 

graph/bundle/schema.ts

@@ -667,9 +667,11 @@ export const VOCAB: readonly VocabSpec[] = [
   { table: 'call_edges', column: 'kind', value: 'DECORATOR_*', languages: P, meaning: 'Applying an unparenthesised decorator; the suffix is the parser\'s decorator kind: BARE, ATTRIBUTE, SUBSCRIPT, EXPRESSION (and CALL/ATTRIBUTE_CALL when the factory expression is not itself a call site). The site is the decorator hash.' },
   { table: 'call_edges', column: 'kind', value: 'METACLASS_CREATION', languages: P, meaning: 'A class statement invokes its metaclass\'s `__new__` / `__init__` at import time, whether the metaclass is written on the statement (`class X(metaclass=M)`) or inherited from a base, and the nearest base\'s `__init_subclass__`. No written call; the site is the class\'s type hash.' },
   { table: 'call_edges', column: 'kind', value: 'PROPERTY_READ', languages: P, meaning: 'Reading `obj.attr` where `attr` is a `@property` runs the getter; reading `Cls.attr` where the METACLASS defines `attr` as a property runs that getter. No written call; the site is the attribute-access expression.' },
+  { table: 'call_edges', column: 'kind', value: 'PROPERTY_WRITE', languages: P, meaning: 'Assigning `obj.attr = v` where `attr` is a `@property` with a setter runs the setter; `del obj.attr` runs its deleter. No written call; the site is the attribute-access expression.' },
   { table: 'call_edges', column: 'kind', value: 'CONTEXT_MANAGER', languages: P, meaning: '`with expr:` runs `__enter__` / `__exit__` (or the async pair). No written call; the site is the context-manager expression.' },
   { table: 'call_edges', column: 'kind', value: 'ITERATION_PROTOCOL', languages: P, meaning: '`for x in expr:` (and comprehensions) runs `__iter__` / `__next__` (or the async pair). No written call; the site is the iterated expression.' },
   { table: 'call_edges', column: 'kind', value: 'SUBSCRIPT_PROTOCOL', languages: P, meaning: '`x[k]` runs `__getitem__` (and `x[k] = v` / `del x[k]` the setter and deleter) of the receiver\'s class. No written call; the site is the subscript expression. Its own kind so it is never counted as a written call.' },
+  { table: 'call_edges', column: 'kind', value: 'BUILTIN_PROTOCOL', languages: P, meaning: '`repr(x)`, `str(x)`, `len(x)`, `hash(x)`, `bool(x)`, `iter(x)`, `next(x)`, `abs(x)`, `format(x)` and `reversed(x)` run the matching dunder of the argument\'s class (str falls back to `__repr__`, bool to `__len__`). The written call is to the builtin; this edge is the dunder it runs. The site is the argument expression. Its own kind so it is never counted as a written call.' },
 
   // entry_points.reason
   { table: 'entry_points', column: 'reason', value: 'main', languages: ['java', 'csharp'], meaning: 'A static `main`. C#: a static `Main`, or the method top-level statements compile to.' },
@@ -748,7 +750,7 @@ export const NOTES: readonly NoteSpec[] = [
   { language: 'typescript', table: 'overrides', note: 'EMPTY — this table is Java-shaped. The TypeScript dispatch envelope is in dispatch_candidates, with basis `nominal` or `structural`.' },
   { language: 'typescript', table: 'type_instantiated', note: 'Every row has how = `new`. Not restricted to client provenance: a type the library constructs is still a type that exists at run time, and dropping it would narrow the envelope unsoundly.' },
   { language: 'typescript', table: 'call_sites', note: 'PROPERTY_READ and PROPERTY_WRITE rows are accessor invocations with no written call: the site is the property-access expression that runs the getter or setter, positioned from the expressions table, and callee_name is NULL because nothing was written; the accessor\'s name is on the callee\'s methods row. Filter them out with kind NOT IN (…) when counting calls.' },
-  { language: 'python', table: 'call_sites', note: 'PROPERTY_READ, CONTEXT_MANAGER, ITERATION_PROTOCOL, METACLASS_CREATION and DYNAMIC_CALL rows are protocol or indirect edges with no written call: their site is the expression that triggers them, and callee_name is always NULL because nothing was written. SUBSCRIPT_CALL is NULL only when the subscript is not a written name (measured 206 of 337 rows on a Python subject). Filter them out with kind NOT IN (…) when counting calls.' },
+  { language: 'python', table: 'call_sites', note: 'PROPERTY_READ, PROPERTY_WRITE, CONTEXT_MANAGER, ITERATION_PROTOCOL, BUILTIN_PROTOCOL, METACLASS_CREATION and DYNAMIC_CALL rows are protocol or indirect edges with no written call: their site is the expression that triggers them, and callee_name is always NULL because nothing was written. SUBSCRIPT_CALL is NULL only when the subscript is not a written name (measured 206 of 337 rows on a Python subject). Filter them out with kind NOT IN (…) when counting calls.' },
   { language: 'python', table: 'call_sites', note: 'The id is an EXPRESSION hash for a written call; a DECORATOR hash (PY_DECORATOR_…) for DECORATOR_APPLICATION and DECORATOR_* sites, positioned at the decorator line; and the class\'s TYPE hash for METACLASS_CREATION, positioned at the class declaration.' },
   { language: 'python', table: 'call_edges', note: 'A `boundary_lib` edge may point at a builtin (callee_provenance builtin, callee_label `builtin:NAME`) or at an unstaged import path (callee_provenance external) — neither has a methods row.' },
   { language: 'java', table: 'call_edges', note: 'A `boundary_lib` edge with callee_provenance external names a method of an ancestor type no staged IR declares (callee_label `external:<type>.<name>`, no methods row). A site whose receiver is declared as such a type is multi_inferred even with one client override: the platform method itself, and the platform\'s own subclasses, are the other possible targets. Stage the library to replace the label with the real method.' },

graph/python/engine/call-edge-generation/call_chain.dl

@@ -156,6 +156,40 @@ method_returns_method("client", m, r) :-
     expr_call_candidate(site, g),
     method_returns_method("client", g, r).
 
+// A function returning a member it LOOKED UP BY NAME: `def get_visitor(self, node): return
+// getattr(self, f"visit_{type(node).__name__}", None)`, called as `f = self.get_visitor(node);
+// f(node)`. The lookup is resolved where it is written (name-resolution.dl's getattr clauses),
+// and the method hands that set back exactly as it would hand back a bare name.
+method_returns_method("client", m, r) :-
+    method_return_value_expr("client", m, e),
+    call_of_expr(e, site), call_name(site, "getattr"),
+    expr_denotes_method("client", e, r).
+
+// ── a call that hands back one of its arguments (py_returns_arg, resolution/builtins.dl) ──
+// Matched through the import that binds the callee, never by the bare name:
+// `functools.update_wrapper(...)` / `t.cast(...)` where the receiver is the name an `import`
+// bound, or `update_wrapper(...)` bound by `from functools import update_wrapper`. A local
+// function that happens to share the name does neither.
+py_passthrough_arg(site, pos) :-
+    call_name(site, fn), call_receiver_object(site, obj),
+    expr_binding("client", rb, ctx, obj), ctx != "STORE", binding_lookup("client", rb, rb2),
+    import_binding("client", rb2, i), import_decl("client", k, mod, _, i),
+    (k = "MODULE_IMPORT" ; k = "MODULE_IMPORT_ALIAS"),   // `import typing as t` is the alias kind
+    py_returns_arg(path, pos), cat(mod, cat(".", fn)) = path.
+py_passthrough_arg(site, pos) :-
+    call_callee_is_value(site), call_callee_expr(site, callee),
+    expr_binding("client", rb, ctx, callee), ctx != "STORE", binding_lookup("client", rb, rb2),
+    import_binding("client", rb2, i), import_decl("client", _, path, _, i),
+    py_returns_arg(path, pos).
+// the expression a value really is, through any number of such calls
+py_passthrough_root(e, e) :- method_return_value_expr("client", _, e).
+py_passthrough_root(e, x) :-
+    py_passthrough_root(e, c), call_of_expr(c, site), py_passthrough_arg(site, pos),
+    call_arg(site, pos, x).
+method_returns_method("client", m, r) :-
+    method_return_value_expr("client", m, e), py_passthrough_root(e, x), x != e,
+    expr_names_method("client", x, r).
+
 // ── decorator_hits_lib(SiteKey, LibMethodHash) ───────────────────────────────
 // `@abstractmethod` names `abc.abstractmethod`, which HAS Python source and is in the
 // staged stdlib IR — it is a library boundary, not a blind spot. A BARE decorator has no
@@ -320,6 +354,25 @@ iter_protocol_edge(src, caller, m) :-
     expr_type("client", src, t), iter_protocol_target(t, m),
     expr_ultimate_method("client", src, caller).
 
+// ── property_write_edge(WriteExprHash, CallerMethodHash, AccessorMethodHash) ──
+// The store and delete halves of the property protocol: `obj.x = v` calls x's setter,
+// `del obj.x` its deleter (type_property_accessor, resolution/attribute-lookup.dl). Same
+// shape as a read, keyed on the access's own name context.
+property_write_edge(e, caller, m) :-
+    expr_node("client", "ATTRIBUTE_ACCESS", _, n, e),
+    expr_name_context("client", "STORE", e),
+    expr_parent("client", e, "ATTRIBUTE_OBJECT", _, obj),
+    expr_type("client", obj, t),
+    type_property_accessor("client", t, n, "PROPERTY_SETTER", m),
+    expr_ultimate_method("client", e, caller).
+property_write_edge(e, caller, m) :-
+    expr_node("client", "ATTRIBUTE_ACCESS", _, n, e),
+    expr_name_context("client", "DEL", e),
+    expr_parent("client", e, "ATTRIBUTE_OBJECT", _, obj),
+    expr_type("client", obj, t),
+    type_property_accessor("client", t, n, "PROPERTY_DELETER", m),
+    expr_ultimate_method("client", e, caller).
+
 // ── property_read_edge(ReadExprHash, CallerMethodHash, GetterMethodHash) ─────
 property_read_edge(e, caller, getter) :-
     expr_node("client", "ATTRIBUTE_ACCESS", _, n, e),
@@ -629,6 +682,33 @@ subscript_protocol_edge(sub, caller, m) :-
     mro_lookup("client", t, "__class_getitem__", m),
     expr_ultimate_method("client", sub, caller).
 
+// ── BUILTIN PROTOCOL — `repr(x)` runs type(x).__repr__ ─────────────────────────
+// A builtin function that exists to call one dunder of its argument: repr, str, len,
+// hash, bool, iter, next, abs, format, reversed. The written call resolves to the builtin
+// (C code, boundary_lib), and the dunder it runs -- a client method, often the one a test
+// exists to check -- had no caller: `assert repr(v) == "<...>"` reached nothing of v's.
+// Only the BARE builtin name (bound to nothing in the client, so it is the builtin), with
+// exactly one argument (`str(b, "utf-8")` decodes and runs no __str__), on an argument the
+// engine can type. str() falls back to __repr__ and bool() to __len__ when the class
+// defines no __str__ / __bool__ of its own, as CPython's type slots do. The builtin names
+// and their dunders are catalogued in resolution/builtins.dl (builtin_protocol_slot).
+builtin_protocol_arg(e, fn, x) :-
+    call_of_expr(e, site), call_callee_is_value(site), call_callee_expr(site, callee),
+    expr_binding("client", b, _, callee), binding_lookup_unresolved("client", b, fn, _),
+    builtin_protocol_slot(fn, _),
+    call_arg(site, "0", x), !call_arg(site, "1", _), !call_kwarg(site, _, _).
+// The edge's site is the ARGUMENT expression, as an iteration's is the iterated one: the
+// call expression is already the written call to the builtin.
+builtin_protocol_edge(x, caller, m) :-
+    builtin_protocol_arg(_, fn, x), builtin_protocol_slot(fn, d),
+    expr_type("client", x, t), mro_lookup("client", t, d, m),
+    expr_ultimate_method("client", x, caller).
+builtin_protocol_edge(x, caller, m) :-
+    builtin_protocol_arg(_, fn, x), builtin_protocol_fallback(fn, d, d2),
+    expr_type("client", x, t), !type_has_member(t, d), mro_lookup("client", t, d2, m),
+    expr_ultimate_method("client", x, caller).
+type_has_member(t, d) :- builtin_protocol_fallback(_, d, _), mro_lookup("client", t, d, _).
+
 // ── THE THREE PROTOCOL EDGES ARE TIERED BY TARGET COUNT, like every other edge ──
 //
 // A property read, a context-manager entry and an iteration all reach their target
@@ -668,9 +748,11 @@ subscript_protocol_edge(sub, caller, m) :-
 // from the graph rather than being re-tiered. A dropped edge is worse than a mislabelled
 // one, and the golden caught it.
 protocol_edge(e, d, caller, m) :- property_read_edge(e, caller, m), method_decl(_, d, _, _, _, m).
+protocol_edge(e, d, caller, m) :- property_write_edge(e, caller, m), method_decl(_, d, _, _, _, m).
 protocol_edge(e, d, caller, m) :- with_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
 protocol_edge(e, d, caller, m) :- iter_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
 protocol_edge(e, d, caller, m) :- subscript_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
+protocol_edge(e, d, caller, m) :- builtin_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
 
 // Aggregate in its own stratum, mirroring site_client_target_count above.
 protocol_edge_target_count(e, d, n) :-
@@ -731,6 +813,12 @@ call_chain_edge(e, caller, "-", getter, "client", cls, "PROPERTY_READ") :-
     property_read_edge(e, caller, getter), method_decl(_, d, _, _, _, getter),
     protocol_edge_class(e, d, cls).
 
+// PROPERTY WRITE — `obj.x = v` runs x's setter and `del obj.x` its deleter. Likewise no
+// call site, and its own kind so it is never counted as a written call.
+call_chain_edge(e, caller, "-", m, "client", cls, "PROPERTY_WRITE") :-
+    property_write_edge(e, caller, m), method_decl(_, d, _, _, _, m),
+    protocol_edge_class(e, d, cls).
+
 // CONTEXT MANAGER — the same shape: an edge with no call site, its own kind so it can
 // never be mistaken for a written call.
 call_chain_edge(cm, caller, "-", m, "client", cls, "CONTEXT_MANAGER") :-
@@ -749,6 +837,12 @@ call_chain_edge(sub, caller, "-", m, "client", cls, "SUBSCRIPT_PROTOCOL") :-
     subscript_protocol_edge(sub, caller, m), method_decl(_, d, _, _, _, m),
     protocol_edge_class(sub, d, cls).
 
+// BUILTIN PROTOCOL — likewise. Its own kind: the written call is to the builtin, and
+// this edge is the dunder that builtin runs, never a second written call.
+call_chain_edge(x, caller, "-", m, "client", cls, "BUILTIN_PROTOCOL") :-
+    builtin_protocol_edge(x, caller, m), method_decl(_, d, _, _, _, m),
+    protocol_edge_class(x, d, cls).
+
 // A site whose caller could not be determined AT ALL would vanish from every rule
 // above. That must be impossible (the parser guarantees a non-empty owner), so it is
 // asserted rather than assumed: any such site is emitted with caller "-" so the

graph/python/engine/config-resolution/knobs.dl

@@ -165,6 +165,14 @@ py_parametrize_deco("parametrize").
 py_parametrize_argnames_kw("argnames").
 py_parametrize_indirect_kw("indirect").
 py_parametrize_indirect_all("True").
+// The VALUES the runner hands in: the decorator's second argument (or `argvalues=`), a
+// fixture's `params=`, a row wrapped as `pytest.param(v, ..., id=...)`, and, inside a fixture
+// declared with params, the attribute `request.param` it reads the current one from.
+py_parametrize_argvalues_kw("argvalues").
+py_fixture_params_kw("params").
+py_param_row_wrapper("param").
+py_fixture_request_param("request").
+py_request_param_attr("param").
 
 // ── py_argnames_sep / py_argnames_lead: how "a, b" separates argument names ──
 // The runner splits the string on "," and strips each piece. Souffle cannot split, so a