Python recall (operators, library bases, context) and --library auto for every language

.github/workflows/ci.yml

@@ -371,6 +371,16 @@ jobs:
         if: matrix.lang == 'csharp'
         run: dotnet build -c Release graph/test/csharp/ground-truth/AxiomCsOracle
 
+      # `--library auto` decompiles a dependency that ships no sources jar; the Java query case for it needs Vineflower
+      - name: fetch the decompiler `--library auto` uses for Java class jars
+        if: matrix.lang == 'java'
+        run: mvn -q dependency:get -Dartifact=org.vineflower:vineflower:1.10.1
+
+      # `--library auto` compiles a NuGet package by decompiling it; the query case for it needs the decompiler
+      - name: install the decompiler `--library auto` uses for NuGet packages
+        if: matrix.lang == 'csharp'
+        run: dotnet tool install -g ilspycmd --version 8.2.0.7535
+
       - name: cache the Soufflé package
         uses: actions/cache@v4
         with:

graph/bundle/SCHEMA.md

@@ -442,7 +442,7 @@ One row per place a call is written (or, for a synthesised edge, the construct t
 - **typescript** — end_line / end_column come from the expression row; the call-site row itself records only the start.
 - **javascript** — caller_id is the parser's enclosing method, or the module initializer for top-level code. end_line / end_column come from the expression row. `require()` is a module edge, not a call site.
 - **typescript** — PROPERTY_READ and PROPERTY_WRITE rows are accessor invocations with no written call: the site is the property-access expression that runs the getter or setter, positioned from the expressions table, and callee_name is NULL because nothing was written; the accessor's name is on the callee's methods row. Filter them out with kind NOT IN (…) when counting calls.
-- **python** — PROPERTY_READ, PROPERTY_WRITE, CONTEXT_MANAGER, ITERATION_PROTOCOL, BUILTIN_PROTOCOL, METACLASS_CREATION and DYNAMIC_CALL rows are protocol or indirect edges with no written call: their site is the expression that triggers them, and callee_name is always NULL because nothing was written. SUBSCRIPT_CALL is NULL only when the subscript is not a written name (measured 206 of 337 rows on a Python subject). Filter them out with kind NOT IN (…) when counting calls.
+- **python** — PROPERTY_READ, PROPERTY_WRITE, CONTEXT_MANAGER, ITERATION_PROTOCOL, BUILTIN_PROTOCOL, OPERATOR_PROTOCOL, TRUTH_PROTOCOL, METACLASS_CREATION and DYNAMIC_CALL rows are protocol or indirect edges with no written call: their site is the expression that triggers them, and callee_name is always NULL because nothing was written. SUBSCRIPT_CALL is NULL only when the subscript is not a written name (measured 206 of 337 rows on a Python subject). Filter them out with kind NOT IN (…) when counting calls.
 - **python** — The id is an EXPRESSION hash for a written call; a DECORATOR hash (PY_DECORATOR_…) for DECORATOR_APPLICATION and DECORATOR_* sites, positioned at the decorator line; and the class's TYPE hash for METACLASS_CREATION, positioned at the class declaration.
 
 ### `call_edges`
@@ -537,6 +537,8 @@ THE GRAPH. One row per (site, resolved target). A site with N possible targets h
 | `ITERATION_PROTOCOL` | python | `for x in expr:` (and comprehensions) runs `__iter__` / `__next__` (or the async pair). No written call; the site is the iterated expression. |
 | `SUBSCRIPT_PROTOCOL` | python | `x[k]` runs `__getitem__` (and `x[k] = v` / `del x[k]` the setter and deleter) of the receiver's class. No written call; the site is the subscript expression. Its own kind so it is never counted as a written call. |
 | `BUILTIN_PROTOCOL` | python | `repr(x)`, `str(x)`, `len(x)`, `hash(x)`, `bool(x)`, `iter(x)`, `next(x)`, `abs(x)`, `format(x)` and `reversed(x)` run the matching dunder of the argument's class (str falls back to `__repr__`, bool to `__len__`). The written call is to the builtin; this edge is the dunder it runs. The site is the argument expression. Its own kind so it is never counted as a written call. |
+| `OPERATOR_PROTOCOL` | python | An operator runs a dunder of its operands: `a + b` the left operand's `__add__` and the right one's `__radd__`, a comparison the left slot and the right one's mirror (`a < b` -> `b.__gt__`, `!=` falls back to `__eq__`), `x in c` the right operand's `__contains__` (or `__iter__`), `-a`/`+a`/`~a` its `__neg__`/`__pos__`/`__invert__`. No written call; the site is the operator expression. |
+| `TRUTH_PROTOCOL` | python | A truth test runs the tested value's `__bool__`, or `__len__` when its class has none: `if x:`, `while x:`, `assert x`, `not x`, a conditional or comprehension condition, and an operand of `and`/`or` (the right one only when the whole expression is tested). No written call; the site is the tested expression. |
 
 **`call_edges.tier` values**
 

graph/bundle/schema.ts

@@ -673,6 +673,8 @@ export const VOCAB: readonly VocabSpec[] = [
   { table: 'call_edges', column: 'kind', value: 'ITERATION_PROTOCOL', languages: P, meaning: '`for x in expr:` (and comprehensions) runs `__iter__` / `__next__` (or the async pair). No written call; the site is the iterated expression.' },
   { table: 'call_edges', column: 'kind', value: 'SUBSCRIPT_PROTOCOL', languages: P, meaning: '`x[k]` runs `__getitem__` (and `x[k] = v` / `del x[k]` the setter and deleter) of the receiver\'s class. No written call; the site is the subscript expression. Its own kind so it is never counted as a written call.' },
   { table: 'call_edges', column: 'kind', value: 'BUILTIN_PROTOCOL', languages: P, meaning: '`repr(x)`, `str(x)`, `len(x)`, `hash(x)`, `bool(x)`, `iter(x)`, `next(x)`, `abs(x)`, `format(x)` and `reversed(x)` run the matching dunder of the argument\'s class (str falls back to `__repr__`, bool to `__len__`). The written call is to the builtin; this edge is the dunder it runs. The site is the argument expression. Its own kind so it is never counted as a written call.' },
+  { table: 'call_edges', column: 'kind', value: 'OPERATOR_PROTOCOL', languages: P, meaning: 'An operator runs a dunder of its operands: `a + b` the left operand\'s `__add__` and the right one\'s `__radd__`, a comparison the left slot and the right one\'s mirror (`a < b` -> `b.__gt__`, `!=` falls back to `__eq__`), `x in c` the right operand\'s `__contains__` (or `__iter__`), `-a`/`+a`/`~a` its `__neg__`/`__pos__`/`__invert__`. No written call; the site is the operator expression.' },
+  { table: 'call_edges', column: 'kind', value: 'TRUTH_PROTOCOL', languages: P, meaning: 'A truth test runs the tested value\'s `__bool__`, or `__len__` when its class has none: `if x:`, `while x:`, `assert x`, `not x`, a conditional or comprehension condition, and an operand of `and`/`or` (the right one only when the whole expression is tested). No written call; the site is the tested expression.' },
 
   // entry_points.reason
   { table: 'entry_points', column: 'reason', value: 'main', languages: ['java', 'csharp'], meaning: 'A static `main`. C#: a static `Main`, or the method top-level statements compile to.' },
@@ -751,7 +753,7 @@ export const NOTES: readonly NoteSpec[] = [
   { language: 'typescript', table: 'overrides', note: 'EMPTY — this table is Java-shaped. The TypeScript dispatch envelope is in dispatch_candidates, with basis `nominal` or `structural`.' },
   { language: 'typescript', table: 'type_instantiated', note: 'Every row has how = `new`. Not restricted to client provenance: a type the library constructs is still a type that exists at run time, and dropping it would narrow the envelope unsoundly.' },
   { language: 'typescript', table: 'call_sites', note: 'PROPERTY_READ and PROPERTY_WRITE rows are accessor invocations with no written call: the site is the property-access expression that runs the getter or setter, positioned from the expressions table, and callee_name is NULL because nothing was written; the accessor\'s name is on the callee\'s methods row. Filter them out with kind NOT IN (…) when counting calls.' },
-  { language: 'python', table: 'call_sites', note: 'PROPERTY_READ, PROPERTY_WRITE, CONTEXT_MANAGER, ITERATION_PROTOCOL, BUILTIN_PROTOCOL, METACLASS_CREATION and DYNAMIC_CALL rows are protocol or indirect edges with no written call: their site is the expression that triggers them, and callee_name is always NULL because nothing was written. SUBSCRIPT_CALL is NULL only when the subscript is not a written name (measured 206 of 337 rows on a Python subject). Filter them out with kind NOT IN (…) when counting calls.' },
+  { language: 'python', table: 'call_sites', note: 'PROPERTY_READ, PROPERTY_WRITE, CONTEXT_MANAGER, ITERATION_PROTOCOL, BUILTIN_PROTOCOL, OPERATOR_PROTOCOL, TRUTH_PROTOCOL, METACLASS_CREATION and DYNAMIC_CALL rows are protocol or indirect edges with no written call: their site is the expression that triggers them, and callee_name is always NULL because nothing was written. SUBSCRIPT_CALL is NULL only when the subscript is not a written name (measured 206 of 337 rows on a Python subject). Filter them out with kind NOT IN (…) when counting calls.' },
   { language: 'python', table: 'call_sites', note: 'The id is an EXPRESSION hash for a written call; a DECORATOR hash (PY_DECORATOR_…) for DECORATOR_APPLICATION and DECORATOR_* sites, positioned at the decorator line; and the class\'s TYPE hash for METACLASS_CREATION, positioned at the class declaration.' },
   { language: 'python', table: 'call_edges', note: 'A `boundary_lib` edge may point at a builtin (callee_provenance builtin, callee_label `builtin:NAME`) or at an unstaged import path (callee_provenance external) — neither has a methods row.' },
   { language: 'java', table: 'call_edges', note: 'A `boundary_lib` edge with callee_provenance external names a method of an ancestor type no staged IR declares (callee_label `external:<type>.<name>`, no methods row). A site whose receiver is declared as such a type is multi_inferred even with one client override: the platform method itself, and the platform\'s own subclasses, are the other possible targets. Stage the library to replace the label with the real method.' },

graph/python/engine/call-edge-generation/call_chain.dl

@@ -709,6 +709,72 @@ builtin_protocol_edge(x, caller, m) :-
     expr_ultimate_method("client", x, caller).
 type_has_member(t, d) :- builtin_protocol_fallback(_, d, _), mro_lookup("client", t, d, _).
 
+// ── OPERATOR PROTOCOL — `a + b`, `a == b`, `x in c`, `-a` run a dunder ────────
+// An operator compiles to BINARY_OP / COMPARE_OP / CONTAINS_OP / UNARY_*, never to a CALL,
+// so there is no written site, and the dunder it runs -- a client `__eq__`, `__or__`,
+// `__contains__` -- had no caller at all: a test asserting `a == b` or composing two
+// strategies with `|` reached nothing of theirs. Same channel as the protocol edges above,
+// its own kind. The site is the operator expression. Both halves of the protocol are
+// emitted, because both can run: the left operand's slot, and the right operand's
+// reflected one when the left is missing it or returns NotImplemented (a comparison's
+// mirror, `a < b` -> b.__gt__). The operators and their dunders are catalogued in
+// resolution/builtins.dl. A chained comparison (`a < b < c`) carries several operators in
+// one node and matches no catalogue row, so it emits nothing rather than a guess.
+operator_protocol_edge(e, caller, m) :-
+    expr_operator("client", op, e), operator_protocol_slot(op, d, _),
+    expr_parent("client", e, "OPERAND_LEFT", _, l), expr_type("client", l, t),
+    mro_lookup("client", t, d, m), expr_ultimate_method("client", e, caller).
+operator_protocol_edge(e, caller, m) :-
+    expr_operator("client", op, e), operator_protocol_slot(op, _, d),
+    expr_parent("client", e, "OPERAND_RIGHT", _, r), expr_type("client", r, t),
+    mro_lookup("client", t, d, m), expr_ultimate_method("client", e, caller).
+// `a != b` on a class with no __ne__ of its own runs __eq__ through object.__ne__.
+operator_protocol_edge(e, caller, m) :-
+    expr_operator("client", op, e), operator_protocol_slot(op, d, _), operator_protocol_fallback(d, d2),
+    expr_parent("client", e, "OPERAND_LEFT", _, l), expr_type("client", l, t),
+    !operator_type_has(t, d), mro_lookup("client", t, d2, m), expr_ultimate_method("client", e, caller).
+operator_type_has(t, d) :- operator_protocol_fallback(d, _), mro_lookup("client", t, d, _).
+// `x in c` runs the RIGHT operand's __contains__, or iterates it when it has none.
+operator_protocol_edge(e, caller, m) :-
+    expr_operator("client", op, e), membership_operator(op), membership_protocol_slot(d),
+    expr_parent("client", e, "OPERAND_RIGHT", _, r), expr_type("client", r, t),
+    mro_lookup("client", t, d, m), expr_ultimate_method("client", e, caller).
+operator_protocol_edge(e, caller, m) :-
+    expr_operator("client", op, e), membership_operator(op), membership_protocol_fallback(d, d2),
+    expr_parent("client", e, "OPERAND_RIGHT", _, r), expr_type("client", r, t),
+    !membership_type_has(t, d), mro_lookup("client", t, d2, m), expr_ultimate_method("client", e, caller).
+membership_type_has(t, d) :- membership_protocol_fallback(d, _), mro_lookup("client", t, d, _).
+// `-a`, `+a`, `~a`
+operator_protocol_edge(e, caller, m) :-
+    expr_node("client", "UNARY_OPERATION", _, _, e), expr_operator("client", op, e), unary_protocol_slot(op, d),
+    expr_parent("client", e, "UNARY_OPERAND", _, x), expr_type("client", x, t),
+    mro_lookup("client", t, d, m), expr_ultimate_method("client", e, caller).
+
+// ── TRUTH PROTOCOL — `if x:` runs type(x).__bool__, or __len__ ───────────────
+// Truth testing has no written call either: `if x:`, `while x:`, `assert x`, `not x`, the
+// condition of `a if x else b` and of a comprehension, and an operand of `and` / `or` all
+// run x's __bool__ -- or __len__ when the class defines no __bool__ (data model,
+// object.__bool__). The left operand of `and`/`or` is always tested; the right one only
+// when the whole expression is (`return a or b` hands b back untested). The site is the
+// tested expression.
+truth_tested(x) :-
+    truth_test_root(rc), expr_root_context("client", rc, x), !expr_has_parent("client", x).
+truth_tested(x) :- truth_test_role(role), expr_parent("client", _, role, _, x).
+truth_tested(x) :-
+    expr_node("client", "UNARY_OPERATION", _, _, e), expr_operator("client", op, e), truth_negation_operator(op),
+    expr_parent("client", e, "UNARY_OPERAND", _, x).
+truth_tested(x) :- expr_node("client", "BOOLEAN_OPERATION", _, _, e), expr_parent("client", e, "OPERAND_LEFT", _, x).
+truth_tested(x) :-
+    expr_node("client", "BOOLEAN_OPERATION", _, _, e), truth_tested(e), expr_parent("client", e, "OPERAND_RIGHT", _, x).
+
+truth_protocol_edge(x, caller, m) :-
+    truth_tested(x), truth_protocol_fallback(d, _), expr_type("client", x, t),
+    mro_lookup("client", t, d, m), expr_ultimate_method("client", x, caller).
+truth_protocol_edge(x, caller, m) :-
+    truth_tested(x), truth_protocol_fallback(d, d2), expr_type("client", x, t),
+    !truth_type_has(t, d), mro_lookup("client", t, d2, m), expr_ultimate_method("client", x, caller).
+truth_type_has(t, d) :- truth_protocol_fallback(d, _), mro_lookup("client", t, d, _).
+
 // ── THE THREE PROTOCOL EDGES ARE TIERED BY TARGET COUNT, like every other edge ──
 //
 // A property read, a context-manager entry and an iteration all reach their target
@@ -753,6 +819,8 @@ protocol_edge(e, d, caller, m) :- with_protocol_edge(e, caller, m), method_decl(
 protocol_edge(e, d, caller, m) :- iter_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
 protocol_edge(e, d, caller, m) :- subscript_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
 protocol_edge(e, d, caller, m) :- builtin_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
+protocol_edge(e, d, caller, m) :- operator_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
+protocol_edge(e, d, caller, m) :- truth_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m).
 
 // Aggregate in its own stratum, mirroring site_client_target_count above.
 protocol_edge_target_count(e, d, n) :-
@@ -843,6 +911,16 @@ call_chain_edge(x, caller, "-", m, "client", cls, "BUILTIN_PROTOCOL") :-
     builtin_protocol_edge(x, caller, m), method_decl(_, d, _, _, _, m),
     protocol_edge_class(x, d, cls).
 
+// OPERATOR PROTOCOL — an operator's dunder. Its own kind: there is no written call.
+call_chain_edge(e, caller, "-", m, "client", cls, "OPERATOR_PROTOCOL") :-
+    operator_protocol_edge(e, caller, m), method_decl(_, d, _, _, _, m),
+    protocol_edge_class(e, d, cls).
+
+// TRUTH PROTOCOL — the __bool__ (or __len__) a truth test runs. Its own kind likewise.
+call_chain_edge(x, caller, "-", m, "client", cls, "TRUTH_PROTOCOL") :-
+    truth_protocol_edge(x, caller, m), method_decl(_, d, _, _, _, m),
+    protocol_edge_class(x, d, cls).
+
 // A site whose caller could not be determined AT ALL would vanish from every rule
 // above. That must be impossible (the parser guarantees a non-empty owner), so it is
 // asserted rather than assumed: any such site is emitted with caller "-" so the

graph/python/engine/framework-behavior/library-callbacks.dl

@@ -76,6 +76,9 @@ lcb_expr_names(e, c) :- lcb_path_of(e, p), lcb_canon(p, c).
 .decl lcb_direct_ext_base(t:symbol, c:symbol)
 lcb_direct_ext_base(t, c) :-
     type_base_slot("client", t, _, "NAME", _, _, bh), base_expr("client", bh, e), lcb_expr_names(e, c).
+// a dotted base, `class Store(abc.Mapping)` after `from collections import abc`: the attribute chain names it
+lcb_direct_ext_base(t, c) :-
+    type_base_slot("client", t, _, "DOTTED_NAME", _, _, bh), base_expr("client", bh, e), lcb_expr_names(e, c).
 lcb_direct_ext_base(t, c) :-
     type_base_slot("client", t, _, "SUBSCRIPT", _, _, bh), base_expr("client", bh, e),
     expr_parent("client", e, "SUBSCRIPT_OBJECT", _, obj), lcb_expr_names(obj, c).