feat: Elixir language support (.ex/.exs)#1264
Conversation
Elixir has no keywords — defmodule/def/use/@SPEC are all macros that parse as plain `call` nodes in elixir-lang/tree-sitter-elixir (prebuilt wasm from tree-sitter-wasms, ABI 14) — so nothing fits the generic functionTypes/classTypes dispatch. The whole language is driven from the visitNode hook, keyed off each call's target identifier, following the erlang extractor's architecture. Extraction: - `defmodule` (and nested defmodules) → namespace nodes carrying the full dotted name; every contained symbol's qualifiedName is `Full.Module::fun`. - `def`/`defp`/`defmacro(p)`/`defguard(p)`/`defdelegate` → function nodes with visibility (defp/defmacrop/defguardp private); multiple clauses of the same function (pattern matching, guards, default args) merge into one node, matching the erlang multi-clause treatment. Both body forms are handled: `do…end` blocks and the one-liner `, do:` keyword. - `@spec` becomes the signature, `@doc`/`@moduledoc` the docstring; attribute subtrees are consumed so type-position expressions (`String.t()` parses as a call) never mint bogus call refs. - `defstruct`/`defexception` fields (keyword and atom-list forms) → field nodes on the module; `%Struct{}` literals → references. - `defprotocol` → namespace; `defimpl Proto, for: Type` → namespace `Proto.Type` plus an implements ref to the protocol. Calls and edges (extractCall elixir branch): - Remote `Mod.Sub.fun(x)` is emitted as `Mod.Sub::fun` — byte-identical to the qualifiedName the module namespace produces — expanding the head segment through the enclosing module's `alias` table (`alias Foo.Bar`, `alias Foo.Bar, as: Baz`, `alias Foo.{A, B}`). - `__MODULE__.fun(...)` → bare name (same-file resolution), like erlang's `?MODULE:fn`. - `&Mod.fun/2` and `&local/1` captures → references (no duplicate call edge); pipes need nothing — the right side is an ordinary call node. - `@behaviour Mod` and `use Mod` → implements refs, resolved module-only in the name matcher (same rule as erlang -behaviour, so an OTP behaviour like GenServer stays unresolved rather than guessed); `import`/`require`/`alias` → imports edges. - "A wrong edge is worse than no edge": calls through a variable receiver (`mod.fun(x)`), `apply/3`, and `GenServer.call(pid, …)` have no static target and stay silent. Bare local calls resolve same-file ONLY — a bare call can only legitimately target its own module (or an import, deliberately unexpanded): letting them fall through to repo-wide bare-name matching linked every Mix `config :app, …` DSL call to some module's config/0 and every ConnTest `get(conn, path)` to an unrelated HTTP client's get. Validated on a real 994-file Elixir umbrella app (6.7k nodes, 19.8k edges, 5.2s): grep-truth vs graph callers matched 23/23 call sites for 5 uniquely-named functions, and a 10-edge random sample verified with zero false edges. Known limitations (v1): user macro expansion, Phoenix/Ecto DSLs, GenServer/process dispatch, import-based resolution of bare calls, and .heex templates are out of scope. Co-Authored-By: Claude <noreply@anthropic.com>
handleAttribute consumed every @-attribute subtree so a compile-time call in a custom attribute value (`@id Product.get_product_name(:x)`) never produced a `calls` edge — the sole recall miss across the 6-repo validation battery. Split attributes into two classes: reserved/META attributes (@SPEC, @doc, @type, @behaviour, @derive, @enforce_keys, … — the full reserved set from the official Module docs) keep being consumed, so type-position expressions like `String.t()` in @SPEC still cannot mint bogus call edges (D3). Every other attribute is a custom attribute whose value is real compile-time code: its value subtree is now walked, so a remote or local call in the value emits a normal `calls` edge attributed to the enclosing module namespace (attributes live at module level). The inner call's target — the attribute name itself — is never treated as a call. Revalidated on crm-proxy: the @calltracker_product_id miss now resolves to Product::get_product_name; calls edges 2613 → 2640 (+27, no explosion); all 20 attribute-driven edges verified correct. Co-Authored-By: Claude <noreply@anthropic.com>
elixir-ecto/ecto 52/56 = 92.9%, methodology calibrated by reproducing the published requests (100.0%) and ripgrep (86.8%) rows with the same build. Residual (4 files) is genuine frontier: OTP application entry, conditional protocol impls, and the Ecto.Query.API/WindowAPI query-DSL pseudo-modules. Co-Authored-By: Claude <noreply@anthropic.com>
…_ struct/captures, one-line modules, local aliases, macro noise)
Found in two code reviews (GLM + Cursor). Verified extraction gaps,
each reproduced by a failing test before the fix:
- guards: emit calls in defguard bodies and def ... when guards (the when
binary_operator right side was never walked)
- %__MODULE__{} struct literals now emit a reference to the enclosing module
- &__MODULE__.fun/arity captures now emit a same-module reference
- defoverridable no longer leaks as a call ref
- Kernel control-flow/process macros (if/unless/case/cond/with/for/raise/try/
send/spawn/self/throw/exit) no longer leak as unresolvable call noise
- defstruct default-value calls (ts: DateTime.utc_now()) are now emitted;
field names never become call edges
- one-line modules (defmodule M, do: ...) now index their body
- alias declared inside a function body now resolves module-wide; alias-table
memo changed to a per-(file,module) map to avoid nested-module re-walk
- dynamically-named defmodule (e.g. defmodule unquote(name) do ...) no longer
silently swallows its body: inner defs are now indexed like top-level defs,
with no fabricated namespace. A literal Module.concat(A, B) / [A, B] name is
statically recovered as A.B and treated as a normal module (F6)
Co-Authored-By: Claude <noreply@anthropic.com>
|
While validating this branch on a production Elixir repo I noticed missing The loss happens later, in the resolver: when such a same-key group straddles the 5000-row resolution batch boundary, the per-batch cleanup (keyed on that 3-tuple, without line/col) deletes the whole group, including rows a later batch hasn't read yet. That's exactly #1265, fixed in #1266 — with #1266 applied on top of this branch, the repo goes from 17/26 to 26/26 edges for that module, with no other changes (+9 edges total, node count identical). Elixir test suites make this pre-existing bug much more likely to fire (many identical-key calls per module), so #1266 is worth landing alongside or before this PR. |
Follow-up to #1220/#1221 and the Elixir entry in #648 — this adds Elixir as a supported language.
What this adds
defmodule, including nested modules with their full dotted name (Foo.Bar.Baz),defprotocol/defimplblocks.def/defp/defmacro/defguardwith correct visibility; multiple clauses of the same function (pattern matching, guards, default-arg heads) merge into one node.@specbecomes the signature,@doc/@moduledocthe docstring,defstructthe fields.Module.fun(...)resolves through the file'saliastable (includingalias Foo.{A, B}andalias ..., as: X),__MODULE__.funlinks within the module,&Mod.fun/aritycaptures and%Struct{}literals becomereferences,@behaviour/usebecomeimplements,import/require/aliasbecome import edges. Calls inside module-attribute values (@key Config.fetch!(...)) are extracted;@spec/@doc-style metadata attributes are consumed soString.t()and friends never become edges.config :app, ...linked to someone'sconfig/0; ConnTest's importedget(conn, path)linked to an HTTP client'sget). The rule removed ~3.2k guessed edges on the first validation repo with zero recall cost on same-module calls. Import-based resolution is deliberately out of scope for v1.apply/3,GenServer.call(pid, ...)) is left unlinked.31 extraction/resolution tests added; full suite green (2393 passed / 0 failed on Node 22). The branch also went through two independent external code reviews before submission; all nine findings (guard-expression calls,
%__MODULE__{}/&__MODULE__.fun/arityreferences, one-linedefmodule ..., do:bodies, dynamically-nameddefmodulebodies — with static recovery of literalModule.concat(A, B)names, function-localalias, declaration/Kernel-macro call noise) were reproduced by a failing test first and are fixed in the last commit.Validation on real code
Validated on 6 production Elixir repositories (~840 Elixir files) from one company codebase, deliberately spanning eras: Elixir ~1.4 (2017) → 1.15, plain Plug, Phoenix 1.4, Phoenix 1.7 + LiveView, Ecto 1–3, umbrella apps, and Elixir+TypeScript/JS mixed repos. Method as in #1220: grep-established ground truth vs graph for statically-resolvable calls (recall), manual audit of random
callsedges (precision), plus a framework-macro false-edge sweep.callsedges, manual audit)Results were uniform across the 1.4→1.15 spectrum — resolution rides on static module names (
alias/fully-qualified), so old code works as well as new.Fair coverage (methodology from the README table, calibrated by reproducing two published rows with the same build — requests 100.0% vs 100%, ripgrep 86.8% vs 86.7%):
elixir-ecto/ecto92.9% (52/56; the 4-file residual is all genuine frontier: the OTP application entry, compile-time conditional protocol impls, and the two query-DSL pseudo-modulesEcto.Query.API/WindowAPI). README row added in this PR. As an application-shaped stress test, the same measurement onplausible/analyticsgives 67.4% on shipping Elixir — the residual there is structural (controllers/LiveView/workers reachable only through routing and scheduling; the router alone holds 419 unresolved refs becauseget/post/livemacros don't bind their module argument), which is precisely what the Phoenix resolver below is for.Known limitations (v1)
import Foo+ barebar()) resolve same-file or not at all — no import expansion yet.apply/3, GenServer/process messaging,Application.get_env-configured modules) — intentionally silent..heextemplates are not parsed. This includes a bare module alias standing as a declarative-macro argument (field :secret, MyApp.EncryptedBinary,belongs_to :site, Site) — no reference is emitted for it yet (deliberate: emitting every capitalized mention risks noise; scoping it to known declarative positions is part of the Phoenix/Ecto follow-up).defdelegatedoes not yet emit an edge to itsto:target;@derivedoes not emit animplementsedge (its targets are almost always out-of-repo protocols likeJason.Encoder).Roadmap / follow-ups
This PR is the language core — the first step of a planned path:
src/resolution/frameworks/phoenix.ts) — route → controller action / LiveView edges (get "/x", StatsController, :funnelwithscope-alias nesting), in the spirit of the Laravel/Rails resolvers. Next PR, in preparation; it will be registered in Tracking: framework-aware route & edge coverage requests #967. Measured headroom on plausible/analytics above; validation will run on production Phoenix apps plus a canonical OSS app + control, per the bar stated there..heextemplates — component/assign references, precedent indocs/design/template-markup-parser.md.Relation to the open Elixir PRs (#228, #871, #1229, #1250)
This is an independent implementation — all four open PRs' code was read before submitting, and this section is meant to make consolidation easier, not to relitigate them. In one line each: #228 takes the same architectural approach as this PR (macro-name dispatch in a visitNode hook) with a narrower feature set; #871/#1250 are one lineage (#1250 = #871 rebased + an
as:-alias fix, validated by its authors on a ~3,400-file production monorepo); #1229 pairs a thinner core with two genuinely valuable pieces no one else has — a tested Phoenix router resolver and HEEx handling.What this PR adds over the extraction-focused ones (verified against #1250's branch):
alias MyApp.Repo+Repo.get(...)andalias MyApp.Accounts.{User, Profile}resolve to the right module here (per-file alias table, memoized); in feat(extraction): Elixir language support (.ex/.exs) #1250 only the explicitalias X, as: Yform resolves — the import node is stored under the canonical name, so the common default-alias style yields no edge.config, imported ConnTestget). The same-file-only rule here removed ~3.2k guessed edges on the first validation repo.@spec→ signature,@doc→ docstring, META vs custom attribute split soString.t()never becomes an edge but calls inside@key Config.fetch!(...)do), clause merging,defguard, protocol/impl blocks, captures and%Struct{}references.$this->dep->method()) never resolve — constructor-injected dependencies produce no call edges #1220: ground truth established by grep per function (recall measured against what should be in the graph, not by inspecting what is), precision on random edge samples, and a framework-macro false-edge sweep — across 6 production repos + the calibrated fair-coverage measurement above. This is what caught the default-alias and bare-call classes in the first place.@waseigo's #1229 (Phoenix resolver + HEEx) is complementary to this core and overlaps with follow-up 1 above — the router resolver there looks portable onto this core, and coordinating/crediting there beats duplicating it.
The branch is built on current
main(no conflicts) with a clean, reviewable history — it's intended to be mergeable as-is. If you'd rather consolidate the Elixir PRs yourself (as with ArkTS/Nix), it works as the reference implementation + validation corpus too — take whatever is useful, same as #1221 — though the commits landing as they are would of course be the preferred outcome.Scope adjustments or a different split are fine on request.