Ghost-merge misses semantic nodes whose source_file is the referencing doc, not the referenced file

[leninherrera94]

Summary

build_from_json()'s ghost-merge pass (the "#1145-extended" logic that collapses a semantic/LLM-extracted duplicate into its AST-canonical twin) only matches on an exact (source_file, label) key. When a semantic extraction pass reads a document that merely mentions a real source file by name in prose (a saved graphify-out/memory/*.md query answer, a runbook like deploy/scripts/provision-host.md, any doc narrating "see App.tsx" or "backup.sh does X"), the new node it creates gets stamped with source_file = the document being read, not the file actually named in the text. That wrong source_file means the (source_file, label) key can never match the real AST node's key, so the ghost-merge never fires — a permanent duplicate node survives every subsequent --update/rebuild, forever.

Concretely, on a real project (NestJS/Next.js/Expo monorepo, ~4600 nodes), this produced ghosts like:

• label App.tsx, source_file = a saved memory query file — instead of merging into the real AST node for apps/customer-app/App.tsx
• label customer-app/index.ts (a doc summary dropped the apps/ root segment) — instead of merging into apps/customer-app/index.ts
• label backup.sh/deploy.sh/init-buckets.sh/restore.sh, source_file = deploy/scripts/provision-host.md — instead of merging into the real script file nodes

These ghosts then poison hyperedges (e.g. a weak_bridge_chain_to_hub hyperedge citing 3 real nodes + 3 of these phantoms) and community/god-node analysis with fake, unbacked entities.

Root cause

In build.py's ghost-merge (inside build_from_json), Pass 1/Pass 2 key exclusively on (source_file, label):

key = (sf, label)
if is_ast:
    ...
    _loc_nodes[key] = nid
else:
    _loc_nodes.setdefault(key, nid)

# Pass 2: find ghosts — non-AST nodes that have an AST canonical twin.
for nid in sorted(node_set):
    ...
    key = (sf, label)
    if key in _loc_collisions:
        continue
    if key in _loc_nodes and _loc_nodes[key] != nid:
        _noloc_nodes[key] = nid

This is correct for genuine same-file re-extraction ghosts (the original #1145 case), but it silently does nothing for the "referenced-not-defined" case above, because the ghost's own source_file is simply wrong — it's the referencing document, not the referenced file.

The codebase already has the exact predicate needed to catch this by label alone: _is_file_node_label(label, source_file) (used elsewhere for file-node label disambiguation) returns True when label is the bare basename of source_file, or a directory-qualified suffix of it (handles a doc dropping a leading path segment, e.g. customer-app/index.ts vs the real apps/customer-app/index.ts). It just isn't wired into the ghost-merge matching at all.

Suggested fix

Add a conservative "Pass 2b" that falls back to matching a ghost by label alone against every AST file-self node (an AST node whose own label already names its own file — i.e. _is_file_node_label(ast_label, ast_source_file) is true), and only remaps when the match is unique (0 or 2+ candidates → leave the ghost alone, same conservatism _loc_collisions already applies for the exact-key ambiguity case).

# In Pass 1, alongside `_loc_nodes[key] = nid` (is_ast branch), also collect:
_ast_file_nodes: list[tuple[str, str]] = []  # (node_id, source_file)
...
if is_ast:
    ...
    _loc_nodes[key] = nid
    if _is_file_node_label(label, sf):
        _ast_file_nodes.append((nid, sf))

# After the existing exact-key _ghost_remap is built, before the removal loop:
if _ast_file_nodes:
    for nid in sorted(node_set):
        if nid in _ghost_remap:
            continue  # already resolved by the exact (sf, label) key
        attrs = G.nodes[nid]
        if attrs.get("_origin") == "ast":
            continue
        label = str(attrs.get("label", "")).strip()
        if not label:
            continue
        matches = {
            ast_id for ast_id, ast_sf in _ast_file_nodes
            if _is_file_node_label(label, ast_sf)
        }
        if len(matches) == 1:
            _ghost_remap[nid] = next(iter(matches))

No other changes needed — _ghost_remap already flows into the existing norm_to_id map that both edge endpoints and hyperedge members resolve through, so this Just Works for both.

Verification

Unit test (positive — unique match merges correctly, edges rewire to the real node):

extraction = {
    'nodes': [
        {'id': 'apps_customer_app_app', 'label': 'App.tsx', '_origin': 'ast', 'file_type': 'code',
         'source_file': 'apps/customer-app/App.tsx', 'source_location': 'L1'},
        {'id': 'apps_customer_app_index', 'label': 'customer-app/index.ts', '_origin': 'ast', 'file_type': 'code',
         'source_file': 'apps/customer-app/index.ts', 'source_location': 'L1'},
        {'id': 'app', 'label': 'App.tsx', '_origin': 'semantic', 'file_type': 'code',
         'source_file': 'graphify-out/memory/query_fake.md', 'source_location': None},
        {'id': 'customer_app_index', 'label': 'customer-app/index.ts', '_origin': 'semantic', 'file_type': 'code',
         'source_file': 'graphify-out/memory/query_fake.md', 'source_location': None},
        {'id': 'some_query_node', 'label': 'Query: ...', '_origin': 'semantic', 'file_type': 'document',
         'source_file': 'graphify-out/memory/query_fake.md', 'source_location': None},
    ],
    'edges': [
        {'source': 'some_query_node', 'target': 'app', 'relation': 'references', 'confidence': 'EXTRACTED'},
        {'source': 'some_query_node', 'target': 'customer_app_index', 'relation': 'references', 'confidence': 'EXTRACTED'},
    ],
    'hyperedges': [],
}
G = build_from_json(extraction, root=None)
# Before fix: nodes include 'app' and 'customer_app_index' as permanent ghosts.
# After fix: G.nodes() == {'apps_customer_app_app', 'apps_customer_app_index', 'some_query_node'}
# and the edges correctly point at the real AST nodes.

Unit test (negative — ambiguous basename correctly left alone): two real files both named index.ts in different directories + a phantom labeled bare index.ts → the phantom is not merged into either (0 or 2+ candidates is the same "no safe winner" rule the existing exact-key path already uses).

Real-world: applied to an existing ~4600-node project graph — 10 long-standing ghost nodes removed (4578 → 4568 nodes) across two separate documents (a saved memory query file, and an unrelated ops runbook), confirming this isn't a narrow edge case but a general failure mode of the "document mentions a file by name" pattern. The two hyperedges that had cited the phantoms now correctly cite only real, AST-backed nodes. Graph health diagnostic clean before and after.

Environment

• graphifyy 0.9.53, installed via uv tool install
• Reproduced via the /graphify skill's standard --update workflow (build_merge → build → build_from_json)