`--update`: re-extracting a file silently breaks cross-file edges from *cached* files that reference it

[pasqualtroncone]

Version: graphifyy 0.9.53 (AST cache v0.9.53-s2), Python 3.12, Linux
Backend: no GEMINI_API_KEY — host agent dispatches general-purpose subagents per
references/extraction-spec.md
Corpus: ~92 files, mixed Python + Markdown/YAML docs, ~126k words

Relationship to existing issues

This is a downstream consequence of the ID churn documented in #3004, but a distinct
failure with a distinct fix, so filing separately rather than as a comment there.

• build_merge: replace-on-re-extract deletes a source's nodes before the new extraction is validated; the #479 guard is skipped under dedup and delegates to a #1951 guard that does not exist in the package #3004 establishes that re-extracting a file mints different node IDs (0–12% ID
  retention across passes). It measures the damage inside the re-extracted file
  (nodes lost). This report measures the damage outside it: edges from other files,
  still in cache, now point at IDs that no longer exist.
• Fixing build_merge: replace-on-re-extract deletes a source's nodes before the new extraction is validated; the #479 guard is skipped under dedup and delegates to a #1951 guard that does not exist in the package #3004 does not fix this. Even a re-extraction that loses no nodes at all
  still breaks referrers if it renames anything.
• diagnose_extraction counts external-package import refs (ref_*) as dangling edges — false "graph may be incomplete/corrupt" warning on JS/TS projects #2191 explains why nobody notices: the dangling counter conflates external-module
  refs with real breakage. That issue asks to "keep the warning for genuinely dangling
  endpoints — that signal is valuable, it is just drowned out today", and notes their
  own semantic edges were 100% valid. This is a worked example of exactly that
  signal, if it's useful for justifying the split.
• build_merge(prune_sources=...) matches raw strings, so --update never prunes deleted files and reports "already clean" #3153 (raw-string prune_sources) also reproduces here as a side effect — see
  "Second-order symptom" below.

What happens

1. Full build. CHANGELOG.md and config.yaml each get cross-file edges into
   README.md concepts.
2. Edit README.md. Run --update.
3. Only README.md is re-extracted. CHANGELOG.md / config.yaml stay cached and
   still hold edges pointing at README.md's old IDs.
4. Those edges are now dangling. No warning, no count, nothing in the diff.

Observed in our corpus, after re-extracting 3 files:

CHANGELOG.md   -> readme_feature_toggle, plan_fase_3_packaging, plan_fase_2_autopaste
CLAUDE.md      -> readme_feature_toggle, readme_config_variables
samples.yaml   -> config_step_roles, config_conditional_route
overlay.yaml   -> config_step_roles

8 broken cross-document edges out of 3770. Two distinct causes, and only the first is
addressable by better ID discipline:

• separator/wording drift: plan_fase_3_packaging → plan_fase3_paquetizacion,
  readme_config_variables → readme_config_env
• legitimate re-conceptualisation: config_step_roles became four separate role
  nodes; config_conditional_route was dropped as a concept. No ID scheme prevents this.

Repair needs referrer ordering — it is not one pass

We repaired it by purging referring files from the semantic cache and re-extracting them
with the current node IDs supplied verbatim in the prompt (instructing the subagent to
copy them exactly and drop any edge whose target is not on the list). Blind
re-extraction just relocates the breakage:

round 1: re-extract the 4 files referencing README.md
         -> fixed the original 8
         -> but re-extracting CLAUDE.md renamed ITS ids,
            breaking 12 edges from 3 files that referenced CLAUDE.md
         net: 8 -> 12  (worse than before)

round 2: re-extract those 3 files
         -> they were leaves (2 inbound edges, both already fresh)
         net: 12 -> 0  (converged)

Round 2 only terminated because those files were leaves. Repair has to walk the referrer
graph leaves-inward; in arbitrary order it can ping-pong, and on cyclic doc-to-doc
references it is not obvious it terminates at all.

Second-order symptom (this is #3153)

During the merge, build_merge emitted 12 warnings of the form:

[graphify] WARNING: node 'X' is minted by two different files — keeping '<label A>'
from 'src/.../file.yaml', dropping '<label B>' from '/abs/path/.../file.yaml'.

Same file, one relative and one absolute — the new chunk's absolute source_file never
matched the graph's relative one, so replace-on-re-extract did not replace; it unioned,
and an arbitrary winner was picked per colliding ID. A clean rebuild afterwards removed
15 superseded nodes and restored 2 that fuzzy dedup had collapsed (1965 → 1952 nodes).
This matches #3153's diagnosis, in new_chunks rather than prune_sources.

Suggested fixes

1. Referrer invalidation on re-extract. When --update re-extracts F, find cached
   fragments holding edges whose target lives in F. Re-queue them, or at minimum print
   the blast radius so the operator knows what just went stale.
2. Supply existing IDs to the extraction prompt. When re-extracting F, pass the
   subagent the current node IDs of files F links into and require verbatim reuse.
   This is what we did by hand; it worked both rounds. Caveat: needs the referrer
   ordering above to be worth much.
3. Count intra-corpus dangling edges separately — this is diagnose_extraction counts external-package import refs (ref_*) as dangling edges — false "graph may be incomplete/corrupt" warning on JS/TS projects #2191, and our 8-vs-294
   ratio is a concrete argument for it.
4. Validate endpoints in build_merge. It currently accepts edges pointing at
   nonexistent nodes silently. A counted warning would surface this at the moment of
   damage instead of runs later.

Measurement note

The health check reports different numbers depending on pipeline stage: after
build_merge, .graphify_extract.json holds post-build edges, so the diagnostic
reports dangling_endpoint_edges: 0 on an --update result, while a clean rebuild of
the identical corpus reports 302. Worth documenting which stage the check is meant for.

Impact

No node loss from this particular path — but cross-document relationships rot quietly,
and those are the edges that make the graph worth more than grep. A corpus maintained
incrementally accumulates this with no signal at all.

Happy to test a patch; reproduces reliably here.