From a9b253db2da671b936dd7981f28f15f2ea903e99 Mon Sep 17 00:00:00 2001 From: AirRocker <264858149+AirRocker@users.noreply.github.com> Date: Fri, 24 Jul 2026 15:54:00 -0500 Subject: [PATCH] skill: require a mandatory-ID gate when re-extracting an already-graphed doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Semantic re-extraction of a doc that already has nodes in graph.json is non-deterministic. On a large project CLAUDE.md, successive --update runs of the same unchanged-except-one-line file returned 75 nodes and then 49, silently dropping a whole documented section. Nothing errored: the loss went into build_merge and would only have surfaced later as a question the graph could no longer answer. It was caught solely because the to_json shrink guard (#479) refused the write. Adds a required three-step gate to the --update flow, in the shared references/update.md fragment plus a pointer from the core template: 1. Snapshot that file's existing node IDs from graph.json before extracting. 2. Pass them to the extraction subagent as MANDATORY must-include IDs, with the baseline node count as an explicit target. Reusing the IDs also suppresses gratuitous ID churn, which orphans saved queries and inflates the merge diff for no semantic gain. 3. Hard-gate the merge: refuse unless node count >= 90% of baseline AND zero mandatory IDs are missing. On failure, re-dispatch naming the missing IDs; two consecutive failures stop and report rather than merging a regression. Also restates that the shrink guard is the authoritative backstop: never force past it without diffing the old and new node sets and being able to name why each removed node is legitimately gone. Fragments are the edited source; graphify/skill*.md, graphify/skills/**, graphify/always_on/** and tools/skillgen/expected/** are regenerated via `python -m tools.skillgen` and `--bless`. Markdown only, no code changes. All five skillgen guards pass (--check, --audit-coverage, --schema-singleton, --monolith-roundtrip, --always-on-roundtrip). Review follow-up (graphify-labs bot, 2026-09-10) — three defects fixed in the gate procedure, each reproduced against a real graph before the edit: * Suffix matching. `endswith(TARGET)` is not anchored to a path boundary, so a bare name adopts every same-named file deeper in the tree. Measured on a real corpus, TARGET='README.md' matched 'workers/README.md' too: baseline 22 nodes instead of 7, and a *perfect* re-extraction of README.md then scored 31.8% of baseline with 15 mandatory IDs "missing" — a hard FAIL on correct output, and the prescribed remedy (re-dispatch naming the missing IDs) would have pushed the subagent to mint another file's nodes under this one's source_file. Now matched exactly, with an ambiguity guard that lists the candidates and stops rather than guessing. * Single shared state file. `.graphify_must_ids.txt` / `.graphify_must_slice.json` were fixed paths, so in a multi-doc delta the second snapshot overwrote the first and that doc went through ungated while the gate still printed PASS — failing open, in the one place this procedure exists to fail closed. State files are now keyed by target, with a one-time clear of stale snapshots. * Only `.graphify_chunk_01.json` was read. Step 3B splits a delta into chunks of 20-25 files (each image its own), so a changed doc can land in any chunk; the gate then read every mandatory ID as missing and failed a clean extraction. The gate now unions all chunks and evaluates every snapshot, counting only the nodes attributed to each target — which also fixes a latent over-count, since the old `len(d['nodes'])` was the whole chunk, not the target's slice. Verified: multi-chunk delta with the target in chunk_02 now PASSes, with negative controls (drop a mandatory node, drop a baseline hyperedge) confirming the gate still FAILs when it should. Markdown only; no code changes. Claude-Session: https://claude.ai/code/session_01B8MXR5KMGHczdCLXB8kvJd --- graphify/skill-agents.md | 2 + graphify/skill-amp.md | 2 + graphify/skill-claw.md | 2 + graphify/skill-codex.md | 2 + graphify/skill-copilot.md | 2 + graphify/skill-droid.md | 2 + graphify/skill-kilo.md | 2 + graphify/skill-kiro.md | 2 + graphify/skill-opencode.md | 2 + graphify/skill-pi.md | 2 + graphify/skill-trae.md | 2 + graphify/skill-vscode.md | 2 + graphify/skill-windows.md | 2 + graphify/skill.md | 2 + graphify/skills/agents/references/update.md | 120 ++++++++++++++++++ graphify/skills/amp/references/update.md | 120 ++++++++++++++++++ graphify/skills/claude/references/update.md | 120 ++++++++++++++++++ graphify/skills/claw/references/update.md | 120 ++++++++++++++++++ graphify/skills/codex/references/update.md | 120 ++++++++++++++++++ graphify/skills/copilot/references/update.md | 120 ++++++++++++++++++ graphify/skills/droid/references/update.md | 120 ++++++++++++++++++ graphify/skills/kilo/references/update.md | 120 ++++++++++++++++++ graphify/skills/kiro/references/update.md | 120 ++++++++++++++++++ graphify/skills/opencode/references/update.md | 120 ++++++++++++++++++ graphify/skills/pi/references/update.md | 120 ++++++++++++++++++ graphify/skills/trae/references/update.md | 120 ++++++++++++++++++ graphify/skills/vscode/references/update.md | 120 ++++++++++++++++++ graphify/skills/windows/references/update.md | 120 ++++++++++++++++++ .../expected/graphify__skill-agents.md | 2 + .../skillgen/expected/graphify__skill-amp.md | 2 + .../skillgen/expected/graphify__skill-claw.md | 2 + .../expected/graphify__skill-codex.md | 2 + .../expected/graphify__skill-copilot.md | 2 + .../expected/graphify__skill-droid.md | 2 + .../skillgen/expected/graphify__skill-kilo.md | 2 + .../skillgen/expected/graphify__skill-kiro.md | 2 + .../expected/graphify__skill-opencode.md | 2 + tools/skillgen/expected/graphify__skill-pi.md | 2 + .../skillgen/expected/graphify__skill-trae.md | 2 + .../expected/graphify__skill-vscode.md | 2 + .../expected/graphify__skill-windows.md | 2 + tools/skillgen/expected/graphify__skill.md | 2 + ...ify__skills__agents__references__update.md | 120 ++++++++++++++++++ ...aphify__skills__amp__references__update.md | 120 ++++++++++++++++++ ...ify__skills__claude__references__update.md | 120 ++++++++++++++++++ ...phify__skills__claw__references__update.md | 120 ++++++++++++++++++ ...hify__skills__codex__references__update.md | 120 ++++++++++++++++++ ...fy__skills__copilot__references__update.md | 120 ++++++++++++++++++ ...hify__skills__droid__references__update.md | 120 ++++++++++++++++++ ...phify__skills__kilo__references__update.md | 120 ++++++++++++++++++ ...phify__skills__kiro__references__update.md | 120 ++++++++++++++++++ ...y__skills__opencode__references__update.md | 120 ++++++++++++++++++ ...raphify__skills__pi__references__update.md | 120 ++++++++++++++++++ ...phify__skills__trae__references__update.md | 120 ++++++++++++++++++ ...ify__skills__vscode__references__update.md | 120 ++++++++++++++++++ ...fy__skills__windows__references__update.md | 120 ++++++++++++++++++ tools/skillgen/fragments/core/core.md | 2 + .../fragments/references/shared/update.md | 120 ++++++++++++++++++ 58 files changed, 3538 insertions(+) diff --git a/graphify/skill-agents.md b/graphify/skill-agents.md index 190827d9ac..8058612dbe 100644 --- a/graphify/skill-agents.md +++ b/graphify/skill-agents.md @@ -675,6 +675,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-amp.md b/graphify/skill-amp.md index 190827d9ac..8058612dbe 100644 --- a/graphify/skill-amp.md +++ b/graphify/skill-amp.md @@ -675,6 +675,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-claw.md b/graphify/skill-claw.md index abd2811d23..60dc60be56 100644 --- a/graphify/skill-claw.md +++ b/graphify/skill-claw.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-codex.md b/graphify/skill-codex.md index af3f723c78..b731e289a1 100644 --- a/graphify/skill-codex.md +++ b/graphify/skill-codex.md @@ -675,6 +675,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-copilot.md b/graphify/skill-copilot.md index abd2811d23..60dc60be56 100644 --- a/graphify/skill-copilot.md +++ b/graphify/skill-copilot.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-droid.md b/graphify/skill-droid.md index fd148d485d..37fbafca40 100644 --- a/graphify/skill-droid.md +++ b/graphify/skill-droid.md @@ -675,6 +675,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-kilo.md b/graphify/skill-kilo.md index 3e70b050a4..62adfea280 100644 --- a/graphify/skill-kilo.md +++ b/graphify/skill-kilo.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-kiro.md b/graphify/skill-kiro.md index abd2811d23..60dc60be56 100644 --- a/graphify/skill-kiro.md +++ b/graphify/skill-kiro.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-opencode.md b/graphify/skill-opencode.md index 91ced60675..27a7029c0a 100644 --- a/graphify/skill-opencode.md +++ b/graphify/skill-opencode.md @@ -670,6 +670,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-pi.md b/graphify/skill-pi.md index abd2811d23..60dc60be56 100644 --- a/graphify/skill-pi.md +++ b/graphify/skill-pi.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-trae.md b/graphify/skill-trae.md index 050667bc20..ac42ac40cf 100644 --- a/graphify/skill-trae.md +++ b/graphify/skill-trae.md @@ -676,6 +676,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-vscode.md b/graphify/skill-vscode.md index 20c7c0835c..cb6612d065 100644 --- a/graphify/skill-vscode.md +++ b/graphify/skill-vscode.md @@ -674,6 +674,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill-windows.md b/graphify/skill-windows.md index b09ecca3c4..6ebf5d5098 100644 --- a/graphify/skill-windows.md +++ b/graphify/skill-windows.md @@ -707,6 +707,8 @@ if (-not (Test-Path graphify-out\.graphify_python)) { Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skill.md b/graphify/skill.md index abd2811d23..60dc60be56 100644 --- a/graphify/skill.md +++ b/graphify/skill.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/graphify/skills/agents/references/update.md b/graphify/skills/agents/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/agents/references/update.md +++ b/graphify/skills/agents/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/amp/references/update.md b/graphify/skills/amp/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/amp/references/update.md +++ b/graphify/skills/amp/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/claude/references/update.md b/graphify/skills/claude/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/claude/references/update.md +++ b/graphify/skills/claude/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/claw/references/update.md b/graphify/skills/claw/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/claw/references/update.md +++ b/graphify/skills/claw/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/codex/references/update.md b/graphify/skills/codex/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/codex/references/update.md +++ b/graphify/skills/codex/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/copilot/references/update.md b/graphify/skills/copilot/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/copilot/references/update.md +++ b/graphify/skills/copilot/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/droid/references/update.md b/graphify/skills/droid/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/droid/references/update.md +++ b/graphify/skills/droid/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/kilo/references/update.md b/graphify/skills/kilo/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/kilo/references/update.md +++ b/graphify/skills/kilo/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/kiro/references/update.md b/graphify/skills/kiro/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/kiro/references/update.md +++ b/graphify/skills/kiro/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/opencode/references/update.md b/graphify/skills/opencode/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/opencode/references/update.md +++ b/graphify/skills/opencode/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/pi/references/update.md b/graphify/skills/pi/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/pi/references/update.md +++ b/graphify/skills/pi/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/trae/references/update.md b/graphify/skills/trae/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/trae/references/update.md +++ b/graphify/skills/trae/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/vscode/references/update.md b/graphify/skills/vscode/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/vscode/references/update.md +++ b/graphify/skills/vscode/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/graphify/skills/windows/references/update.md b/graphify/skills/windows/references/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/graphify/skills/windows/references/update.md +++ b/graphify/skills/windows/references/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skill-agents.md b/tools/skillgen/expected/graphify__skill-agents.md index 190827d9ac..8058612dbe 100644 --- a/tools/skillgen/expected/graphify__skill-agents.md +++ b/tools/skillgen/expected/graphify__skill-agents.md @@ -675,6 +675,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-amp.md b/tools/skillgen/expected/graphify__skill-amp.md index 190827d9ac..8058612dbe 100644 --- a/tools/skillgen/expected/graphify__skill-amp.md +++ b/tools/skillgen/expected/graphify__skill-amp.md @@ -675,6 +675,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-claw.md b/tools/skillgen/expected/graphify__skill-claw.md index abd2811d23..60dc60be56 100644 --- a/tools/skillgen/expected/graphify__skill-claw.md +++ b/tools/skillgen/expected/graphify__skill-claw.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-codex.md b/tools/skillgen/expected/graphify__skill-codex.md index af3f723c78..b731e289a1 100644 --- a/tools/skillgen/expected/graphify__skill-codex.md +++ b/tools/skillgen/expected/graphify__skill-codex.md @@ -675,6 +675,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-copilot.md b/tools/skillgen/expected/graphify__skill-copilot.md index abd2811d23..60dc60be56 100644 --- a/tools/skillgen/expected/graphify__skill-copilot.md +++ b/tools/skillgen/expected/graphify__skill-copilot.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-droid.md b/tools/skillgen/expected/graphify__skill-droid.md index fd148d485d..37fbafca40 100644 --- a/tools/skillgen/expected/graphify__skill-droid.md +++ b/tools/skillgen/expected/graphify__skill-droid.md @@ -675,6 +675,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-kilo.md b/tools/skillgen/expected/graphify__skill-kilo.md index 3e70b050a4..62adfea280 100644 --- a/tools/skillgen/expected/graphify__skill-kilo.md +++ b/tools/skillgen/expected/graphify__skill-kilo.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-kiro.md b/tools/skillgen/expected/graphify__skill-kiro.md index abd2811d23..60dc60be56 100644 --- a/tools/skillgen/expected/graphify__skill-kiro.md +++ b/tools/skillgen/expected/graphify__skill-kiro.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-opencode.md b/tools/skillgen/expected/graphify__skill-opencode.md index 91ced60675..27a7029c0a 100644 --- a/tools/skillgen/expected/graphify__skill-opencode.md +++ b/tools/skillgen/expected/graphify__skill-opencode.md @@ -670,6 +670,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-pi.md b/tools/skillgen/expected/graphify__skill-pi.md index abd2811d23..60dc60be56 100644 --- a/tools/skillgen/expected/graphify__skill-pi.md +++ b/tools/skillgen/expected/graphify__skill-pi.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-trae.md b/tools/skillgen/expected/graphify__skill-trae.md index 050667bc20..ac42ac40cf 100644 --- a/tools/skillgen/expected/graphify__skill-trae.md +++ b/tools/skillgen/expected/graphify__skill-trae.md @@ -676,6 +676,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-vscode.md b/tools/skillgen/expected/graphify__skill-vscode.md index 20c7c0835c..cb6612d065 100644 --- a/tools/skillgen/expected/graphify__skill-vscode.md +++ b/tools/skillgen/expected/graphify__skill-vscode.md @@ -674,6 +674,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill-windows.md b/tools/skillgen/expected/graphify__skill-windows.md index b09ecca3c4..6ebf5d5098 100644 --- a/tools/skillgen/expected/graphify__skill-windows.md +++ b/tools/skillgen/expected/graphify__skill-windows.md @@ -707,6 +707,8 @@ if (-not (Test-Path graphify-out\.graphify_python)) { Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skill.md b/tools/skillgen/expected/graphify__skill.md index abd2811d23..60dc60be56 100644 --- a/tools/skillgen/expected/graphify__skill.md +++ b/tools/skillgen/expected/graphify__skill.md @@ -678,6 +678,8 @@ fi Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/expected/graphify__skills__agents__references__update.md b/tools/skillgen/expected/graphify__skills__agents__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__agents__references__update.md +++ b/tools/skillgen/expected/graphify__skills__agents__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__amp__references__update.md b/tools/skillgen/expected/graphify__skills__amp__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__amp__references__update.md +++ b/tools/skillgen/expected/graphify__skills__amp__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__claude__references__update.md b/tools/skillgen/expected/graphify__skills__claude__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__claude__references__update.md +++ b/tools/skillgen/expected/graphify__skills__claude__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__claw__references__update.md b/tools/skillgen/expected/graphify__skills__claw__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__claw__references__update.md +++ b/tools/skillgen/expected/graphify__skills__claw__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__codex__references__update.md b/tools/skillgen/expected/graphify__skills__codex__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__codex__references__update.md +++ b/tools/skillgen/expected/graphify__skills__codex__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__copilot__references__update.md b/tools/skillgen/expected/graphify__skills__copilot__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__copilot__references__update.md +++ b/tools/skillgen/expected/graphify__skills__copilot__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__droid__references__update.md b/tools/skillgen/expected/graphify__skills__droid__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__droid__references__update.md +++ b/tools/skillgen/expected/graphify__skills__droid__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__kilo__references__update.md b/tools/skillgen/expected/graphify__skills__kilo__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__kilo__references__update.md +++ b/tools/skillgen/expected/graphify__skills__kilo__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__kiro__references__update.md b/tools/skillgen/expected/graphify__skills__kiro__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__kiro__references__update.md +++ b/tools/skillgen/expected/graphify__skills__kiro__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__opencode__references__update.md b/tools/skillgen/expected/graphify__skills__opencode__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__opencode__references__update.md +++ b/tools/skillgen/expected/graphify__skills__opencode__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__pi__references__update.md b/tools/skillgen/expected/graphify__skills__pi__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__pi__references__update.md +++ b/tools/skillgen/expected/graphify__skills__pi__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__trae__references__update.md b/tools/skillgen/expected/graphify__skills__trae__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__trae__references__update.md +++ b/tools/skillgen/expected/graphify__skills__trae__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__vscode__references__update.md b/tools/skillgen/expected/graphify__skills__vscode__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__vscode__references__update.md +++ b/tools/skillgen/expected/graphify__skills__vscode__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/expected/graphify__skills__windows__references__update.md b/tools/skillgen/expected/graphify__skills__windows__references__update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/expected/graphify__skills__windows__references__update.md +++ b/tools/skillgen/expected/graphify__skills__windows__references__update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: diff --git a/tools/skillgen/fragments/core/core.md b/tools/skillgen/fragments/core/core.md index c527a12563..39709db59d 100644 --- a/tools/skillgen/fragments/core/core.md +++ b/tools/skillgen/fragments/core/core.md @@ -601,6 +601,8 @@ Before running any subcommand below (`--update`, `--cluster-only`, `query`, `pat Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. +> **Re-extracting an already-graphed doc requires the mandatory-ID gate** (`references/update.md` → "REQUIRED: mandatory-ID gate for re-extracted docs"): snapshot that file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count drops below 90% of baseline or any ID is missing. Semantic re-extraction is non-deterministic and drops whole sections without it. + --- ## For /graphify query diff --git a/tools/skillgen/fragments/references/shared/update.md b/tools/skillgen/fragments/references/shared/update.md index 3632fd4126..a7f8f1b57b 100644 --- a/tools/skillgen/fragments/references/shared/update.md +++ b/tools/skillgen/fragments/references/shared/update.md @@ -65,6 +65,126 @@ If `code_only` is True: print `[graphify update] Code-only changes detected - sk If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. +### REQUIRED: mandatory-ID gate for re-extracted docs + +Semantic re-extraction of an already-graphed doc is non-deterministic: the same file can +yield 75 nodes on one run and 49 on the next, dropping whole sections with no error. The +loss lands in `build_merge` and only surfaces later as a question the graph can no longer +answer. Apply this gate to **every changed doc/paper that already has nodes in +`graph.json`**. Do not skip it for a small edit — a one-line change re-extracts the whole +file. + +**1 — Snapshot the file's existing node IDs before extracting.** First clear any +snapshots left behind by an earlier run — **once**, before the first target of this run: + +```bash +rm -f graphify-out/.graphify_must_*.json graphify-out/.graphify_must_ids_*.txt +``` + +Then run the snapshot **once per changed doc**. The state files are keyed by target: a +single fixed path would let the second doc's snapshot overwrite the first's, leaving that +file silently ungated while the gate still printed PASS. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +TARGET = 'DOC_PATH' # substitute: the changed doc, VERBATIM as it appears in source_file +g = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +t = norm(TARGET) +# Match source_file EXACTLY. endswith() is not anchored to a path boundary, so a bare +# name silently adopts every same-named file deeper in the tree ('README.md' is a +# suffix of 'workers/README.md'), inflating the baseline with IDs the subagent cannot +# emit and failing the gate on a perfect extraction. +exact = [n for n in g['nodes'] if norm(n.get('source_file','')) == t] +other = sorted({norm(n.get('source_file','')) for n in g['nodes'] + if norm(n.get('source_file','')).endswith(t) and norm(n.get('source_file','')) != t}) +if not exact and other: + print('AMBIGUOUS TARGET - no source_file equals %r. Candidates:' % TARGET) + for c in other: print(' ', c) + print('Re-run with one of these verbatim; do not suffix-match.') + raise SystemExit(1) +if other: + print('NOTE: %d other file(s) end with %r - matched exactly, not by suffix.' % (len(other), TARGET)) +ids = sorted(n['id'] for n in exact) +SLUG = re.sub(r'[^A-Za-z0-9]+', '_', TARGET).strip('_') +Path('graphify-out/.graphify_must_ids_%s.txt' % SLUG).write_text('\n'.join(ids), encoding='utf-8') +Path('graphify-out/.graphify_must_%s.json' % SLUG).write_text( + json.dumps({'target': TARGET, 'ids': ids}, ensure_ascii=False), encoding='utf-8') +print(f'baseline: {len(ids)} existing node(s) for {TARGET}') +" +``` + +A count of 0 means the file is new — skip the gate and extract normally. + +**2 — Pass the IDs to the subagent as mandatory.** Append to the prompt from +`references/extraction-spec.md`: + +- the baseline count as an explicit target ("the prior build extracted N nodes from this + file; land at approximately N — under 90% of N means you under-extracted, go back and + cover the sections you skimmed"), +- the full ID list verbatim, labelled **MANDATORY — every one of these existed in the + prior graph and MUST appear in your output, reusing the ID verbatim**, +- an instruction to verify every mandatory ID is present *before* writing CHUNK_PATH. + +Reusing the IDs also suppresses gratuitous ID churn on re-extraction, which orphans saved +queries and inflates the merge diff for no semantic gain. + +**3 — Hard-gate the merge.** After the chunks land and before `build_merge`, require of +**every** snapshotted target: node count ≥ 90% of baseline, and zero mandatory IDs missing. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path +# Read EVERY chunk, not just _01. Step 3B splits a delta into chunks of 20-25 files (and +# each image gets its own), so a changed doc can land in any chunk; reading only _01 +# reports a perfect extraction as a total loss. Then gate every snapshot, so a multi-doc +# update cannot leave one of its docs unchecked. +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +snaps = sorted(glob.glob('graphify-out/.graphify_must_*.json')) +if not chunks: + print('GATE: FAIL - no chunk files on disk'); raise SystemExit(1) +nodes = [] +for c in chunks: + nodes += json.loads(Path(c).read_text(encoding='utf-8')).get('nodes', []) +ids = {n['id'] for n in nodes} +def norm(p): return str(p).replace(chr(92), '/') # chr(92) = backslash, unquotable here +bases = [json.loads(Path(s).read_text(encoding='utf-8')) for s in snaps] +targets = [norm(b['target']) for b in bases] +def owner(sf): + # Chunk source_file is the FILE_LIST path (absolute), so anchor on a path boundary -- + # and when two targets both match, the LONGEST wins, or every 'workers/README.md' node + # would also be counted against the root 'README.md'. + sf = norm(sf) + cand = [x for x in targets if sf == x or sf.endswith('/' + x)] + return max(cand, key=len) if cand else None +allok = True +for base in bases: + t, must = norm(base['target']), base['ids'] + own = [n for n in nodes if owner(n.get('source_file','')) == t] + missing = [m for m in must if m not in ids] + shrunk = len(own) < 0.9 * len(must) + ok = not (shrunk or missing) + allok = allok and ok + print('%s: nodes=%d baseline=%d missing=%d -> %s' + % (base['target'], len(own), len(must), len(missing), 'PASS' if ok else 'FAIL')) + if shrunk: print(' SHRINK: node count below 90% of baseline') + if missing: print(' MISSING:', ' '.join(missing[:20])) +print('GATE:', 'PASS' if allok else 'FAIL') +" +``` + +On **FAIL**, do not merge: re-dispatch the extraction with the same mandatory list and a +sharper completeness instruction naming the missing IDs. Two consecutive failures — stop +and report to the user rather than merging a regressed graph. + +**The `to_json` shrink guard (#479) stays authoritative.** It is the backstop, not a +nuisance. Never pass `force=True` to clear it until you have diffed the old and new node +sets and can name why each removed node is legitimately gone (file deleted, section +removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a +legitimate shrink; a missing section is not. If no new files exist (only deletions), create an empty extraction so the merge step can prune: