Skip to content

skill: mandatory-ID gate for re-extracted docs on --update - #2162

Open
AirRocker wants to merge 1 commit into
Graphify-Labs:v8from
AirRocker:feat/mandatory-id-gate
Open

skill: mandatory-ID gate for re-extracted docs on --update#2162
AirRocker wants to merge 1 commit into
Graphify-Labs:v8from
AirRocker:feat/mandatory-id-gate

Conversation

@AirRocker

@AirRocker AirRocker commented Jul 24, 2026

Copy link
Copy Markdown

Problem

Semantic re-extraction of a doc that already has nodes in graph.json is non-deterministic. Running --update twice on the same large project CLAUDE.md — changed by one line between runs — returned 75 nodes on one run and 49 on the next, silently dropping an entire documented section.

Nothing errored. The loss went straight into build_merge and would only have surfaced much later as a question the graph could no longer answer. It was caught solely because the to_json shrink guard (#479) refused the write.

The failure mode is quiet by construction: the extraction "succeeds", the chunk is valid JSON, and the merge is happy. Only a node-count comparison against the prior build reveals it.

Fix

Adds a required three-step gate to the --update flow, applied to any changed doc/paper that already has nodes in the graph:

  1. Snapshot that file's existing node IDs from graph.json before extracting.
  2. Pass them to the subagent as mandatory must-include IDs, with the baseline node count as an explicit target ("the prior build extracted N nodes; land at ~N — under 90% means you under-extracted").
  3. Hard-gate the merge — refuse unless node count ≥ 90% of baseline and zero mandatory IDs are missing. On failure, re-dispatch naming the missing IDs; two consecutive failures stop and report rather than merging a regression.

It also restates that the shrink guard is the authoritative backstop: never force=True past it without diffing the old and new node sets and being able to name why each removed node is legitimately gone (file deleted, section removed, ID renamed with a verified replacement). ID churn on a re-extracted file is a legitimate shrink; a missing section is not.

Reusing the prior IDs has a second benefit beyond completeness: it suppresses gratuitous ID churn on re-extraction (claude_weather_proxy_workerclaude_workers_weather_proxy), which orphans saved queries and inflates the merge diff for no semantic gain.

Relationship to #3203 / 0.9.53

0.9.53 ships a per-source, count-level guard on the graphify extract path. build_merge now flags any re-extracted source whose semantic node count strictly drops, and the extract CLI arms the shrink guard on that flag: the write is refused and the file's manifest stamp is withheld, so the file re-dispatches on the next run. I re-ran my #3203 repro against 0.9.53 and it does exactly that (verification comment).

So what this gate still covers is the residual, and there are two parts to it:

1. The default skill --update path. Only graphify extract acts on the new flag (_handle_unverified_semantic_shrink in cli.py). The skill's --update flow calls build_merge directly (tools/skillgen/fragments/references/shared/update.md) and never reads the flag. That call also passes no dedup, so #2497's identity diff is skipped on this path too (it is gated if had_graph and not dedup: in build.py). On a skill --update, an under-extracted doc is at best caught afterwards, by the whole-graph to_json count check (#479). That check can tell you something shrank, but not what, and it can't see the loss at all when growth elsewhere in the same run offsets it.

2. The count-neutral identity shape. When a re-extract returns the same number of nodes for a file under new IDs, a per-source count check passes it by construction. Upstream's own test_build_merge_equal_count_different_ids_not_flagged marks that shape as by design. On 0.9.53 it still writes: exit 0, 22 → 22 nodes, and the same 9 edges dropped as in the original repro. The mandatory-ID gate compares identity rather than count, so on the skill path that shape gets re-dispatched instead of merged.

In short, 0.9.53 guards strict shrinks on graphify extract. This gate covers the skill --update path, where that guard doesn't arm, and the count-neutral shape, which no count check can see.

Evidence

  • The 75 → 49 collapse above, caught only by the shrink guard.
  • After adopting the gate: three consecutive regression-free doc updates on the same project.
  • First run with the gate codified: 111/111 mandatory IDs present, node count held exactly, merge clean, health check clean.
  • A second, worse collapse (−26 nodes, dropping an entire documented API-contract section) was caught by the gate before merge and fixed by re-dispatching with the ID list — the graph never regressed.

Scope

Markdown only — no code or CLI changes.

  • Hand-edited: tools/skillgen/fragments/references/shared/update.md (the gate) and tools/skillgen/fragments/core/core.md (a pointer from the --update section).
  • Everything else in the diff is regenerated via python -m tools.skillgen + --bless.

All five skillgen guards pass locally: --check, --audit-coverage, --schema-singleton, --monolith-roundtrip, --always-on-roundtrip.

Note on CI

tests/test_labeling.py::test_label_communities_batches_when_over_batch_size may show red. It is pre-existing and unrelated to this PR — re-confirmed on an unmodified v8 checkout at 50556ba, where it reproduces with:

pytest tests/test_skillgen.py tests/test_labeling.py

Cause: label_communities runs batches in a thread pool (graphify/llm.py, workers = max(1, min(max_concurrency, n_batches))), and for backend="gemini" concurrency stays > 1. The test appends to calls from inside the worker, so it records completion order but asserts dispatch order ([100, 100, 50]); under a different thread schedule it observes [100, 50, 100]. It passes when its file runs alone, which is why it usually goes unnoticed.

Happy to split that into its own issue/PR if useful — I left it untouched here to keep this diff to instructions only.

@AirRocker

Copy link
Copy Markdown
Author

Rebased onto current v8 (50556ba) — clean, no conflicts, and the diff is unchanged at 58 files / +1943.

Verified locally on the rebased branch: all five skillgen guards pass (--check, --audit-coverage, --schema-singleton, --monolith-roundtrip, --always-on-roundtrip), and pytest tests/ gives 4218 passed / 3 skipped. The one failure is the pre-existing test_label_communities_batches_when_over_batch_size flake described in the PR body — it reproduces the same way on an unmodified v8 checkout, so it isn't coming from this branch. The Actions run currently shows action_required, so CI is waiting on a workflow approval rather than on anything in the branch.

I've also added a short section to the description on how this sits alongside #2497, since that landed after the PR was opened and is clearly the better version of the count check. The reason I think it's still worth having both: the revived guard is gated if had_graph and not dedup: (build.py:1823) while dedup defaults to True (build.py:1553), so on a default --update the identity-level check is skipped by design, and a silent doc-node loss is still only caught afterwards by the to_json count comparison — which can tell you something shrank, but not which nodes went or whether it was legitimate ID churn.

Very happy to rework the shape if a lighter touch would fit the project better — an opt-in flag, a different default, or a config knob would all be fine by me. The protection is what I care about, not the mechanism. No rush on any of this.

@graphify-labs-staging graphify-labs-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).


Graphify review — findings

This PR adds an identical advisory callout block to each of the per-agent Graphify skill markdown files (skill-agents, skill-amp, skill-claw, etc.) plus the base skill.md, inserted after the --update/--cluster-only explanation. The note directs readers to a "mandatory-ID gate" procedure in references/update.md for re-extracting already-graphed documents. The changes also appear to expand references/update.md itself with the corresponding gate documentation. The surface area is documentation-only across the graphify skill variants and their shared reference file.

No blocking issues surfaced. 1 lower-confidence candidate did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 875 functions depend on the 875 functions this change touches.

Health — grade A; no new coupling hotspots.

Verification — 875 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 875 function(s) in the blast radius were not formally verified this run

@AirRocker

Copy link
Copy Markdown
Author

Rebased onto current v8 (adf814f). It applied cleanly and the diff is unchanged at 58 files / +1943. All five skillgen guards pass locally, including --check on the Windows skill, which upstream had touched since the last rebase. pytest tests/ is fully green run the way CI runs it (5538 passed / 13 skipped), and the label_communities flake noted in the description didn't trigger this time.

I've also updated the description for the post-0.9.53 picture. The count-level guard now ships on the extract CLI path, so this gate's remaining scope is the skill --update path plus the count-neutral ID-churn shape.

The fresh push needs a workflow approval before CI runs, whenever that's convenient. Genuinely no rush. I'm still happy to reshape this however suits the project, whether that's an opt-in flag, a different default, or a config knob.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.


Graphify review — findings

Adds a mandatory-ID gate warning to the --update/--cluster-only section of every graphify skill variant and its skillgen fixtures, requiring re-extraction of an already-graphed doc to snapshot the file's existing node IDs, pass them to the subagent as must-include, and refuse the merge if the node count falls below 90% of baseline or any ID goes missing. Documents that this guards against non-deterministic semantic re-extraction silently dropping whole sections, with the full procedure spelled out in references/update.md.

Worth a look

  • Mandatory-ID gate uses a single shared state file for every re-extracted documentgraphify/skills/agents/references/update.md:84 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Mandatory-ID snapshot can include unrelated files with matching suffixgraphify/skills/agents/references/update.md:86 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Baseline snapshot matches node IDs by endswith(TARGET) causing suffix collisionsgraphify/skills/agents/references/update.md:87 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Gate reads only .graphify_chunk_01.json, missing IDs in later chunksgraphify/skills/agents/references/update.md:105 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Mandatory-ID gate uses a single shared state file for every re-extracted documentgraphify/skills/amp/references/update.md:84 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 875 functions depend on the 875 functions this change touches.

Health — grade A; no new coupling hotspots.

Verification — 875 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 875 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

266 of 266 test file(s) selected (100%) via static blast radius.

Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.

  • tests/test_affected_cli.py — full-run-safety
  • tests/test_affected_member_seed.py — full-run-safety
  • tests/test_agents_platform.py — full-run-safety
  • tests/test_analyze.py — full-run-safety
  • tests/test_anthropic_custom_endpoint.py — full-run-safety
  • tests/test_antigravity_install.py — full-run-safety
  • tests/test_apm_fallback_version.py — full-run-safety
  • tests/test_architecture_doc.py — full-run-safety
  • tests/test_astro_extraction.py — full-run-safety
  • tests/test_astro_import_ids.py — full-run-safety
  • tests/test_atomic_canvas_export.py — full-run-safety
  • tests/test_atomic_version_stamp.py — full-run-safety
  • tests/test_atomic_writes.py — full-run-safety
  • tests/test_backend_extras.py — full-run-safety
  • tests/test_benchmark.py — full-run-safety
  • tests/test_benchmark_raw_graph.py — full-run-safety
  • tests/test_build.py — full-run-safety
  • tests/test_build_merge_hyperedges_and_prune.py — full-run-safety
  • tests/test_build_merge_shrink_guard.py — full-run-safety
  • tests/test_builtin_global_type_refs.py — full-run-safety
  • tests/test_cache.py — full-run-safety
  • tests/test_callflow_html.py — full-run-safety
  • tests/test_cargo_introspect.py — full-run-safety
  • tests/test_carried_hyperedge_remap.py — full-run-safety
  • tests/test_case_sensitive_resolution.py — full-run-safety
  • tests/test_charmap_encoding.py — full-run-safety
  • tests/test_chunking.py — full-run-safety
  • tests/test_cjs_module_extension.py — full-run-safety
  • tests/test_claude_cli_backend.py — full-run-safety
  • tests/test_claude_md.py — full-run-safety
  • tests/test_cli_broken_pipe.py — full-run-safety
  • tests/test_cli_export.py — full-run-safety
  • tests/test_cli_help.py — full-run-safety
  • tests/test_cluster.py — full-run-safety
  • tests/test_codebuddy.py — full-run-safety
  • tests/test_community_hub_labels.py — full-run-safety
  • tests/test_community_labels_skill.py — full-run-safety
  • tests/test_confidence.py — full-run-safety
  • tests/test_corrupt_graph_json.py — full-run-safety
  • tests/test_cpp_nested_and_cli.py — full-run-safety
  • tests/test_cpp_objc_cross_file_calls.py — full-run-safety
  • tests/test_cpp_preprocess.py — full-run-safety
  • tests/test_cross_extension_reexport_self_cycle.py — full-run-safety
  • tests/test_cross_language_call_resolution.py — full-run-safety
  • tests/test_cross_repo_member_calls.py — full-run-safety
  • tests/test_cross_repo_shared_types.py — full-run-safety
  • tests/test_csharp_call_site_generic_args.py — full-run-safety
  • tests/test_csharp_enum_members.py — full-run-safety
  • tests/test_csharp_field_generic_args.py — full-run-safety
  • tests/test_csharp_generic_callsites.py — full-run-safety
  • … and 216 more

non-code file(s) changed (graphify/skill-agents.md, graphify/skill-amp.md, graphify/skill-claw.md, graphify/skill-codex.md, graphify/skill-copilot.md …) → running the full suite for safety (a code graph can't see config/fixture/data deps)

changed code file(s) with no mapped test (graphify/skill-agents.md, graphify/skill-amp.md, graphify/skill-claw.md, graphify/skill-codex.md, graphify/skill-copilot.md …) — a coverage gap or a missing link — running the full suite rather than only the selected tests

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

…hed doc

Semantic re-extraction of a doc that already has nodes in graph.json is
non-deterministic. On a large project CLAUDE.md, successive --update runs
of the same unchanged-except-one-line file returned 75 nodes and then 49,
silently dropping a whole documented section. Nothing errored: the loss
went into build_merge and would only have surfaced later as a question the
graph could no longer answer. It was caught solely because the to_json
shrink guard (Graphify-Labs#479) refused the write.

Adds a required three-step gate to the --update flow, in the shared
references/update.md fragment plus a pointer from the core template:

1. Snapshot that file's existing node IDs from graph.json before extracting.
2. Pass them to the extraction subagent as MANDATORY must-include IDs, with
   the baseline node count as an explicit target. Reusing the IDs also
   suppresses gratuitous ID churn, which orphans saved queries and inflates
   the merge diff for no semantic gain.
3. Hard-gate the merge: refuse unless node count >= 90% of baseline AND zero
   mandatory IDs are missing. On failure, re-dispatch naming the missing IDs;
   two consecutive failures stop and report rather than merging a regression.

Also restates that the shrink guard is the authoritative backstop: never
force past it without diffing the old and new node sets and being able to
name why each removed node is legitimately gone.

Fragments are the edited source; graphify/skill*.md, graphify/skills/**,
graphify/always_on/** and tools/skillgen/expected/** are regenerated via
`python -m tools.skillgen` and `--bless`. Markdown only, no code changes.

All five skillgen guards pass (--check, --audit-coverage, --schema-singleton,
--monolith-roundtrip, --always-on-roundtrip).

Review follow-up (graphify-labs bot, 2026-09-10) — three defects fixed in the
gate procedure, each reproduced against a real graph before the edit:

* Suffix matching. `endswith(TARGET)` is not anchored to a path boundary, so a
  bare name adopts every same-named file deeper in the tree. Measured on a real
  corpus, TARGET='README.md' matched 'workers/README.md' too: baseline 22 nodes
  instead of 7, and a *perfect* re-extraction of README.md then scored 31.8% of
  baseline with 15 mandatory IDs "missing" — a hard FAIL on correct output, and
  the prescribed remedy (re-dispatch naming the missing IDs) would have pushed
  the subagent to mint another file's nodes under this one's source_file. Now
  matched exactly, with an ambiguity guard that lists the candidates and stops
  rather than guessing.

* Single shared state file. `.graphify_must_ids.txt` / `.graphify_must_slice.json`
  were fixed paths, so in a multi-doc delta the second snapshot overwrote the
  first and that doc went through ungated while the gate still printed PASS —
  failing open, in the one place this procedure exists to fail closed. State
  files are now keyed by target, with a one-time clear of stale snapshots.

* Only `.graphify_chunk_01.json` was read. Step 3B splits a delta into chunks of
  20-25 files (each image its own), so a changed doc can land in any chunk; the
  gate then read every mandatory ID as missing and failed a clean extraction.
  The gate now unions all chunks and evaluates every snapshot, counting only the
  nodes attributed to each target — which also fixes a latent over-count, since
  the old `len(d['nodes'])` was the whole chunk, not the target's slice.

Verified: multi-chunk delta with the target in chunk_02 now PASSes, with
negative controls (drop a mandatory node, drop a baseline hyperedge) confirming
the gate still FAILs when it should. Markdown only; no code changes.

Claude-Session: https://claude.ai/code/session_01B8MXR5KMGHczdCLXB8kvJd
@AirRocker
AirRocker force-pushed the feat/mandatory-id-gate branch from 7992600 to a9b253d Compare September 12, 2026 17:34
@AirRocker

Copy link
Copy Markdown
Author

Thanks for this — the review earned its keep. All four distinct findings are real, and three of them I could reproduce against a live graph before touching a word of the text. Fixed in a9b253d, force-pushed over 7992600.

1. Single shared state file for every re-extracted document (…/update.md:84, and the same finding on the amp variant)

Real, and the worst of the four, because it is the one that fails open. .graphify_must_ids.txt was a fixed path, so in a delta with two changed docs the second snapshot overwrote the first, and that doc went through the merge ungated while the gate still printed GATE: PASS — a silent hole in the one procedure whose whole job is to fail closed. Reproduced on a 3,545-node graph: snapshot doc A (2 mandatory IDs), snapshot doc B (27), and A's IDs are simply gone from disk.

One clarification on the framing, because it changes what the fix has to be: this isn't a concurrency race. Nothing writes these files concurrently — the extraction subagents write CHUNK_PATH, while the snapshot and the gate both run in the host session, sequentially. It is a plain sequential overwrite, which is why keying the state file by target is sufficient and no locking is needed.

Fix: state files are keyed by target (.graphify_must_<slug>.json), with a one-time clear of stale snapshots before the first target of a run.

2 + 3. Suffix collisions from endswith(TARGET) (…:86 and …:87)

Real — and the sharpest of the three to demonstrate, because it needs no operator error at all. endswith is not anchored to a path boundary, so a source_file that is a bare root-level name is a suffix of every same-named file deeper in the tree. On one of our real corpora, TARGET = 'README.md' — which is exactly how the root README appears in source_file, so there is nothing to mistype — also matched workers/README.md:

gate (endswith) baseline : 22 nodes
true (exact)   baseline :  7 nodes
FOREIGN nodes pulled in  : 15  (from workers/README.md)

The consequence is a hard FAIL on correct output: a perfect re-extraction of README.md lands at 31.8% of the inflated baseline with 15 mandatory IDs reported missing. And the remedy the procedure then prescribes makes it worse — "re-dispatch naming the missing IDs" tells the subagent to emit another file's nodes under this file's source_file, which replace-on-re-extract would then attribute to the wrong file.

Fix: match source_file exactly. Where no exact match exists but suffix candidates do, the snapshot prints the candidates and stops rather than guessing.

4. Gate reads only .graphify_chunk_01.json (…:105)

Real. One correction to the mechanism, because it shaped the fix: a document never splits across chunks. Step 3B chunks by file — 20–25 files per chunk, and each image gets its own — so one file always lands in exactly one chunk. Multiple chunks come from a delta with many semantic files, and then the target doc can sit in _02 while the gate reads _01, sees none of its mandatory IDs, and fails a clean extraction. Not hypothetical: a recent --update on our side dispatched two doc subagents on a 14-file delta.

Fix: the gate now unions every .graphify_chunk_*.json and evaluates every snapshot in one pass. That also closed a latent over-count nobody flagged — the old len(d['nodes']) was the whole chunk, so for any chunk holding more than one file the 90% shrink check was measured against the wrong number. It now counts only the nodes attributed to each target, longest-suffix wins, so workers/README.md nodes are not counted against README.md.

Verification

Two-chunk delta with the target in _02 now passes with correct per-doc counts (7/7 and 15/15), and two negative controls confirm the gate can still fail: drop one mandatory node → FAIL with the ID named; drop one baseline hyperedge → FAIL. All five skillgen guards pass. pytest tests/ is 5412 passed / 95 skipped — the only failures are the four test_ollama_retry_cap.py tests that need the uninstalled openai extra and reproduce identically on a pristine checkout, plus one scales_linearly timing test that passes in isolation.

Worth saying plainly: this gate text runs in production for us, across a six-project fleet, in a hardened superset of this fragment. So these are not edits to a proposal — the identical three fixes went into the live copy in the same pass, which is also where the reproductions above came from.

CI is showing action_required on the new SHA, so it is waiting on a workflow approval whenever that is convenient. No rush on any of it, and as before I am happy to reshape the whole thing if a lighter touch would suit the project better.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Formal verification. 2 change(s) tested, no difference found (not proven).


Graphify review — findings

Adds a mandatory-ID gate callout to the --update/re-extraction section of every graphify skill variant and its expected fixtures, telling the agent to snapshot an already-graphed doc's node IDs, pass them to the subagent as must-include, and refuse the merge if node count drops below 90% of baseline or any ID goes missing. Explains the fallback rationale: semantic re-extraction is non-deterministic and silently drops whole sections without the gate, with the full procedure documented in references/update.md.

No blocking issues surfaced. 49 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 875 functions depend on the 875 functions this change touches.

Health — grade A; no new coupling hotspots.

Verification — 875 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 875 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

266 of 266 test file(s) selected (100%) via static blast radius.

Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.

  • tests/test_affected_cli.py — full-run-safety
  • tests/test_affected_member_seed.py — full-run-safety
  • tests/test_agents_platform.py — full-run-safety
  • tests/test_analyze.py — full-run-safety
  • tests/test_anthropic_custom_endpoint.py — full-run-safety
  • tests/test_antigravity_install.py — full-run-safety
  • tests/test_apm_fallback_version.py — full-run-safety
  • tests/test_architecture_doc.py — full-run-safety
  • tests/test_astro_extraction.py — full-run-safety
  • tests/test_astro_import_ids.py — full-run-safety
  • tests/test_atomic_canvas_export.py — full-run-safety
  • tests/test_atomic_version_stamp.py — full-run-safety
  • tests/test_atomic_writes.py — full-run-safety
  • tests/test_backend_extras.py — full-run-safety
  • tests/test_benchmark.py — full-run-safety
  • tests/test_benchmark_raw_graph.py — full-run-safety
  • tests/test_build.py — full-run-safety
  • tests/test_build_merge_hyperedges_and_prune.py — full-run-safety
  • tests/test_build_merge_shrink_guard.py — full-run-safety
  • tests/test_builtin_global_type_refs.py — full-run-safety
  • tests/test_cache.py — full-run-safety
  • tests/test_callflow_html.py — full-run-safety
  • tests/test_cargo_introspect.py — full-run-safety
  • tests/test_carried_hyperedge_remap.py — full-run-safety
  • tests/test_case_sensitive_resolution.py — full-run-safety
  • tests/test_charmap_encoding.py — full-run-safety
  • tests/test_chunking.py — full-run-safety
  • tests/test_cjs_module_extension.py — full-run-safety
  • tests/test_claude_cli_backend.py — full-run-safety
  • tests/test_claude_md.py — full-run-safety
  • tests/test_cli_broken_pipe.py — full-run-safety
  • tests/test_cli_export.py — full-run-safety
  • tests/test_cli_help.py — full-run-safety
  • tests/test_cluster.py — full-run-safety
  • tests/test_codebuddy.py — full-run-safety
  • tests/test_community_hub_labels.py — full-run-safety
  • tests/test_community_labels_skill.py — full-run-safety
  • tests/test_confidence.py — full-run-safety
  • tests/test_corrupt_graph_json.py — full-run-safety
  • tests/test_cpp_nested_and_cli.py — full-run-safety
  • tests/test_cpp_objc_cross_file_calls.py — full-run-safety
  • tests/test_cpp_preprocess.py — full-run-safety
  • tests/test_cross_extension_reexport_self_cycle.py — full-run-safety
  • tests/test_cross_language_call_resolution.py — full-run-safety
  • tests/test_cross_repo_member_calls.py — full-run-safety
  • tests/test_cross_repo_shared_types.py — full-run-safety
  • tests/test_csharp_call_site_generic_args.py — full-run-safety
  • tests/test_csharp_enum_members.py — full-run-safety
  • tests/test_csharp_field_generic_args.py — full-run-safety
  • tests/test_csharp_generic_callsites.py — full-run-safety
  • … and 216 more

non-code file(s) changed (graphify/skill-agents.md, graphify/skill-amp.md, graphify/skill-claw.md, graphify/skill-codex.md, graphify/skill-copilot.md …) → running the full suite for safety (a code graph can't see config/fixture/data deps)

changed code file(s) with no mapped test (graphify/skill-agents.md, graphify/skill-amp.md, graphify/skill-claw.md, graphify/skill-codex.md, graphify/skill-copilot.md …) — a coverage gap or a missing link — running the full suite rather than only the selected tests

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

Could not verify: Could not verify build.

The verifier did not have enough to check build, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 9 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly AttributeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify build\_merge.

The verifier did not have enough to check build\_merge, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `graph_path` is annotated `str | Path | None` — outside the synthesizable primitive/collection set

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

Could not verify: Could not verify deduplicate\_entities.

The verifier did not have enough to check deduplicate\_entities, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 9 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_llm\_tiebreak.

The verifier did not have enough to check \_llm\_tiebreak, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 200 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify extract.

The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_extract\_parallel.

The verifier did not have enough to check \_extract\_parallel, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_apply\_symbol\_resolution\_facts.

The verifier did not have enough to check \_apply\_symbol\_resolution\_facts, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_collect\_js\_symbol\_resolution\_facts.

The verifier did not have enough to check \_collect\_js\_symbol\_resolution\_facts, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `facts` is annotated `_SymbolResolutionFacts` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_collect\_python\_symbol\_resolution\_facts.

The verifier did not have enough to check \_collect\_python\_symbol\_resolution\_facts, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_contained\_in\_package.

The verifier did not have enough to check \_contained\_in\_package, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `resolved` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_find\_js\_config.

The verifier did not have enough to check \_find\_js\_config, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `start_dir` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_find\_js\_project\_anchor.

The verifier did not have enough to check \_find\_js\_project\_anchor, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `start_dir` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_find\_workspace\_root.

The verifier did not have enough to check \_find\_workspace\_root, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `start_dir` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_go\_import\_path\_for\_file.

The verifier did not have enough to check \_go\_import\_path\_for\_file, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `source_file` is annotated `str | Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_js\_source\_path.

The verifier did not have enough to check \_js\_source\_path, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_load\_package\_imports.

The verifier did not have enough to check \_load\_package\_imports, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `start_dir` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_parse\_python\_tree.

The verifier did not have enough to check \_parse\_python\_tree, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_read\_tsconfig\_aliases.

The verifier did not have enough to check \_read\_tsconfig\_aliases, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `tsconfig` is annotated `Path` — outside the synthesizable primitive/collection set

No difference found (not proven): No behavior difference found in \_resolve\_c\_include\_path (not a proof).

The verifier ran both versions of \_resolve\_c\_include\_path on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_resolve\_cross\_file\_imports.

The verifier did not have enough to check \_resolve\_cross\_file\_imports, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 2 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

Could not verify: Could not verify \_resolve\_python\_namespace\_dir.

The verifier did not have enough to check \_resolve\_python\_namespace\_dir, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `current_path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_source\_key.

The verifier did not have enough to check \_source\_key, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

No difference found (not proven): No behavior difference found in \_walk\_python\_tree (not a proof).

The verifier ran both versions of \_walk\_python\_tree on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: concolic exploration (CrossHair). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify extract\_rust.

The verifier did not have enough to check extract\_rust, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant