Surface unclassified files instead of a silent successful run (#3511) - #3512
Surface unclassified files instead of a silent successful run (#3511)#3512ayushcodes10 wants to merge 2 commits into
Conversation
detect() already returns every file it saw but could not classify, no supported extension or shebang, but nothing read it: a corpus mostly in an unsupported language got the same well covered verdict as one that was actually extracted, silently. Adds an Unclassified line to the Corpus Check section naming the count and the biggest offending extensions. Toward Graphify-Labs#3511. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
graphify extract has printed this since Graphify-Labs#1692, but the update and watch rebuild path never did, so a corpus with no extractor for its language, Lean 4 in the reported case, rebuilt successfully with those files completely absent and nothing said about it. Prints the same wording from the extract command, and threads the count into the local detection summary the rebuild path already builds for GRAPH_REPORT.md, which previously dropped it. Fixes Graphify-Labs#3511. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
Surfaces files that detect() saw but couldn't classify (no supported extension or shebang) in two places that previously dropped them silently: the corpus report now appends an Unclassified: N file(s) line with the top three extensions when any exist, and the watch/update _rebuild_code path prints a skip notice listing up to six filenames and threads unclassified through the detection dict it builds. When there are no unclassified files, both paths behave exactly as before.
Worth a look
- Unclassified files are hidden when corpus check has a warning —
graphify/report.py:146· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 644 functions depend on the 437 functions this change touches.
Health — this change adds coupling hotspots:
- new:
_rebuild_code()— 116 callers, 51 callees - new:
dispatch_command()— 2 callers, 124 callees - new:
generate()— 34 callers, 7 callees - new:
run_pipeline()— 8 callers, 13 callees - new:
make_inputs()— 16 callers, 5 callees - new:
watch()— 5 callers, 7 callees - new:
_reconcile_graph_html()— 6 callers, 5 callees - new:
load_learning_for_report()— 4 callers, 3 callees - …and 5 more — each is listed as a finding
Verification — 644 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 474 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
9 of 268 test file(s) selected (3%) via static blast radius.
tests/test_confidence.py— impacttests/test_hypergraph.py— impacttests/test_labeling.py— impacttests/test_pipeline.py— impacttests/test_report.py— impact, changed-testtests/test_report_gap_thresholds.py— impacttests/test_semantic_similarity.py— impacttests/test_watch.py— impact, changed-testtests/test_watch_manifest_location.py— impact
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
Could not verify: Could not verify generate.
The verifier did not have enough to check generate, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 200 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly ValueError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_rebuild\_code.
The verifier did not have enough to check \_rebuild\_code, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `watch_path` is annotated `Path` — outside the synthesizable primitive/collection set
· 13 more finding(s) on lines outside this diff (see the check run).
Summary
Fixes #3511.
detect()already computes and returnsunclassified— every file it saw but could not classify (no supported extension or shebang) — but nothing surfaced it. On a corpus in a language graphify has no extractor for (the report's example: a 260-file Lean 4 repo),graphify update .exits 0, prints a normal-looking "Rebuilt: N nodes..." summary, andGRAPH_REPORT.md's Corpus Check says "corpus is large enough that graph structure adds value" — describing the 4 shell scripts and 2 READMEs that happened to be classifiable, with the other 260 files invisible and unmentioned.graphify extractalready had this reporting, added by #1692 — the gap is specifically theupdate/watch rebuild path (graphify/watch.py), which never got it, plusGRAPH_REPORT.md's Corpus Check section, which had no field for it at all regardless of which command wrote the graph.graphify/report.py: adds anUnclassified: N file(s) not represented in the graph (top: .ext A, .ext B)line to the Corpus Check section wheneverdetection_resultcarries any.graphify/watch.py:_rebuild_codenow prints the same "N file(s) not classified..." lineextractalready prints, and threads the count into the local detection summary it builds forGRAPH_REPORT.md(which previously only carriedtotal_files/total_words, droppingunclassifiedeven thoughdetect()had already computed it).Scoped to just the reporting gap, matching the issue's own "Suggested fix" section. Left
graph.json'sunclassified_filesfield (the issue's third suggested surface) out of this PR — threading it through would mean passingdetection_resultdown throughbuild/build_from_json/to_json's call chain, which is a much larger, more invasive change than the two display-only additions here; happy to follow up separately if wanted. Also left the issue's separate "should graphify parse Lean 4" question alone — the reporter raised it explicitly as a discussion point for a maintainer architecture call, not as part of this fix.Test plan
tests/test_report.py: new tests confirming the Corpus Check section includes the unclassified line (with per-extension breakdown) when present, and omits it entirely when absent (backward compatible).tests/test_watch.py: new test confirming_rebuild_codeprints the unclassified count and filenames for a mixed.py/.leancorpus..pyfile plus several.leanfiles,_rebuild_codecalled directly): confirmed both the console line and theGRAPH_REPORT.mdCorpus Check line appear correctly.python3 -m pytest -q— 5423 passed, only the pre-existing unrelated failures (test_ollama_retry_cap.pymissingopenaiin this env, one flaky timing assertion intest_ts_import_type_arguments.py).python3 -m tools.skillgen --check— OK.🤖 Generated with Claude Code
https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh