Skip to content

Surface unclassified files instead of a silent successful run (#3511) - #3512

Open
ayushcodes10 wants to merge 2 commits into
Graphify-Labs:v8from
ayushcodes10:fix-3511-surface-unclassified-files
Open

Surface unclassified files instead of a silent successful run (#3511)#3512
ayushcodes10 wants to merge 2 commits into
Graphify-Labs:v8from
ayushcodes10:fix-3511-surface-unclassified-files

Conversation

@ayushcodes10

Copy link
Copy Markdown
Contributor

Summary

Fixes #3511. detect() already computes and returns unclassified — every file it saw but could not classify (no supported extension or shebang) — but nothing surfaced it. On a corpus in a language graphify has no extractor for (the report's example: a 260-file Lean 4 repo), graphify update . exits 0, prints a normal-looking "Rebuilt: N nodes..." summary, and GRAPH_REPORT.md's Corpus Check says "corpus is large enough that graph structure adds value" — describing the 4 shell scripts and 2 READMEs that happened to be classifiable, with the other 260 files invisible and unmentioned.

graphify extract already had this reporting, added by #1692 — the gap is specifically the update/watch rebuild path (graphify/watch.py), which never got it, plus GRAPH_REPORT.md's Corpus Check section, which had no field for it at all regardless of which command wrote the graph.

  • graphify/report.py: adds an Unclassified: N file(s) not represented in the graph (top: .ext A, .ext B) line to the Corpus Check section whenever detection_result carries any.
  • graphify/watch.py: _rebuild_code now prints the same "N file(s) not classified..." line extract already prints, and threads the count into the local detection summary it builds for GRAPH_REPORT.md (which previously only carried total_files/total_words, dropping unclassified even though detect() had already computed it).

Scoped to just the reporting gap, matching the issue's own "Suggested fix" section. Left graph.json's unclassified_files field (the issue's third suggested surface) out of this PR — threading it through would mean passing detection_result down through build/build_from_json/to_json's call chain, which is a much larger, more invasive change than the two display-only additions here; happy to follow up separately if wanted. Also left the issue's separate "should graphify parse Lean 4" question alone — the reporter raised it explicitly as a discussion point for a maintainer architecture call, not as part of this fix.

Test plan

  • tests/test_report.py: new tests confirming the Corpus Check section includes the unclassified line (with per-extension breakdown) when present, and omits it entirely when absent (backward compatible).
  • tests/test_watch.py: new test confirming _rebuild_code prints the unclassified count and filenames for a mixed .py/.lean corpus.
  • Manually reproduced the issue's shape end to end (a .py file plus several .lean files, _rebuild_code called directly): confirmed both the console line and the GRAPH_REPORT.md Corpus Check line appear correctly.
  • Full suite: python3 -m pytest -q — 5423 passed, only the pre-existing unrelated failures (test_ollama_retry_cap.py missing openai in this env, one flaky timing assertion in test_ts_import_type_arguments.py).
  • python3 -m tools.skillgen --check — OK.

🤖 Generated with Claude Code

https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh

ayushcodes10 and others added 2 commits September 12, 2026 18:58
detect() already returns every file it saw but could not classify, no
supported extension or shebang, but nothing read it: a corpus mostly
in an unsupported language got the same well covered verdict as one
that was actually extracted, silently. Adds an Unclassified line to
the Corpus Check section naming the count and the biggest offending
extensions. Toward Graphify-Labs#3511.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
graphify extract has printed this since Graphify-Labs#1692, but the update and
watch rebuild path never did, so a corpus with no extractor for its
language, Lean 4 in the reported case, rebuilt successfully with
those files completely absent and nothing said about it. Prints the
same wording from the extract command, and threads the count into
the local detection summary the rebuild path already builds for
GRAPH_REPORT.md, which previously dropped it. Fixes Graphify-Labs#3511.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Formal verification. No changes could be formally verified in this run.


Graphify review — findings

Surfaces files that detect() saw but couldn't classify (no supported extension or shebang) in two places that previously dropped them silently: the corpus report now appends an Unclassified: N file(s) line with the top three extensions when any exist, and the watch/update _rebuild_code path prints a skip notice listing up to six filenames and threads unclassified through the detection dict it builds. When there are no unclassified files, both paths behave exactly as before.

Worth a look

  • Unclassified files are hidden when corpus check has a warninggraphify/report.py:146 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 644 functions depend on the 437 functions this change touches.

Health — this change adds coupling hotspots:

  • new: _rebuild_code() — 116 callers, 51 callees
  • new: dispatch_command() — 2 callers, 124 callees
  • new: generate() — 34 callers, 7 callees
  • new: run_pipeline() — 8 callers, 13 callees
  • new: make_inputs() — 16 callers, 5 callees
  • new: watch() — 5 callers, 7 callees
  • new: _reconcile_graph_html() — 6 callers, 5 callees
  • new: load_learning_for_report() — 4 callers, 3 callees
  • …and 5 more — each is listed as a finding

Verification — 644 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 474 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

9 of 268 test file(s) selected (3%) via static blast radius.

  • tests/test_confidence.py — impact
  • tests/test_hypergraph.py — impact
  • tests/test_labeling.py — impact
  • tests/test_pipeline.py — impact
  • tests/test_report.py — impact, changed-test
  • tests/test_report_gap_thresholds.py — impact
  • tests/test_semantic_similarity.py — impact
  • tests/test_watch.py — impact, changed-test
  • tests/test_watch_manifest_location.py — impact

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

Could not verify: Could not verify generate.

The verifier did not have enough to check generate, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 200 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly ValueError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_rebuild\_code.

The verifier did not have enough to check \_rebuild\_code, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `watch_path` is annotated `Path` — outside the synthesizable primitive/collection set

· 13 more finding(s) on lines outside this diff (see the check run).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

detect() already tracks unclassified files; nothing surfaces them, so a 260-file Lean 4 repo reports as a successful run

1 participant