fix(builder): do not commit a file's hash when its parse produced no data - #2556
Merged
Conversation
…data Same failure mode as #2435 (a file_hashes row that outlives the data it describes), but on the codegraph build --incremental (scoped, and more generally any) build path rather than codegraph watch, and independent of that fix. handleScopedBuild purges a changed file's nodes/edges before parsing runs, deliberately leaving its OLD file_hashes row alone (the #1731 deferred-commit design: a hash only advances once the data it describes has been rebuilt to match). commitFileHashes/build_file_hash_entries then built their hash lists from the changed-file set directly, with no check that each file actually produced data — so a file whose extraction failed outright (worker crash, unreadable, unsupported/missing grammar) still got its hash committed as if the (missing) new state matched disk, silently and permanently hiding the loss from every later incremental build. Both engines shared this bug: the JS orchestration pipeline (commitFileHashes) via ctx.filesToParse, and the fully-native Rust pipeline (NativeDatabase::build_graph -> run_pipeline) via its own build_file_hash_entries, called independently as the primary native build path (JS runPipelineStages is only the fallback when native build_graph is unavailable or throws). Fix: skip a changed file's hash entry when it has no corresponding entry in fileSymbols/file_symbols — the reliable signal that extraction actually ran and produced (possibly empty) output, as opposed to no entry at all meaning it never ran. This preserves #1068's distinct guarantee (a file that parses successfully but legitimately produces zero symbols DOES get an entry, with empty definitions/exports, and must still get its hash committed or the no-op fast-skip pre-flight would reject it as missing). The purge itself is not deferred (a much larger change, since the purge and the parse live in different pipeline stages) — the surviving stale hash is what forces the next incremental build to reprocess the file and recover the data. docs check acknowledged Impact: 3 functions changed, 6 affected
Contributor
Greptile SummaryThe PR prevents incremental builds from committing a changed file’s hash when parsing produced no output, preserving the stale hash needed to retry and recover on a later build.
Confidence Score: 5/5The PR appears safe to merge, with both build engines preserving the deferred-hash invariant across failed and legitimately empty parses. The new membership checks align with the parser result contracts: failed parses produce no map entry, successful empty parses do, and withheld hashes cause subsequent incremental builds to retry rather than silently accepting missing graph data. Important Files Changed
Sequence DiagramsequenceDiagram
participant Build as Incremental Build
participant Parse as Parser
participant Graph as Graph DB
Build->>Graph: Purge changed file data
Build->>Parse: Parse changed file
alt Parse produces FileSymbols entry
Parse-->>Build: Symbols or valid empty output
Build->>Graph: Rebuild nodes and edges
Build->>Graph: Commit new file hash
else Parse produces no entry
Parse-->>Build: No output
Build-->>Graph: Keep previous file hash
Note over Build,Graph: Next incremental build detects the mismatch and retries
end
Reviews (1): Last reviewed commit: "fix(builder): do not commit a file's has..." | Re-trigger Greptile |
Contributor
Codegraph Impact Analysis3 functions changed → 6 callers affected across 4 files
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Same failure mode as #2435 (a
file_hashesrow that outlives the data it describes), but on thecodegraph build --incremental(scoped, and more generally any) build path rather thancodegraph watch— pre-existing and independent of that fix.handleScopedBuildpurges a changed file's nodes/edges before parsing runs, deliberately leaving its OLDfile_hashesrow alone (the #1731 deferred-commit design: a hash only advances once the data it describes has been rebuilt to match).commitFileHashes/build_file_hash_entriesthen built their hash lists from the changed-file set directly, with no check that each file actually produced data — so a file whose extraction failed outright (worker crash, unreadable, unsupported/missing grammar) still got its hash committed as if the (missing) new state matched disk, silently and permanently hiding the loss from every later incremental build.Both engines shared this bug
commitFileHashes) viactx.filesToParse.NativeDatabase::build_graph→run_pipeline) via its ownbuild_file_hash_entries— this is the primary native build path (JSrunPipelineStagesis only the fallback when nativebuild_graphis unavailable or throws), so this genuinely needed a mirrored fix, not just a JS-side one.Fix
Skip a changed file's hash entry when it has no corresponding entry in
fileSymbols/file_symbols— the reliable signal that extraction actually ran and produced (possibly empty) output, as opposed to no entry at all meaning it never ran.This preserves #1068's distinct guarantee: a file that parses successfully but legitimately produces zero symbols (empty file, parser no-op, optional-language grammar unavailable) DOES get an entry (with empty
definitions/exports), and must still get its hash committed — otherwise the no-op fast-skip pre-flight would reject it as missing fromfile_hashesand force a full rebuild.buildFileHashes's newparsedRelPathsparameter is optional specifically so #1068's own direct unit test (which doesn't track parse outcomes) keeps its original unconditional behavior.The purge itself is not deferred (the issue's own "stronger" option, but a much larger change since the purge and the parse live in different pipeline stages) — the surviving stale hash is what forces the next incremental build to reprocess the file and recover the data.
Tests
tests/integration/issue-2441-scoped-build-failed-parse-data-loss.test.ts: drives a real scopedbuildGraph({ incremental: true, scope: [...] }), forcing one file's WASM parse to fail via a targetedWasmWorkerPool.parsemock. Verifies the stale (pre-edit) hash survives the failed build, then that a subsequent successful scoped build actually reprocesses and recovers the file's symbols and edges. A second test confirms the hash still commits normally on success.build_file_hash_entries: one confirming a file with nofile_symbolsentry is skipped, one confirming a legitimately-empty-but-parsed file is still included (the fast-skip: file_hashes drops files when grammar isn't installed (breaks fast-skip permanently) #1068-equivalent case).cargo clippy/cargo fmt --checkclean.Closes #2441