Skip to content

fix(builder): do not commit a file's hash when its parse produced no data - #2556

Merged
carlos-alm merged 1 commit into
mainfrom
fix/issue-2441
Aug 17, 2026
Merged

fix(builder): do not commit a file's hash when its parse produced no data#2556
carlos-alm merged 1 commit into
mainfrom
fix/issue-2441

Conversation

@carlos-alm

Copy link
Copy Markdown
Contributor

Summary

Same failure mode as #2435 (a file_hashes row that outlives the data it describes), but on the codegraph build --incremental (scoped, and more generally any) build path rather than codegraph watch — pre-existing and independent of that fix.

handleScopedBuild purges a changed file's nodes/edges before parsing runs, deliberately leaving its OLD file_hashes row alone (the #1731 deferred-commit design: a hash only advances once the data it describes has been rebuilt to match). commitFileHashes/build_file_hash_entries then built their hash lists from the changed-file set directly, with no check that each file actually produced data — so a file whose extraction failed outright (worker crash, unreadable, unsupported/missing grammar) still got its hash committed as if the (missing) new state matched disk, silently and permanently hiding the loss from every later incremental build.

Both engines shared this bug

  • The JS orchestration pipeline (commitFileHashes) via ctx.filesToParse.
  • The fully-native Rust pipeline (NativeDatabase::build_graphrun_pipeline) via its own build_file_hash_entries — this is the primary native build path (JS runPipelineStages is only the fallback when native build_graph is unavailable or throws), so this genuinely needed a mirrored fix, not just a JS-side one.

Fix

Skip a changed file's hash entry when it has no corresponding entry in fileSymbols/file_symbols — the reliable signal that extraction actually ran and produced (possibly empty) output, as opposed to no entry at all meaning it never ran.

This preserves #1068's distinct guarantee: a file that parses successfully but legitimately produces zero symbols (empty file, parser no-op, optional-language grammar unavailable) DOES get an entry (with empty definitions/exports), and must still get its hash committed — otherwise the no-op fast-skip pre-flight would reject it as missing from file_hashes and force a full rebuild. buildFileHashes's new parsedRelPaths parameter is optional specifically so #1068's own direct unit test (which doesn't track parse outcomes) keeps its original unconditional behavior.

The purge itself is not deferred (the issue's own "stronger" option, but a much larger change since the purge and the parse live in different pipeline stages) — the surviving stale hash is what forces the next incremental build to reprocess the file and recover the data.

Tests

  • tests/integration/issue-2441-scoped-build-failed-parse-data-loss.test.ts: drives a real scoped buildGraph({ incremental: true, scope: [...] }), forcing one file's WASM parse to fail via a targeted WasmWorkerPool.parse mock. Verifies the stale (pre-edit) hash survives the failed build, then that a subsequent successful scoped build actually reprocesses and recovers the file's symbols and edges. A second test confirms the hash still commits normally on success.
  • Two new Rust unit tests directly on build_file_hash_entries: one confirming a file with no file_symbols entry is skipped, one confirming a legitimately-empty-but-parsed file is still included (the fast-skip: file_hashes drops files when grammar isn't installed (breaks fast-skip permanently) #1068-equivalent case).
  • Revert-verified both fixes independently (temporarily reverted each, confirmed the exact targeted tests fail, restored).
  • Full suites green: 338 JS test files / 5401 tests; 1080 Rust tests. cargo clippy/cargo fmt --check clean.

Closes #2441

…data

Same failure mode as #2435 (a file_hashes row that outlives the data it
describes), but on the codegraph build --incremental (scoped, and more
generally any) build path rather than codegraph watch, and independent
of that fix.

handleScopedBuild purges a changed file's nodes/edges before parsing runs,
deliberately leaving its OLD file_hashes row alone (the #1731
deferred-commit design: a hash only advances once the data it describes
has been rebuilt to match). commitFileHashes/build_file_hash_entries then
built their hash lists from the changed-file set directly, with no check
that each file actually produced data — so a file whose extraction failed
outright (worker crash, unreadable, unsupported/missing grammar) still got
its hash committed as if the (missing) new state matched disk, silently
and permanently hiding the loss from every later incremental build.

Both engines shared this bug: the JS orchestration pipeline
(commitFileHashes) via ctx.filesToParse, and the fully-native Rust pipeline
(NativeDatabase::build_graph -> run_pipeline) via its own
build_file_hash_entries, called independently as the primary native build
path (JS runPipelineStages is only the fallback when native build_graph is
unavailable or throws).

Fix: skip a changed file's hash entry when it has no corresponding entry
in fileSymbols/file_symbols — the reliable signal that extraction actually
ran and produced (possibly empty) output, as opposed to no entry at all
meaning it never ran. This preserves #1068's distinct guarantee (a file
that parses successfully but legitimately produces zero symbols DOES get
an entry, with empty definitions/exports, and must still get its hash
committed or the no-op fast-skip pre-flight would reject it as missing).

The purge itself is not deferred (a much larger change, since the purge
and the parse live in different pipeline stages) — the surviving stale
hash is what forces the next incremental build to reprocess the file and
recover the data.

docs check acknowledged

Impact: 3 functions changed, 6 affected
@greptile-apps

greptile-apps Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR prevents incremental builds from committing a changed file’s hash when parsing produced no output, preserving the stale hash needed to retry and recover on a later build.

  • Applies equivalent parse-outcome filtering to the TypeScript/WASM and native Rust pipelines.
  • Preserves hash commits for successfully parsed files that legitimately contain no symbols.
  • Adds scoped-build integration coverage and focused Rust unit tests for failed and empty parse outcomes.

Confidence Score: 5/5

The PR appears safe to merge, with both build engines preserving the deferred-hash invariant across failed and legitimately empty parses.

The new membership checks align with the parser result contracts: failed parses produce no map entry, successful empty parses do, and withheld hashes cause subsequent incremental builds to retry rather than silently accepting missing graph data.

Important Files Changed

Filename Overview
crates/codegraph-core/src/domain/graph/builder/pipeline.rs Filters native hash entries by successful parse-output membership while retaining valid empty parses; focused unit tests cover both outcomes.
src/domain/graph/builder/stages/insert-nodes.ts Gates deferred TypeScript-side hash commits on fileSymbols membership without changing metadata-only healing or reverse-dependency handling.
tests/integration/issue-2441-scoped-build-failed-parse-data-loss.test.ts Reproduces a scoped WASM parse failure and verifies that a later successful incremental build restores symbols, edges, and the hash.

Sequence Diagram

sequenceDiagram
  participant Build as Incremental Build
  participant Parse as Parser
  participant Graph as Graph DB
  Build->>Graph: Purge changed file data
  Build->>Parse: Parse changed file
  alt Parse produces FileSymbols entry
    Parse-->>Build: Symbols or valid empty output
    Build->>Graph: Rebuild nodes and edges
    Build->>Graph: Commit new file hash
  else Parse produces no entry
    Parse-->>Build: No output
    Build-->>Graph: Keep previous file hash
    Note over Build,Graph: Next incremental build detects the mismatch and retries
  end
Loading

Reviews (1): Last reviewed commit: "fix(builder): do not commit a file's has..." | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown
Contributor

Codegraph Impact Analysis

3 functions changed6 callers affected across 4 files

  • iterFileHashRecords in src/domain/graph/builder/stages/insert-nodes.ts:162 (3 transitive callers)
  • buildFileHashes in src/domain/graph/builder/stages/insert-nodes.ts:230 (3 transitive callers)
  • commitFileHashes in src/domain/graph/builder/stages/insert-nodes.ts:503 (4 transitive callers)

@carlos-alm
carlos-alm merged commit 9db49c0 into main Aug 17, 2026
34 checks passed
@carlos-alm
carlos-alm deleted the fix/issue-2441 branch August 17, 2026 12:20
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A scoped build that fails to parse a changed file purges it and commits its hash as up to date

1 participant