Skip to content

fix: rebuild crashes with "cannot start a transaction within a transaction"#181

Open
realkotob wants to merge 1 commit intotirth8205:mainfrom
realkotob:fix/sqlite-nested-transaction-purge
Open

fix: rebuild crashes with "cannot start a transaction within a transaction"#181
realkotob wants to merge 1 commit intotirth8205:mainfrom
realkotob:fix/sqlite-nested-transaction-purge

Conversation

@realkotob
Copy link
Copy Markdown

@realkotob realkotob commented Apr 9, 2026

The problem

Running code-review-graph build on a large repo (~9k files) after an earlier interrupted build crashes almost immediately with:

sqlite3.OperationalError: cannot start a transaction within a transaction

I hit this on a big source tree on my Macbook M1 Pro. The first build ran through the parse phase to completion, then I Ctrl-C'd it while community detection was stuck (separate perf issue — the file-based fallback is quadratic on large graphs). On the retry, the rebuild blew up on the very first file:

INFO: Progress: ... files parsed ... (never gets here)
Traceback (most recent call last):
  ...
  File ".../incremental.py", line 397, in full_build
      store.store_file_nodes_edges(...)
  File ".../graph.py", line 239, in store_file_nodes_edges
      self._conn.execute("BEGIN IMMEDIATE")
sqlite3.OperationalError: cannot start a transaction within a transaction

Why it happens

full_build starts by purging any files that used to exist in the DB but no longer exist on disk:

for stale in existing_files - current_abs:
    store.remove_file_data(stale)

remove_file_data issues two DELETE statements and returns without committing. Under Python's default (legacy) sqlite3 isolation mode, those DELETEs silently open an implicit transaction. The very next store_file_nodes_edges() call then runs an explicit BEGIN IMMEDIATE — and SQLite rejects it because one is already open.

The reason this went undetected is that the purge loop is usually a no-op: if every file still exists on disk, existing_files - current_abs is empty and the buggy path never runs. My interrupted build had left the DB with thousands of file rows whose absolute paths no longer matched the set the rebuild was computing, so every single one was treated as stale and the connection was left dirty before parsing even started.

The fix

Two parts:

  1. incremental.pyfull_build now calls store.commit() once after draining the purge loop, so the parse phase starts with a clean connection.
  2. graph.pystore_file_nodes_edges checks conn.in_transaction and flushes any dangling implicit transaction before its BEGIN IMMEDIATE. Defensive: if any future code path leaves DML uncommitted, the atomic per-file replace still works instead of crashing the build.

Test plan

  • New regression test test_store_file_nodes_edges_after_dangling_dml primes an implicit transaction via remove_file_data and asserts store_file_nodes_edges still succeeds. Fails on main, passes with the fix.
  • uv run pytest tests/test_graph.py tests/test_incremental.py — 45 passed
  • uv run pytest tests/test_tools.py tests/test_integration_v2.py — 66 passed
  • Manually reproduced the crash on a real repo and confirmed the fix allows the rebuild to proceed past the purge loop.

…ction"

Running `code-review-graph build` on a large repo (~9k files) after an
earlier interrupted build would crash almost immediately with:

    sqlite3.OperationalError: cannot start a transaction within a
    transaction

I hit this on the Godot source tree. The first build ran to completion
on the parse phase, then I Ctrl-C'd it while community detection was
stuck (separate perf issue). On the retry, the rebuild blew up on the
very first file.

Why it happens:

`full_build` starts by purging any files that used to exist in the DB
but no longer exist on disk. The purge loop calls
`store.remove_file_data()` for each stale path, which issues two
`DELETE` statements and returns without committing. Under Python's
default (legacy) sqlite3 isolation mode, those DELETEs silently open an
implicit transaction. The next `store_file_nodes_edges()` call then runs
an explicit `BEGIN IMMEDIATE` — and SQLite rejects it because one is
already open.

The reason nobody had tripped this before is that the purge loop is
usually a no-op: if every file still exists, `existing_files -
current_abs` is empty. But my interrupted build had left the DB with
thousands of file rows whose absolute paths no longer matched the set
the rebuild was computing, so every single one hit the purge path and
the connection was left dirty before parsing even started.

Fix is two parts:

  1. `full_build` now calls `store.commit()` once after the purge loop,
     so the parse phase starts with a clean connection.
  2. `store_file_nodes_edges` checks `conn.in_transaction` and flushes
     any dangling implicit transaction before its `BEGIN IMMEDIATE`.
     Defensive: if any future code path leaves DML uncommitted, the
     atomic per-file replace still works instead of crashing the build.

Adds a regression test that primes an implicit transaction via
`remove_file_data` and then asserts `store_file_nodes_edges` still
succeeds.
ghiemer added a commit to ghiemer/code-review-graph that referenced this pull request Apr 10, 2026
…205#135, tirth8205#181)

Python's sqlite3 with default isolation_level="" silently opens a
DEFERRED transaction on the first DML statement. When remove_file_data()
is called for stale/deleted files, the DELETE statements open an implicit
transaction that is never committed. The subsequent
store_file_nodes_edges() then executes BEGIN IMMEDIATE inside that
already-open transaction, causing:

  sqlite3.OperationalError: cannot start a transaction within a transaction

This affects both full_build (stale-file purge loop) and
incremental_update (deleted-file handling).

Fix:
- graph.py: Add in_transaction guard before BEGIN IMMEDIATE in
  store_file_nodes_edges() to flush any pending implicit transaction
- graph.py: Add public rollback() method to GraphStore
- incremental.py: Commit after stale-file purge loop in full_build()
- incremental.py: Commit after deleted-file removal in incremental_update()

Tests:
- test_store_after_remove_no_transaction_error (single remove → store)
- test_store_after_multiple_removes_no_transaction_error (bulk removes → store)

Closes tirth8205#135, tirth8205#181, relates to tirth8205#162, tirth8205#193, tirth8205#94

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant