codegraph status repairs indexes during a live rebuild, causing SQLITE_BUSY_SNAPSHOT (517) #2913

Summary

In CodeGraph 1.6.2, running codegraph status during codegraph index can make reference resolution fail with:

Failed to index: database is locked

The status command opens the database read-write and runs open-time crash recovery. During an active bulk-load window, that recovery recreates secondary indexes that the indexer deliberately dropped. A resolver transaction that has already read its node endpoints can then fail when it attempts to write edges, with SQLite extended error 517 (SQLITE_BUSY_SNAPSHOT).

The status CLI also proceeds while another live process owns both rebuild.pid and writer.pid.

Environment and observed failure

• @colbymchenry/codegraph 1.6.2, installed through npm, darwin-arm64 platform bundle.
• macOS on Apple Silicon; built-in node:sqlite backend, WAL journal mode.
• The application repository uses a Nix development shell. CodeGraph startup worked there, and the isolated reproduction below uses CodeGraph's bundled Node directly without Nix.
• A real rebuild scanned 5,380 files, finished parsing, then failed at approximately 8% of reference resolution while a separate diagnostic codegraph status check overlapped it.
• A later rebuild completed.

The overlapping status check is a plausible trigger for the real failure, rather than proof of its exact scheduling. The synthetic reproduction below deterministically demonstrates the defect using the installed CLI and database/resolver code.

Relevant code paths in v1.6.2

1. The status command calls CodeGraph.open(projectPath) without readOnly: true and without checking the rebuild guard.
2. CodeGraph.open passes the unset read-only option into DatabaseConnection.open.
3. DatabaseConnection.open runs migrations, healBulkNodeLoad(), healBulkSecondaryIndexes(), and WAL maintenance unless opened read-only.
4. beginBulkEdgeLoad() intentionally drops five secondary edge indexes; healBulkSecondaryIndexes() recreates them on the status connection.
5. QueryBuilder.insertEdges reads existing endpoint IDs before inserting rows inside a transaction. The SQLite adapter begins that transaction with deferred BEGIN. A concurrent schema write after that read makes the subsequent write fail with SQLITE_BUSY_SNAPSHOT.

The database already has WAL and busy_timeout = 5000. Increasing the timeout does not make an invalidated snapshot writable.

Deterministic reproduction using the installed CLI

This creates and deletes only a disposable synthetic database. It does not index a source repository or stop any daemon. The hook around getExistingNodeIds controls scheduling: after the real endpoint read and before the real edge insert, it invokes the actual codegraph status CLI in a separate process. The installed implementation is not modified.

Save as repro.cjs:

const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { spawnSync } = require('node:child_process');

const bundle = process.env.CODEGRAPH_BUNDLE;
if (!bundle) throw new Error('Set CODEGRAPH_BUNDLE to the platform bundle directory');

const { DatabaseConnection } = require(path.join(bundle, 'lib/dist/db/index.js'));
const { QueryBuilder } = require(path.join(bundle, 'lib/dist/db/queries.js'));

const root = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-status-repro-'));
let writer;

try {
  const dataDir = path.join(root, '.codegraph');
  fs.mkdirSync(dataDir);
  writer = DatabaseConnection.initialize(path.join(dataDir, 'codegraph.db'));
  const db = writer.getDb();

  for (const id of ['a', 'b']) {
    db.prepare(
      'INSERT INTO nodes ' +
      '(id,kind,name,qualified_name,file_path,language,' +
      'start_line,end_line,start_column,end_column,updated_at) ' +
      'VALUES (?,?,?,?,?,?,?,?,?,?,?)'
    ).run(id, 'function', id, id, 'fixture.ts', 'typescript',
          1, 1, 0, 1, Date.now());
  }

  // The observer is a separate process; both locks name this live writer.
  const lock = JSON.stringify({
    pid: process.pid, mode: 'rebuild', startedAt: Date.now(), ready: false
  });
  fs.writeFileSync(path.join(dataDir, 'rebuild.pid'), lock);
  fs.writeFileSync(path.join(dataDir, 'writer.pid'), lock);

  writer.beginBulkEdgeLoad();
  const queries = new QueryBuilder(db);
  const getIds = queries.getExistingNodeIds.bind(queries);

  queries.getExistingNodeIds = (...args) => {
    const ids = getIds(...args);
    const observer = spawnSync(
      path.join(bundle, 'node'),
      [
        '--disable-warning=ExperimentalWarning',
        path.join(bundle, 'lib/dist/bin/codegraph.js'),
        'status', root, '--json'
      ],
      { encoding: 'utf8', timeout: 15000, cwd: root }
    );
    console.log(JSON.stringify({
      statusExit: observer.status,
      returnedStatistics: observer.stdout.includes('nodeCount'),
      stderr: observer.stderr.trim()
    }));
    return ids;
  };

  try {
    queries.insertEdges([{ source: 'a', target: 'b', kind: 'calls' }]);
    console.log('Writer: PASS');
  } catch (error) {
    console.log(JSON.stringify({
      writer: error.message,
      sqliteExtendedCode: error.errcode
    }));
  }
} finally {
  if (writer) writer.close();
  fs.rmSync(root, { recursive: true, force: true });
}

For the npm-installed macOS arm64 bundle:

bundle="$(npm root -g)/@colbymchenry/codegraph/node_modules/@colbymchenry/codegraph-darwin-arm64"
CODEGRAPH_BUNDLE="$bundle" "$bundle/node" --disable-warning=ExperimentalWarning repro.cjs

Observed output with the same interleaving:

{"statusExit":0,"returnedStatistics":true,"stderr":""}
{"writer":"database is locked","sqliteExtendedCode":517}

A separate paired test replaced the CLI observer with a second DatabaseConnection.open(dbPath, { readOnly }) at the same point:

Expected behavior / suggested direction

• Status must not mutate the index, repair schemas, migrate, or launch write maintenance.
• Opening status with CodeGraph.open(projectPath, { readOnly: true }) is the demonstrated direction for preventing this particular interference.
• Read CLI commands should use a consistent read-only policy and respect the rebuild lifecycle; if a full rebuild makes a coherent read unavailable, return rebuild-in-progress guidance instead of repairing the live database.
• Add a regression test with an active bulk-load window and a status subprocess. Verify that status leaves intentionally dropped indexes alone and the writer transaction succeeds.
• Preserve crash repair for legitimate writer-owned recovery after an interrupted run.

Related reports checked

• SQLite "database is locked" error on concurrent MCP tool calls #238 concerns concurrent MCP queries and older WAL/busy-timeout handling.
• Crash recovery does not restore secondary indexes dropped by bulk parse #1556 concerns restoring indexes after an actual interrupted bulk load.
• Interrupted bulk-load recovery can be repeatedly watchdog-killed while rebuilding indexes #1887 concerns watchdog termination during recovery.
• codegraph index fails with "database file is in use" when MCP server is still running #1325 concerns a still-running MCP server preventing database recreation.
• PR feat(db): read-only consumption of prebuilt indices via immutable fallback #1863 concerns consuming non-writable prebuilt artifacts; this reproduction uses a writable database.

I did not find an existing issue for status-triggered repair during a live rebuild. An earlier daemon-stop verification message was also encountered during investigation, but its cause was not confirmed and is not claimed to be explained by this reproduction.

AI-assisted diagnosis and reproduction; no application source or private repository data is included.