fix(driver-sqlite-wasm)!: a file wasm SQLite cannot read is refused at connect, naming the file, instead of being renamed and replaced with an empty database

.changeset/25019-wasm-unreadable-file-refused.md

@@ -0,0 +1,18 @@
+---
+'@objectstack/driver-sqlite-wasm': patch
+'@objectstack/service-datasource': patch
+---
+
+fix(driver-sqlite-wasm)!: a file wasm SQLite cannot read is refused at connect, naming the file, and left as it was found, instead of being renamed and replaced with an empty database (#25019)
+
+Clause-②: no (narrowing)
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) No metadata moves: no spec key, authorable spelling, object definition or stored shape is added, removed, renamed or re-shaped, and no exported declaration is removed or narrowed (`WasmSqliteConnection` and `SqliteWasmDriver` keep their shapes; the private quarantine method is gone), so there is nothing for `objectstack migrate meta` to rewrite. What changes is the answer wasm SQLite's connect gives for a database file it cannot read: a rejection naming the file where it renamed the file and resolved over an empty database. The remedy is the datasource's file, never a rewrite of a package source or a stored row. -->
+
+**BREAKING** (wasm SQLite's connect accept set narrows). `WasmSqliteConnection.open()` used to answer a file sql.js could not read by renaming it to `<file>.corrupt-<timestamp>` and starting an empty database in its place, with one warning. That self-heal was written for wasm SQLite's own torn writes, which the atomic flush has since closed; what still reached it was an operator's file on a declared datasource. A datasource declared `driver: 'sqlite-wasm'` over such a file read `available` over the empty replacement, in development and production alike, and a declared `sqlite` datasource on a development boot whose native addon cannot load stepped down to wasm SQLite and met the same rename. The native driver already refuses the same files at connect.
+
+- **FROM:** a file sql.js cannot read (plain text, another format, a database truncated inside its header or short of the pages its schema names) was moved to `<file>.corrupt-<timestamp>`; `connect()` resolved over an empty database, and the datasource read `available`. On a development boot whose native addon cannot load, `resolveSqliteDriver` (`@objectstack/service-datasource`) returned that wasm driver, and when wasm SQLite failed for any reason it went on to the in-memory driver.
+- **TO:** `SqliteWasmDriver.connect()` rejects with the native driver's `SqliteDatabaseFileError`: `code: 'SQLITE_NOTADB'` for bytes that are not a database, `code: 'SQLITE_CORRUPT'` for a database it cannot read, `filename`, and a message naming the file and how to move it aside. Nothing is renamed, nothing is written beside it, and a stale write-ahead log beside a refused file stays where it is. `WasmSqliteConnection.open()` rejects with the same code and `filename`. `resolveSqliteDriver` re-throws that refusal at the wasm rung instead of trying the in-memory driver, so the datasource's connect fails and its connection state says so. Any other wasm failure, sql.js itself not loading first among them, still goes on to the in-memory driver.
+- **Unchanged:** a real database opens and answers; an absent file is created; an empty file opens as an empty database; a native addon that cannot load over a real or absent file still steps down to wasm SQLite.
+- **The fix for an operator:** restore the database from a backup, or move the unreadable file aside yourself (`mv '<file>' '<file>.unreadable'`) and start again: an absent file is created empty on the next connect. A development server whose own wasm image is damaged no longer starts itself clean; this is that step, done by hand.

packages/drivers/driver-sqlite-wasm/src/sqlite-wasm-25019-unreadable-file-refused.test.ts

@@ -0,0 +1,164 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * A file wasm SQLite cannot read is refused at connect, naming the file, with
+ * the code the native driver answers for the same file, and is left exactly as
+ * it was found (#25019): nothing renamed, nothing written beside it, no empty
+ * database in its place.
+ *
+ * Before, `WasmSqliteConnection.open()` moved such a file to
+ * `<file>.corrupt-<timestamp>` and started an empty database, so a declared
+ * `sqlite-wasm` datasource over an operator's file read as connected, in
+ * development and production alike.
+ *
+ * Real sql.js and real files in a scratch directory.
+ */
+
+import { afterEach, beforeEach, describe, expect, it } from 'vitest';
+import { mkdtempSync, readdirSync, readFileSync, rmSync, truncateSync, writeFileSync, statSync, existsSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+import { SqliteWasmDriver, WasmSqliteConnection } from './index.js';
+
+let dir: string;
+const drivers: SqliteWasmDriver[] = [];
+
+beforeEach(() => { dir = mkdtempSync(join(tmpdir(), 'wasm-25019-')); });
+
+afterEach(async () => {
+  await Promise.all(drivers.splice(0).map((d) => d.disconnect().catch(() => {})));
+  rmSync(dir, { recursive: true, force: true });
+});
+
+function driverFor(filename: string): SqliteWasmDriver {
+  const driver = new SqliteWasmDriver({ filename, persist: 'on-write', logger: { warn: () => {} } });
+  drivers.push(driver);
+  return driver;
+}
+
+/** A real database written by this driver, holding one row in `t`. Its size in bytes. */
+async function seedDatabase(filename: string): Promise<number> {
+  const seed = new SqliteWasmDriver({ filename, persist: 'on-disconnect' });
+  await seed.connect();
+  await seed.execute('CREATE TABLE t (a INTEGER)');
+  await seed.execute('INSERT INTO t (a) VALUES (7)');
+  await seed.disconnect();
+  return statSync(filename).size;
+}
+
+/** `connect()`'s rejection; a failure when it resolves instead. */
+async function connectRefusal(driver: SqliteWasmDriver): Promise<any> {
+  try {
+    await driver.connect();
+  } catch (error) {
+    return error;
+  }
+  throw new Error('connect() resolved over a file wasm SQLite cannot read');
+}
+
+/** The driver's refusal: the native driver's class and code, the file named, and how to move it aside. */
+function expectFileRefusal(error: any, filename: string, code: 'SQLITE_NOTADB' | 'SQLITE_CORRUPT'): void {
+  expect(error?.name).toBe('SqliteDatabaseFileError');
+  expect(error?.code).toBe(code);
+  expect(error?.filename).toBe(filename);
+  expect(String(error?.message)).toContain(`SQLite database file '${filename}'`);
+  expect(String(error?.message)).toContain(
+    `To start from an empty database, move the file aside yourself (mv '${filename}' '${filename}.unreadable') and start again; to keep its data, restore a copy that opens`,
+  );
+}
+
+describe('a file wasm SQLite cannot read is refused at connect and left as it was found (#25019)', () => {
+  it('plain text: SQLITE_NOTADB, naming the file', async () => {
+    const filename = join(dir, 'archive.db');
+    const bytes = Buffer.from('id,name\n1,not a database\n'.repeat(20));
+    writeFileSync(filename, bytes);
+
+    const refusal = await connectRefusal(driverFor(filename));
+    expectFileRefusal(refusal, filename, 'SQLITE_NOTADB');
+    expect(String(refusal.message)).toContain('was left as it is');
+
+    expect(readdirSync(dir)).toEqual(['archive.db']);
+    expect(readFileSync(filename).equals(bytes)).toBe(true);
+  });
+
+  it('a database truncated inside its 100-byte header: SQLITE_CORRUPT, naming the file', async () => {
+    const filename = join(dir, 'archive.db');
+    await seedDatabase(filename);
+    truncateSync(filename, 50);
+    const bytes = readFileSync(filename);
+
+    expectFileRefusal(await connectRefusal(driverFor(filename)), filename, 'SQLITE_CORRUPT');
+
+    expect(readdirSync(dir)).toEqual(['archive.db']);
+    expect(readFileSync(filename).equals(bytes)).toBe(true);
+  });
+
+  it('a database short of the pages its schema names: SQLITE_CORRUPT, naming the file', async () => {
+    const filename = join(dir, 'archive.db');
+    const size = await seedDatabase(filename);
+    truncateSync(filename, Math.floor(size / 2));
+    const bytes = readFileSync(filename);
+
+    expectFileRefusal(await connectRefusal(driverFor(filename)), filename, 'SQLITE_CORRUPT');
+
+    expect(readdirSync(dir)).toEqual(['archive.db']);
+    expect(readFileSync(filename).equals(bytes)).toBe(true);
+  });
+
+  it('a refused file keeps the write-ahead log beside it: nothing is parked', async () => {
+    const filename = join(dir, 'archive.db');
+    writeFileSync(filename, 'not a database');
+    const wal = Buffer.from('a stale write-ahead log');
+    writeFileSync(`${filename}-wal`, wal);
+
+    expectFileRefusal(await connectRefusal(driverFor(filename)), filename, 'SQLITE_NOTADB');
+
+    expect(readdirSync(dir).sort()).toEqual(['archive.db', 'archive.db-wal']);
+    expect(readFileSync(`${filename}-wal`).equals(wal)).toBe(true);
+  });
+
+  it('WasmSqliteConnection.open() itself rejects with the code and the file, and writes nothing', async () => {
+    const filename = join(dir, 'archive.db');
+    writeFileSync(filename, 'not a database');
+
+    const conn = new WasmSqliteConnection({ filename, logger: { warn: () => {} } });
+    let refusal: any;
+    await conn.open().catch((e: unknown) => { refusal = e; });
+    expect(refusal?.code).toBe('SQLITE_NOTADB');
+    expect(refusal?.filename).toBe(filename);
+    expect(String(refusal?.message)).toContain(`mv '${filename}' '${filename}.unreadable'`);
+    expect(String((refusal?.cause as Error)?.message)).toBe('file is not a database');
+
+    expect(readdirSync(dir)).toEqual(['archive.db']);
+    expect(readFileSync(filename, 'utf8')).toBe('not a database');
+  });
+});
+
+describe('CONTROLS: what keeps today\'s behaviour (#25019)', () => {
+  it('a real database opens and answers', async () => {
+    const filename = join(dir, 'archive.db');
+    await seedDatabase(filename);
+    const driver = driverFor(filename);
+    await driver.connect();
+    expect(await driver.execute('SELECT a FROM t')).toEqual([{ a: 7 }]);
+  });
+
+  it('an absent file is created', async () => {
+    const filename = join(dir, 'new', 'archive.db');
+    const driver = driverFor(filename);
+    await driver.connect();
+    await driver.execute('CREATE TABLE t (a INTEGER)');
+    await driver.flush();
+    expect(existsSync(filename)).toBe(true);
+  });
+
+  it('an empty file opens as an empty database', async () => {
+    const filename = join(dir, 'archive.db');
+    writeFileSync(filename, '');
+    const driver = driverFor(filename);
+    await driver.connect();
+    expect(await driver.execute('SELECT 1 AS one')).toEqual([{ one: 1 }]);
+    expect(readdirSync(dir).filter((name) => name.includes('.corrupt-'))).toEqual([]);
+  });
+});

packages/drivers/driver-sqlite-wasm/src/sqlite-wasm-driver-durability.test.ts

@@ -21,8 +21,10 @@ import { SqliteWasmDriver } from '../src/index.js';
  *
  *  1. Flushes must be ATOMIC. A non-atomic overwrite torn by a mid-write kill
  *     left "database disk image is malformed" on the next boot.
- *  2. An already-corrupt on-disk image must SELF-HEAL: quarantine the bad file
- *     and boot on a fresh database instead of failing every query forever.
+ *  2. An already-corrupt on-disk image is REFUSED at connect, naming the file,
+ *     and left exactly as it was found (#25019). It used to self-heal: the open
+ *     renamed it to `<file>.corrupt-<timestamp>` and booted on an empty
+ *     database, which read as connected over an operator's file it replaced.
  *  3. `undefined` bindings must persist as SQL NULL rather than throwing sql.js's
  *     raw-string "Wrong API use : tried to bind a value of an unknown type".
  */
@@ -70,11 +72,11 @@ describe('SqliteWasmDriver durability & resilience', () => {
     expect(rows[0].name).toBe('A');
   });
 
-  it('self-heals a corrupt database image by quarantining it and booting fresh', async () => {
+  it('refuses a corrupt database image at connect, naming the file, and leaves it as it was found', async () => {
     const dir = newDir();
     const file = join(dir, 'corrupt.db');
     // A file that starts with the SQLite magic header but is otherwise garbage:
-    // sql.js opens it, then blows up on the first page read ("malformed").
+    // sql.js opens it, then fails the first page read.
     const junk = Buffer.concat([
       Buffer.from('SQLite format 3\0', 'binary'),
       Buffer.alloc(4096, 0xab),
@@ -90,18 +92,21 @@ describe('SqliteWasmDriver durability & resilience', () => {
       }),
     );
 
-    // Boot must succeed and queries must work on the fresh database.
-    await driver.initObjects([{ name: 'acct', fields: { name: { type: 'string' } } }]);
-    await (driver as any).knex('acct').insert({ id: 'a1', name: 'healed' });
-    const rows = await (driver as any).knex('acct');
-    expect(rows.length).toBe(1);
+    let refusal: any;
+    await driver.connect().catch((e: unknown) => { refusal = e; });
+    expect(refusal?.name).toBe('SqliteDatabaseFileError');
+    expect(refusal?.code).toBe('SQLITE_NOTADB');
+    expect(refusal?.filename).toBe(file);
+    expect(String(refusal?.message)).toContain(`'${file}'`);
+    // The prescription: the operator moves the file aside, since the open no longer does.
+    expect(String(refusal?.message)).toContain(
+      `move the file aside yourself (mv '${file}' '${file}.unreadable')`,
+    );
 
-    // The corrupt bytes were preserved in a sibling quarantine file, and a
-    // warning was surfaced.
-    const quarantined = readdirSync(dir).filter((f) => f.includes('.corrupt-'));
-    expect(quarantined.length).toBe(1);
-    expect(readFileSync(join(dir, quarantined[0]))).toEqual(junk);
-    expect(warnings.some((w) => w.includes('corrupt'))).toBe(true);
+    // Nothing renamed, nothing written beside it, the bytes untouched.
+    expect(readdirSync(dir)).toEqual(['corrupt.db']);
+    expect(readFileSync(file)).toEqual(junk);
+    expect(warnings.some((w) => w.includes('Quarantined'))).toBe(false);
   });
 
   it('persists an undefined binding as SQL NULL instead of throwing', async () => {

packages/drivers/driver-sqlite-wasm/src/wasm-connection.ts

@@ -86,6 +86,55 @@ async function loadSqlJs(
   return cachedSqlJs;
 }
 
+/**
+ * SQLite's own text for `SQLITE_NOTADB` (`sqlite3_errstr(26)`). sql.js throws
+ * a bare `Error` carrying SQLite's message and no result code, so this text is
+ * the only form the code reaches this file in.
+ */
+const SQLITE_NOTADB_ERRSTR = 'file is not a database';
+
+/**
+ * The refusal {@link WasmSqliteConnection.open} throws for a file sql.js cannot
+ * read (#25019), with the SQLite code the native driver answers for the same
+ * file: `SQLITE_NOTADB` when SQLite says the bytes are not a database, and
+ * `SQLITE_CORRUPT` for every other reason a read of the image failed ("database
+ * disk image is malformed", a `quick_check` that does not answer `ok`).
+ *
+ * The code is what makes it one refusal with the native driver's:
+ * `SqlDriver.connect()` names the file with its own words for that code, and
+ * `resolveSqliteDriver` (`@objectstack/service-datasource`) re-throws it rather
+ * than step down. The message is the rest of the answer, and the one thing only
+ * this engine can say: it no longer moves the file aside for the operator, so
+ * it says how to do that.
+ */
+function unreadableDatabaseFileRefusal(
+  filename: string,
+  cause: unknown,
+): UnreadableDatabaseFileRefusal {
+  const reason =
+    typeof cause === 'string' ? cause : (cause as Error | undefined)?.message ?? String(cause);
+  // `cause` is attached by hand: the `ErrorOptions` overload is ES2022 and this
+  // package compiles against ES2020's lib.
+  const refusal = new Error(
+    `wasm SQLite cannot read it (${reason}). To start from an empty database, move the ` +
+      `file aside yourself (mv '${filename}' '${filename}.unreadable') and start again; ` +
+      `to keep its data, restore a copy that opens`,
+  ) as UnreadableDatabaseFileRefusal;
+  refusal.code = reason === SQLITE_NOTADB_ERRSTR ? 'SQLITE_NOTADB' : 'SQLITE_CORRUPT';
+  refusal.filename = filename;
+  refusal.cause = cause;
+  return refusal;
+}
+
+/** SQLite's own result-code names for a file it cannot read, as the native driver carries them. */
+type UnreadableDatabaseFileCode = 'SQLITE_NOTADB' | 'SQLITE_CORRUPT';
+
+type UnreadableDatabaseFileRefusal = Error & {
+  code: UnreadableDatabaseFileCode;
+  filename: string;
+  cause: unknown;
+};
+
 /**
  * A sql.js-backed connection that exposes the `prepare`/`exec`/`close`
  * subset used by Knex's SQLite dialect. Mutations are queued through a
@@ -198,24 +247,37 @@ export class WasmSqliteConnection {
       return;
     }
 
-    await this.quarantineOrphanedWal();
-
-    // Open the on-disk bytes, but guard against a corrupt image. A torn write
-    // (process killed mid-flush before atomic writes existed) or otherwise
-    // damaged file makes `new SQL.Database(bytes)` either throw ("file is not a
-    // database") or open a handle whose every query fails with "database disk
-    // image is malformed" — which, for a background dispatcher on a tick loop,
-    // means the same error spammed forever with no path to recovery. Detect it
-    // once at open, quarantine the bad file, and start fresh so the dev server
-    // becomes usable again instead of wedging.
+    // Open the on-disk bytes, and refuse a file sql.js cannot read (#25019).
+    // A file that is not a database makes the first read throw "file is not a
+    // database"; a damaged one throws "database disk image is malformed" or
+    // fails `quick_check`. Either way every later query would answer the same,
+    // so the open fails here, naming the file, with the code the native driver
+    // answers for the same file (see `unreadableDatabaseFileRefusal`).
+    //
+    // The file is left exactly as it was found. This open used to move it to
+    // `<file>.corrupt-<timestamp>` and start an empty database in its place: a
+    // self-heal written for torn wasm images, which `atomicWriteFile` has since
+    // closed. What still reached it was an operator's file on a declared
+    // datasource — an explicit `sqlite-wasm` one in any mode, and the
+    // development step-down from a native addon that cannot load — which then
+    // read as connected over an empty database.
+    let candidate: Database | undefined;
     try {
-      const candidate = new SQL.Database(bytes);
+      candidate = new SQL.Database(bytes);
       this.assertReadable(candidate);
-      this.db = candidate;
     } catch (err) {
-      await this.quarantineCorruptFile(err);
-      this.db = new SQL.Database();
+      try {
+        candidate?.close();
+      } catch {
+        /* the refusal below is the answer */
+      }
+      throw unreadableDatabaseFileRefusal(this.filename, err);
     }
+
+    // Only a file this open accepts gets its stale log parked: a refused file
+    // keeps everything beside it as well.
+    await this.quarantineOrphanedWal();
+    this.db = candidate;
   }
 
   /**
@@ -233,35 +295,6 @@ export class WasmSqliteConnection {
     }
   }
 
-  /**
-   * Move a corrupt database file aside so its bytes are preserved for
-   * post-mortem while a fresh, empty database takes its place. Best-effort:
-   * failures here must not prevent the server from booting on a clean DB.
-   */
-  private async quarantineCorruptFile(cause: unknown): Promise<void> {
-    if (!this.fs) return;
-    const reason =
-      typeof cause === 'string' ? cause : (cause as Error)?.message ?? String(cause);
-    const backup = `${this.filename}.corrupt-${Date.now()}`;
-    try {
-      await this.fs.rename(this.filename, backup);
-      this.logger.warn(
-        `[driver-sqlite-wasm] Database image at ${this.filename} is corrupt ` +
-          `(${reason}). Quarantined to ${backup} and starting from an empty ` +
-          `database so the server can boot.`,
-      );
-    } catch (renameErr) {
-      // Could not move it aside (e.g. permissions) — overwrite is still better
-      // than looping forever on a malformed image. Warn loudly and continue.
-      this.logger.warn(
-        `[driver-sqlite-wasm] Database image at ${this.filename} is corrupt ` +
-          `(${reason}) and could not be quarantined (${String(renameErr)}). ` +
-          `Starting from an empty database; the corrupt file will be overwritten ` +
-          `on the next flush.`,
-      );
-    }
-  }
-
   /**
    * Move a write-ahead log left behind by a *real* SQLite aside (#3941).
    *