test(driver-mongodb): gate the mongod-backed suites behind an opt-in env var (#5517) (#5538)

The seven suites that need a real MongoDB now skip by default and print why.
`mongodb-memory-server` downloads a ~123 MB binary on first use, and on a cold
cache two vitest workers raced that download: the winner renamed
`<archive>.tgz.downloading`, the loser's `rename` threw ENOENT inside an
unawaited `async` listener (`MongoBinaryDownload.js:413`), so the enclosing
promise never settled AND the ENOENT surfaced as a process-level unhandled
rejection. That is one event, not the two the issue describes: the never-settling
promise is the "timed out after 120s" skip, and the unhandled rejection is what
turned all-green runs into `exit 1` and ejected unrelated PRs from the merge
queue.

Per the maintainer's decision, the download is retired rather than made
single-flight (#5499 freezes investment in this family): no globalSetup
pre-download, no cross-worker lock, no workflow cache warming.

- `OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1` opts in (`OS_TEST_*` + `_ENABLED`,
  the `OS_TEST_MULTI_ORG_ENABLED` shape, Prime Directive #9).
- The gate is checked BEFORE `mongodb-memory-server` is imported at all, so a
  default run starts zero downloads; the new gate suite pins that by counting
  library evaluations.
- Notices go to `process.stderr`, not `console.warn`: measured on vitest 4.1.10,
  the default reporter renders nothing a passing/skipped file logs through
  `console`, which would have made the retirement a silent skip.
- `installAbandonedDownloadGuard` swallows the abandoned-download ENOENT — and
  only it — on the opt-in path, re-raising everything else because vitest steps
  aside once another `unhandledRejection` listener exists.
- `mongodb-findone-query` / `mongodb-pagination-conformance` called
  `MongoMemoryServer.create()` directly (bypassing the gate and the deadline);
  both now go through `createTestMongod`.
- Coverage consequence recorded in `scripts/check-driver-conformance.mjs`: the
  mongo cells stay CONSUMED on their imports, but the temporal and pagination
  case-sets now run only opt-in.


Claude-Session: https://claude.ai/code/session_01BWS4heBoAitLmzCLhcYdbK

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Author: os-zhuang

packages/plugins/driver-mongodb/README.md

@@ -199,13 +199,32 @@ kernel.use(mongodbPlugin, {
 ## Development
 
 ```bash
-# Run tests
+# Run tests (the suites that need a real mongod SKIP — see below)
 pnpm test
 
+# Run every suite, including the ones that need a real mongod
+OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1 pnpm test
+
 # Build
 pnpm build
 ```
 
+### The mongod-backed suites are opt-in (#5517)
+
+Seven suites here need a real MongoDB, which `mongodb-memory-server` provides by
+downloading a ~123 MB binary on first use. With a cold cache, two vitest workers
+downloaded it at the same time and the loser's `rename` failed as an unhandled
+rejection — turning an all-green run into `exit 1` and ejecting unrelated PRs
+from the merge queue. Those suites are therefore gated behind
+`OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1`: without it they skip, each printing
+one line that names this issue and the switch, and **no download starts**.
+
+The rest of the package's tests — filter translation, the shared filter-logic
+conformance case-set over the emitted documents, sort specs, tenancy guard,
+temporal helpers — run by default and need no binary. The gate lives in
+`src/test-mongod.ts`, which documents the mechanism and what a default run gives
+up.
+
 ## License
 
 Apache-2.0. See [LICENSING.md](../../../LICENSING.md).

packages/plugins/driver-mongodb/src/mongodb-driver.test.ts

@@ -7,13 +7,14 @@ import { MongoDBDriver } from './mongodb-driver.js';
 import { createTestMongod } from './test-mongod.js';
 
 // `mongodb-memory-server` downloads a real MongoDB binary from
-// fastdl.mongodb.org on first use. In sandboxed / offline CI that download can
-// fail — or HANG, which this suite could not express until `createTestMongod`
-// put a deadline on the wait (see that module); both now land on the same skip.
-// When it does we **skip** this suite rather than failing the whole package's
-// test run (and, with it, the monorepo `Test Core` job). The startup is
-// attempted once here so availability is known at collection time — a throwing
-// `beforeAll` would *fail* every test instead of skipping it.
+// fastdl.mongodb.org on first use, which is why this suite is OPT-IN since
+// #5517: two workers downloading it at once made an all-green run `exit 1` and
+// ejected unrelated PRs from the merge queue. `createTestMongod` skips this
+// suite — printing why — unless `OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1`, and
+// an opted-in download that fails or HANGS lands on the same skip rather than
+// stalling the job. The acquisition happens once here so availability is known
+// at collection time — a throwing `beforeAll` would *fail* every test instead
+// of skipping it.
 const sharedMongod: MongoMemoryServer | undefined = await createTestMongod('MongoDBDriver');
 
 describe.skipIf(!sharedMongod)('MongoDBDriver', () => {

packages/plugins/driver-mongodb/src/mongodb-filter-logic-conformance.test.ts

@@ -16,11 +16,12 @@
  *
  * The same table is driven server-free by
  * `mongodb-filter-logic-translation.test.ts`, which is the half that always
- * runs. This one skips when the mongod binary cannot be fetched (the
- * `createTestMongod` convention every suite in this package uses — a blocked or
- * hanging download costs a skipped suite, not a stalled test job). **A skip is
- * not a pass**: on a machine without the binary, the translation suite is the
- * whole proof, which is exactly why it carries the priority half.
+ * runs. This one is OPT-IN since #5517: it needs the mongod binary, whose
+ * concurrent download turned green runs into `exit 1`, so `createTestMongod`
+ * skips it unless `OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1` (and still skips,
+ * rather than stalling, when an opted-in download is blocked or hanging).
+ * **A skip is not a pass**: by default the translation suite is the whole proof,
+ * which is exactly why it carries the priority half.
  */
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';

packages/plugins/driver-mongodb/src/mongodb-findone-query.test.ts

@@ -24,24 +24,22 @@
  * because an untouched collection comes back in insertion order whether or not
  * a sort went out.
  *
- * Skips itself when the mongod binary cannot be fetched — the convention the
- * other suites in this package already use. A skip is not a pass.
+ * Needs a real mongod, so it is OPT-IN since #5517 — `createTestMongod` prints
+ * why and this suite skips unless `OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1`. It
+ * used to call `MongoMemoryServer.create()` itself, which both started a
+ * download in every default run and bypassed the package's shared deadline. A
+ * skip is not a pass.
  */
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
-import { MongoMemoryServer } from 'mongodb-memory-server';
+import type { MongoMemoryServer } from 'mongodb-memory-server';
 import { MongoDBDriver } from './mongodb-driver.js';
 import { FINDONE_CASES, FINDONE_ROWS } from './mongodb-findone-cases.js';
+import { createTestMongod } from './test-mongod.js';
 
-let sharedMongod: MongoMemoryServer | undefined;
-try {
-  sharedMongod = await MongoMemoryServer.create({ instance: { launchTimeout: 60_000 } });
-} catch (err) {
-  console.warn(
-    '[driver-mongodb] Skipping findOne query-execution suite — mongodb-memory-server could not '
-      + `start: ${(err as Error)?.message ?? String(err)}`,
-  );
-}
+const sharedMongod: MongoMemoryServer | undefined = await createTestMongod(
+  'findOne query-execution',
+);
 
 describe.skipIf(!sharedMongod)('driver-mongodb — findOne executes the whole query', () => {
   const mongod = sharedMongod as MongoMemoryServer;

packages/plugins/driver-mongodb/src/mongodb-memory-server-gate.test.ts

@@ -0,0 +1,217 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#5517] The binary-dependent suites are OPT-IN, and the gate is checked before
+ * anything can start a download.
+ *
+ * This is the suite that survives the retirement, so it has to pin the two
+ * properties the retirement is worth anything for:
+ *
+ *   1. **Nothing is even imported when the gate is closed.** A `describe.skipIf`
+ *      alone would not have been enough: the download starts at MODULE LOAD, so
+ *      a gate evaluated after `import { MongoMemoryServer } from …` would skip
+ *      the tests and still fetch 123 MB. The mock factory below counts library
+ *      evaluations, so re-adding a static value import to `test-mongod.ts` turns
+ *      the first test red.
+ *   2. **The abandoned-download rejection is swallowed, and only it.** The
+ *      guard's listener suppresses vitest's own unhandled-rejection reporting for
+ *      the whole worker (vitest steps aside when a second listener exists), so a
+ *      guard that swallowed indiscriminately would hide real failures. The
+ *      re-raise path is asserted, not assumed.
+ *
+ * Test ORDER matters in the first three cases: case 1 asserts the library has
+ * never been evaluated, and case 3 is what evaluates it.
+ */
+
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+/** Counts evaluations of `mongodb-memory-server` and lets a case choose what `create()` does. */
+const lib = vi.hoisted(() => ({
+  imported: 0,
+  create: undefined as undefined | (() => Promise<unknown>),
+}));
+
+vi.mock('mongodb-memory-server', () => {
+  lib.imported++;
+  return {
+    MongoMemoryServer: {
+      create: () => {
+        if (!lib.create) throw new Error('no create() behaviour was configured by this test');
+        return lib.create();
+      },
+    },
+  };
+});
+
+import {
+  MONGOD_TESTS_ENV,
+  createTestMongod,
+  disposeAbandonedDownloadGuard,
+  installAbandonedDownloadGuard,
+  isAbandonedDownloadRejection,
+  mongodSkipReason,
+  mongodTestsEnabled,
+  printMongodNotice,
+} from './test-mongod.js';
+
+/** The exact shape `fs.promises.rename` throws at MongoBinaryDownload.js:413. */
+function renameEnoent(
+  path = '/home/runner/.cache/mongodb-binaries/mongodb-linux-x86_64-ubuntu2404-8.2.6.tgz.downloading',
+): Error {
+  return Object.assign(
+    new Error(`ENOENT: no such file or directory, rename '${path}' -> '${path.slice(0, -12)}'`),
+    { code: 'ENOENT', syscall: 'rename', errno: -2, path, dest: path.slice(0, -12) },
+  );
+}
+
+describe('[#5517] the mongod opt-in gate', () => {
+  let warnings: string[];
+
+  beforeEach(() => {
+    warnings = [];
+    // The notices go to stderr, NOT through `console` — vitest's default reporter
+    // renders nothing a passing/skipped file logs via `console`, which would make
+    // the skip silent in exactly the runs that matter. `printMongodNotice`
+    // documents the measurement.
+    vi.spyOn(process.stderr, 'write').mockImplementation((chunk: unknown) => {
+      warnings.push(String(chunk).trimEnd());
+      return true;
+    });
+    delete process.env[MONGOD_TESTS_ENV];
+    lib.create = undefined;
+  });
+
+  afterEach(() => {
+    vi.restoreAllMocks();
+    delete process.env[MONGOD_TESTS_ENV];
+    // Cases that open the gate install the guard on the REAL process. Leaving it
+    // behind would make vitest defer its own unhandled-rejection reporting for
+    // every later file in this worker.
+    disposeAbandonedDownloadGuard();
+  });
+
+  it('skips without touching the library at all when the switch is unset', async () => {
+    expect(mongodTestsEnabled()).toBe(false);
+
+    const mongod = await createTestMongod('gate probe');
+
+    expect(mongod).toBeUndefined();
+    expect(warnings).toHaveLength(1);
+    expect(warnings[0]).toContain('#5517');
+    expect(warnings[0]).toContain(MONGOD_TESTS_ENV);
+    expect(warnings[0]).toContain('SKIP gate probe');
+    // The property that makes the skip cost nothing: the module that would
+    // download the binary was never evaluated.
+    expect(lib.imported).toBe(0);
+  });
+
+  it('treats a set-but-not-"1" value as OFF and says so instead of skipping quietly', async () => {
+    process.env[MONGOD_TESTS_ENV] = 'true';
+
+    const mongod = await createTestMongod('gate probe');
+
+    expect(mongod).toBeUndefined();
+    expect(lib.imported).toBe(0);
+    expect(warnings[0]).toContain('is set to "true", which does NOT enable it');
+    expect(mongodSkipReason('x', { [MONGOD_TESTS_ENV]: '' })).not.toContain('does NOT enable it');
+  });
+
+  it('imports the library and returns a server once the switch is on', async () => {
+    process.env[MONGOD_TESTS_ENV] = '1';
+    const stub = { getUri: () => 'mongodb://stub', stop: async () => {} };
+    lib.create = async () => stub;
+
+    const mongod = await createTestMongod('gate probe');
+
+    expect(mongod).toBe(stub);
+    expect(lib.imported).toBe(1);
+    expect(warnings).toEqual([]);
+  });
+
+  it('still degrades to a named skip when an opted-in acquisition fails', async () => {
+    process.env[MONGOD_TESTS_ENV] = '1';
+    lib.create = async () => {
+      throw renameEnoent();
+    };
+
+    const mongod = await createTestMongod('gate probe');
+
+    expect(mongod).toBeUndefined();
+    expect(warnings[0]).toContain('SKIP gate probe');
+    expect(warnings[0]).toContain('could not start');
+  });
+
+  it('prints on stderr, not through the console vitest silences on a green run', () => {
+    const sink: string[] = [];
+    printMongodNotice('injected', (chunk) => sink.push(chunk));
+    expect(sink).toEqual(['injected\n']);
+
+    const consoleWarn = vi.spyOn(console, 'warn').mockImplementation(() => {});
+    printMongodNotice('default sink');
+    expect(warnings).toContain('default sink');
+    expect(consoleWarn).not.toHaveBeenCalled();
+  });
+});
+
+describe('[#5517] the abandoned-download rejection guard', () => {
+  it('recognises the rename ENOENT of a lost download race', () => {
+    expect(isAbandonedDownloadRejection(renameEnoent())).toBe(true);
+  });
+
+  it('does not recognise anything else', () => {
+    expect(isAbandonedDownloadRejection(undefined)).toBe(false);
+    expect(isAbandonedDownloadRejection(null)).toBe(false);
+    expect(isAbandonedDownloadRejection('ENOENT: rename')).toBe(false);
+    expect(isAbandonedDownloadRejection(new Error('boom'))).toBe(false);
+    // Right code and syscall, but a real file rather than a download temp file.
+    expect(
+      isAbandonedDownloadRejection(
+        Object.assign(new Error('x'), { code: 'ENOENT', syscall: 'rename', path: '/tmp/a.json' }),
+      ),
+    ).toBe(false);
+    // A different syscall on the download temp file is somebody else's bug.
+    expect(
+      isAbandonedDownloadRejection(
+        Object.assign(new Error('x'), { code: 'ENOENT', syscall: 'unlink', path: '/c/x.tgz.downloading' }),
+      ),
+    ).toBe(false);
+  });
+
+  it('swallows the abandoned download and re-raises every other rejection', () => {
+    const listeners: ((reason: unknown) => void)[] = [];
+    const host = {
+      on: (_event: 'unhandledRejection', listener: (reason: unknown) => void) => {
+        listeners.push(listener);
+      },
+      off: (_event: 'unhandledRejection', listener: (reason: unknown) => void) => {
+        listeners.splice(listeners.indexOf(listener), 1);
+      },
+    };
+    const reraised: unknown[] = [];
+    const warned: string[] = [];
+
+    installAbandonedDownloadGuard({
+      host,
+      reraise: (reason) => reraised.push(reason),
+      warn: (message) => warned.push(message),
+    });
+    // One suite per worker installs it; the rest must not stack listeners.
+    installAbandonedDownloadGuard({ host, reraise: () => {}, warn: () => {} });
+    expect(listeners).toHaveLength(1);
+
+    listeners[0](renameEnoent());
+    expect(reraised).toEqual([]);
+    expect(warned).toHaveLength(1);
+    expect(warned[0]).toContain('#5517');
+
+    const real = new Error('a genuine unhandled rejection');
+    listeners[0](real);
+    // Vitest has stepped aside because this listener exists, so the guard owes
+    // the run a failure for anything it does not recognise.
+    expect(reraised).toEqual([real]);
+    expect(warned).toHaveLength(1);
+
+    disposeAbandonedDownloadGuard();
+    expect(listeners).toHaveLength(0);
+  });
+});

packages/plugins/driver-mongodb/src/mongodb-pagination-conformance.test.ts

@@ -25,24 +25,23 @@
  */
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
-import { MongoMemoryServer } from 'mongodb-memory-server';
+import type { MongoMemoryServer } from 'mongodb-memory-server';
 import {
   PAGINATION_ALL_IDS,
   PAGINATION_CASES,
   PAGINATION_ROWS,
   PAGINATION_UNORDERED_CASES,
 } from '@objectstack/spec/data';
 import { MongoDBDriver } from './mongodb-driver.js';
-
-let sharedMongod: MongoMemoryServer | undefined;
-try {
-  sharedMongod = await MongoMemoryServer.create({ instance: { launchTimeout: 60_000 } });
-} catch (err) {
-  console.warn(
-    '[driver-mongodb] Skipping pagination conformance — mongodb-memory-server could not start: ' +
-      `${(err as Error)?.message ?? String(err)}`,
-  );
-}
+import { createTestMongod } from './test-mongod.js';
+
+// The live half needs a real mongod, so it is OPT-IN since #5517 (see
+// `test-mongod.ts`). It used to call `MongoMemoryServer.create()` itself, which
+// both started a download in every default run and bypassed the package's shared
+// deadline. The sort-spec half below runs regardless — it needs no server.
+const sharedMongod: MongoMemoryServer | undefined = await createTestMongod(
+  'pagination conformance',
+);
 
 describe.skipIf(!sharedMongod)('driver-mongodb — paged reads are a partition of the result set', () => {
   const mongod = sharedMongod as MongoMemoryServer;