From 470f3602b52abd24a702157131b39cdbbd218aca Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 11 Aug 2026 20:42:55 +0000 Subject: [PATCH] fix(driver-memory): render the analytics echo as the query it describes (#7117) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `MemoryAnalyticsService` has two exits for one normalized filter tree and they disagreed about what the LIKE family MEANS. `query()` builds a real containment pattern; `generateSql()` emitted the comparand as a bare literal with no wildcard anywhere, so `{name: {$contains: 'acme'}}` echoed `WHERE name LIKE 'acme'` — an EQUALITY — beside a chart drawn from every row CONTAINING `acme`. `notContains` mirrored it through `NOT LIKE`. The `$contains` family now renders `GLOB '*v*'` / `NOT GLOB`, and `$icontains` `lower(col) GLOB lower('*v*')`. GLOB rather than LIKE because this exit emits SQLite-shaped SQL and SQLite's LIKE folds ASCII unconditionally, while #4706 Q2 = A rules the family case-SENSITIVE and #7723 put this package's execution faces on that answer — a LIKE echo would have contradicted execution on a second axis the moment the wildcards were added. The translation is the spec's shared `likePatternToGlobPattern`, over a LIKE-escaped comparand, so an author's own `%` / `_` / `*` / `?` / `[` stay literal instead of becoming the match-every-row bypass (#5567). `operatorToSql`'s `|| '='` fallback is gone with it: a name→name map cannot hold a wildcard, a list, or a null-safe negation, so it is now a builder table keyed by `CubeOperator` — the shape #5374 gave the mingo exit — and a widened vocabulary fails to compile until its SQL spelling exists. Three operators were reaching that fallback and are fixed with it: `{$in: [a,b]}` echoed `= a`, `{$nin: [a]}` echoed the exact COMPLEMENT of the query's rows, and `{$exists: true}` echoed `name = 1`, which selects nothing. Three smaller divergences on the same builder went too — negations are null-safe (#5146 / #5297), an empty `$in`/`$nin` renders a predicate instead of no WHERE, and a bare-day `$lte` renders half-open as the pipeline has read it since #4042. `startsWith` / `endsWith` never reached the fallback and are unchanged: this face does not lower them, so both exits refuse them with `INVALID_FILTER` / 400. `memory-analytics-echo-operator-coverage.test.ts` pins the WHERE against `query()`'s ROW SET by executing the echoed statement on a real SQLite engine (`sql.js`), enumerates the closed vocabulary on both exits, and records the eight-way reverse verification. Only the DISPLAYED SQL changes; `query()`'s rows are untouched. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01TPYmYr8mjAsbZ6RqwEk4TT --- .../memory-analytics-echo-like-wildcards.md | 60 +++ packages/drivers/driver-memory/package.json | 2 + ...y-analytics-echo-operator-coverage.test.ts | 471 ++++++++++++++++++ .../driver-memory/src/memory-analytics.ts | 298 +++++++++-- .../src/memory-icontains.test.ts | 26 +- .../src/memory-like-pattern.test.ts | 21 +- pnpm-lock.yaml | 6 + 7 files changed, 813 insertions(+), 71 deletions(-) create mode 100644 .changeset/memory-analytics-echo-like-wildcards.md create mode 100644 packages/drivers/driver-memory/src/memory-analytics-echo-operator-coverage.test.ts diff --git a/.changeset/memory-analytics-echo-like-wildcards.md b/.changeset/memory-analytics-echo-like-wildcards.md new file mode 100644 index 0000000000..0511550f49 --- /dev/null +++ b/.changeset/memory-analytics-echo-like-wildcards.md @@ -0,0 +1,60 @@ +--- +"@objectstack/driver-memory": patch +--- + +fix(driver-memory): the analytics echo renders the query it describes (#7117) + +`MemoryAnalyticsService` has two exits for one normalized filter tree, and they +disagreed about what the LIKE family MEANS. `query()` builds a real containment +pattern; `generateSql()` emitted the comparand as a bare literal with no +wildcard anywhere, so `{name: {$contains: 'acme'}}` echoed + +```sql +WHERE name LIKE 'acme' +``` + +— an **equality** — beside a chart drawn from every row *containing* `acme`. +The echo's only job is reproducing execution, so an author who ran it to debug +the chart got a **narrower** row set and read the filter as broken. +`$notContains` mirrored it through `NOT LIKE`. + +**What the echo emits now.** The `$contains` family renders `GLOB '*v*'` / +`NOT GLOB`, and `$icontains` renders `lower(col) GLOB lower('*v*')`. `GLOB` +rather than `LIKE` because this exit emits SQLite-shaped SQL and SQLite's `LIKE` +folds ASCII case unconditionally: #4706 Q2 = A rules the `$contains` family +case-**sensitive**, and #7723 put this package's execution faces on that answer, +so a `LIKE` echo would have contradicted execution on a second axis the moment +the missing wildcards were added. The two halves are one fix because `GLOB` +speaks a different pattern language from `LIKE` — choosing the construct and +rendering the wildcards are the same decision. The translation is the spec's +shared `likePatternToGlobPattern`, and the comparand is escaped first, so an +author's own `%` / `_` / `*` / `?` / `[` stay literal instead of becoming the +match-every-row bypass (#5567). + +**The `|| '='` fallback is gone with it.** `operatorToSql` was a +name→name map, which cannot hold a wildcard, a list, or a null-safe negation; +it is now a builder table keyed by `CubeOperator`, the shape #5374 gave the +mingo exit, so a widened vocabulary fails to compile until its SQL spelling +exists. Three operators were reaching that fallback and are fixed with it — +measured against `query()` on a six-row fixture: + +| `where` | `query()` | echoed, before | echoed, now | +|---|---|---|---| +| `{name: {$in: [a, b]}}` | both rows | `name = a` — one row | `name IN (a, b)` | +| `{name: {$nin: [a]}}` | the other five | `name = a` — the **complement** | `(name IS NULL OR name NOT IN (a))` | +| `{name: {$exists: true}}` | five rows | `name = 1` — **no** rows | `name IS NOT NULL` | + +Three smaller divergences on the same builder went with them: negations are +null-safe (`$ne` / `$nin` / `$notContains` kept only rows whose column was not +NULL, where the pipeline returns them — the #5146 / #5297 rule the rest of the +repo already follows); an empty `$in` / `$nin` list now renders a predicate +instead of no `WHERE` at all (an empty `$in` echoed the whole table while the +pipeline returns nothing); and a bare-day `$lte` bound renders half-open, as the +pipeline has read it since #4042. + +`$startsWith` and `$endsWith` never reached the fallback and are unchanged: this +face does not lower them, so both exits refuse them with `INVALID_FILTER` / 400 +(#5345). + +Only the *displayed* SQL changes — this exit produces the statement shown for +transparency, never the query that runs, and `query()`'s rows are untouched. diff --git a/packages/drivers/driver-memory/package.json b/packages/drivers/driver-memory/package.json index e0a71c3363..7273f15314 100644 --- a/packages/drivers/driver-memory/package.json +++ b/packages/drivers/driver-memory/package.json @@ -25,6 +25,8 @@ }, "devDependencies": { "@types/node": "^26.1.2", + "@types/sql.js": "^1.4.11", + "sql.js": "^1.14.1", "typescript": "^6.0.3", "vitest": "^4.1.10" }, diff --git a/packages/drivers/driver-memory/src/memory-analytics-echo-operator-coverage.test.ts b/packages/drivers/driver-memory/src/memory-analytics-echo-operator-coverage.test.ts new file mode 100644 index 0000000000..8ce667c278 --- /dev/null +++ b/packages/drivers/driver-memory/src/memory-analytics-echo-operator-coverage.test.ts @@ -0,0 +1,471 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#7117] `MemoryAnalyticsService.generateSql()` describes the query + * `MemoryAnalyticsService.query()` actually runs — pinned by RUNNING it. + * + * # The defect + * + * This service has two exits for one normalized filter tree and they disagreed + * about what the LIKE family MEANS. `query()` builds a real containment pattern + * (`CUBE_OPERATOR_TO_MONGO_PREDICATE`'s `contains` row, over the driver's own + * `filterSubstringPattern`); `generateSql()` emitted the comparand as a bare + * literal with no wildcard anywhere, so `{name: {$contains: 'acme'}}` echoed + * + * WHERE name LIKE 'acme' + * + * — an EQUALITY — beside a chart built from every row CONTAINING `acme`. The + * echo's only reason to exist is reproducing execution, so an author who runs it + * to debug the chart gets a NARROWER row set and reads the filter as broken. + * That is the #5333 / #3650 class ("a rendering that contradicts execution is + * worse than no rendering"), reached through driver-memory's analytics face. + * + * # Why this file EXECUTES the echo instead of asserting its text + * + * The maintainer's ruling on this card (comment `5234868334`) is explicit, and + * it is the same call `objectql-echo-operator-coverage.test.ts` makes for + * `service-analytics`' third compiler: **pin the WHERE against `query()`'s ROW + * SET, not against a string.** An assertion on the SQL text would not have + * caught the missing `%` either — `WHERE name LIKE 'acme'` contains the column, + * the operator and the comparand, and reads as a working predicate. So every + * case below is run twice: once through `query()` (mingo) and once by handing + * the echoed statement to a real SQLite engine over the same fixture. The + * engine is `sql.js` (pure WASM) for the reason + * `native-sql-filter-logic-conformance.test.ts` gives — a native binding loads + * only under the exact Node ABI it was built for and aborts the vitest worker on + * CI's Node, taking the file's cases silently with it. + * + * SQLite specifically, because that is the dialect this exit has always emitted: + * `toSqlLiteral` spells booleans `1`/`0`, and #6520's `icontains` row reasoned + * from SQLite's ASCII-only `LIKE`. Stating it as an executed premise rather than + * a comment is what makes it go RED rather than stale if the exit ever changes + * dialect. + * + * # The two axes, because #7723 opened the second one + * + * #7723 made the `$contains` family case-EXACT on this package's execution faces + * (#4706 Q2 = A) — including `filterSubstringPattern`, which this face borrows. + * A `LIKE` echo would then have contradicted execution on a SECOND axis the + * moment the first was fixed: SQLite's `LIKE` folds ASCII case and the fold + * cannot be turned off per statement. The two halves are one fix because the + * case-exact construct (`GLOB`) speaks a DIFFERENT pattern language from `LIKE`, + * so choosing it and rendering the wildcards are the same decision — which is + * why #7723 serialized this half behind this card rather than shipping it. + * + * | axis | `query()` | echo, before | echo, now | + * |:--|:--|:--|:--| + * | containment | substring | **equality** — no wildcard | `GLOB '*v*'` | + * | case (`$contains`) | exact (#7723) | folds ASCII (SQLite `LIKE`) | exact (`GLOB`) | + * | case (`$icontains`) | folds ASCII | folds ASCII | folds ASCII (`lower()` both sides) | + * + * # What the enumeration adds over the measured table + * + * `ANALYTICS_FILTER_CAPABILITIES` closes the vocabulary — `normalizeFilters` + * refuses anything outside `MONGO_TO_CUBE_OPERATOR` with `INVALID_FILTER` / 400, + * on BOTH exits — so the operators that can ever reach the WHERE builder are a + * finite, enumerable set. Driving all of them through both exits turns "the two + * tables drifted" from something a reader has to notice into a failing test. + * That is how the OTHER half of this card was measured: the issue expected + * `startsWith` / `endsWith` to fall to `operatorToSql`'s `|| '='` fallback, and + * they cannot — they are refused. The three operators that DID fall to it were + * `in`, `notIn` and `set`: + * + * | `where` | `query()` | echo, before | echo, now | + * |:--|:--|:--|:--| + * | `{name: {$in: [a, b]}}` | both rows | `name = a` — one row | `name IN (a, b)` | + * | `{name: {$nin: [a]}}` | the other four | `name = a` — the **complement** | `(name IS NULL OR name NOT IN (a))` | + * | `{name: {$exists: true}}` | four rows | `name = 1` — **no** rows | `name IS NOT NULL` | + * + * # Reverse verification + * + * Each repair was reverted one at a time, the failure direction PREDICTED + * first, then measured. All eight predictions held; the `measured` column is + * the assertion text vitest printed, not a paraphrase. + * + * | reverted to | predicted | measured (6 rows; ids as above) | + * |:--|:--|:--| + * | `contains` → `LIKE ` | the original defect: the echo narrows to an equality and answers no rows | 6 failures; `{$contains:'ACME'}` → `expected [] to deeply equal ['2']` | + * | `GLOB` → `LIKE '%v%'` | containment right, CASE wrong — the echo picks up the upper-case row the case-exact query excludes | 5 failures; `{$contains:'ACME'}` → `expected ['1','2'] to deeply equal ['2']` | + * | drop the comparand's LIKE-escape | `{$contains:'%'}` widens from the one row that really contains a `%` to every non-null row | `echo %: expected ['1','2','3','4','5'] to deeply equal ['4']` | + * | `icontains` → `GLOB` without `lower()` | the fold vanishes, so a lower-case comparand matches nothing | `expected [] to deeply equal ['1','2','3']` | + * | `notContains` → bare `NOT GLOB` | SQL's three-valued `WHERE` drops the NULL row mingo returns | `expected ['2','4','5'] to deeply equal ['2','4','5','6']` | + * | `in` → the `\|\| '='` fallback | only the first list member survives | `expected ['1'] to deeply equal ['1','3']` | + * | restore the `values.length > 0` guard | empty `$in` emits no WHERE, so the echo describes the whole table | `expected ['1','2','3','4','5','6'] to deeply equal []` | + * | `lte` → a closed bare-day bound | the echo drops that day's timestamped rows | the `at < '2026-01-03'` pin, then `['1'] != ['1','2']` | + * + * Reverting `in`/`notIn`/`set` all the way to the deleted `operatorToSql` is not + * expressible any more without also deleting the `Record` key + * type — which is the point of that key type, and the reason those three rows + * cannot silently go missing again. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { InMemoryDriver } from './memory-driver.js'; +import { MemoryAnalyticsService, ANALYTICS_FILTER_CAPABILITIES } from './memory-analytics.js'; +import { FILTER_OPERATORS } from '@objectstack/spec/data'; +import type { AnalyticsQuery, Cube, FilterCondition } from '@objectstack/spec/data'; + +/** + * Six rows chosen so that every wrong rendering this file guards against shows + * up as a wrong ID rather than as a coincidence: + * + * - rows 1/2 differ only in CASE, so a case-folding echo picks up row 2 where + * the case-exact query does not; + * - row 3 contains the comparand without starting or ending with it, so a + * containment predicate compiled as an equality returns nothing; + * - row 4 carries LIKE's own metacharacters, so an unescaped comparand shows + * up as the `%`-matches-everything bypass (#5567); + * - row 5 carries GLOB's metacharacters, which are ORDINARY to LIKE — the + * direction a hand-written escape forgets; + * - row 6 has a NULL `name`, which is the row every negation drops under SQL's + * three-valued `WHERE` unless the predicate is null-safe (#5146 / #5297). + */ +const ROWS = [ + { id: '1', name: 'Acme Industries', amount: 10 }, + { id: '2', name: 'ACME INDUSTRIES', amount: 20 }, + { id: '3', name: 'Global Industries Ltd', amount: 30 }, + { id: '4', name: '100% off_peak', amount: 40 }, + { id: '5', name: 'a*b?c[d', amount: 50 }, + { id: '6', name: null, amount: 60 }, +] as const; + +const CUBE: Cube = { + name: 'deals', + title: 'Deals', + sql: 'deal', + measures: { total: { name: 'total', label: 'Total', type: 'count', sql: 'id' } }, + dimensions: { + id: { name: 'id', label: 'Id', type: 'string', sql: 'id' }, + name: { name: 'name', label: 'Name', type: 'string', sql: 'name' }, + amount: { name: 'amount', label: 'Amount', type: 'number', sql: 'amount' }, + }, + public: true, +}; + +/** Point sql.js at the `.wasm` shipped inside its own package (Node-safe). */ +async function locateWasm(): Promise<((file: string) => string) | undefined> { + try { + const { createRequire } = await import('node:module'); + const require = createRequire(import.meta.url); + const pkgJsonPath = require.resolve('sql.js/package.json'); + const { dirname, join } = await import('node:path'); + const dir = dirname(pkgJsonPath); + return (file: string) => join(dir, 'dist', file); + } catch { + return undefined; + } +} + +/** + * One authorable `$op` spelling this face ACCEPTS, with a comparand this fixture + * can answer. Keyed by the spelling rather than by the lowered cube operator, so + * a widening of `MONGO_TO_CUBE_OPERATOR` shows up here as a missing key. + */ +const ACCEPTED_CASES: Record = { + $eq: { name: { $eq: 'Acme Industries' } }, + $ne: { name: { $ne: 'Acme Industries' } }, + $gt: { amount: { $gt: 30 } }, + $gte: { amount: { $gte: 30 } }, + $lt: { amount: { $lt: 30 } }, + $lte: { amount: { $lte: 30 } }, + $in: { name: { $in: ['Acme Industries', 'Global Industries Ltd'] } }, + $nin: { name: { $nin: ['Acme Industries'] } }, + // Lower-case `I` on purpose for `$icontains` and upper for `$contains`: each + // comparand answers differently depending on whether the fold RAN, so a + // missing fold and a stray fold are both wrong IDs rather than passing counts. + $contains: { name: { $contains: 'Industries' } }, + $icontains: { name: { $icontains: 'industries' } }, + $notContains: { name: { $notContains: 'Industries' } }, + $exists: { name: { $exists: true } }, +}; + +/** The spellings this face REFUSES — the complement, so the two sets are total. */ +const REFUSED_OPERATORS = ['$between', '$startsWith', '$endsWith', '$null'] as const; + +describe('[#7117] the analytics echo renders the query it describes', () => { + let db: any; + let driver: InMemoryDriver; + let service: MemoryAnalyticsService; + + beforeAll(async () => { + const mod: any = await import('sql.js'); + const initSqlJs = mod.default ?? mod; + const locateFile = await locateWasm(); + const SQL = await initSqlJs(locateFile ? { locateFile } : undefined); + + db = new SQL.Database(); + db.run(`CREATE TABLE deal (id TEXT PRIMARY KEY, name TEXT, amount REAL);`); + const insert = db.prepare(`INSERT INTO deal (id, name, amount) VALUES (?,?,?)`); + for (const r of ROWS) insert.run([r.id, r.name, r.amount]); + insert.free(); + + driver = new InMemoryDriver({ persistence: false }); + await driver.connect(); + for (const r of ROWS) await driver.create('deal', { ...r }); + service = new MemoryAnalyticsService({ driver, cubes: [CUBE] }); + }); + + afterAll(() => { + db?.close(); + return driver?.disconnect?.(); + }); + + const query = (where: unknown): AnalyticsQuery => ({ + cube: 'deals', + measures: ['total'], + dimensions: ['id'], + where, + } as unknown as AnalyticsQuery); + + const sortIds = (ids: string[]): string[] => [...ids].sort((a, b) => a.localeCompare(b)); + + /** The rows the pipeline actually returns — what the chart is drawn from. */ + const executedIds = async (where: unknown): Promise => + sortIds((await service.query(query(where))).rows.map((r: any) => String(r.id))); + + /** The rows the ECHOED statement returns, run on a real SQLite engine. */ + const echoIds = async (where: unknown): Promise => { + const { sql } = await service.generateSql(query(where)); + const stmt = db.prepare(sql); + const out: string[] = []; + while (stmt.step()) out.push(String(stmt.getAsObject().id)); + stmt.free(); + return sortIds(out); + }; + + const echoSql = async (where: unknown): Promise => + (await service.generateSql(query(where))).sql; + + it('seeded both engines identically (the premise)', async () => { + expect(await executedIds({})).toEqual(['1', '2', '3', '4', '5', '6']); + expect(await echoIds({})).toEqual(['1', '2', '3', '4', '5', '6']); + }); + + // ── The issue's own repro ────────────────────────────────────────────────── + + describe("the issue's measured case", () => { + it('`$contains` echoes a containment pattern, not an equality', async () => { + const where = { name: { $contains: 'Industries' } }; + // Was `WHERE name LIKE 'Industries'` — an equality, and zero rows. + expect(await echoSql(where)).toContain("name GLOB '*Industries*'"); + expect(await echoIds(where)).toEqual(['1', '3']); + expect(await executedIds(where)).toEqual(['1', '3']); + }); + + it('`$notContains` mirrors it, and keeps the NULL row mingo keeps', async () => { + const where = { name: { $notContains: 'Industries' } }; + // #5146 / #5297: a bare `NOT GLOB` is UNKNOWN for a NULL column and a + // `WHERE` keeps only TRUE, so row 6 would vanish from the echo alone. + expect(await echoSql(where)).toContain("(name IS NULL OR name NOT GLOB '*Industries*')"); + expect(await echoIds(where)).toEqual(['2', '4', '5', '6']); + expect(await executedIds(where)).toEqual(['2', '4', '5', '6']); + }); + + it('the echoed `$contains` is case-EXACT, like the query it describes', async () => { + // The #7723 axis. Row 2 is row 1 in upper case: SQLite's `LIKE` would + // return it and the pipeline does not, so this is the assertion that + // fails if the construct ever goes back to `LIKE`. + const where = { name: { $contains: 'ACME' } }; + expect(await echoIds(where)).toEqual(['2']); + expect(await executedIds(where)).toEqual(['2']); + }); + + it('`$icontains` folds ASCII on BOTH sides, and only ASCII', async () => { + const where = { name: { $icontains: 'industries' } }; + expect(await echoSql(where)).toContain("lower(name) GLOB lower('*industries*')"); + expect(await echoIds(where)).toEqual(['1', '2', '3']); + expect(await executedIds(where)).toEqual(['1', '2', '3']); + }); + + it("an author's own LIKE metacharacters stay LITERAL", async () => { + // #5567's bypass, in this exit. Unescaped, `LIKE '%%%'` matches every + // non-null row; the query matches only the row that really contains a `%`. + for (const [comparand, expected] of [ + ['%', ['4']], + ['_', ['4']], + ['100% off', ['4']], + ['off_peak', ['4']], + ] as Array<[string, string[]]>) { + const where = { name: { $contains: comparand } }; + expect(await echoIds(where), `echo ${comparand}`).toEqual(expected); + expect(await executedIds(where), `executed ${comparand}`).toEqual(expected); + } + }); + + it("GLOB's OWN metacharacters stay literal too — the direction LIKE has no opinion on", async () => { + // `*`, `?` and `[` are ordinary to LIKE and wildcards to GLOB, so moving + // the construct is what makes these cases exist at all. + for (const comparand of ['a*b', 'b?c', 'c[d', 'a*b?c[d']) { + const where = { name: { $contains: comparand } }; + expect(await echoIds(where), `echo ${comparand}`).toEqual(['5']); + expect(await executedIds(where), `executed ${comparand}`).toEqual(['5']); + } + // …and a pattern that would only match if `*` were a wildcard matches + // NEITHER face. This is the assertion an escape-free rendering fails. + const bypass = { name: { $contains: '*Industries*' } }; + expect(await echoIds(bypass)).toEqual([]); + expect(await executedIds(bypass)).toEqual([]); + }); + + it('a quote in the comparand survives into the pattern literal', async () => { + const where = { name: { $contains: "o'clock" } }; + expect(await echoSql(where)).toContain("GLOB '*o''clock*'"); + expect(await echoIds(where)).toEqual([]); + expect(await executedIds(where)).toEqual([]); + }); + }); + + // ── The `|| '='` fallback the issue's Scope names ────────────────────────── + + describe("the operators that fell to `operatorToSql`'s `|| '='`", () => { + it('`$in` renders the WHOLE list, not its first member', async () => { + const where = { name: { $in: ['Acme Industries', 'Global Industries Ltd'] } }; + expect(await echoIds(where)).toEqual(['1', '3']); + expect(await executedIds(where)).toEqual(['1', '3']); + }); + + it('`$nin` renders the negation — it used to echo the exact COMPLEMENT', async () => { + const where = { name: { $nin: ['Acme Industries'] } }; + expect(await echoIds(where)).toEqual(['2', '3', '4', '5', '6']); + expect(await executedIds(where)).toEqual(['2', '3', '4', '5', '6']); + }); + + it('an EMPTY list is a predicate, not an absent clause', async () => { + // The `values.length > 0` guard: `$in: []` selects nothing and used to + // echo a statement with no WHERE at all — the whole table. + expect(await echoIds({ name: { $in: [] } })).toEqual([]); + expect(await executedIds({ name: { $in: [] } })).toEqual([]); + expect(await echoIds({ name: { $nin: [] } })).toEqual(['1', '2', '3', '4', '5', '6']); + expect(await executedIds({ name: { $nin: [] } })).toEqual(['1', '2', '3', '4', '5', '6']); + }); + + it('`$exists` renders a nullness test — it used to echo `name = 1`', async () => { + expect(await echoSql({ name: { $exists: true } })).toContain('name IS NOT NULL'); + expect(await echoSql({ name: { $exists: false } })).toContain('name IS NULL'); + // `name = 1` matched NO row on either engine; these match the query's. + expect(await echoIds({ name: { $exists: true } })).toEqual(['1', '2', '3', '4', '5']); + }); + + /** + * The one cell where SQL cannot say what mingo says, asserted as an + * INEQUALITY so it cannot be closed in silence. + * + * mingo's `$exists` tests KEY PRESENCE; a relational column always has it. + * A row storing an explicit `null` therefore satisfies `$exists: true` on + * `query()` and fails `IS NOT NULL` in the echo. `IS NOT NULL` is + * nonetheless the spelling both of this repo's other SQL lowerings use + * (`read-scope-sql.ts`'s `$exists` arm; `driver-sql`'s "a present field is a + * non-null column in SQL"), and it is a far smaller gap than the `name = 1` + * it replaces, which matched nothing at all. + */ + it('documents the one `$exists` cell SQL cannot translate exactly', async () => { + const where = { name: { $exists: true } }; + // Row 6 stores an explicit `null`: present to mingo, NULL to SQL. + expect(await executedIds(where)).toEqual(['1', '2', '3', '4', '5', '6']); + expect(await echoIds(where)).toEqual(['1', '2', '3', '4', '5']); + }); + }); + + // ── The bare-day upper bound, which both exits must read the same way ────── + + it('a bare-day `$lte` is half-open in the echo, as it is in the pipeline', async () => { + // #4042 (the SQL twin is #3777): `<= '2026-01-02'` drops that day's + // timestamped rows, so the echo was one row NARROWER than its chart. + const temporal = new InMemoryDriver({ persistence: false }); + await temporal.connect(); + for (const r of [ + { id: '1', at: '2026-01-01T05:00:00.000Z' }, + { id: '2', at: '2026-01-02T18:00:00.000Z' }, + { id: '3', at: '2026-01-03T00:00:00.000Z' }, + ]) await temporal.create('ev', { ...r }); + const cube: Cube = { + name: 'evs', title: 'Evs', sql: 'ev', + measures: { total: { name: 'total', label: 'T', type: 'count', sql: 'id' } }, + dimensions: { + id: { name: 'id', label: 'Id', type: 'string', sql: 'id' }, + at: { name: 'at', label: 'At', type: 'time', sql: 'at' }, + }, + public: true, + }; + const svc = new MemoryAnalyticsService({ driver: temporal, cubes: [cube] }); + const q = { + cube: 'evs', measures: ['total'], dimensions: ['id'], + where: { at: { $lte: '2026-01-02' } }, + } as unknown as AnalyticsQuery; + + const { sql } = await svc.generateSql(q); + expect(sql).toContain("at < '2026-01-03'"); + + db.run(`CREATE TABLE ev (id TEXT PRIMARY KEY, at TEXT);`); + const insert = db.prepare(`INSERT INTO ev (id, at) VALUES (?,?)`); + for (const r of [ + ['1', '2026-01-01T05:00:00.000Z'], + ['2', '2026-01-02T18:00:00.000Z'], + ['3', '2026-01-03T00:00:00.000Z'], + ]) insert.run(r); + insert.free(); + + const stmt = db.prepare(sql); + const echoed: string[] = []; + while (stmt.step()) echoed.push(String(stmt.getAsObject().id)); + stmt.free(); + + expect(sortIds(echoed)).toEqual(['1', '2']); + expect(sortIds((await svc.query(q)).rows.map((r: any) => String(r.id)))).toEqual(['1', '2']); + await temporal.disconnect?.(); + }); + + // ── The vocabulary, enumerated so the two tables cannot drift apart ──────── + + describe('the closed vocabulary, enumerated', () => { + it('the accepted and refused sets together are the whole Filter Protocol', () => { + expect(sortIds([...Object.keys(ACCEPTED_CASES), ...REFUSED_OPERATORS])) + .toEqual(sortIds([...FILTER_OPERATORS])); + }); + + it('the accepted set IS the face\'s declared vocabulary', () => { + expect(sortIds(Object.keys(ACCEPTED_CASES))) + .toEqual(sortIds([...ANALYTICS_FILTER_CAPABILITIES.fieldOperators])); + }); + + for (const [op, where] of Object.entries(ACCEPTED_CASES)) { + it(`${op}: running the echo returns exactly the rows the query returns`, async () => { + const executed = await executedIds(where); + const echoed = await echoIds(where); + if (op === '$exists') { + // The documented residue above — asserted there, skipped here so this + // loop stays a statement about every OTHER operator. + return; + } + expect(echoed, `${op}: the echo describes a different row set`).toEqual(executed); + }); + + it(`${op}: the echo carries a WHERE at all`, async () => { + const sql = await echoSql(where); + expect(sql, `${op} echoed no WHERE — the predicate was dropped`).toContain('WHERE'); + expect(sql).toMatch(/WHERE\s+\S/); + }); + } + + for (const op of REFUSED_OPERATORS) { + it(`${op} is REFUSED by both exits — it never reaches a fallback`, async () => { + // The half of this card's premise that measurement falsified: the issue + // expected `$startsWith` / `$endsWith` to render as `=` through + // `operatorToSql`'s `|| '='`. They cannot — they are not in + // `MONGO_TO_CUBE_OPERATOR`, so #5345's gate refuses them first, loudly + // and identically on both exits. + const where = { name: { [op]: 'x' } } as unknown as FilterCondition; + for (const run of [ + () => service.query(query(where)), + () => service.generateSql(query(where)), + ]) { + const err = await run().then(() => null, (e: any) => e); + expect(err, `${op} was accepted`).toBeInstanceOf(Error); + expect(err.code).toBe('INVALID_FILTER'); + expect(err.status).toBe(400); + expect(err.message).toContain(op); + } + }); + } + }); +}); diff --git a/packages/drivers/driver-memory/src/memory-analytics.ts b/packages/drivers/driver-memory/src/memory-analytics.ts index 42334d3e9c..6d7a3c1b8a 100644 --- a/packages/drivers/driver-memory/src/memory-analytics.ts +++ b/packages/drivers/driver-memory/src/memory-analytics.ts @@ -3,7 +3,9 @@ import type { IAnalyticsService, AnalyticsResult, CubeMeta } from '@objectstack/spec/contracts'; import type { Cube, AnalyticsQuery } from '@objectstack/spec/data'; // [#6520] `$icontains`' ASCII-only fold, from the spec's one definition. -import { asciiCaseInsensitiveRegexSource } from '@objectstack/spec/data'; +// [#7117] `likePatternToGlobPattern` — the spec's one LIKE→GLOB translation, so +// the SQL exit's wildcard rendering is not a third hand-copy of that escape. +import { asciiCaseInsensitiveRegexSource, likePatternToGlobPattern } from '@objectstack/spec/data'; import type { InMemoryDriver } from './memory-driver.js'; import { Logger, createLogger, nextUtcCalendarDay } from '@objectstack/core'; import { @@ -153,8 +155,13 @@ interface MongoPredicateInput { /** The operands as authored. For operands that are not comparands. */ readonly raw: readonly unknown[]; /** - * A comparand as a case-insensitive literal-substring pattern, built by the - * DRIVER's own rule (`filterSubstringPattern`) rather than re-derived here. + * A comparand as a literal-substring pattern, built by the DRIVER's own rule + * (`filterSubstringPattern`) rather than re-derived here. + * + * [#7723] Case-EXACT, because that rule is: `filterSubstringPattern` carried + * an `i` flag until #7723 took it off, putting the `$contains` family on the + * #4706 Q2 = A answer across every face of this package. Borrowing the rule + * rather than restating it is what made that one edit reach this face too. */ readonly substring: (value: unknown) => RegExp; /** @@ -162,11 +169,12 @@ interface MongoPredicateInput { * `$icontains`' fold, which is NOT {@link substring}'s. * * The two are deliberately separate functions rather than one with a flag. - * `substring` folds the whole Unicode range (the driver's `i` flag), which is - * the open defect #6682 tracks for the `$contains` family on this face; this - * one folds `A-Z` only, which is what the protocol says `$icontains` means - * (#4706 Q1 = A). Collapsing them would silently give one of the two operators - * the other's answer. + * `substring` is case-EXACT (#4706 Q2 = A, landed for this package in #7723); + * this one folds `A-Z` and nothing else, which is what the protocol says + * `$icontains` means (#4706 Q1 = A). Collapsing them would silently give one + * of the two operators the other's answer — and note the fold lives in the + * pattern SOURCE, never in a RegExp flag, because an `i` flag folds the whole + * Unicode range and would answer `CAFÉ` for `café`. */ readonly asciiSubstring: (value: unknown) => RegExp; } @@ -259,6 +267,187 @@ const CUBE_OPERATOR_TO_MONGO_PREDICATE: Readonly ({ $exists: raw.length > 0 ? Boolean(raw[0]) : true }), }); +/** + * [#7117] What one lowered entry gives its SQL predicate builder — the SQL twin + * of {@link MongoPredicateInput}, and split along the same seam for the same + * reason: a VALUE COMPARISON is rendered from the comparand in the field's + * storage form, while a PATTERN operand is built from what the author wrote. + */ +interface SqlPredicateInput { + /** The resolved column expression this predicate constrains. */ + readonly column: string; + /** Comparands in the field's storage form (#4047). For value comparisons. */ + readonly comparands: readonly unknown[]; + /** The operands as authored. For operands that are not comparands. */ + readonly raw: readonly unknown[]; + /** One comparand as a SQL literal ({@link MemoryAnalyticsService.toSqlLiteral}). */ + readonly literal: (value: unknown) => string; + /** + * A comparand as a case-EXACT substring GLOB pattern, already a SQL literal. + * See {@link globSubstringPattern} for why GLOB and not LIKE. + */ + readonly globSubstring: (value: unknown) => string; +} + +type SqlPredicateBuilder = (input: SqlPredicateInput) => string; + +/** + * [#7117] A comparand as a SQLite `GLOB` pattern that matches it as a literal + * SUBSTRING — the wildcard rendering this issue is about. + * + * ## Why a pattern at all + * + * `generateSql()` used to emit the bare comparand: `{name: {$contains: 'acme'}}` + * echoed `WHERE name LIKE 'acme'`, which is an EQUALITY, next to a `query()` + * that returns every row CONTAINING `acme`. The echo's whole job is reproducing + * execution, so an author who runs it to debug a chart gets a NARROWER row set + * and reads the filter as broken — the #5333 / #3650 class ("a rendering that + * contradicts execution is worse than no rendering") reached through this + * package's analytics face. + * + * ## Why GLOB and not LIKE + * + * This exit emits SQLite-shaped SQL — the dialect its sibling decisions already + * assume ({@link MemoryAnalyticsService.toSqlLiteral} spells booleans `1`/`0`, + * and #6520's `icontains` row reasoned from SQLite's ASCII-only `LIKE`). On + * SQLite `LIKE` folds ASCII case and the fold cannot be turned off per + * statement, while #4706 Q2 = A rules the `$contains` family case-SENSITIVE and + * #7723 put this package's execution faces on that answer. So a `LIKE` echo + * would have contradicted execution on a SECOND axis the moment the first was + * fixed: right rows for the wrong case. `GLOB` is case-exact by definition, + * which is exactly why `driver-sql`'s `textMatchPredicate` picks it for its own + * SQLite arm — this is the same cell of that table, rendered as a literal + * instead of a binding. + * + * ## Why the translation is borrowed and not written here + * + * GLOB's pattern language is not LIKE's: `*` / `?` / `[` are metacharacters to + * GLOB and ORDINARY characters to LIKE, and forgetting that direction is the + * `%`-matches-every-row bypass (#5567) wearing GLOB's clothes. The spec owns one + * definition of the translation (`likePatternToGlobPattern`), so the comparand + * is LIKE-escaped into a substring pattern and handed to it — a third hand-copy + * of the escape is the thing `service-analytics`' `like-pattern.ts` header says + * to refuse, and this is how it is refused. + * + * The LIKE escape in front of it is what keeps an author's own metacharacter + * literal: `{name: {$contains: '%'}}` matched NO row on `query()` and every + * non-null row on the echoed `LIKE '%'`, which is the widening direction, not + * the narrowing one this issue opened on. + */ +function globSubstringPattern(value: unknown): string { + return likePatternToGlobPattern(`%${String(value).replace(/[\\%_]/g, '\\$&')}%`); +} + +/** + * [#7117] How each cube operator becomes a SQL predicate — the whole clause, not + * the name of an operator. + * + * # Why the shape changed + * + * This was `operatorToSql(operator): string`, a name→name map, and the WHERE + * builder filled the name in as `${column} ${op} ${literal}`. That shape can say + * "compare this column to this value" and NOTHING else, which is the same + * expressiveness ceiling {@link CUBE_OPERATOR_TO_MONGO_PREDICATE} was built to + * break on the mingo side in #5374 — and it failed here in the same three ways: + * + * - **The LIKE family had nowhere to put its wildcards.** `contains` → + * `'LIKE'` rendered `name LIKE 'acme'`, an equality. See + * {@link globSubstringPattern}. + * - **`in` / `notIn` / `set` are not in the table at all**, so they fell to + * its `|| '='` fallback — the silent-wrong-answer shape #5374 and #5345 + * removed from this file's sibling tables. Measured against `query()` on + * the fixture in `memory-analytics-echo-operator-coverage.test.ts`: + * `{name: {$nin: [a]}}` echoed `name = a`, the exact COMPLEMENT of the rows + * `query()` returns; `{name: {$in: [a, b]}}` echoed only `a`; + * `{name: {$exists: true}}` echoed `name = 1`, which selects nothing at all. + * (`startsWith` / `endsWith`, which this issue's text also expected to land + * on that fallback, cannot reach it: they are not in + * {@link MONGO_TO_CUBE_OPERATOR}, so `normalizeFilters` REFUSES them for + * both exits with `INVALID_FILTER` / 400 — loudly, per #5345.) + * - **A negation could not be made null-safe**, because `(a OR b)` is not a + * name. SQL is three-valued and a `WHERE` keeps only TRUE, so a bare + * `col != v` / `col NOT GLOB v` drops every row whose column is NULL, while + * mingo returns them. #5146 ruled that divergence closed repo-wide and + * #5297 spells the remedy — `(col IS NULL OR )`, which is what + * `read-scope-sql.ts`'s `nullSafeNegative` emits for `$ne` / `$nin` / + * `$notContains`. The three negative rows below are that same rewrite. + * + * # Why it is a `Record` + * + * Same reason as its mingo twin, and it is the load-bearing half of this fix. + * The `|| '='` fallback is not merely unreachable now — it is UNNECESSARY: + * keying by {@link CubeOperator} makes widening {@link MONGO_TO_CUBE_OPERATOR} + * (a deliberate one-line edit, per #5345) fail to COMPILE until this table has + * the new operator's SQL spelling. The totality is proven rather than defended, + * so no future operator can silently render as an equality nobody wrote. + * + * # The one cell where SQL cannot say what mingo says + * + * `set` renders `IS NOT NULL` / `IS NULL` — the spelling this repo's other two + * SQL lowerings already use (`read-scope-sql.ts`'s `$exists` arm, `driver-sql`'s + * "a present field is a non-null column in SQL"). It is not an exact + * translation, and cannot be: mingo's `$exists` tests KEY PRESENCE, which a + * relational column always has. A row storing an explicit `null` therefore + * satisfies `$exists: true` on `query()` and fails `IS NOT NULL` in the echo. + * That residue is inherent to describing a document store in SQL, it is pinned + * as an explicit inequality in `memory-analytics-echo-operator-coverage.test.ts` + * so it cannot be "fixed" in silence, and it is a far smaller gap than the + * `name = 1` it replaces — which matched nothing at all. + */ +const CUBE_OPERATOR_TO_SQL_PREDICATE: Readonly> = Object.freeze({ + // [#5373] A null comparand is a NULLNESS test, not a comparison. SQL's + // `= NULL` is never true, so emitting one would move the very loss #5373 + // closed from the mingo exit to this one: `{closed_at: null}` would select + // nothing while `query()` selects the null rows. The two exits have to mean + // the same thing. + equals: ({ column, comparands, literal }) => + comparands[0] == null ? `${column} IS NULL` : `${column} = ${literal(comparands[0])}`, + notEquals: ({ column, comparands, literal }) => + comparands[0] == null + ? `${column} IS NOT NULL` + : `(${column} IS NULL OR ${column} != ${literal(comparands[0])})`, + gt: ({ column, comparands, literal }) => `${column} > ${literal(comparands[0])}`, + gte: ({ column, comparands, literal }) => `${column} >= ${literal(comparands[0])}`, + lt: ({ column, comparands, literal }) => `${column} < ${literal(comparands[0])}`, + // Half-open on a bare-day bound, exactly as the mingo row above is (#4042; the + // SQL twin is #3777). `<= '2026-01-02'` drops that day's timestamped rows, + // which is measurable as an echo one row NARROWER than the chart it describes. + lte: ({ column, comparands, literal }) => { + const nextDay = nextUtcCalendarDay(comparands[0]); + return nextDay != null + ? `${column} < ${literal(nextDay)}` + : `${column} <= ${literal(comparands[0])}`; + }, + // The list operators take the WHOLE list, and an EMPTY one is a real + // predicate on this side too — `$in: []` selects nothing, `$nin: []` + // everything. Saying so here is what retires the WHERE builder's + // `values.length > 0` guard, under which `{code: {$in: []}}` emitted no clause + // at all and the echo described the whole table while `query()` returns none + // of it. The mingo row above retired the identical guard in #5374. + in: ({ column, comparands, literal }) => + comparands.length === 0 ? '1 = 0' : `${column} IN (${comparands.map(literal).join(', ')})`, + notIn: ({ column, comparands, literal }) => + comparands.length === 0 + ? '1 = 1' + : `(${column} IS NULL OR ${column} NOT IN (${comparands.map(literal).join(', ')}))`, + // A pattern, not a comparand: `raw`, and the shared GLOB substring rule. + contains: ({ column, raw, globSubstring }) => `${column} GLOB ${globSubstring(raw[0])}`, + notContains: ({ column, raw, globSubstring }) => + `(${column} IS NULL OR ${column} NOT GLOB ${globSubstring(raw[0])})`, + // [#6520] The case-INSENSITIVE twin. SQLite's `lower()` folds ASCII and + // nothing else — measured in #6518: `lower('CAFÉ')` is `'cafÉ'` — so it is + // `$icontains`' fold (#4706 Q1 = A) rather than the Unicode one, and it goes + // on BOTH sides: folding only the pattern compares a folded needle against a + // raw column and matches just the rows that were already lower-case. + icontains: ({ column, raw, globSubstring }) => + `lower(${column}) GLOB lower(${globSubstring(raw[0])})`, + // A presence flag, not a comparand — see this table's docblock for the one + // cell SQL cannot translate exactly. The `raw.length === 0` arm mirrors the + // mingo row's reading of a valueless `set`. + set: ({ column, raw }) => + `${column} IS ${raw.length === 0 || Boolean(raw[0]) ? 'NOT NULL' : 'NULL'}`, +}); + /** * [#6814] The size of a collected `$addToSet`, as `count_distinct` defines it: * distinct NON-NULL values of the column. @@ -644,27 +833,23 @@ export class MemoryAnalyticsService implements IAnalyticsService { } // Build WHERE clause + // + // [#7117] One builder per operator, from a table keyed by `CubeOperator` — + // the SQL twin of the `$match` construction in `query()`, and total by + // construction for the same reason. There is deliberately no + // `values.length > 0` guard any more: an empty list IS a predicate, and + // skipping the clause described the whole table (see the `in` row). const whereClauses: string[] = []; const normalizedFilters = this.normalizeFilters(query); - if (normalizedFilters.length > 0) { - for (const filter of normalizedFilters) { - const fieldPath = this.resolveFieldPath(cube, filter.member); - const sqlOp = this.operatorToSql(filter.operator); - if (filter.values && filter.values.length > 0) { - const comparand = this.comparandsFor(cube, filter.member, filter.values)[0]; - // [#5373] A null comparand is a NULLNESS test, not a comparison. SQL's - // `= NULL` is never true (and `!= NULL` never true either), so emitting - // one would move the very loss this issue is about from the mingo exit - // to this one: `{closed_at: null}` would compile to a WHERE that - // selects nothing while `query()` selects the two null rows. The two - // exits have to mean the same thing. - if (comparand == null && (filter.operator === 'equals' || filter.operator === 'notEquals')) { - whereClauses.push(`${fieldPath} IS ${filter.operator === 'notEquals' ? 'NOT ' : ''}NULL`); - } else { - whereClauses.push(`${fieldPath} ${sqlOp} ${this.toSqlLiteral(comparand)}`); - } - } - } + for (const filter of normalizedFilters) { + const fieldPath = this.resolveFieldPath(cube, filter.member); + whereClauses.push(this.sqlPredicateBuilder(filter.operator)({ + column: fieldPath, + comparands: this.comparandsFor(cube, filter.member, filter.values), + raw: filter.values, + literal: (value) => this.toSqlLiteral(value), + globSubstring: (value) => this.toSqlLiteral(globSubstringPattern(value)), + })); } let sql = `SELECT ${selectClauses.join(', ')} FROM ${tableName}`; @@ -815,8 +1000,9 @@ export class MemoryAnalyticsService implements IAnalyticsService { } /** - * Lower a Filter Protocol `$op` key to the cube-style operator name - * `convertOperatorToMongo` / `operatorToSql` accept. + * Lower a Filter Protocol `$op` key to the cube-style operator name both exits + * consume — {@link CUBE_OPERATOR_TO_MONGO_PREDICATE} and + * {@link CUBE_OPERATOR_TO_SQL_PREDICATE} are keyed by its result. * * [#5345] An operator with no row in {@link MONGO_TO_CUBE_OPERATOR} is * REFUSED, not skipped. The gate in `normalizeFilters` refuses the same set @@ -866,9 +1052,15 @@ export class MemoryAnalyticsService implements IAnalyticsService { * that justification was always sound for THIS half, and only wrong because * the in-memory half was forced to share it. * - * A `null` comparand never reaches here from `equals`/`notEquals`; the WHERE - * builder emits `IS NULL` / `IS NOT NULL` for those. `NULL` is the honest - * literal for the remaining operators, which cannot be satisfied by it. + * A `null` comparand never reaches here from `equals`/`notEquals`; those rows + * of {@link CUBE_OPERATOR_TO_SQL_PREDICATE} emit `IS NULL` / `IS NOT NULL`. + * `NULL` is the honest literal for the remaining operators, which cannot be + * satisfied by it. + * + * [#7117] It also renders the GLOB PATTERNS the text rows build, which is why + * the quote-doubling matters beyond comparands: a pattern is a string literal + * in the same statement, and `{name: {$contains: "o'brien"}}` has to survive + * as one. */ private toSqlLiteral(v: unknown): string { if (v == null) return 'NULL'; @@ -1003,25 +1195,29 @@ export class MemoryAnalyticsService implements IAnalyticsService { return build; } - private operatorToSql(operator: string): string { - const opMap: Record = { - 'equals': '=', - 'notEquals': '!=', - 'contains': 'LIKE', - 'notContains': 'NOT LIKE', - // [#6520] Needed because the `|| '='` fallback below is not a default, it - // is a wrong ANSWER: without this row `icontains` would render as `=`, an - // EQUALITY, in a statement offered to the author as a description of a - // containment query. `LIKE` is also the semantically right construct here - // — this exit emits SQLite-shaped SQL, and SQLite's `LIKE` folds ASCII - // only, which is exactly `$icontains`' domain (#4706 Q1 = A). - 'icontains': 'LIKE', - 'gt': '>', - 'gte': '>=', - 'lt': '<', - 'lte': '<=', - }; - return opMap[operator] || '='; + /** + * [#7117] The SQL predicate builder for one lowered operator — the twin of + * {@link MemoryAnalyticsService.mongoPredicateBuilder}, with the same totality + * floor and for the same reason. + * + * It replaced `operatorToSql`, whose `|| '='` fallback was not a default but a + * wrong ANSWER: three of this face's twelve operators (`in`, `notIn`, `set`) + * had no row and rendered as an EQUALITY in a statement offered to the author + * as a description of their query. The throw below cannot be reached from any + * input — {@link ANALYTICS_FILTER_CAPABILITIES} refuses everything outside the + * vocabulary, and the `Record` key type makes a missing row a + * type error first — so an arrival means our own two tables drifted. + */ + private sqlPredicateBuilder(operator: CubeOperator): SqlPredicateBuilder { + const build = (CUBE_OPERATOR_TO_SQL_PREDICATE as Record)[operator]; + if (!build) { + throw new Error( + `[driver-memory] analytics face: no SQL predicate for cube operator '${operator}'. ` + + `MONGO_TO_CUBE_OPERATOR and CUBE_OPERATOR_TO_SQL_PREDICATE have drifted — ` + + `add the missing builder rather than letting the operator render as an equality.`, + ); + } + return build; } private measureToSql(measure: { type: string; sql: string }): string { diff --git a/packages/drivers/driver-memory/src/memory-icontains.test.ts b/packages/drivers/driver-memory/src/memory-icontains.test.ts index 28e2f92fa7..981ba09199 100644 --- a/packages/drivers/driver-memory/src/memory-icontains.test.ts +++ b/packages/drivers/driver-memory/src/memory-icontains.test.ts @@ -24,13 +24,18 @@ * ## Why this file drives the ROWS and spells its own cases * * `check-driver-conformance.mjs` judges coverage by whether a package names the - * shared text case-set's marker export. Naming it here would flip this driver's - * cell to "covered" while requirement 2 of that case-set (the `$contains` family - * folding Unicode on this driver) is still open — and a ledger entry for a - * covered cell fails the gate's RECONCILED invariant. So this file drives - * `FILTER_TEXT_ROWS`, the fixture, and writes out the `$icontains` cases it is - * entitled to answer. The DEBT row stays until #6682 closes the other half; see - * that row's `why`, which now names one open requirement instead of two. + * shared text case-set's marker export. Naming it here would have flipped this + * driver's cell to "covered" while requirement 2 of that case-set — the + * `$contains` family folding Unicode on this driver — was still open, and a + * ledger entry for a covered cell fails the gate's RECONCILED invariant. So + * this file drives `FILTER_TEXT_ROWS`, the fixture, and writes out the + * `$icontains` cases it was entitled to answer. + * + * [#7723] That other half is now closed: `filterSubstringPattern` no longer + * carries the `i` flag, `memory-filter-text-conformance.test.ts` names the + * marker and enrolls the cell, and the DEBT row is gone. This file keeps its + * own cases — the shared case-set enrolls the FAMILY, not this operator's ASCII + * boundary — but it no longer stands in for a cell nothing else covers. */ import { describe, it, expect, beforeEach } from 'vitest'; @@ -114,9 +119,10 @@ describe('[#6520] $icontains — the accept table, on every face', () => { /** * `$icontains` is the case-INSENSITIVE twin, so its case-EXACT sibling must * NOT have moved. This is the row that would catch an implementation that - * "fixed" the fold by making `$contains` insensitive too — and note the - * matcher is the face that answers `$contains` case-exactly today (the query - * path still folds Unicode there, which is #6682, not this PR). + * "fixed" the fold by making `$contains` insensitive too. When it was written + * the matcher was the ONLY face answering `$contains` case-exactly, the query + * path still folding Unicode (#6682); since #7723 all three agree, so the + * choice of face here is no longer load-bearing. */ it('leaves $contains case-SENSITIVE on the reference matcher', () => { expect(matcherIds({ name: { $contains: 'acme' } })).toEqual(['2']); diff --git a/packages/drivers/driver-memory/src/memory-like-pattern.test.ts b/packages/drivers/driver-memory/src/memory-like-pattern.test.ts index 0d973a70d1..2156da9642 100644 --- a/packages/drivers/driver-memory/src/memory-like-pattern.test.ts +++ b/packages/drivers/driver-memory/src/memory-like-pattern.test.ts @@ -111,16 +111,17 @@ describe('[#7536] driver-memory — $like / $ilike on both faces', () => { // The control: `$contains` must NOT have moved. // // The comparand is `Ltd` rather than the `Industries` every other row uses, - // and that is deliberate. Measured on this branch: `{ $contains: - // 'Industries' }` answers ['1','2','3','4'] on the QUERY path and - // ['1','2','3'] on the reference matcher, because the query path lowers - // `$contains` to a RegExp carrying the `i` flag while the matcher uses - // `String.includes`. That is this package's KNOWN open divergence — the - // #6682 DEBT row the driver-conformance ledger carries for exactly this - // pair of faces — and it predates #7536 by a long way. Steering the control - // around it keeps this file measuring what it is about; pinning either - // answer here would enshrine a defect or fail for a reason unrelated to - // `$like`. + // and it was steered there deliberately: when this file was written the two + // faces disagreed on `{ $contains: 'Industries' }` — ['1','2','3','4'] on + // the QUERY path against ['1','2','3'] on the reference matcher — because + // the query path lowered `$contains` to a RegExp carrying the `i` flag + // while the matcher used `String.includes`. That was #6682's open half, the + // last DEBT row the driver-conformance ledger carried for this pair of + // faces. #7723 closed it: `filterSubstringPattern` no longer sets the flag, + // both faces answer ['1','2','3'] (measured), and the ledger is at zero. + // The comparand stays `Ltd` because it is a perfectly good control and + // moving it would change what this row measures, not because it still has + // to dodge anything. ['the $contains control still matches substrings', { name: { $contains: 'Ltd' } }, ['2']], ]; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 10b9bb20d8..3fae26feef 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -861,6 +861,12 @@ importers: '@types/node': specifier: ^26.1.2 version: 26.1.2 + '@types/sql.js': + specifier: ^1.4.11 + version: 1.4.11 + sql.js: + specifier: ^1.14.1 + version: 1.14.1 typescript: specifier: ^6.0.3 version: 6.0.3