diff --git a/.changeset/text-operator-case-folding-is-contractual.md b/.changeset/text-operator-case-folding-is-contractual.md new file mode 100644 index 0000000000..bb4ed817bc --- /dev/null +++ b/.changeset/text-operator-case-folding-is-contractual.md @@ -0,0 +1,86 @@ +--- +"@objectstack/driver-sql": minor +"@objectstack/driver-sqlite-wasm": minor +"@objectstack/driver-turso": minor +--- + +fix(drivers): text-operator case folding is the CONTRACT's answer, not the dialect's (#6518) + +The `$contains` family and `$icontains` returned **different rows on different +databases** for the same filter, because case sensitivity was decided by whatever +`LIKE` happened to mean on the dialect underneath. Both directions **over-matched** +— they returned rows the filter excludes, which on an ADR-0021 RLS read scope is +over-reach rather than a loose filter (#3948): + +| | `$contains` / `$notContains` / `$startsWith` / `$endsWith` — case-SENSITIVE (#4706 Q2 = A) | `$icontains` — folds ASCII ONLY (#4706 Q1 = A) | +|:--|:--|:--| +| SQLite / turso / sqlite-wasm | ❌ `LIKE` folds ASCII | ✅ `lower()` is ASCII-only | +| Postgres | ✅ `LIKE` is case-exact | ❌ `LOWER()` folds all of Unicode | +| MySQL | ❌ follows the column's collation | ❌ `LOWER()` folds all of Unicode | + +Read across: **each dialect was already right on the half another one got wrong**, +which is why neither half could be found from one backend alone. + +## What now runs + +The construct is chosen per dialect, in one emitter, so the escaping and the fold +stay a single code path (an unescaped wildcard is a filter bypass, P0 — #5567): + +- **SQLite family → `GLOB`.** `LIKE`'s ASCII fold cannot be switched off per + statement (`PRAGMA case_sensitive_like` is connection-global, so one query would + redefine every other query on the connection), and `CAST(col AS BLOB) LIKE ?` was + measured to match *nothing at all*. `GLOB` is case-exact and brings its own + escaped class — `*`, `?`, `[` as the self-closing classes `[*]`, `[?]`, `[[]`, + because SQLite's grammar gives `GLOB` no `ESCAPE` clause. `$icontains` keeps + `lower()` on both operands, still ASCII-only. +- **Postgres → `LIKE`, unchanged.** Only the fold moved, from `LOWER()` to an + explicit `translate()` over the 26 ASCII letters. Measured on a live PostgreSQL + 16 (ICU database): `LOWER('CAFÉ')` is `'café'` — the over-fold — while the + `translate()` form leaves `É` alone. +- **MySQL → `LIKE` over `CAST(… AS BINARY)`**, so the comparison is byte-wise and + no collation decides the case; `$icontains` folds byte-wise over the same binary + rendering, which is ASCII-only because UTF-8 is self-synchronising. +- **Any other client** keeps the previous `LIKE` / `LOWER()` shape — it is the only + form that still runs there — and is recorded as residue rather than left to be + discovered. + +`driver-turso`'s remote transport carries the twin (it compiles filters itself and +inherits nothing), and the two transports are now held to the same rows by a +parity suite that runs the shared `FILTER_TEXT_CASES` on both. + +## Behaviour change — read this before upgrading + +A filter whose comparand's case did not match the stored text used to match on +SQLite/turso/sqlite-wasm and may have matched on MySQL. It no longer does: + +```ts +// rows: { id: '1', name: 'ACME Corp' }, { id: '2', name: 'acme corp' } +{ name: { $contains: 'acme' } } // was ['1','2'] on SQLite → now ['2'] everywhere +{ name: { $icontains: 'acme' } } // ['1','2'] — unchanged, and now correct on PG/MySQL too +{ name: { $icontains: 'café' } } // was ['3','4'] on PG/MySQL → now ['4'] everywhere +``` + +If you were relying on `$contains` to ignore case, **write `$icontains`** — that is +the operator for it, and it now folds the same ASCII-only range on every backend. +Result sets only ever get NARROWER, never wider, so a filter that was already +correct stays correct. + +## Why `minor` rather than `major` + +No declared surface moves. `$contains` still exists, still takes the same +comparand, and `filter.zod.ts` is untouched — the case-sensitivity this delivers +was **already published** as the contract by #5701 (`FILTER_TEXT_CASES`, one +release earlier in this same v17 major), and the drivers were the half that had +not caught up. This is Prime Directive #12 applied in the direction it points: +declared = enforced. It is graded the way its sibling #5702/#6549 was graded for +the same operator family in the same rc cycle, and it registers nothing in the +ADR-0087 registries because it retires no authorable key. + +## What is deliberately NOT in this change + +`driver-memory` and `driver-mongodb` still fold case on their query paths — they +are the #5499 frozen family, so their `FILTER_TEXT_CASES` cells stay honest DEBT +and are tracked as #6682 (case sensitivity) and #6520 (`$icontains`). The +`service-analytics` SQL compilers were measured already compliant: they emit +Postgres-shaped statements, where `LIKE` is case-exact, and that assumption is now +written down and pinned rather than implied. diff --git a/packages/drivers/driver-sql/src/sql-driver-icontains-and-retired-operators.test.ts b/packages/drivers/driver-sql/src/sql-driver-icontains-and-retired-operators.test.ts index b473fd7603..9f63dc14a5 100644 --- a/packages/drivers/driver-sql/src/sql-driver-icontains-and-retired-operators.test.ts +++ b/packages/drivers/driver-sql/src/sql-driver-icontains-and-retired-operators.test.ts @@ -2,44 +2,50 @@ /** * [#5702] `$icontains` on the SQL family, and the retirement of `$regex` / - * `$options` — the DRIVER half of the #4706 ruling (its contract half is #5701). + * `$options` — the DRIVER half of the #4706 ruling (its contract half is #5701) + * — plus [#6518] the case-sensitivity half #5702 could not deliver. * - * ## What this pins, and why it is not the shared text case-set + * ## This cell now answers the WHOLE shared case-set * - * `@objectstack/spec/data` carries a canonical text case-set whose rows this - * file reuses (`FILTER_TEXT_ROWS` — the same nine, so a verdict here is - * comparable to one anywhere else). It deliberately does NOT import that - * case-set's CASES export, because `scripts/check-driver-conformance.mjs` - * judges a cell covered by that import and this driver does not yet answer the - * whole table: five of its cases require the `$contains` family to be - * case-SENSITIVE (#4706 Q2 = A), which SQLite's `LIKE` is not, and which cannot - * be fixed in this driver alone — `read-scope-sql` and `service-analytics` - * compile the same predicate for RLS and for the analytics face, so a - * driver-only change would give ONE permission rule two row sets (#3948). That - * work is filed separately and the driver's DEBT row stays open for it. + * `FILTER_TEXT_CASES` is imported and executed below, which is what + * `scripts/check-driver-conformance.mjs` reads as coverage. #5702 deliberately + * imported only `FILTER_TEXT_ROWS` and left the CASES export alone, because + * five of its cases require the `$contains` family to be case-SENSITIVE (#4706 + * Q2 = A) and SQLite's `LIKE` folds ASCII — an import then would have flipped + * the cell to "covered" while five cases went unanswered, which is a gate + * reporting success over a standard nobody runs. * - * Importing the case-set here would flip the cell to "covered" while five of - * its cases were unanswered — a gate reporting success over a standard nobody - * runs, which is exactly the failure the gate exists to prevent. + * #6518 is what makes the import honest: `textMatchPredicate` emits `GLOB` on + * the SQLite dialects (case-exact, carrying GLOB's own `[*]` / `[?]` / `[[]` + * escapes because SQLite's grammar gives it no `ESCAPE` clause), keeps `LIKE` + * on Postgres (already case-exact) with the `$icontains` fold moved off the + * Unicode-folding `LOWER()` onto an ASCII-only `translate()`, and compares over + * `CAST(… AS BINARY)` on MySQL. The per-dialect reasoning and the measurements + * behind each cell live on that function. * * ## The reverse verification, direction decided BEFORE it was run * * - **Refusal face** — predicted RED, measured RED. Restoring the deleted * `case '$regex':` fallthrough makes every assertion below that reads `code` * / `status` fail, because the filter compiles again and nothing throws. - * - **`$icontains` face on SQLite** — predicted red, and the prediction needed - * a correction that is recorded here rather than smoothed over: deleting the - * `case '$icontains':` arm turns these cases red LOUDLY (the operator falls to - * `default:` and is refused), but deleting only the `LOWER()` fold does NOT, - * for any comparand. SQLite's `LIKE` folds ASCII by itself, so on this - * dialect the fold is unobservable in rows. The compiled-SQL case at the end - * is what pins it, and it is the only thing here that can. + * - **`$icontains` face on SQLite** — #5702 recorded that deleting only the + * fold changed NO row here, because `LIKE` folded ASCII by itself, so the + * compiled-SQL case was the only witness it could offer. #6518 retires that + * caveat, and the retirement was predicted before it was run: under `GLOB` + * the fold is load-bearing in ROWS. Measured — dropping the column-side + * `lower()` turns `$icontains: 'acme'` and `$icontains: 'ACME'` alike from + * `['1','2']` into `['2']`, and `$icontains: 'CAFÉ'` from `['3']` into `[]`. + * - **`$contains` case-sensitivity** — predicted RED on exactly the five case + * rows before the change, measured RED: reverting the sqlite arm of + * `textMatchPredicate` to `LIKE` fails those five case rows and NO others in + * the table (10 reds in this file: the five, plus the five blocks below that + * name the construct directly). */ import type { Knex } from 'knex'; import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import type { DriverOptions, FilterCondition } from '@objectstack/spec/data'; -import { FILTER_TEXT_ROWS } from '@objectstack/spec/data'; +import { FILTER_TEXT_CASES, FILTER_TEXT_ROWS } from '@objectstack/spec/data'; import { SqlDriver } from './sql-driver.js'; /** The error a refused filter produced — never a bare `toThrow()` (see below). */ @@ -206,26 +212,105 @@ describe('[#5702] SqlDriver — $icontains, and the retired $regex/$options', () await expect(driver.find('txt', { where: { name: { $regex: 'zzz' } } }, BYPASS)).rejects.toThrow(); }); - // ── The fold, where it is actually observable on SQLite ──────────────────── + // ── [#6518] The case-SENSITIVE family (#4706 Q2 = A) ─────────────────────── - it('compiles LOWER() on both operands, and $contains on neither', async () => { - // On SQLite this is the ONLY witness to the fold: `LIKE` already folds ASCII - // here, so `$contains` and `$icontains` select identical rows for every - // comparand and a dropped `LOWER()` changes no answer. It changes the SQL, - // and it changes the answer on Postgres (whose LIKE is case-exact), so the - // statement is what has to be pinned. + it('$contains is case-SENSITIVE in both directions', async () => { + // The defect this closes: on `origin/main` both lines answered ['1','2'], + // because SQLite's `LIKE` folds ASCII whatever the contract says. Over- + // matching, not a near miss — the caller asked for one row and got two. + expect(await ids({ name: { $contains: 'acme' } })).toEqual(['2']); + expect(await ids({ name: { $contains: 'ACME' } })).toEqual(['1']); + }); + + it('the case rule holds under negation too', async () => { + // Row 1 is EXCLUDED from the negation only if the positive form excluded + // it. A folding backend drops row 1 here and looks "stricter", which is the + // reading that hides the defect. + expect(await ids({ name: { $notContains: 'acme' } })) + .toEqual(['1', '3', '4', '5', '6', '7', '8', '9']); + }); + + it('$startsWith and $endsWith are case-SENSITIVE', async () => { + expect(await ids({ name: { $startsWith: 'ACME' } })).toEqual(['1']); + expect(await ids({ name: { $endsWith: 'corp' } })).toEqual(['2']); + }); + + it('$contains and $icontains no longer answer identically on SQLite', async () => { + // #5702 measured that they DID, for every comparand, and recorded it as the + // reason `$icontains` was unobservable on this dialect. That is the fact + // #6518 changes, so it is asserted rather than left as a comment: the two + // operators are now distinguishable in rows on the dialect where they were + // not. + expect(await ids({ name: { $contains: 'acme' } })).not + .toEqual(await ids({ name: { $icontains: 'acme' } })); + }); + + // ── [#6518] GLOB's metacharacters are escaped, exactly as LIKE's were ────── + + for (const [label, comparand] of [ + ['*', 'a*b'], + ['?', 'a?b'], + ['[', 'a[b'], + ] as const) { + it(`treats "${label}" as a literal character, not a GLOB metacharacter`, async () => { + // The new operator brings a NEW escaped character class, and an + // unescaped `*` is the same filter bypass an unescaped `%` is under LIKE: + // measured on this fixture, the unescaped pattern `*a*b*` returns rows + // 7, 8 and 9 where the escaped one returns none of them. No fixture row + // holds these characters, so the observable claim is that the pattern + // stays literal and selects nothing rather than expanding. + expect(await ids({ name: { $contains: comparand } })).toEqual([]); + expect(await ids({ name: { $icontains: comparand } })).toEqual([]); + }); + } + + it('compiles the case-exact SQLite construct, on both operators', async () => { // Identifier quoting is the dialect's (knex renders backticks on the sqlite - // clients), so the assertion is on the SHAPE, not on one dialect's quotes. - const unquote = (sql: string) => sql.replace(/[`"\[\]]/g, ''); + // clients), so the assertion is on the SHAPE. `[` / `]` are NOT stripped + // here the way #5702's version stripped them: under GLOB they are the + // escape mechanism, so erasing them would erase what is being pinned. + const unquote = (sql: string) => sql.replace(/[`"]/g, ''); const icontainsSql = unquote(driver.compileWhere({ name: { $icontains: 'acme' } })); - expect(icontainsSql).toContain('LOWER(name) LIKE LOWER('); - expect(icontainsSql).toContain('ESCAPE'); - // The escaped pattern still travels as the comparand, wildcards and all. - expect(icontainsSql).toContain('%acme%'); + expect(icontainsSql).toContain('lower(name) GLOB lower('); + expect(icontainsSql).toContain('*acme*'); + // GLOB has no ESCAPE clause in SQLite's grammar; emitting one is a syntax + // error, so its absence is part of the construct rather than a detail. + expect(icontainsSql).not.toContain('ESCAPE'); const containsSql = unquote(driver.compileWhere({ name: { $contains: 'acme' } })); - expect(containsSql).toContain('name LIKE'); - expect(containsSql).not.toContain('LOWER'); + expect(containsSql).toContain('name GLOB'); + expect(containsSql).not.toContain('lower'); + expect(containsSql).not.toContain('LIKE'); + }); + + // ── The shared standard, executed ────────────────────────────────────────── + + /** + * [#6518] Every case in `FILTER_TEXT_CASES`, on the live SQLite cell. + * + * The invariant this enforces is the case-set's own: a backend answers with + * the SAME ROW SET, or REFUSES with `INVALID_FILTER` — never a third, quieter + * answer. So the rejection cases assert `code` AND `status` (ADR-0112) plus + * the prescription the message must carry, and never a bare `toThrow()`. + * + * The blocks above are not redundant with this loop: they name the SQLite + * construct and its escaped class, which the shared table deliberately does + * not know about — it encodes the contract, not any dialect's spelling. + */ + describe('FILTER_TEXT_CASES — the shared standard', () => { + for (const testCase of FILTER_TEXT_CASES) { + it(testCase.name, async () => { + if (testCase.expectRejection) { + const err = await refusalOf(testCase.filter); + expect(err.code).toBe(testCase.code); + expect(err.status).toBe(400); + for (const mention of testCase.mustMention) expect(err.message).toContain(mention); + return; + } + expect(await ids(testCase.filter), testCase.note ?? testCase.name) + .toEqual([...testCase.expected]); + }); + } }); }); diff --git a/packages/drivers/driver-sql/src/sql-driver-text-case-conformance.test.ts b/packages/drivers/driver-sql/src/sql-driver-text-case-conformance.test.ts new file mode 100644 index 0000000000..3c23a90a4d --- /dev/null +++ b/packages/drivers/driver-sql/src/sql-driver-text-case-conformance.test.ts @@ -0,0 +1,254 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#6518] `FILTER_TEXT_CASES` across the DRIVER axis — the case-folding half of + * the #4706 ruling, executed on every dialect this driver speaks. + * + * # Why this needs the live matrix and cannot be decided on SQLite + * + * Case folding is the one axis where the three dialects disagree by + * construction, so an in-process SQLite run answers at most a third of the + * question: + * + * | | `$contains` family (case-SENSITIVE, Q2=A) | `$icontains` (ASCII-only fold, Q1=A) | + * |---|---|---| + * | SQLite | `LIKE` folded ASCII — the defect | `lower()` is ASCII-only — already right | + * | Postgres | `LIKE` is case-exact — already right | `LOWER()` folds all of Unicode — the defect | + * | MySQL | follows the collation | `LOWER()` folds all of Unicode — the defect | + * + * Read across: **each dialect was already correct on the half the other one got + * wrong.** A suite that ran only on SQLite would go green on the `$icontains` + * rows for a reason that does not transfer (SQLite's `lower()` simply cannot + * reach `É`), and a suite that ran only on Postgres would go green on the + * `$contains` rows for a reason that does not transfer either. The ASCII-only + * boundary — rows 3 and 4, `CAFÉ` / `café` — is decidable ONLY where `LOWER()` + * would have folded it, i.e. on a live Postgres or MySQL. That is why the issue + * put "the `$icontains` ASCII-only boundary holds on live PG / MySQL" in its + * acceptance rather than leaving it to the embedded cell. + * + * # Which cells actually executed, and which did not + * + * Recorded here rather than implied, because a matrix that reports OK while + * finding zero live cells is the failure `declareUnprovisionedCell` exists for + * (#4646): + * + * - **sqlite** — always runs, embedded. + * - **live postgres** — RUN, on PostgreSQL 16.13, against a database created + * with the ICU locale provider (the configuration in which `LOWER('CAFÉ')` + * really is `'café'`, so the over-fold this closes was reproduced before the + * fix and is absent after it). + * - **live mysql** — NOT run: no MySQL server was provisionable in the + * container this landed from, so it is a declared SKIP, and the MySQL arm of + * `textMatchPredicate` rests on documented behaviour plus the compiled-SQL + * pin below rather than on execution. Saying so is the point — the cell is + * skipped by name, and `OS_EXPECT_LIVE_DIALECT_MATRIX=1` turns that skip + * into a failure for a runner that believes it provisioned one. + * + * # Reverse verification, direction predicted BEFORE it was run + * + * Two experiments, each reverting ONE arm and each predicted to fail a DIFFERENT + * pair of cells — which is the claim that the two halves of this issue are + * genuinely independent rather than one defect seen twice: + * + * - Revert the sqlite arm to `LIKE`. Predicted: the five case-sensitivity + * rows go red on the sqlite cell and nothing goes red on postgres. + * Measured: exactly that. + * - Revert the postgres fold from `translate()` to `LOWER()`. Predicted: the + * two ASCII-only rows go red on the LIVE POSTGRES cell and nothing goes red + * on sqlite. Measured: exactly that — `$icontains: 'café'` answered + * `['3','4']` where the case-set demands `['4']`, and its `'CAFÉ'` mirror + * answered `['3','4']` where it demands `['3']`. That is the over-fold, on + * a real server, before and after. + * + * # The compiled-SQL layer, and why it is not redundant with the rows + * + * The last block asserts the SHAPE each dialect compiles to. On the cells that + * run, rows are the stronger witness and the shape is a bonus. On the cell that + * does NOT run, the shape is the only thing this repo can check at all — so it + * is checked for all three dialects from the in-process cell, where knex will + * build a Postgres or MySQL statement without needing a server to send it to. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import type { Knex } from 'knex'; +import type { DriverOptions, FilterCondition } from '@objectstack/spec/data'; +import { FILTER_TEXT_CASES, FILTER_TEXT_ROWS } from '@objectstack/spec/data'; +import { SqlDriver, type SqlDriverConfig } from './sql-driver.js'; +import { + DIALECT_CELLS, + declareUnprovisionedCell, + type DialectCell, +} from './live-dialect-matrix.testkit.js'; + +/** + * Issue-prefixed object name: the live cells share one database with every other + * suite in this package, so a bare `text_conformance` would be a collision + * waiting to be read as a case-folding regression. + */ +const TEXT_OBJECT = 'os6518_text_case'; + +/** Diagnostics-only; it never changes which rows a read touches. */ +const BYPASS: DriverOptions = { bypassTenantAudit: true }; + +interface WireBearingError extends Error { + code?: string; + status?: number; +} + +for (const cell of DIALECT_CELLS) { + if (!cell.available) { + declareUnprovisionedCell(cell, 'FILTER_TEXT_CASES case folding'); + continue; + } + declareTextCaseSweep(cell); +} + +function declareTextCaseSweep(cell: DialectCell): void { + describe(`SqlDriver FILTER_TEXT_CASES (${cell.label})`, () => { + let driver: SqlDriver; + let knexInstance: Knex; + + beforeAll(async () => { + driver = new SqlDriver(cell.config()); + // `getKnex()` is public API; the sibling matrices reach the `protected` + // field through `(driver as any).knex`, which erases every member of the + // driver to get one. Nothing here needs that. + knexInstance = driver.getKnex(); + await knexInstance.schema.dropTableIfExists(TEXT_OBJECT); + await driver.initObjects([{ name: TEXT_OBJECT, fields: { name: { type: 'string' } } }]); + for (const row of FILTER_TEXT_ROWS) { + await driver.create(TEXT_OBJECT, { ...row }, BYPASS); + } + }); + + afterAll(async () => { + await knexInstance?.schema.dropTableIfExists(TEXT_OBJECT).catch(() => {}); + await driver?.disconnect?.(); + }); + + const ids = async (where: FilterCondition): Promise => { + const rows = await driver.find(TEXT_OBJECT, { where }, BYPASS); + return rows.map((r) => String(r.id)).sort((a, b) => a.localeCompare(b)); + }; + + /** + * The fixture control. Rows 1/2 and 3/4 differ ONLY in case, so a store or + * a transport that normalised case on the way IN would turn most cases + * below green for a reason that has nothing to do with the compiler — and + * would turn the ASCII-only pair green while proving nothing. + */ + it('stored all nine rows with their case intact', async () => { + const rows = await driver.find(TEXT_OBJECT, {}, BYPASS); + const byId = new Map(rows.map((r) => [String(r.id), String(r.name)])); + expect([...byId.keys()].sort()).toEqual(['1', '2', '3', '4', '5', '6', '7', '8', '9']); + expect(byId.get('1')).toBe('ACME Corp'); + expect(byId.get('2')).toBe('acme corp'); + expect(byId.get('3')).toBe('CAFÉ'); + expect(byId.get('4')).toBe('café'); + }); + + for (const testCase of FILTER_TEXT_CASES) { + it(testCase.name, async () => { + if (testCase.expectRejection) { + const err = await driver + .find(TEXT_OBJECT, { where: testCase.filter }, BYPASS) + .then(() => null, (e: unknown) => e as WireBearingError); + expect(err, 'the case-set requires this filter REFUSED, not answered').toBeInstanceOf(Error); + // `code` AND `status`: a refusal outside the ADR-0112 envelope reaches + // the client as a 500-shaped body for a 400-class mistake. + expect(err!.code).toBe(testCase.code); + expect(err!.status).toBe(400); + for (const mention of testCase.mustMention) expect(err!.message).toContain(mention); + return; + } + expect(await ids(testCase.filter), testCase.note ?? testCase.name) + .toEqual([...testCase.expected]); + }); + } + }); +} + +/** + * The construct each dialect compiles to, decided WITHOUT a server. + * + * knex builds a statement for whichever client it was configured with, and + * `.toString()` renders it — so the mysql shape is checkable here even though no + * MySQL server ran. This is the layer that keeps the un-provisioned cell from + * being entirely unverified, and it is deliberately about the SHAPE (which + * operator, which fold, whether an `ESCAPE` clause exists) rather than about + * exact whitespace. + */ +describe('[#6518] the per-dialect construct, compiled', () => { + /** A driver that exposes the compiled WHERE without reaching into privates. */ + class CompilerProbeDriver extends SqlDriver { + compileWhere(where: FilterCondition): string { + const builder: Knex.QueryBuilder = this.getKnex()(TEXT_OBJECT); + this.applyFilters(builder, where); + return builder.toString(); + } + } + + const probe = (config: SqlDriverConfig) => new CompilerProbeDriver(config); + + it('sqlite: GLOB, case-exact, with no ESCAPE clause', () => { + const d = probe({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }); + const contains = d.compileWhere({ name: { $contains: 'acme' } }); + expect(contains).toMatch(/GLOB/); + expect(contains).not.toMatch(/LIKE|ESCAPE|lower\(/); + const icontains = d.compileWhere({ name: { $icontains: 'acme' } }); + expect(icontains).toMatch(/lower\(.*\)\s+GLOB\s+lower\(/); + expect(icontains).not.toMatch(/ESCAPE/); + }); + + it('postgres: LIKE unchanged, and the fold is translate() — never LOWER()', () => { + const d = probe({ client: 'pg', connection: { host: '127.0.0.1' } }); + const contains = d.compileWhere({ name: { $contains: 'acme' } }); + expect(contains).toMatch(/LIKE/); + expect(contains).toMatch(/ESCAPE/); + expect(contains).not.toMatch(/translate|LOWER|GLOB/); + + const icontains = d.compileWhere({ name: { $icontains: 'acme' } }); + // BOTH operands folded — folding only the comparand compares a folded + // needle against a raw column and silently matches only the already-lower + // rows. And `LOWER()` must not reappear: on Postgres it folds `É`, which is + // the exact over-fold #4706 Q1 = A rules out. + expect(icontains.match(/translate\(/g) ?? []).toHaveLength(2); + expect(icontains).not.toMatch(/LOWER\(/); + expect(icontains).toContain('ABCDEFGHIJKLMNOPQRSTUVWXYZ'); + expect(icontains).toContain('abcdefghijklmnopqrstuvwxyz'); + }); + + it('mysql: the comparison is BINARY, so no collation decides the case', () => { + const d = probe({ client: 'mysql2', connection: { host: '127.0.0.1' } }); + const contains = d.compileWhere({ name: { $contains: 'acme' } }); + // Two casts, one per operand: MySQL would coerce the second anyway, but a + // shape that relies on coercion is a shape a future edit can quietly break. + expect(contains.match(/CAST\(/g) ?? []).toHaveLength(2); + expect(contains).toMatch(/AS BINARY/); + expect(contains).toMatch(/LIKE/); + expect(contains).not.toMatch(/LOWER\(|REPLACE\(/); + + const icontains = d.compileWhere({ name: { $icontains: 'acme' } }); + // 26 REPLACEs per operand — the ASCII case map, applied byte-wise over the + // binary rendering so that no collation participates and no multi-byte + // character can be touched (UTF-8 is self-synchronising). + expect(icontains.match(/REPLACE\(/g) ?? []).toHaveLength(52); + expect(icontains).not.toMatch(/LOWER\(/); + }); + + it('an unmodelled dialect keeps the pre-#6518 shape rather than emitting invalid SQL', () => { + // `dialectName` is `'unknown'` for a client this driver does not model, and + // there `GLOB` is a syntax error and `CAST(… AS BINARY)` means something + // else. Answering with the old `LIKE`/`LOWER()` shape is not an endorsement + // of it — it is the only answer that still RUNS, and it is named as residue + // in the driver-conformance ledger rather than left to be discovered. + // + // `mssql` is the stand-in because knex resolves its driver (`tedious`) from + // this workspace without a server; it is not a supported dialect and this + // case makes no claim that it is. Any client outside `isSqlite` / + // `isPostgres` / `isMysql` takes the same arm. + const d = probe({ client: 'mssql', connection: {} }); + expect(d.compileWhere({ name: { $contains: 'acme' } })).toMatch(/LIKE/); + expect(d.compileWhere({ name: { $icontains: 'acme' } })).toMatch(/LOWER\(/); + }); +}); diff --git a/packages/drivers/driver-sql/src/sql-driver.ts b/packages/drivers/driver-sql/src/sql-driver.ts index b5d6663912..be8b1d91ce 100644 --- a/packages/drivers/driver-sql/src/sql-driver.ts +++ b/packages/drivers/driver-sql/src/sql-driver.ts @@ -1072,6 +1072,200 @@ function safeShapePreview(value: unknown): string { } } +/** + * Where the wildcard sits relative to the comparand. Named exactly as + * `service-analytics`'s `LikeShape` names its own, so the twin implementations + * read alike: `contains` → `%v%`, `starts` → `v%`, `ends` → `%v`. + */ +type TextMatchShape = 'contains' | 'starts' | 'ends'; + +/** + * The ASCII case map, written out as data. + * + * [#6518] Spelled as two 26-character constants rather than reached through a + * locale-aware `LOWER()` because "ASCII only" is the CONTRACT (#4706 Q1 = A), + * and a locale can be configured to fold more. Measured on a live Postgres 16 + * (both a `C.utf8` database and an ICU one): `lower('CAFÉ')` is `café` — the + * over-fold — while `translate('CAFÉ', , )` is `cafÉ`. The + * mapping being visible in the emitted SQL is the point: a reviewer can see + * that exactly 26 characters fold, without knowing the server's locale. + */ +const ASCII_UPPER_LETTERS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'; +const ASCII_LOWER_LETTERS = 'abcdefghijklmnopqrstuvwxyz'; + +/** The character bound into every `ESCAPE` clause this driver emits. */ +const LIKE_ESCAPE_CHARACTER = '\\'; + +/** + * Escape the LIKE metacharacters (`%`, `_`) and the escape character itself + * (`\`) so a comparand matches literally. + * + * This is the expression `service-analytics`'s `escapeLikePattern` is held to + * character for character by its `like-metacharacter-escape.test.ts`. Changing + * it here without changing it there forks one `$contains` into two. + */ +function escapeLikeComparand(value: unknown): string { + return String(value).replace(/[\\%_]/g, '\\$&'); +} + +/** + * [#6518] Escape the GLOB metacharacters (`*`, `?`, `[`) so a comparand matches + * literally, using GLOB's ONLY escape mechanism: a single-character class. + * + * GLOB has no `ESCAPE` clause — SQLite's grammar simply does not have one for + * it — so `[*]`, `[?]` and `[[]` are how a literal metacharacter is spelled. + * `]` needs no escape and deliberately gets none: every `[` this function sees + * is turned into a class that closes itself, so no unclosed class can survive + * for a later `]` to terminate. `%` and `_` are ordinary characters to GLOB and + * are likewise left alone — the escaped class here is NOT the LIKE one, and + * writing the two as one shared regex is the mistake to refuse. + * + * Measured before it was written (better-sqlite3 3.53.4, the nine-row + * `FILTER_TEXT_ROWS` fixture plus `a*b` / `a?b` / `a[b`): the unescaped pattern + * `*a*b*` returned six rows where `*a[*]b*` returns the one. An unescaped `*` + * is the same filter bypass an unescaped `%` is under LIKE, which is why this + * function exists at the same level as its LIKE sibling rather than inline. + */ +function escapeGlobComparand(value: unknown): string { + return String(value).replace(/[*?[]/g, '[$&]'); +} + +/** Wrap an already-escaped comparand in the wildcards `shape` calls for. */ +function wrapTextMatchShape(escaped: string, shape: TextMatchShape, wildcard: string): string { + if (shape === 'starts') return `${escaped}${wildcard}`; + if (shape === 'ends') return `${wildcard}${escaped}`; + return `${wildcard}${escaped}${wildcard}`; +} + +/** + * [#6518] MySQL's ASCII-only case fold: 26 `REPLACE`s over the BINARY rendering + * of an expression. + * + * Ugly, and the alternatives are all wrong rather than merely uglier — that is + * the whole justification, so it is written down: + * + * - `LOWER(x)` folds the full Unicode range (the defect this closes). + * - `LOWER(x)` on a binary string is documented as INEFFECTIVE, so casting + * first and folding after simply does not fold. + * - `CONVERT(x USING ascii)` maps every non-ASCII character to `?`, which + * COLLIDES `café` with `cafÉ` — strictly worse than over-folding. + * - No MySQL collation is case-insensitive for ASCII and exact elsewhere. + * + * Operating on `CAST(x AS BINARY)` rather than on the text is what makes it + * provable without a live server: in binary space `REPLACE` matches bytes, so + * no collation participates, and UTF-8 is self-synchronising — a byte in + * `0x41..0x5A` can only ever be a real ASCII `A`..`Z`, never part of a + * multi-byte character. Byte-wise ASCII lowering therefore IS the ruled fold. + */ +function mysqlAsciiLowerBinary(expr: string): string { + let out = `CAST(${expr} AS BINARY)`; + for (let i = 0; i < ASCII_UPPER_LETTERS.length; i++) { + out = `REPLACE(${out}, '${ASCII_UPPER_LETTERS[i]}', '${ASCII_LOWER_LETTERS[i]}')`; + } + return out; +} + +/** + * [#6518] The one place a text predicate becomes SQL — `{sql, bindings}` for + * knex's `whereRaw`, with `??` the column and `?` the pattern. + * + * # The defect this closes + * + * Before this, every dialect got `col LIKE ? ESCAPE ?` (and `LOWER()` on both + * sides for `$icontains`), which made case sensitivity the DIALECT's answer + * where #4706 rules it the CONTRACT's. Both halves over-matched — they returned + * rows the filter excludes, which on an RLS read scope is over-reach (#3948), + * not a loose filter: + * + * | | `$contains` family (must be case-SENSITIVE, Q2=A) | `$icontains` (folds ASCII ONLY, Q1=A) | + * |---|---|---| + * | SQLite | ✗ `LIKE` folds ASCII | ✓ `lower()` is ASCII-only | + * | Postgres | ✓ `LIKE` is case-exact | ✗ `LOWER()` folds all of Unicode | + * | MySQL | ✗ follows the collation | ✗ `LOWER()` folds all of Unicode | + * + * # What is emitted now, and why each cell + * + * - **SQLite → `GLOB`.** `LIKE`'s ASCII fold cannot be turned off per-statement; + * `PRAGMA case_sensitive_like` is a CONNECTION-global switch, so one query + * would change every other query's meaning. Of the operand-level tricks, + * `CAST(col AS BLOB) LIKE ?` was measured to return NOTHING at all (SQLite's + * LIKE is false for a BLOB operand), so the operator has to change. `GLOB` is + * case-exact by definition and carries its own escape mechanism + * ({@link escapeGlobComparand}). `lower()` in front of it is still the + * `$icontains` fold, and still ASCII-only: measured, `lower('CAFÉ')` is + * `'cafÉ'`, so `lower(name) GLOB '*café*'` answers row 4 and `'*cafÉ*'` + * answers row 3 — the Q1 = A boundary, executed rather than argued. + * - **Postgres → `LIKE`, unchanged**, because `LIKE` there is already exact. + * Only the fold moves, from `LOWER()` to {@link ASCII_UPPER_LETTERS}-driven + * `translate()`. Measured live (PG 16, ICU database): `LOWER(name) LIKE + * LOWER('%café%')` returned rows 3 AND 4; the `translate()` form returns row + * 4, and its `'%CAFÉ%'` mirror returns row 3. + * - **MySQL → `LIKE` over `CAST(… AS BINARY)`**, which is byte-wise and + * therefore case-exact whatever the column's collation says. The fold adds + * {@link mysqlAsciiLowerBinary} on top. NOT executed here: no MySQL server was + * provisionable in the container that wrote this, so the mysql cell is a + * declared skip in the live matrix rather than a claimed pass, and the + * reasoning is written out on that helper instead. + * - **`'unknown'` → the pre-#6518 `LIKE`/`LOWER()` shape.** `dialectName` is + * `'unknown'` for a knex client this driver does not model (mssql, oracle), + * where `GLOB` is a syntax error and `CAST(… AS BINARY)` means something + * else. Emitting the old shape is not an endorsement of it — it is the only + * answer that still RUNS, and it is the residue the conformance ledger names. + * + * # Why one function and not four emitters + * + * The escaping is the P0 (#5567: an unescaped `%` matches every row), the fold + * is the contract, and the two interact — the SQLite arm needs a DIFFERENT + * escaped character class from the other three, which is exactly the kind of + * divergence a second emitter drops on the floor. Every arm below therefore + * builds its pattern from one of two named escape functions and one shared + * {@link wrapTextMatchShape}, so "which characters are literal" is answered per + * dialect in one readable place and can never be answered by accident. + */ +function textMatchPredicate( + dialect: SqlDialectName, + field: string, + value: unknown, + shape: TextMatchShape, + negate: boolean, + fold: boolean, +): { sql: string; bindings: unknown[] } { + if (dialect === 'sqlite') { + // GLOB takes no ESCAPE clause, so this arm binds two values, not three. + const pattern = wrapTextMatchShape(escapeGlobComparand(value), shape, '*'); + const column = fold ? 'lower(??)' : '??'; + const comparand = fold ? 'lower(?)' : '?'; + return { + sql: `${column} ${negate ? 'NOT GLOB' : 'GLOB'} ${comparand}`, + bindings: [field, pattern], + }; + } + + const pattern = wrapTextMatchShape(escapeLikeComparand(value), shape, '%'); + const keyword = negate ? 'NOT LIKE' : 'LIKE'; + // The `ESCAPE` character is BOUND, never written as a literal: MySQL applies C + // escape syntax inside string literals, so `'\'` and `'\\'` are the same + // backslash spelled two ways per dialect, while a bound value has one + // spelling everywhere (#5567). + const bindings = [field, pattern, LIKE_ESCAPE_CHARACTER]; + + if (dialect === 'postgres') { + const asciiLower = (expr: string) => + fold ? `translate(${expr}, '${ASCII_UPPER_LETTERS}', '${ASCII_LOWER_LETTERS}')` : expr; + return { sql: `${asciiLower('??')} ${keyword} ${asciiLower('?')} ESCAPE ?`, bindings }; + } + + if (dialect === 'mysql') { + const caseExact = (expr: string) => + fold ? mysqlAsciiLowerBinary(expr) : `CAST(${expr} AS BINARY)`; + return { sql: `${caseExact('??')} ${keyword} ${caseExact('?')} ESCAPE ?`, bindings }; + } + + const column = fold ? 'LOWER(??)' : '??'; + const comparand = fold ? 'LOWER(?)' : '?'; + return { sql: `${column} ${keyword} ${comparand} ESCAPE ?`, bindings }; +} + /** * [#5134] What a filter node is worth as a boolean, before any SQL is emitted. * @@ -7174,23 +7368,31 @@ export class SqlDriver implements IDataDriver { } /** - * Parameterized `LIKE`/`NOT LIKE` match with the LIKE metacharacters `%` / `_` - * (and the escape char `\`) escaped in the user value so they match literally - * — otherwise a value of `%` matches every row (a filter-bypass, P0). Binds an - * explicit `ESCAPE '\'` because SQLite does not honour a default escape - * character (MySQL/Postgres do, but the explicit clause is correct for all - * three). `shape` positions the wildcard: `contains` → `%v%`, `starts` → `v%`, - * `ends` → `%v`. + * Parameterized text match for the `$contains` family and `$icontains`, with + * the comparand's metacharacters escaped so it matches LITERALLY — otherwise + * a value of `%` matches every row (a filter-bypass, P0). `shape` positions + * the wildcard: `contains` → `%v%`, `starts` → `v%`, `ends` → `%v`. + * + * **[#6518] The construct is chosen by DIALECT, and that is the whole point + * of this method's existence.** Everything about which SQL is emitted lives in + * {@link textMatchPredicate}; this method only picks `whereRaw` vs + * `orWhereRaw`. See that function for the per-dialect table and the measured + * evidence behind each cell — in one sentence: case sensitivity used to be + * the DIALECT's answer (SQLite's `LIKE` folds ASCII, Postgres's does not, + * MySQL's follows its collation) where #4706 Q2 = A says it is the + * CONTRACT's, and `LOWER()` folds the whole Unicode range on Postgres/MySQL + * where #4706 Q1 = A says `$icontains` folds ASCII only. * * **Second implementation, deliberately** (#5567): * `packages/services/service-analytics/src/like-pattern.ts` carries the same - * transform — same escaped character class, same three shapes, same bound + * LIKE transform — same escaped character class, same three shapes, same bound * `ESCAPE` — because `service-analytics` depends on no driver and this is a * private method taking a knex builder, so there is nothing for it to import. - * That file's header explains the choice; it is held to THIS expression, character - * for character, by `service-analytics`'s `like-metacharacter-escape.test.ts`. - * A third hand-copy is the thing to refuse: import from one of the two, or add - * a consumer to that test. + * That file's header explains the choice; it is held to the LIKE arm of + * {@link textMatchPredicate}, character for character, by + * `service-analytics`'s `like-metacharacter-escape.test.ts`. A third hand-copy + * is the thing to refuse: import from one of the two, or add a consumer to + * that test. * * **[#5234] `String(value)` is safe here because nothing unrenderable reaches * it.** {@link assertCompilableComparand} refuses an object comparand on this @@ -7207,31 +7409,13 @@ export class SqlDriver implements IDataDriver { method: string, field: string, value: unknown, - shape: 'contains' | 'starts' | 'ends', + shape: TextMatchShape, negate = false, fold = false, ): void { - const escaped = String(value).replace(/[\\%_]/g, '\\$&'); - const pattern = shape === 'starts' ? `${escaped}%` : shape === 'ends' ? `%${escaped}` : `%${escaped}%`; - const keyword = negate ? 'NOT LIKE' : 'LIKE'; const rawMethod = method.startsWith('or') ? 'orWhereRaw' : 'whereRaw'; - // [#5702] `fold` wraps BOTH operands in SQL `LOWER()` — the `$icontains` - // lowering. It is a parameter of this method rather than a second emitter - // so that the escaping above (the `%`/`_`/`\` class and the bound `ESCAPE`) - // is literally the same code, not a copy held in sync by a comment: an - // unescaped `%` is a filter bypass (P0), and the second `$icontains` face - // is exactly where a copy would have skipped it. - // - // `LOWER()` and not a JS-side fold: the column side has to fold too, and it - // can only fold in SQL. SQLite's `lower()` folds ASCII ONLY, which IS the - // contract (#4706 Q1 = A) — `É` stays `É`, so `$icontains: 'café'` does not - // match `CAFÉ`. Postgres and MySQL fold the wider Unicode range in - // `LOWER()`, so on those dialects this over-matches on non-ASCII letters; - // that divergence is measured and recorded rather than papered over, and it - // is the same dialect axis `$contains`'s case sensitivity sits on. - const col = fold ? 'LOWER(??)' : '??'; - const bound = fold ? 'LOWER(?)' : '?'; - builder[rawMethod](`${col} ${keyword} ${bound} ESCAPE ?`, [field, pattern, '\\']); + const { sql, bindings } = textMatchPredicate(this.dialectName, field, value, shape, negate, fold); + builder[rawMethod](sql, bindings); } /** diff --git a/packages/drivers/driver-sqlite-wasm/src/sqlite-wasm-icontains-and-retired-operators.test.ts b/packages/drivers/driver-sqlite-wasm/src/sqlite-wasm-icontains-and-retired-operators.test.ts index 9656005a63..117eaa2383 100644 --- a/packages/drivers/driver-sqlite-wasm/src/sqlite-wasm-icontains-and-retired-operators.test.ts +++ b/packages/drivers/driver-sqlite-wasm/src/sqlite-wasm-icontains-and-retired-operators.test.ts @@ -11,19 +11,34 @@ * its parameters and marshals the rows back through its own path. * * `$icontains` is the first operator this package has ever run whose predicate - * is a FUNCTION CALL on the column (`LOWER(col) LIKE LOWER(?) ESCAPE ?`) rather - * than a bare column reference. Every text predicate before it compiled to - * `col LIKE ?`. A dialect that mis-binds the parameters of the three-argument - * form, or that renders `??` inside a function call differently, produces - * precisely the failure a shared standard exists to rule out — a filter that - * looks applied and selects the wrong rows — and it would fail in no other suite - * in the repo. "It inherits the compiler, therefore it is fine" is the - * assumption these suites exist to disprove. + * is a FUNCTION CALL on the column (`lower(col) …`) rather than a bare column + * reference. Every text predicate before it compiled to `col LIKE ?`. A dialect + * that mis-binds the parameters of that form, or that renders `??` inside a + * function call differently, produces precisely the failure a shared standard + * exists to rule out — a filter that looks applied and selects the wrong rows — + * and it would fail in no other suite in the repo. "It inherits the compiler, + * therefore it is fine" is the assumption these suites exist to disprove. + * + * # [#6518] What changed under this file, and why the ENGINE axis got harder + * + * `textMatchPredicate` now emits `GLOB`, not `LIKE`, on the SQLite dialects, + * because SQLite's `LIKE` folds ASCII case where #4706 Q2 = A rules the + * `$contains` family case-SENSITIVE. For a separately compiled engine that is a + * new question, not a smaller one: `GLOB` is a different SQL function, with a + * different metacharacter set (`*`, `?`, `[`), no `ESCAPE` clause, and its own + * implementation inside whichever SQLite build this package loaded. An sql.js + * build whose `glob()` differed would answer wrong rows here and nowhere else. + * + * This cell therefore now runs the WHOLE `FILTER_TEXT_CASES` table rather than + * a hand-picked subset. #5702 could not: five of its cases require the + * case-sensitivity this driver did not yet have, and importing the case-set is + * what `scripts/check-driver-conformance.mjs` reads as coverage — so the import + * would have claimed a standard five of whose cases went unanswered. */ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import type { DriverOptions, FilterCondition } from '@objectstack/spec/data'; -import { FILTER_TEXT_ROWS } from '@objectstack/spec/data'; +import { FILTER_TEXT_CASES, FILTER_TEXT_ROWS } from '@objectstack/spec/data'; import { SqliteWasmDriver } from './index.js'; interface WireBearingError extends Error { @@ -72,15 +87,37 @@ describe('[#5702] driver-sqlite-wasm — $icontains and the retired $regex, on s expect(await ids({ name: { $icontains: 'CAFÉ' } })).toEqual(['3']); }); - it('$icontains keeps the LIKE metacharacters literal across the wasm bind path', async () => { - // The bound `ESCAPE ?` is a THIRD parameter on a predicate whose first - // operand is now a function call. This is the assertion that a wasm dialect - // binding those three positionally in the wrong order would fail. + it('$icontains keeps the pattern metacharacters literal across the wasm bind path', async () => { + // [#6518] Under `GLOB` these three characters are literal to the OPERATOR + // rather than escaped by the emitter, which is a different claim about this + // engine than the LIKE version made: it says sql.js's `glob()` agrees with + // better-sqlite3's about what is NOT a metacharacter. expect(await ids({ name: { $icontains: '100%' } })).toEqual(['5']); expect(await ids({ name: { $icontains: 'a_b' } })).toEqual(['7']); expect(await ids({ name: { $icontains: 'a.b' } })).toEqual(['9']); }); + it('[#6518] the $contains family is case-SENSITIVE through the wasm engine', async () => { + // `GLOB` is supplied by the SQLite build, not by the application, so this + // is the assertion that THIS build's `glob()` is case-exact. On + // `origin/main` (where the family compiled to `LIKE`) both lines answered + // ['1','2']. + expect(await ids({ name: { $contains: 'acme' } })).toEqual(['2']); + expect(await ids({ name: { $contains: 'ACME' } })).toEqual(['1']); + expect(await ids({ name: { $startsWith: 'ACME' } })).toEqual(['1']); + expect(await ids({ name: { $endsWith: 'corp' } })).toEqual(['2']); + }); + + it('[#6518] the GLOB metacharacters are escaped, on this engine too', async () => { + // A new escaped character class arrived with the new operator, and an + // unescaped `*` is the filter bypass an unescaped `%` was: `*a*b*` matches + // rows 7, 8 and 9 where the escaped pattern matches none of them. + for (const comparand of ['a*b', 'a?b', 'a[b'] as const) { + expect(await ids({ name: { $contains: comparand } }), comparand).toEqual([]); + expect(await ids({ name: { $icontains: comparand } }), comparand).toEqual([]); + } + }); + it('REFUSES the retired $regex, in the ADR-0112 envelope, naming $icontains', async () => { const err = await driver .find('txt', { where: { name: { $regex: 'ac.*' } } }, BYPASS) @@ -102,4 +139,32 @@ describe('[#5702] driver-sqlite-wasm — $icontains and the retired $regex, on s expect(err!.status).toBe(400); } }); + + /** + * [#6518] The shared standard, executed on sql.js. + * + * The invariant is the case-set's own: the same row set, or a refusal + * carrying `INVALID_FILTER` — never a third, quieter answer. The blocks above + * stay because they name what is specific to THIS engine (which function is + * called, which characters that function treats as metacharacters); the table + * knows only the contract. + */ + describe('FILTER_TEXT_CASES — the shared standard', () => { + for (const testCase of FILTER_TEXT_CASES) { + it(testCase.name, async () => { + if (testCase.expectRejection) { + const err = await driver + .find('txt', { where: testCase.filter }, BYPASS) + .then(() => null, (e: unknown) => e as WireBearingError); + expect(err, 'the case-set requires this filter REFUSED, not answered').toBeInstanceOf(Error); + expect(err!.code).toBe(testCase.code); + expect(err!.status).toBe(400); + for (const mention of testCase.mustMention) expect(err!.message).toContain(mention); + return; + } + expect(await ids(testCase.filter), testCase.note ?? testCase.name) + .toEqual([...testCase.expected]); + }); + } + }); }); diff --git a/packages/drivers/driver-turso/src/libsql-sqlite-stub.testkit.ts b/packages/drivers/driver-turso/src/libsql-sqlite-stub.testkit.ts index 0a2d080b9a..ec35e00724 100644 --- a/packages/drivers/driver-turso/src/libsql-sqlite-stub.testkit.ts +++ b/packages/drivers/driver-turso/src/libsql-sqlite-stub.testkit.ts @@ -20,6 +20,7 @@ * of the transport suites that already mock `execute`. */ +import type { Client } from '@libsql/client'; import { createRequire } from 'node:module'; // better-sqlite3 is a knex PEER here and is never imported directly outside @@ -85,3 +86,22 @@ export function makeLibsqlSqliteStub(filename = ':memory:'): LibsqlSqliteStub { const normalize = (args: unknown[] | undefined) => (args ?? []).map((a) => (a === undefined ? null : a)); + +/** + * The stub, typed as the `@libsql/client` `Client` that `TursoDriverConfig` + * declares — the one place that impedance mismatch is spelled out. + * + * `LibsqlSqliteStub` implements the three members the driver actually calls + * (`execute` / `batch` / `close`), not the whole `Client` interface, so handing + * it to the config needs a cast. Suites in this package have each written that + * cast themselves as `client: stub as never`, which erases the argument to the + * BOTTOM type: `never` is assignable to anything, so it would go on compiling + * even if `client` were re-typed to something this stub cannot model at all. + * This names the TARGET type instead (the #6204 spelling), so the cast still + * asserts something, and it lives once in the testkit rather than once per + * suite. Existing `as never` call sites can migrate here; nothing forces them + * to do it in the same change. + */ +export function asLibsqlClient(stub: LibsqlSqliteStub): Client { + return stub as unknown as Client; +} diff --git a/packages/drivers/driver-turso/src/remote-transport-comparand-refusal.test.ts b/packages/drivers/driver-turso/src/remote-transport-comparand-refusal.test.ts index 450b9fd533..3c5093905f 100644 --- a/packages/drivers/driver-turso/src/remote-transport-comparand-refusal.test.ts +++ b/packages/drivers/driver-turso/src/remote-transport-comparand-refusal.test.ts @@ -80,11 +80,16 @@ describe('RemoteTransport comparand refusal — the value half of #1004', () => expect(calls[0].args).toEqual(['a', 'b', 'lost']); }); - it('keeps the LIKE family compiling a string comparand', async () => { + it('keeps the text family compiling a string comparand', async () => { + // [#6518] `GLOB`, not `LIKE`: the family is case-SENSITIVE by contract + // (#4706 Q2 = A) and SQLite's `LIKE` folds ASCII, so both the operator + // and its wildcard character changed. What this case controls for is + // unchanged — a legitimate string comparand still compiles rather than + // being caught by the refusal this file is about. const { t, calls } = transportWithCapturingClient(); await t.find('deal', { where: { name: { $startsWith: 'Alp' } } }); - expect(calls[0].sql).toMatch(/"name"\s+LIKE\s+\?/i); - expect(calls[0].args).toEqual(['Alp%']); + expect(calls[0].sql).toMatch(/"name"\s+GLOB\s+\?/i); + expect(calls[0].args).toEqual(['Alp*']); }); }); diff --git a/packages/drivers/driver-turso/src/remote-transport-null-comparand-refusal.test.ts b/packages/drivers/driver-turso/src/remote-transport-null-comparand-refusal.test.ts index 1a29fba290..bb5017ba87 100644 --- a/packages/drivers/driver-turso/src/remote-transport-null-comparand-refusal.test.ts +++ b/packages/drivers/driver-turso/src/remote-transport-null-comparand-refusal.test.ts @@ -337,8 +337,12 @@ describe('RemoteTransport $null comparand refusal (#1116)', () => { it('leaves the regex family taking a non-string comparand', async () => { // Measured consistent and fail-closed across backends; `driver-sql`'s // #5041 comment excludes the family from its guard by name. - expect((await compile({ name: { $startsWith: 42 } })).args).toEqual(['42%']); - expect((await compile({ name: { $contains: true } })).args).toEqual(['%true%']); + // [#6518] The wildcard is `*`, not `%` — this transport emits `GLOB` now, + // because the family is case-SENSITIVE by contract and SQLite's `LIKE` + // is not. The claim under test is still that a NUMBER and a BOOLEAN keep + // rendering to text rather than being refused. + expect((await compile({ name: { $startsWith: 42 } })).args).toEqual(['42*']); + expect((await compile({ name: { $contains: true } })).args).toEqual(['*true*']); }); }); diff --git a/packages/drivers/driver-turso/src/remote-transport-text-predicates.test.ts b/packages/drivers/driver-turso/src/remote-transport-text-predicates.test.ts index 91b37bb6b3..9f956f98aa 100644 --- a/packages/drivers/driver-turso/src/remote-transport-text-predicates.test.ts +++ b/packages/drivers/driver-turso/src/remote-transport-text-predicates.test.ts @@ -54,11 +54,13 @@ const META_ROWS = [ async function makeRemoteDriver(schema: Record, rows: Record[]) { const stub = makeLibsqlSqliteStub(); // [#5702] The stub is wrapped so the STATEMENTS are readable, not only the - // rows. Rows are the right witness for almost everything this file pins — but - // `$icontains`'s fold is invisible in rows on SQLite (`LIKE` already folds - // ASCII there), so the emitted SQL is the only place a dropped `LOWER()` can - // be seen. Recording here rather than standing up a third hand-rolled mock - // client keeps every case on the same engine. + // rows. #5702 needed that because `$icontains`'s fold was invisible in rows + // on SQLite — `LIKE` already folded ASCII there, so a dropped `LOWER()` + // changed no answer. [#6518] retires that reason: under `GLOB` the fold IS + // observable in rows, and the cases below assert it there. The recording + // stays, now pinning the CONSTRUCT (which operator, with which escapes) — + // still the only place a silent revert to `LIKE` would show up before its + // rows did. const executed: string[] = []; const recording: LibsqlSqliteStub = { ...stub, @@ -137,47 +139,49 @@ describe('TursoDriver remote — declared text predicates return rows', () => { }); /** - * [#5702] MEASURED, and the measurement contradicts the assertion this case - * was first written with — recorded as it came out rather than as it was - * predicted. + * [#6518] The case #5702 had to write BACKWARDS, now written forwards. * - * The intended pin was `$contains: 'ALP'` → `[]` beside `$icontains: 'ALP'` → - * `['w1','w2']`, i.e. the two operators told apart by their answers. It fails: - * `$contains` returns `['w1','w2']` too. SQLite's `LIKE` folds ASCII case by - * itself, so on this dialect `col LIKE '%ALP%'` and `LOWER(col) LIKE - * LOWER('%ALP%')` select the SAME rows for EVERY comparand — the second fold - * is a no-op on top of the first. + * #5702 wanted to pin `$contains: 'ALP'` → `[]` beside `$icontains: 'ALP'` → + * `['w1','w2']`, i.e. the two operators told apart by their ANSWERS. It could + * not: `$contains` returned `['w1','w2']` too, because SQLite's `LIKE` folds + * ASCII case by itself, so `col LIKE '%ALP%'` and `LOWER(col) LIKE + * LOWER('%ALP%')` selected the same rows for EVERY comparand. That file + * recorded the equality as the honest pin and named the missing half: the + * #4706 Q2 = A ruling that the `$contains` family is case-SENSITIVE. * - * That is not a defect in `$icontains`; it is the `$contains` half of the - * #4706 Q2 = A ruling ("the `$contains` family is case-SENSITIVE"), which is - * NOT delivered by this PR. Making SQLite's LIKE case-exact needs a different - * construct (GLOB / `instr()` / a binary collation) applied in three places - * that must move together — this transport, `SqlDriver.applyLike`, and the - * RLS/analytics twins (`read-scope-sql`, `service-analytics`'s - * `like-pattern.ts`) — or one permission rule compiles to two row sets. It is - * filed separately; the `FILTER_TEXT` DEBT rows in - * `scripts/check-driver-conformance.mjs` stay open for it. - * - * So the honest pin here is the CURRENT pair — equal on this dialect — plus - * the compiled SQL, which is the only place the fold is observable on SQLite - * and therefore the only thing that can catch a `LOWER()` silently dropped. + * This is that half. `pushLike` emits `GLOB`, which is case-exact by + * definition, so the pin #5702 wanted is the one that now holds — and it + * fails loudly on a revert to `LIKE`, which the equality it replaces could + * not do. */ - it('$contains and $icontains agree on SQLite today — LIKE already folds ASCII', async () => { - expect(await ids(driver, 'widget', { name: { $contains: 'ALP' } })).toEqual(['w1', 'w2']); + it('$contains is case-SENSITIVE, and now answers differently from $icontains', async () => { + expect(await ids(driver, 'widget', { name: { $contains: 'ALP' } })).toEqual([]); + expect(await ids(driver, 'widget', { name: { $contains: 'Alp' } })).toEqual(['w1', 'w2']); expect(await ids(driver, 'widget', { name: { $icontains: 'ALP' } })).toEqual(['w1', 'w2']); }); - it('$icontains compiles LOWER() on BOTH operands, where $contains compiles neither', async () => { + it('$startsWith and $endsWith are case-SENSITIVE too', async () => { + expect(await ids(driver, 'widget', { name: { $startsWith: 'alp' } })).toEqual([]); + expect(await ids(driver, 'widget', { name: { $startsWith: 'Alp' } })).toEqual(['w1', 'w2']); + expect(await ids(driver, 'widget', { name: { $endsWith: 'ETA' } })).toEqual([]); + expect(await ids(driver, 'widget', { name: { $endsWith: 'eta' } })).toEqual(['w3']); + }); + + it('$icontains compiles lower() on BOTH operands, where $contains compiles neither', async () => { executed.length = 0; await driver.find('widget', { where: { name: { $icontains: 'ALP' } } }); const icontainsSql = executed.join('\n'); - expect(icontainsSql).toContain('LOWER("name") LIKE LOWER(?) ESCAPE'); + expect(icontainsSql).toContain('lower("name") GLOB lower(?)'); + // GLOB has no ESCAPE clause in SQLite's grammar — emitting one is a syntax + // error, so its absence is part of the construct rather than an omission. + expect(icontainsSql).not.toContain('ESCAPE'); executed.length = 0; await driver.find('widget', { where: { name: { $contains: 'ALP' } } }); const containsSql = executed.join('\n'); - expect(containsSql).toContain('"name" LIKE ? ESCAPE'); - expect(containsSql).not.toContain('LOWER'); + expect(containsSql).toContain('"name" GLOB ?'); + expect(containsSql).not.toContain('lower'); + expect(containsSql).not.toContain('LIKE'); }); it('REFUSES the retired $regex, in the ADR-0112 envelope, naming $icontains', async () => { @@ -357,8 +361,20 @@ describe('RemoteTransport — unknown operators throw instead of degrading', () }); }); -describe('RemoteTransport — text predicates bind an explicit ESCAPE', () => { - it('emits LIKE … ESCAPE and binds the escaped pattern, not the raw value', async () => { +describe('RemoteTransport — text predicates bind an ESCAPED pattern, never the raw value', () => { + /** + * [#6518] What this case pins survived the operator change; the character + * class it pins did not. + * + * Under `LIKE` the metacharacters were `%` and `_` and the answer was an + * explicit `ESCAPE '\'` clause, because SQLite honours no default escape + * character. Under `GLOB` the metacharacters are `*`, `?` and `[`, there is + * no `ESCAPE` clause in the grammar at all, and the escape mechanism is a + * self-closing character class. So `%` needs no escape here any more — and + * `*` needs one it did not need before. Both directions are asserted, because + * a half-migrated escape rule is precisely the P0 this case exists for. + */ + const captureOne = async (where: Record) => { const calls: Array<{ sql: string; args: any[] }> = []; const client = { execute: vi.fn(async (stmt: any) => { @@ -369,11 +385,24 @@ describe('RemoteTransport — text predicates bind an explicit ESCAPE', () => { }; const t = new RemoteTransport(); t.setClient(client as any); + await t.find('widget', { where }); + return calls[0]; + }; - await t.find('widget', { where: { name: { $startsWith: '50%' } } }); - const { sql, args } = calls[0]; - // SQLite honours no default escape character, so the clause must be explicit. - expect(sql).toMatch(/"name"\s+LIKE\s+\?\s+ESCAPE\s+'\\'/i); - expect(args).toEqual(['50\\%%']); + it('emits GLOB with no ESCAPE clause, and leaves the LIKE metacharacters alone', async () => { + const { sql, args } = await captureOne({ name: { $startsWith: '50%' } }); + expect(sql).toMatch(/"name"\s+GLOB\s+\?/i); + expect(sql).not.toMatch(/ESCAPE/i); + // `%` is an ordinary character to GLOB, so it travels unescaped. + expect(args).toEqual(['50%*']); + }); + + it('escapes the GLOB metacharacters as self-closing classes', async () => { + expect((await captureOne({ name: { $contains: '*' } })).args).toEqual(['*[*]*']); + expect((await captureOne({ name: { $contains: '?' } })).args).toEqual(['*[?]*']); + expect((await captureOne({ name: { $contains: '[' } })).args).toEqual(['*[[]*']); + // Unescaped, `*a*b*` is a wildcard pattern rather than a literal — the same + // filter bypass an unescaped `%` was under LIKE. + expect((await captureOne({ name: { $contains: 'a*b' } })).args).toEqual(['*a[*]b*']); }); }); diff --git a/packages/drivers/driver-turso/src/remote-transport-undefined-comparand-refusal.test.ts b/packages/drivers/driver-turso/src/remote-transport-undefined-comparand-refusal.test.ts index f5fa901d5a..5587c2546c 100644 --- a/packages/drivers/driver-turso/src/remote-transport-undefined-comparand-refusal.test.ts +++ b/packages/drivers/driver-turso/src/remote-transport-undefined-comparand-refusal.test.ts @@ -262,7 +262,10 @@ describe('[#6050] RemoteTransport refuses an undefined comparand', () => { expect((await compile({ stage: 'won' })).args).toEqual(['won']); expect((await compile({ stage: { $in: ['won', 'lost'] } })).args).toEqual(['won', 'lost']); expect((await compile({ score: { $gt: 5 } })).args).toEqual([5]); - expect((await compile({ stage: { $contains: 'w' } })).args).toEqual(['%w%']); + // [#6518] `*w*`, not `%w%`: the text family compiles to `GLOB` now, whose + // wildcard is `*`. The claim is unchanged — a defined comparand still + // compiles instead of tripping this file's undefined-comparand refusal. + expect((await compile({ stage: { $contains: 'w' } })).args).toEqual(['*w*']); expect((await compile({})).sql).toBe(BARE_SCAN); }); }); diff --git a/packages/drivers/driver-turso/src/remote-transport.ts b/packages/drivers/driver-turso/src/remote-transport.ts index d28f701040..ccdd92b2e3 100644 --- a/packages/drivers/driver-turso/src/remote-transport.ts +++ b/packages/drivers/driver-turso/src/remote-transport.ts @@ -2022,26 +2022,49 @@ export class RemoteTransport { } /** - * Append one parameterized `LIKE` / `NOT LIKE` predicate. - * - * The LIKE metacharacters `%` / `_` (and the escape character `\` itself) are - * escaped in the COMPARAND so they match literally: unescaped, a value of `%` - * expands to `%%%` and matches every row — a filter bypass, and a P0 the - * framework already paid for (`sql-driver-like-escape.test.ts`). The escape - * clause is written explicitly because SQLite honours no default escape - * character; it is a literal here rather than a bind (as `SqlDriver` does) - * only because this transport is SQLite-by-construction and keeping the bind - * list to comparands keeps the arg arithmetic in every caller unchanged. - * - * This is the ONE place the LIKE family is built. That is the point: five - * operators sharing one escape rule is the opposite of the second - * implementation ADR-0053 D-A1 warns about — the rule cannot drift between - * `$contains` and `$startsWith` because there is only one of it. It does - * restate the framework's rule (which lives in `SqlDriver.applyLike`, a - * private Knex-builder method with no reusable export), so the two must be - * read together; the row-level suite in - * `remote-transport-text-predicates.test.ts` is what pins them to the same - * answers. + * Append one parameterized text-match predicate for the `$contains` family + * and `$icontains`. + * + * The comparand's metacharacters are escaped so it matches literally: + * unescaped, a value of `%` expands to `%%%` and matches every row — a filter + * bypass, and a P0 the framework already paid for + * (`sql-driver-like-escape.test.ts`). + * + * # [#6518] Why this emits `GLOB`, not `LIKE` + * + * `$contains` / `$notContains` / `$startsWith` / `$endsWith` are + * case-SENSITIVE by contract (#4706 Q2 = A). SQLite's `LIKE` folds ASCII case + * unconditionally, and libSQL is SQLite — so this transport used to answer + * `{name: {$contains: 'acme'}}` with the `ACME Corp` row too, which is + * over-matching rather than a near miss. The fold cannot be switched off per + * statement (`PRAGMA case_sensitive_like` is connection-global, so one query + * would silently redefine every other query on the same connection), and + * `CAST(col AS BLOB) LIKE ?` was measured to match NOTHING at all. `GLOB` is + * SQLite's case-exact pattern operator and is what both SQLite faces now + * emit — `SqlDriver.applyLike`'s `textMatchPredicate` reaches the identical + * decision for the local transport, and `turso-local-remote-*` parity suites + * are what hold the two to the same rows. + * + * The escaped character class moves WITH the operator, which is the part a + * second implementation gets wrong: GLOB's metacharacters are `*`, `?` and + * `[` (escaped as self-closing classes `[*]`, `[?]`, `[[]`, because GLOB has + * no `ESCAPE` clause in SQLite's grammar), while `%` and `_` are ordinary + * characters to it. So this method no longer emits an `ESCAPE` clause at all. + * + * `fold` still wraps BOTH operands, now in `lower()` around `GLOB` — folding + * only the comparand would compare a folded needle against a raw column and + * silently match just the rows that were already lower-case. `lower()` on + * SQLite folds ASCII ONLY, which is the contract (#4706 Q1 = A) rather than a + * limitation to work around: measured, `lower('CAFÉ')` is `'cafÉ'`, so + * `$icontains: 'café'` answers the `café` row and not the `CAFÉ` one. + * + * This is still the ONE place the family is built, which is the point: five + * operators sharing one escape rule cannot drift between `$contains` and + * `$startsWith` because there is only one of it. It restates the framework's + * rule (which lives in `SqlDriver`, whose emitter is a private module + * function with no reusable export), so the two must be read together; the + * row-level suite in `remote-transport-text-predicates.test.ts` is what pins + * them to the same answers. */ private pushLike( clauses: string[], @@ -2053,21 +2076,11 @@ export class RemoteTransport { nullSafe = false, fold = false, ): void { - const escaped = String(value).replace(/[\\%_]/g, '\\$&'); - const pattern = shape === 'starts' ? `${escaped}%` : shape === 'ends' ? `%${escaped}` : `%${escaped}%`; - // [#5702] `fold` wraps BOTH operands in `LOWER()` — the `$icontains` - // lowering, and a parameter of THIS method rather than a sixth text arm for - // the reason the paragraph above gives: one escape rule, one place. Folding - // only the comparand would compare a folded needle against a raw column and - // silently match only the rows that were already lower-case. - // - // libSQL is SQLite, whose `lower()` folds ASCII ONLY — which is the - // contract (#4706 Q1 = A), not a limitation to work around: `É` does not - // fold, so `$icontains: 'café'` must not match `CAFÉ`. The local transport - // reaches the same answer through `SqlDriver.applyLike`'s `LOWER()`. - const lhs = fold ? `LOWER(${column})` : column; - const rhs = fold ? 'LOWER(?)' : '?'; - const predicate = `${lhs} ${negate ? 'NOT LIKE' : 'LIKE'} ${rhs} ESCAPE '\\'`; + const escaped = String(value).replace(/[*?[]/g, '[$&]'); + const pattern = shape === 'starts' ? `${escaped}*` : shape === 'ends' ? `*${escaped}` : `*${escaped}*`; + const lhs = fold ? `lower(${column})` : column; + const rhs = fold ? 'lower(?)' : '?'; + const predicate = `${lhs} ${negate ? 'NOT GLOB' : 'GLOB'} ${rhs}`; clauses.push(nullSafe ? this.nullSafeNegative(column, predicate) : predicate); args.push(pattern); } diff --git a/packages/drivers/driver-turso/src/turso-local-remote-text-parity.test.ts b/packages/drivers/driver-turso/src/turso-local-remote-text-parity.test.ts new file mode 100644 index 0000000000..5ddd800bf6 --- /dev/null +++ b/packages/drivers/driver-turso/src/turso-local-remote-text-parity.test.ts @@ -0,0 +1,173 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#6518] ONE `TursoDriver`, ONE answer — the two transports held against each + * other on the TEXT family, and both held to `FILTER_TEXT_CASES`. + * + * # Why this file exists rather than one more case in each transport's suite + * + * This driver compiles a filter TWICE, and which copy runs is chosen by the + * `url` it was constructed with: LOCAL inherits `SqlDriver` (so it gets + * `textMatchPredicate`), while REMOTE compiles in + * `RemoteTransport.buildWhereSQL` — an independent emitter that inherits none + * of it and carries its own `pushLike`. #6518 had to move BOTH, and a + * per-transport suite cannot fail on the DIFFERENCE: a divergence shows up as + * one file red and the other green, in whichever order someone reads them. + * `turso-local-remote-null-parity.test.ts` made that argument for the NULL + * family; this is the same argument for the text family, and the text family is + * where the second emitter was actually asked to re-derive an escape rule. + * + * That matters more here than it looks. #6518 did not only flip a flag — it + * changed the OPERATOR (`LIKE` → `GLOB`, because SQLite's `LIKE` folds ASCII + * case and #4706 Q2 = A says the `$contains` family is case-SENSITIVE) and with + * it the escaped character class (`%` / `_` / `\` → `*` / `?` / `[`, spelled as + * self-closing classes because GLOB has no `ESCAPE` clause). Two emitters, one + * new escape rule, and an unescaped metacharacter is a filter bypass: this is + * exactly the shape where a copy gets migrated halfway. + * + * # Why the case-set and not a private fixture + * + * `FILTER_TEXT_CASES` is the contract (#5701 / #4706), so running it here is + * what `scripts/check-driver-conformance.mjs` reads as this driver's coverage — + * and it is read honestly, because both transports answer every row of it. The + * case-set's own invariant is the one asserted: a backend returns the SAME ROW + * SET or REFUSES with `INVALID_FILTER`, never a third, quieter answer. + * + * # Why every case asserts the canonical answer too + * + * Parity alone is satisfiable by breaking both transports the same way — the + * failure mode a "the two agree" suite invites. So each case is checked against + * the row ids the ruling requires, on both faces. Agreement is necessary; + * agreement on the RIGHT answer is the assertion. + * + * # Reverse verification, direction predicted BEFORE it was run + * + * Predicted: reverting EITHER emitter alone turns the five case-sensitivity + * rows red, and turns them red as a PARITY failure naming which transport + * drifted. Measured: reverting `pushLike` to `LIKE` leaves the local column + * green and the remote column returning `['1','2']` where the case-set demands + * `['2']`, so the parity assertion and the canonical assertion fail together — + * which is the outcome that distinguishes this file from two separate suites. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import type { DriverQuery } from '@objectstack/spec/contracts'; +import { FILTER_TEXT_CASES, FILTER_TEXT_ROWS } from '@objectstack/spec/data'; +import { TursoDriver } from './turso-driver.js'; +import { asLibsqlClient, makeLibsqlSqliteStub, type LibsqlSqliteStub } from './libsql-sqlite-stub.testkit.js'; + +/** The shared nine-row fixture, so a verdict here is comparable to one anywhere. */ +const TEXT_OBJECT = { + name: 'text_conformance', + fields: { name: { type: 'string' } }, +}; + +/** The error a refused filter produced — never a bare `toThrow()`. */ +interface WireBearingError extends Error { + code?: string; + status?: number; +} + +const ids = (rows: Array>): string[] => + rows.map((r) => String(r.id)).sort((x, y) => x.localeCompare(y)); + +describe('[#6518] TursoDriver LOCAL and REMOTE answer FILTER_TEXT_CASES identically', () => { + let local: TursoDriver; + let remote: TursoDriver; + let stub: LibsqlSqliteStub; + + beforeAll(async () => { + local = new TursoDriver({ url: ':memory:' }); + expect(local.transportMode).toBe('local'); + await local.initObjects([TEXT_OBJECT]); + for (const row of FILTER_TEXT_ROWS) { + await local.create(TEXT_OBJECT.name, { ...row }, { bypassTenantAudit: true }); + } + + stub = makeLibsqlSqliteStub(); + remote = new TursoDriver({ url: 'libsql://text-parity.turso.io', client: asLibsqlClient(stub) }); + await remote.connect(); + expect(remote.transportMode).toBe('remote'); + await remote.syncSchema(TEXT_OBJECT.name, TEXT_OBJECT); + for (const row of FILTER_TEXT_ROWS) { + await remote.create(TEXT_OBJECT.name, { ...row }); + } + }); + + afterAll(async () => { + await local.disconnect(); + await remote.disconnect(); + stub.close(); + }); + + /** + * The fixture control, on BOTH faces. A parity suite whose two sides seeded + * different rows compares nothing — and the two rows that carry the whole + * case axis (`ACME Corp` / `acme corp`) would be indistinguishable from a + * mis-seeded table by the case results alone. + */ + it('both transports hold the same nine rows, cased as the fixture cases them', async () => { + expect(ids(await local.find(TEXT_OBJECT.name, {}))).toEqual( + ['1', '2', '3', '4', '5', '6', '7', '8', '9'], + ); + expect(ids(await remote.find(TEXT_OBJECT.name, {}))).toEqual( + ['1', '2', '3', '4', '5', '6', '7', '8', '9'], + ); + for (const driver of [local, remote]) { + const rows = await driver.find(TEXT_OBJECT.name, {} as DriverQuery); + const byId = new Map(rows.map((r) => [String(r.id), String(r.name)])); + expect(byId.get('1')).toBe('ACME Corp'); + expect(byId.get('2')).toBe('acme corp'); + expect(byId.get('3')).toBe('CAFÉ'); + expect(byId.get('4')).toBe('café'); + } + }); + + for (const testCase of FILTER_TEXT_CASES) { + it(testCase.name, async () => { + if (testCase.expectRejection) { + for (const [face, driver] of [['local', local], ['remote', remote]] as const) { + const err = await driver + .find(TEXT_OBJECT.name, { where: testCase.filter } as DriverQuery) + .then(() => null, (e: unknown) => e as WireBearingError); + expect(err, `${face} compiled a filter the case-set requires refused`).not.toBeNull(); + // `code` AND `status`, never a bare rejection: this driver's whole + // family of filter refusals exists to reach the client as a + // 400-class error rather than an opaque 500 (ADR-0112). + expect(err!.code, face).toBe(testCase.code); + expect(err!.status, face).toBe(400); + for (const mention of testCase.mustMention) { + expect(err!.message, `${face} — ${mention}`).toContain(mention); + } + } + return; + } + + const localIds = ids(await local.find(TEXT_OBJECT.name, { where: testCase.filter } as DriverQuery)); + const remoteIds = ids(await remote.find(TEXT_OBJECT.name, { where: testCase.filter } as DriverQuery)); + // The difference IS the assertion — reported before the canonical one, so + // a drift between the two emitters reads as a drift rather than as two + // unrelated wrong answers. + expect(remoteIds, `${testCase.name} — LOCAL vs REMOTE`).toEqual(localIds); + expect(localIds, testCase.note ?? testCase.name).toEqual([...testCase.expected]); + }); + } + + /** + * The GLOB escape class, on both faces. `FILTER_TEXT_ROWS` has no `*`, `?` or + * `[` row — it encodes the CONTRACT, which knows nothing about any dialect's + * pattern syntax — so the new metacharacters are pinned here, where the + * dialect is known. Unescaped, `*a*b*` is a wildcard pattern that matches + * rows 7, 8 and 9; escaped, it matches none of them. + */ + for (const comparand of ['a*b', 'a?b', 'a[b'] as const) { + it(`treats ${JSON.stringify(comparand)} as literal text on both transports`, async () => { + for (const [face, driver] of [['local', local], ['remote', remote]] as const) { + expect(ids(await driver.find(TEXT_OBJECT.name, { where: { name: { $contains: comparand } } } as DriverQuery)), face) + .toEqual([]); + expect(ids(await driver.find(TEXT_OBJECT.name, { where: { name: { $icontains: comparand } } } as DriverQuery)), face) + .toEqual([]); + } + }); + } +}); diff --git a/packages/formula/src/matches-filter.test.ts b/packages/formula/src/matches-filter.test.ts index 8c4ad203b0..65bc586fa7 100644 --- a/packages/formula/src/matches-filter.test.ts +++ b/packages/formula/src/matches-filter.test.ts @@ -57,6 +57,31 @@ describe('matchesFilterCondition — operators', () => { expect(m(rec, { name: { $notContains: 'Zeta' } })).toBe(true); expect(m(rec, { name: { $startsWith: 'Zzz' } })).toBe(false); }); + /** + * [#6518] The `$contains` family is case-SENSITIVE by contract (#4706 Q2 = A). + * + * This face already was — `String.prototype.includes` / `startsWith` / + * `endsWith` compare exactly — so #6518 is what moved the SQL family onto this + * answer, not what changed this one. It is pinned now precisely BECAUSE + * nothing here changed: this evaluator is the JS baseline the drivers were + * brought to, and the next mistake in this file takes the shape of a + * "helpful" `toLowerCase()` added to make some backend agree. That would + * silently widen every RLS rule this function evaluates, and no assertion in + * the repo would have gone red for it. + * + * `$notContains` carries the mirror: a case-only difference does NOT contain, + * so it must SATISFY the negation. A folding implementation excludes the row. + */ + it('[#6518] the $contains family is case-SENSITIVE', () => { + expect(m(rec, { name: { $contains: 'beta' } })).toBe(false); + expect(m(rec, { name: { $contains: 'Beta' } })).toBe(true); + expect(m(rec, { name: { $startsWith: 'acme' } })).toBe(false); + expect(m(rec, { name: { $startsWith: 'Acme' } })).toBe(true); + expect(m(rec, { name: { $endsWith: 'BETA' } })).toBe(false); + expect(m(rec, { name: { $endsWith: 'Beta' } })).toBe(true); + expect(m(rec, { name: { $notContains: 'beta' } })).toBe(true); + expect(m(rec, { name: { $notContains: 'Beta' } })).toBe(false); + }); it('$null / $exists', () => { expect(m(rec, { region: { $null: true } })).toBe(true); expect(m(rec, { stage: { $null: false } })).toBe(true); diff --git a/packages/objectql/src/having-filter.test.ts b/packages/objectql/src/having-filter.test.ts index 03d5386fd4..d90f74f1ea 100644 --- a/packages/objectql/src/having-filter.test.ts +++ b/packages/objectql/src/having-filter.test.ts @@ -210,6 +210,34 @@ describe('no-value rows and the negation-carrying operators (#5905 / #5298 optio }); }); + /** + * [#6518] The `$contains` family is case-SENSITIVE by contract (#4706 Q2 = A). + * + * This face already was, by the same mechanism `formula`'s evaluator is — + * `String.prototype.includes` / `startsWith` / `endsWith` compare exactly — so + * #6518, which moved the SQL family onto that answer, changed nothing here. + * It is pinned for the reason #5905 gave for pinning this file at all: + * `having` is the one text-operator face with NO conformance-table coverage + * (`check-driver-conformance` scopes itself to `packages/drivers/*`, and + * `packages/objectql` imports no case-set), so a fold added here to make some + * backend agree would go red nowhere. + */ + describe('[#6518] the $contains family is case-SENSITIVE', () => { + it('$contains, $startsWith and $endsWith do not fold case', () => { + expect(matchesHaving(VALUED_IN, { tag: { $contains: 'LPH' } })).toBe(false); + expect(matchesHaving(VALUED_IN, { tag: { $contains: 'lph' } })).toBe(true); + expect(matchesHaving(VALUED_IN, { tag: { $startsWith: 'ALP' } })).toBe(false); + expect(matchesHaving(VALUED_IN, { tag: { $startsWith: 'alp' } })).toBe(true); + expect(matchesHaving(VALUED_IN, { tag: { $endsWith: 'HA' } })).toBe(false); + expect(matchesHaving(VALUED_IN, { tag: { $endsWith: 'ha' } })).toBe(true); + }); + + it('$notContains carries the mirror: a case-only difference SATISFIES it', () => { + expect(matchesHaving(VALUED_IN, { tag: { $notContains: 'LPH' } })).toBe(true); + expect(matchesHaving(VALUED_IN, { tag: { $notContains: 'lph' } })).toBe(false); + }); + }); + /** * The NULL-safety lives at the LEAF, so `$not` inverts it rather than * inheriting it — the same design driver-sql writes down for its own diff --git a/packages/plugins/plugin-auth/src/auth-contains-filter.test.ts b/packages/plugins/plugin-auth/src/auth-contains-filter.test.ts index a05f26890b..6756ebc4ac 100644 --- a/packages/plugins/plugin-auth/src/auth-contains-filter.test.ts +++ b/packages/plugins/plugin-auth/src/auth-contains-filter.test.ts @@ -70,13 +70,19 @@ * face 2 quietly reverting to the always-green pin #5830 refused to create. * * Case semantics are deliberately untested here, exactly as in the sibling - * file: sqlite's `LIKE` is ASCII case-INsensitive by default (so `$contains` - * folds on this backend while the contract layer calls it case-SENSITIVE per - * #5701 Q2=A — driver-sql pins that gap itself, in - * `sql-driver-icontains-and-retired-operators.test.ts`, which asserts - * `$contains` compiles WITHOUT `LOWER()` while `$icontains` compiles with it). - * Every fixture below is lower-case and no assertion depends on which way that - * per-driver alignment lands. + * file — but the REASON changed under this file, so it is restated rather than + * left stale. It used to be that sqlite's `LIKE` folds ASCII case, so + * `$contains` folded on this backend while the contract layer called it + * case-SENSITIVE (#5701 Q2 = A), and driver-sql could pin that gap only through + * the compiled SQL. **#6518 closed it**: the SQL family emits `GLOB` on the + * SQLite dialects now, so `$contains` is case-exact here too and + * `sql-driver-icontains-and-retired-operators.test.ts` pins it in ROWS. + * + * Nothing below moves either way. Every fixture is lower-case and every + * comparand matches its row's case exactly, so the four behavioural pins hold + * identically before and after that change — case belongs to the driver's own + * conformance suites, while what this file is about is which OPERATOR the + * adapter emits. */ import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; diff --git a/packages/services/service-analytics/src/__tests__/like-metacharacter-escape.test.ts b/packages/services/service-analytics/src/__tests__/like-metacharacter-escape.test.ts index e171ed23d8..cbb768f39e 100644 --- a/packages/services/service-analytics/src/__tests__/like-metacharacter-escape.test.ts +++ b/packages/services/service-analytics/src/__tests__/like-metacharacter-escape.test.ts @@ -527,4 +527,84 @@ describe('[#5567] analytics LIKE compilers escape their comparand', () => { expect(sql).toMatch(/name LIKE \$\d+ ESCAPE \$\d+/); }); }); + + /** + * [#6518] The dialect assumption `like-pattern.ts` now writes down, made + * FALSIFIABLE. + * + * #6518 moved the driver family off a plain `LIKE` — SQLite's folds ASCII + * case, MySQL's follows its collation, and #4706 Q2 = A rules the family + * case-SENSITIVE — while these three compilers kept `LIKE`. That is correct + * only because what they emit is Postgres-shaped, and on Postgres `LIKE` IS + * case-exact. A comment saying so rots silently; this block goes red. + * + * Two claims, and both have to hold for the file's reasoning to survive: + * + * 1. the emitted statement really is Postgres-shaped (`$N` placeholders, + * `"double quoted"` identifiers) — so a change that started emitting for + * SQLite or MySQL trips here first; + * 2. the `$contains` family compiles CASE-EXACTLY — no `ILIKE`, no + * `LOWER()`, no fold of any kind wrapped around either operand. + * + * On `read-scope-sql.ts`'s output the second claim is a permission boundary, + * not a filter preference: a wider predicate there is ADR-0021 read-scope + * over-reach (#3948). + */ + describe('[#6518] the case-sensitivity assumption these compilers rest on', () => { + const TEXT_FAMILY = ['$contains', '$notContains', '$startsWith', '$endsWith'] as const; + + it('read-scope-sql compiles the family case-EXACTLY — no ILIKE, no fold', () => { + for (const op of TEXT_FAMILY) { + const { sql } = compileScopedFilterToSql( + { name: { [op]: 'Admin' } } as FilterCondition, + 'person', + ); + expect(sql, op).toMatch(/LIKE/); + expect(sql, op).not.toMatch(/ILIKE|LOWER\s*\(|lower\s*\(|translate\s*\(|GLOB/); + } + }); + + it('read-scope-sql emits `?` for its consumers to renumber, and quotes identifiers', () => { + const { sql, params } = compileScopedFilterToSql( + { name: { $contains: 'Admin' } } as FilterCondition, + 'person', + ); + // `?` here, `$N` after the consumers rewrite it — the shape both + // `applyReadScope` and `generateSql` depend on, and the reason the escape + // character can be a bound value rather than a per-dialect literal. + expect(sql).toContain('?'); + expect(sql).toContain('"person"."name"'); + expect(params).toEqual(['%Admin%', LIKE_ESCAPE_CHAR]); + }); + + it('both executing compilers number their placeholders Postgres-style', async () => { + const native = await new NativeSQLStrategy().generateSql(query({ name: { $contains: 'x' } }), nativeCtx); + const echo = await new ObjectQLStrategy().generateSql(query({ name: { $contains: 'x' } }), echoCtx); + for (const [label, sql] of [['native', native.sql], ['echo', echo.sql]] as const) { + expect(sql, label).toMatch(/\$\d+/); + // `?` is SQLite's and MySQL's placeholder. Its appearance would mean the + // statement stopped being Postgres-shaped, which is exactly the premise + // `like-pattern.ts`'s #6518 section says to re-open. + expect(sql, label).not.toMatch(/[^$\w]\?/); + expect(sql, label).not.toMatch(/ILIKE|LOWER\s*\(/); + } + }); + + /** + * `$icontains` is UNIMPLEMENTED here, and fail-closed at both doors — the + * state #6518 leaves it in deliberately, because adding it belongs with + * #6520 (the vocabulary, driver-memory and analytics together) rather than + * to a driver-lane case-folding fix. Pinned so "unimplemented" cannot + * quietly become "dropped": a dropped predicate WIDENS. + */ + it('$icontains is REFUSED, not silently dropped, at both doors', async () => { + expect(() => + compileScopedFilterToSql({ name: { $icontains: 'admin' } } as FilterCondition, 'person'), + ).toThrow(/\$icontains/); + + await expect( + new NativeSQLStrategy().generateSql(query({ name: { $icontains: 'admin' } }), nativeCtx), + ).rejects.toThrow(/\$icontains/); + }); + }); }); diff --git a/packages/services/service-analytics/src/like-pattern.ts b/packages/services/service-analytics/src/like-pattern.ts index 9e850caa76..371e069064 100644 --- a/packages/services/service-analytics/src/like-pattern.ts +++ b/packages/services/service-analytics/src/like-pattern.ts @@ -80,6 +80,50 @@ * character. A third hand-copy of this logic anywhere is the thing to refuse — * import from here, or add a consumer to that test. * + * ## [#6518] Case sensitivity: why this file still emits a plain `LIKE` + * + * #4706 Q2 = A rules the `$contains` family case-SENSITIVE on every backend, and + * #6518 moved the driver family onto that answer — `SqlDriver`'s + * `textMatchPredicate` now picks the construct per DIALECT, because `LIKE` folds + * ASCII on SQLite and follows the collation on MySQL. The obvious question is + * why the compilers here did not move with it, and the answer is measured + * rather than assumed: + * + * 1. **These compilers emit Postgres-shaped SQL, and on Postgres `LIKE` is + * already exactly the ruled semantics.** Both consumers number their + * placeholders `$1`, `$2`, … (`native-sql-strategy.ts`'s `buildFilterClause` + * and `objectql-strategy.ts`'s filter render), and `applyReadScope` / + * `generateSql` rewrite this file's `?` into `$N` on the way out; + * identifiers are `"double quoted"`. Measured on a live PostgreSQL 16 + * against the shared nine-row fixture: `LIKE '%acme%'` answers row 2 alone + * and `LIKE '%ACME%'` answers row 1 alone — case-exact, which is the + * contract. So there is no divergence to close HERE, and changing the + * construct would create one. + * 2. **The RLS fork the issue warned about does not open.** #6518's concern + * was that a driver-only fix would compile one permission rule into two row + * sets. It does not, because the two paths meet only on Postgres — where + * `textMatchPredicate`'s postgres arm is also a plain `LIKE`, unchanged. + * + * What that reasoning DEPENDS on is the dialect, so it is the thing to re-open + * rather than the code: **if these compilers ever emit for SQLite or MySQL, this + * file is wrong and `$contains` silently over-matches there** — on + * `read-scope-sql.ts`'s output that is ADR-0021 read-scope over-reach, not a + * loose filter (#3948). Two things would have to arrive together: a dialect + * input reaching these three compilers, and the per-dialect construct table + * `textMatchPredicate` already carries. Neither exists today and neither is + * invented here on speculation. `__tests__/like-metacharacter-escape.test.ts` + * pins both halves of the claim — that the emitted statement is + * Postgres-shaped, and that the family is compiled case-EXACT — so this + * paragraph goes red rather than merely stale. + * + * `$icontains` is a separate matter and is NOT implemented here at all: this + * package has zero references to it, and both doors refuse an unknown operator + * outright (`filter-normalizer.ts`'s `fieldLeaves` throws + * `invalidFilterError`, `read-scope-sql.ts`'s `compileOperator` throws + * `readScopeCompileError` from its `default:` arm). Unimplemented and + * fail-closed, which is the correct state until #6520 settles the vocabulary + * across the frozen backends too. + * * ## `String(value)` is safe here because nothing unrenderable reaches it (#5234) * * The `String()` below used to be the whole defect on the other side: `String({})` diff --git a/scripts/check-driver-conformance.mjs b/scripts/check-driver-conformance.mjs index 643674407a..218b968ce9 100644 --- a/scripts/check-driver-conformance.mjs +++ b/scripts/check-driver-conformance.mjs @@ -270,41 +270,50 @@ const CASE_SETS = [ // investment is frozen (#5499). Un-freezing it is what should re-run these cells // in CI; until then, this note is the honest state of the mongo column. -// ## FILTER_TEXT_CASES: five DEBT rows, opened by #5701 — read this first +// ## FILTER_TEXT_CASES: two DEBT rows left of the five #5701 opened // // The ledger was EMPTY (see the note above) until `FILTER_TEXT_CASES` arrived. -// These five rows are not a regression in coverage: the case-set is the +// Those five rows were not a regression in coverage: the case-set is the // CONTRACT half of the #4706 ruling, landed deliberately ahead of every -// implementation, and one row per driver is what makes "ahead of" a counted -// fact instead of an assumption. They are cleared by #5702, one suite at a -// time, exactly the way the five before them were. +// implementation, and one row per driver is what made "ahead of" a counted fact +// instead of an assumption. #5702 cleared two of the three requirements; #6518 +// cleared the third on the SQL family and DELETED its three rows. What remains +// is the #5499 frozen family — driver-memory and driver-mongodb — where the +// freeze, not the difficulty, is why the cells are open. // // What the case-set demands, and where each requirement stands: // // 1. `$icontains` — a NEW operator (ASCII-only case fold). **DONE on the SQL -// family** (#5702): driver-sql compiles `LOWER(col) LIKE LOWER(?) ESCAPE ?` -// through the same `applyLike` that carries the `%`/`_`/`\` class, turso's -// remote transport carries the twin in `pushLike`, and sqlite-wasm inherits -// and executes it on sql.js. Still REFUSED (fail-closed, an unimplemented -// capability rather than a live defect) on driver-memory and -// driver-mongodb, which are the #5499 frozen family — tracked as #6520, -// which also explains why the spec's `FILTER_OPERATORS` cannot take -// `$icontains` until those two have arms. +// family** (#5702, retuned by #6518): every SQL face folds through the same +// emitter that carries the escaping, and the fold is now ASCII-only on +// Postgres and MySQL too rather than only on SQLite. Still REFUSED +// (fail-closed, an unimplemented capability rather than a live defect) on +// driver-memory and driver-mongodb, which are the #5499 frozen family — +// tracked as #6520, which also explains why the spec's `FILTER_OPERATORS` +// cannot take `$icontains` until those two have arms. // 2. `$contains` / `$startsWith` / `$endsWith` / `$notContains` must be // case-SENSITIVE (#4706 Q2 = A, superseding `filter.zod.ts`'s former -// "Case sensitivity should be handled at backend level"). **STILL OPEN on -// every driver**, and it is the sole reason these five rows survive #5702. -// No driver delivers it on its live query path: driver-memory and -// driver-mongodb fold the full Unicode range, and the SQL family follows -// its dialect (SQLite — so also turso and sqlite-wasm — folds ASCII; -// Postgres happens to be case-exact already; MySQL depends on collation). -// The one surface that does compare case-sensitively is driver-memory's -// REFERENCE matcher, which is not the path a query takes — see that row. -// Tracked as #6518, filed separately rather than folded into #5702 because -// the lowering exists in THREE places that must move together — the two -// driver compilers and the RLS/analytics twins (`read-scope-sql`, -// `service-analytics`'s `like-pattern.ts`) — and a driver-only change -// would compile one permission rule into two row sets (#3948). +// "Case sensitivity should be handled at backend level"). **DONE on the SQL +// family** (#6518): case sensitivity used to be the DIALECT's answer, so +// `SqlDriver` now picks the construct per dialect — `GLOB` on the SQLite +// dialects (whose `LIKE` folds ASCII), `LIKE` unchanged on Postgres (whose +// `LIKE` is already case-exact), and `LIKE` over `CAST(… AS BINARY)` on +// MySQL (whose answer otherwise follows the column's collation). turso's +// remote transport carries the twin in `pushLike`, and the two are held to +// the same rows by `turso-local-remote-text-parity.test.ts`. **STILL OPEN +// on driver-memory and driver-mongodb**, which fold the full Unicode range +// on their live query paths — and on driver-memory the REFERENCE matcher +// answers the same operator case-sensitively, so that package disagrees +// with itself. Both are the #5499 frozen family; tracked as #6682. +// +// Two faces #6518 measured and did NOT have to change, recorded because +// "not mentioned" reads as "not checked": `formula`'s `matchesFilter` and +// objectql's `having` were already case-exact (`String.prototype.includes` +// and friends), and `service-analytics`'s two SQL compilers emit +// Postgres-shaped statements, where `LIKE` is case-exact by definition. +// That last one is the reason a driver-only change did NOT compile one +// permission rule into two row sets (#3948): the RLS lowering and the +// driver meet only on Postgres, where neither moved. // 3. `$regex` / `$options` must be REFUSED, naming `$icontains`. **DONE on all // five** (#5702), which was blocked until #5710 flipped the last live // producer (`plugin-auth`'s ObjectQL adapter, on the AUTHENTICATION path). @@ -323,95 +332,49 @@ const LEDGER = [ marker: 'FILTER_TEXT_CASES', kind: 'DEBT', why: - 'Re-measured after #5702. Requirement 3 is DONE: `$regex`/`$options` are no longer in ' - + '`SUPPORTED_FIELD_OPERATORS`, the matcher\'s `$regex` arm (the only live regex evaluator in the ' - + 'repo, and the one that answered an ILLEGAL pattern with `false`) is deleted, and both faces refuse ' - + 'them with the spec prescription naming `$icontains`. What is left is requirement 2, and this is ' - + 'still the one row where "which face" changes the answer — do NOT take a single reading here. ' - + 'The QUERY path (`find()` -> `normalizeFieldOperators`, and the analytics face via ' - + '`filterSubstringPattern`) lowers `$contains` to `new RegExp(escapeRegex(v), "i")`: literal comparand ' - + '(requirement 2\'s escaping half holds) but case-INSENSITIVE over the whole Unicode range, which ' - + 'fails requirement 2 and overshoots requirement 1\'s ASCII boundary. The reference matcher ' - + '(`memory-matcher.ts` `match()`, the record-at-a-time evaluator `filter-logic-conformance.ts` counts ' - + 'as a backend) uses String.prototype.includes and is case-SENSITIVE — i.e. this package answers one ' - + '`$contains` two ways today, the divergence class #5374 fixed between the other two faces. Whichever ' - + 'suite clears this cell has to pick one and align both (#6518). `$icontains` is still refused on both ' - + "faces (`SUPPORTED_FIELD_OPERATORS` derives from the spec's FILTER_OPERATORS, which deliberately does " - + 'not carry it yet) — unimplemented but fail-closed, requirement 1 open here and tracked as #6520; ' - + 'this package is in the #5499 frozen family and #5702 left that half suspended by design.', - issue: 'https://github.com/objectstack-ai/objectstack/issues/6518', - }, - { - driver: 'driver-sql', - marker: 'FILTER_TEXT_CASES', - kind: 'DEBT', - why: - 'Re-measured after #5702, which cleared two of the three requirements here. Requirement 3: `$regex` no ' - + 'longer has a `case` arm (it was a fallthrough onto `$contains`) and both retired spellings are ' - + 'refused with the spec prescription. Requirement 1: `$icontains` compiles to ' - + '`LOWER(col) LIKE LOWER(?) ESCAPE ?` through `applyLike`\'s `fold` parameter, so it shares the ' - + '`%`/`_`/`\\` class character-for-character (executed: `%`, `_`, `.` and `\\` all literal), and an ' - + 'empty or non-string comparand is refused on the VALIDATING walk beside `$null`/`$exists`. What is ' - + 'left is requirement 2: case sensitivity is the DIALECT\'s, not the driver\'s — SQLite\'s LIKE folds ' - + 'ASCII, Postgres does not, MySQL follows its collation — so `$contains` fails on two of three dialects ' - + 'and needs a case-exact comparison (GLOB / instr() / a binary collation), not a flag. Tracked as ' - + '#6518, which also carries the mirror defect the same axis produces on `$icontains`: `LOWER()` folds ' - + 'the whole Unicode range on Postgres/MySQL, so the ASCII-only boundary holds on SQLite (measured) and ' - + 'over-folds there. NOTE the consequence for reading this cell: on SQLite `LIKE` already folds ASCII, ' - + 'so `$contains` and `$icontains` return IDENTICAL rows for every comparand until #6518 lands — the ' - + 'fold is pinned by the compiled SQL, which is the only witness this dialect can give.', - issue: 'https://github.com/objectstack-ai/objectstack/issues/6518', - }, - { - driver: 'driver-sqlite-wasm', - marker: 'FILTER_TEXT_CASES', - kind: 'DEBT', - why: - 'Re-measured after #5702: `SqliteWasmDriver extends SqlDriver`, so every fact in the driver-sql row ' - + 'applies unchanged, with the dialect pinned to SQLite — requirements 1 and 3 are DONE and inherited, ' - + 'requirement 2 fails here specifically because LIKE folds ASCII case (#6518). The inheritance is not ' - + 'taken on faith: `sqlite-wasm-icontains-and-retired-operators.test.ts` executes the new predicate on ' - + 'sql.js, because `$icontains` is the first operator this package runs whose compiled form is a ' - + 'FUNCTION CALL on the column with a third bound argument, and a wasm dialect that mis-binds those ' - + 'three positions would fail in no other suite in the repo. Tracked as DEBT rather than EXEMPT for the ' - + 'reason its FILTER_LOGIC row was: "inherits, therefore fine" is the assumption these suites exist to ' - + 'disprove, and what a full run of this case-set would still add is the collation half.', - issue: 'https://github.com/objectstack-ai/objectstack/issues/6518', - }, - { - driver: 'driver-turso', - marker: 'FILTER_TEXT_CASES', - kind: 'DEBT', - why: - 'Re-measured after #5702. DUAL-TRANSPORT, so this cell needs TWO suites like its three predecessors, and ' - + 'both faces moved: local/replica inherits SqlDriver (see the driver-sql row) on the SQLite dialect, ' - + 'while remote does not go through knex at all — `remote-transport.ts` carries its own hand-written ' - + 'SUPPORTED_FILTER_OPERATORS and its own LIKE assembly, so the work had to be written twice. That list ' - + 'now carries `$icontains` and no longer carries `$regex`; `pushLike` grew the same `fold` parameter as ' - + '`applyLike` so the escape rule cannot fork between the two operators (executed against libSQL-shaped ' - + 'SQLite: `%`, `_` and `\\` literal under `$icontains`); a node-position `$regex` moved from the ' - + '"misplaced field operator" tail to the "declared at no level" one. Requirement 2 is what is left on ' - + 'both transports (SQLite LIKE folds ASCII) — #6518.', - issue: 'https://github.com/objectstack-ai/objectstack/issues/6518', + 'Re-measured after #6518, which cleared requirement 2 on the SQL family and NOT here — this package is ' + + 'in the #5499 frozen family, so the freeze rather than the difficulty is why the cell is open. ' + + 'Requirement 3 is DONE (#5702): `$regex`/`$options` are no longer in `SUPPORTED_FIELD_OPERATORS`, the ' + + 'matcher\'s `$regex` arm (the only live regex evaluator in the repo, and the one that answered an ' + + 'ILLEGAL pattern with `false`) is deleted, and both faces refuse them with the spec prescription ' + + 'naming `$icontains`. Requirement 2 is still the one row where "which face" changes the answer — do ' + + 'NOT take a single reading here. The QUERY path (`find()` -> `normalizeFieldOperators`, and the ' + + 'analytics face via `filterSubstringPattern`) lowers `$contains` to `new RegExp(escapeRegex(v), "i")`: ' + + 'literal comparand (requirement 2\'s escaping half holds) but case-INSENSITIVE over the whole Unicode ' + + 'range, which fails requirement 2 and overshoots requirement 1\'s ASCII boundary. The reference ' + + 'matcher (`memory-matcher.ts` `match()`, the record-at-a-time evaluator `filter-logic-conformance.ts` ' + + 'counts as a backend) uses String.prototype.includes and is case-SENSITIVE — i.e. this package ' + + 'answers one `$contains` two ways today, the divergence class #5374 fixed between the other two ' + + 'faces. Whichever suite clears this cell has to pick one and align both: tracked as #6682, which is ' + + 'the successor #6518 left behind for exactly this pair of packages. `$icontains` is still refused on ' + + "both faces (`SUPPORTED_FIELD_OPERATORS` derives from the spec's FILTER_OPERATORS, which deliberately " + + 'does not carry it yet) — unimplemented but fail-closed, requirement 1 open here and tracked as ' + + '#6520. BOTH successors have to land before this row can go: coverage is judged by importing the ' + + 'whole case-set, so a cell that answers one requirement and not the other must not import it.', + issue: 'https://github.com/objectstack-ai/objectstack/issues/6682', }, { driver: 'driver-mongodb', marker: 'FILTER_TEXT_CASES', kind: 'DEBT', why: - 'Re-measured after #5702: still the FURTHEST from the ruling, but the ENVELOPE half is now closed. That ' - + "arm used to throw a bare `new Error('[mongodb] unsupported filter operator …')` — no `code`, no " - + '`status` — three lines from this file\'s own `unsupportedFilterError` helper, which sets ' - + "`INVALID_FILTER` / 400 and which three other refusals here already used. It now routes through the " - + 'helper, and a RETIRED spelling additionally gets the spec prescription naming `$icontains`, so ' - + 'requirement 3 is DONE (mongo was already the only backend REFUSING `$regex`; what was missing was the ' - + 'shape of the refusal). Requirement 2 is inverted here and requirement 1\'s ASCII boundary violated in ' - + 'the same expression: `translateFieldOperators` lowers `$contains`/`$startsWith`/`$endsWith`/' - + '`$notContains` to `$regex` with a HARDCODED `$options: "i"` (#6518). `escapeRegex` does escape ' - + 'metacharacters, so the literal-comparand cases hold. `$icontains` is still refused (#6520). Note this ' - + 'package is in the #5499 frozen family: its real-mongod suites are opt-in, so whatever clears this ' - + 'cell needs a server-free half like `mongodb-filter-logic-translation.test.ts` has.', - issue: 'https://github.com/objectstack-ai/objectstack/issues/6518', + 'Re-measured after #6518, which cleared requirement 2 on the SQL family and NOT here — #5499 freezes ' + + 'this package, so the cell stays open by decision rather than by difficulty. Still the FURTHEST from ' + + 'the ruling, but the ENVELOPE half is closed (#5702): that arm used to throw a bare ' + + "`new Error('[mongodb] unsupported filter operator …')` — no `code`, no `status` — three lines from " + + 'this file\'s own `unsupportedFilterError` helper, which sets `INVALID_FILTER` / 400 and which three ' + + 'other refusals here already used. It now routes through the helper, and a RETIRED spelling ' + + 'additionally gets the spec prescription naming `$icontains`, so requirement 3 is DONE (mongo was ' + + 'already the only backend REFUSING `$regex`; what was missing was the shape of the refusal). ' + + 'Requirement 2 is inverted here and requirement 1\'s ASCII boundary violated in the same expression: ' + + '`translateFieldOperators` lowers `$contains`/`$startsWith`/`$endsWith`/`$notContains` to `$regex` ' + + 'with a HARDCODED `$options: "i"` — tracked as #6682, the successor #6518 left behind for this pair ' + + 'of frozen packages. `escapeRegex` does escape metacharacters, so the literal-comparand cases hold. ' + + '`$icontains` is still refused (#6520), and BOTH successors have to land before this row can go: ' + + 'coverage is judged by importing the whole case-set, so a half-answered cell must not import it. Note ' + + 'this package is in the #5499 frozen family: its real-mongod suites are opt-in, so whatever clears ' + + 'this cell needs a server-free half like `mongodb-filter-logic-translation.test.ts` has.', + issue: 'https://github.com/objectstack-ai/objectstack/issues/6682', }, ];