Skip to content

fix(drivers): text-operator case folding is the contract's answer, not the dialect's (#6518) - #6706

Merged
os-zhuang merged 3 commits into
mainfrom
claude/issue-6518-contains-case-folding
Aug 8, 2026
Merged

fix(drivers): text-operator case folding is the contract's answer, not the dialect's (#6518)#6706
os-zhuang merged 3 commits into
mainfrom
claude/issue-6518-contains-case-folding

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #6518

$contains / $notContains / $startsWith / $endsWith are case-SENSITIVE by contract (#4706 Q2 = A), and $icontains folds ASCII only (#4706 Q1 = A). Neither held: case sensitivity was whatever LIKE happened to mean on the dialect underneath. Both directions over-matched — rows the filter excludes came back — which on an ADR-0021 RLS read scope is over-reach, not a loose filter (#3948).

$contains family — case-SENSITIVE $icontains — ASCII-only fold
SQLite / turso / sqlite-wasm LIKE folds ASCII lower() is ASCII-only
Postgres LIKE is case-exact LOWER() folds all of Unicode
MySQL ❌ follows the collation LOWER() folds all of Unicode

Read across: each dialect was already right on the half another one got wrong. That is why neither half was findable from one backend, and why the acceptance asked for a live PG/MySQL cell.

Premise re-verified against post-#6549 origin/main

#6549 (82397b6) merged the morning this was dispatched and rewrote applyLike / pushLike (adding the fold axis) and the five DEBT rows. Re-measured on d13f627: the premise still holds. applyLike still emitted one LIKE … ESCAPE ? for every dialect and LOWER() for the fold; the five case rows of FILTER_TEXT_CASES were still unanswered and still the sole reason the five DEBT rows survived. premise_still_valid: true.

What is emitted now

One emitter (textMatchPredicate), construct chosen by dialect, so escaping and folding stay a single code path — an unescaped wildcard is a filter bypass, P0 (#5567):

  • SQLite family → GLOB. PRAGMA case_sensitive_like is connection-global (one query would redefine every other query on the connection), so the fold cannot be switched off per statement. Of the operand-level tricks, CAST(col AS BLOB) LIKE ? was measured to match nothing at all — SQLite's LIKE is false for a BLOB operand — so the operator has to change. GLOB is case-exact and brings its own escaped class: *, ?, [[*], [?], [[], because SQLite's grammar gives GLOB no ESCAPE clause. $icontains keeps lower() on both operands.
  • Postgres → LIKE, unchanged (already case-exact). Only the fold moved, from LOWER() to an explicit translate() over the 26 ASCII letters — the mapping is visible in the emitted SQL, so its ASCII-only-ness is structural rather than a property of the server's locale.
  • MySQL → LIKE over CAST(… AS BINARY), byte-wise so no collation decides the case; the fold is 26 REPLACEs over that same binary rendering. Byte-wise ASCII lowering is the ruled fold because UTF-8 is self-synchronising — a byte in 0x41..0x5A can only ever be a real ASCII A..Z. The rejected alternatives are written out on the helper (LOWER() over-folds; LOWER() on a binary string is documented ineffective; CONVERT(… USING ascii) collides café with cafÉ, strictly worse).
  • Any other client keeps the pre-drivers(sql family): 文本算子的大小写折叠是「方言的」而非「契约的」—— $contains 在 SQLite 过折叠、$icontains 在 PG/MySQL 过折叠 #6518 shape — the only form that still runs there — recorded as residue in the ledger rather than left to be discovered.

Per-face verdict table

Every face from the dispatch inventory. "Not mentioned" would read as "missed", so all of them are here.

# face verdict evidence
1 driver-sql applyLiketextMatchPredicate 已改 Per-dialect construct table. All 17 FILTER_TEXT_CASES green on sqlite and on live PG.
2 driver-turso RemoteTransport.pushLike 已改 Independent compiler, moved in the same batch to GLOB + glob escapes. New parity suite pins it to the local face on every case.
driver-sqlite-wasm 已改 (inherited, executed) Inherits face 1; the whole case-set now runs on sql.js. GLOB is supplied by the SQLite build, not the application, so "it inherits" is not taken on faith.
3 service-analytics read-scope-sql 本就合规 Emits ? that both consumers rewrite to $N, with "double quoted" identifiers — Postgres-shaped, and on Postgres LIKE is the ruled semantics (measured live: %acme% → row 2 only, %ACME% → row 1 only). Assumption now written down in like-pattern.ts and pinned by new cases, so it fails rather than rots.
4 service-analytics filter-normalizer + shared like-pattern.ts (+ native-sql-strategy, objectql-strategy) 本就合规 Same Postgres-shaped evidence; both executing compilers assert $N placeholders and no ILIKE/LOWER(.
5 formula matchesFilterCondition 本就合规 (now pinned) String.prototype.includes / startsWith / endsWith compare exactly. Nothing changed — which is exactly why it is pinned: this is the JS baseline the drivers were brought to, and the next mistake here is a "helpful" toLowerCase() that no assertion would have caught.
half objectql having-filter 本就合规 (now pinned) Same mechanism. Pinned because it is the ONE text face with no conformance-table coverage (check-driver-conformance scopes itself to packages/drivers/*), so a fold added here would go red nowhere.
frozen driver-memory (memory-driver.ts RegExp-'i'; memory-matcher.ts includes; analytics face) 明确不在范围 + #5499 Freeze restated as a ruling in the dispatch. Still folds full Unicode on the query path while its reference matcher is case-exact — one package, two answers. DEBT row kept and re-pointed.
frozen driver-mongodb (mongodb-filter.ts hard-coded $options: 'i') 明确不在范围 + #5499 Same. DEBT row kept and re-pointed.

$icontains on faces 3/4, per the dispatch's (d): verified fail-closed, unchanged. The package has zero $icontains references; filter-normalizer.ts's fieldLeaves throws invalidFilterError and read-scope-sql.ts's compileOperator throws from its default: arm. Both are now asserted, so "unimplemented" cannot quietly become "dropped" — a dropped predicate widens. Adding the operator there stays #6520's programme.

Reverse verification — direction predicted BEFORE each run

Three experiments, each reverting one arm, each predicted to fail a different set of cells. That the sets are disjoint is the claim that this issue really is two independent defects rather than one seen twice.

experiment predicted measured
sqlite arm GLOBLIKE exactly the 5 case-sensitivity rows red on sqlite, nothing red on postgres ✅ exactly those 5 case rows (+5 blocks naming the construct directly = 10 reds in that file), no other case in the table moved
drop the column-side lower() (sqlite) the fold becomes observable in ROWS — retiring #5702's "unobservable on SQLite" caveat $icontains: 'acme' and 'ACME' both ['1','2']['2']; $icontains: 'CAFÉ' ['3'][]
postgres fold translate()LOWER() exactly the 2 ASCII-only rows red on the live postgres cell, nothing red on sqlite $icontains: 'café' answered ['3','4'] where the case-set demands ['4']; the 'CAFÉ' mirror answered ['3','4'] where it demands ['3']

The CAST(col AS BLOB) LIKE ? candidate was also measured before being written off, on better-sqlite3 3.53.4 against the nine-row fixture: every shape returned []. GLOB's escape class was measured the same way — unescaped *a*b* returned 6 rows where *a[*]b* returns 1.

Which live cells executed, and which did not

  • sqlite — ran (embedded).
  • live postgresRAN, on a PostgreSQL 16.13 provisioned in-container, database created with the ICU locale provider — the configuration where LOWER('CAFÉ') genuinely is 'café', so the over-fold was reproduced before the fix and is absent after. All 17 cases + the fixture-control green (OS_TEST_POSTGRES_URL set).
  • live mysqlNOT run. No MySQL server was provisionable here (no mysqld, no working Docker daemon). It is a declared skip by name via the testkit's declareUnprovisionedCell, never a faked green, and OS_EXPECT_LIVE_DIALECT_MATRIX=1 turns that skip into a failure for a runner that believes it provisioned one. CI's Temporal Conformance (live PG + MySQL) job already sets both URLs, so the mysql cell executes there. Its arm rests meanwhile on documented behaviour plus a compiled-SQL pin — knex builds a MySQL statement without a server, so the shape (two CAST(… AS BINARY), 52 REPLACEs under the fold, no LOWER() is asserted in-process.

Pin sweep

Swept the whole repo in one pass; every flipped pin asserts the new substance, none was merely deleted, and every refusal assertion is verbatim.

file what moved
driver-sql/sql-driver-icontains-and-retired-operators.test.ts compiled-SQL pin LOWER(…) LIKE LOWER(lower(…) GLOB lower(; now imports FILTER_TEXT_CASES; added case-sensitivity, negation, glob-escape blocks
driver-sql/sql-driver-text-case-conformance.test.ts new — the case-set across DIALECT_CELLS (#5589 conventions) + the per-dialect compiled shape
driver-sqlite-wasm/sqlite-wasm-icontains-…test.ts now imports FILTER_TEXT_CASES; added case-sensitivity + glob-escape blocks on sql.js
driver-turso/remote-transport-text-predicates.test.ts "the two operators agree on SQLite today" → they now differ; LIKE … ESCAPE shape → GLOB, plus a new block asserting */?/[ are escaped and % is not
driver-turso/turso-local-remote-text-parity.test.ts new — case-set on BOTH transports, difference asserted first
driver-turso/remote-transport-{comparand,null-comparand,undefined-comparand}-refusal.test.ts incidental pattern pins %…%*…*, LIKEGLOB; each keeps the claim it was making
formula/matches-filter.test.ts, objectql/having-filter.test.ts new case-sensitivity pins on faces that were already compliant
service-analytics/__tests__/like-metacharacter-escape.test.ts new block pinning the Postgres-shape + case-exactness assumption, and $icontains fail-closed at both doors
scripts/check-driver-conformance.mjs three rows deleted, two re-graded (below)

Searched for and found no other fixture, REST-layer test or doc example asserting a folded row set.

DEBT re-grade

driver              FILTER_TEXT
driver-memory       DEBT   → re-pointed to #6682 (+ #6520)
driver-mongodb      DEBT   → re-pointed to #6682 (+ #6520)
driver-sql          ok
driver-sqlite-wasm  ok
driver-turso        ok
check-driver-conformance: OK — 28 covered cell(s), 2 in the DEBT ledger, 0 exempt.

The two frozen rows could not point at #6518 (it closes here), and #6520 covers only requirement 1 ($icontains). Requirement 2 for those two packages had no successor, so one was filed unassigned, search-first-deduped: #6682. Both rows now say explicitly that both successors must land before the row can go, because coverage is judged by importing the whole case-set — a half-answered cell must not import it.

Changeset level — graded, with reasoning

minor on the three driver packages. Not a downgrade to dodge the ADR-0087 gate: no declared surface moves. $contains still exists, takes the same comparand, and filter.zod.ts is untouched — the case-sensitivity delivered here was already published as the contract by #5701 one release earlier in this same open v17 major, and the drivers were the half that had not caught up. Prime Directive #12 in the direction it points: declared = enforced. Walking the gate's four dispositions honestly, none applies (registered: nothing registered; unpublished: the packages are published; already-registered: no id; no-migration-prescription: refused, since the body does carry migration guidance) — which is the gate saying this is not an ADR-0087-class change. Same grade its sibling #5702/#6549 took for the same operator family in the same rc cycle. The gate confirms: this PR adds no declared-breaking changeset.

The behaviour change is stated prominently in the changeset body regardless: result sets only ever get narrower, and anyone relying on $contains to ignore case should write $icontains.

Gates — enumerated from .github/workflows/lint.yml, run one by one

pnpm lint                                    OK (exit 0)
turbo run typecheck (packages/*, apps/*)     OK — 120 successful, 120 total
pnpm --filter @objectstack/spec exec tsc     OK
pnpm test (whole repo)                       OK — 135 successful, 135 total
check:driver-conformance                     OK — 28 covered, 2 DEBT, 0 exempt
check:query-options-erasure                  OK — test surface 263 sites (at the ceiling, not under it)
check:type-check-debt                        OK — 34 entries re-measured, none above its number
check:engine-double-contract                 OK — 98 pinned, 133 DEBT, 2 exempt
check:nul-bytes                              OK — 6228 files
check:adr-0087-registration                  OK — no declared-breaking changeset
check:empty-changeset                        OK — 1 declaring changeset added

Also green, each run individually: check:slot-lookup, check:verify-stand-in, check:doc-authoring, check:docs-audit-scope, check:role-word, check:quick-reference-counts, check:adr-anchors, check:org-identifier, check:authz-resolver, check:service-providers, check:route-envelope, check:error-code-casing, check:wildcard-fallthrough, check:meta-type-normalized, check:init-service-contract, check:durability-log-level, check:startup-registry-verdict, check:objectui-changeset, check:release-notes, check:release-body, check:node-version, check:workflow-status-functions, check:shard-attestation, check:published-files, check:resume-authority-declared, check:merge-driver, check:spec-parsed-alias, check:stall-guard, check:type-check-coverage, check:i18n, check:i18n-coverage, check:app-nav-i18n, check:skill-frame-sync, check:skill-compatibility, check:prerelease-pins, and the spec-scoped set (check:generated --reconcile-only, check:skill-docs, check:spec-changes, check:upgrade-guide, check:authorable-surface, check:docs, check:skill-refs, check:react-blocks, check:api-surface, check:exported-any, check:dual-source-exports, check:skill-examples), plus check:doc-formula-expressions, examples typecheck and the downstream-contract typecheck.

One red, and it is not in lint.yml: check:objectui-pin-fresh reports the objectui pin as stale. It lives in release.yml / objectui-pin-freshness.yml (the Version-Packages/release path), is a function of repo state versus objectui's main rather than of this diff, and refreshing it would be exactly the pin-bump rider this change must not carry. Left untouched and reported.

Constraints honoured

  • packages/spec source: zero changes — the case-set already encoded the ruled contract, so no spec-side change turned out to be necessary.
  • content/docs/releases/: untouched.
  • No pin bumps.
  • Zero new as any / as never / @ts-ignore. The one as unknown as added is the sanctioned fix(drivers): 聚合函数拒收带上 ADR-0112 信封,并把两类条件分开措辞 (#5907) #6204 named-member spelling, and it replaces rather than adds erosion: asLibsqlClient() in the turso testkit names the Client target type once, where the pre-existing suites each wrote client: stub as never — a cast to the bottom type, which would keep compiling even if client were re-typed to something the stub cannot model. Existing call sites can migrate to it; nothing forces them to here. The new matrix suite reaches knex through the public getKnex() rather than the (driver as any).knex its sibling matrices use.
  • No new fake engines were introduced, so no assertEngineDeleteDispatch site was needed; every new/rewritten rejection case asserts code and status (ADR-0112), never a bare toThrow.

Successor repricing (必答项)

issue effect why
#6409count_distinct in the SQL family unaffected Aggregation projection; shares no code with the text-predicate emitter and no dialect-selection seam with it.
#6543 — unique-violation regex → shared @objectstack/types predicate unaffected Error-message parsing on the write path. Different file, different direction; nothing here touches unique-violation.ts.
#6402 — turso options?: any narrowing very slightly easier remote-transport.ts gains no new any, and the new parity suite exercises both transports through the typed driver surface on 22 cases — narrowing work there now has a row-level regression net it did not have. Not a blocker either way.
#6401GroupByNode.alias unaffected Group-by AST; no overlap with the filter emitter.

Open questions (adjudicated here, per the dispatch — not blocking)

  1. translate() vs lower(x COLLATE "C") on Postgres. Both measured ASCII-only on live PG 16 (C.utf8 and ICU databases). Chose translate(): its ASCII-only-ness is structural — the 26-character mapping is visible in the emitted SQL — where COLLATE "C" is a property of a collation definition a reader has to go look up. Long-term-project axis, and it also makes the MySQL arm read as the same idea rather than an unrelated trick.
  2. MySQL's 26 REPLACEs are verbose (~52 per $icontains predicate). Accepted: the alternatives are wrong rather than merely uglier (spelled out on the helper), the predicate was a full scan either way, and correctness that a reviewer can verify by reading beats brevity that needs a live server to trust.
  3. 'unknown' dialect keeps the old shape. GLOB is a syntax error and CAST(… AS BINARY) means something else outside the three modelled clients, so the old form is the only one that runs. Recorded as residue in the ledger and pinned in the matrix rather than left silent.
  4. service-analytics was not given a dialect seam. Measured: the package genuinely has no dialect input, and its compilers emit Postgres-shaped SQL where the ruled semantics already hold, so adding one would have been speculative surface for no behaviour change. The dependency is now the thing written down — like-pattern.ts says in as many words that the file is wrong the day these compilers emit for SQLite or MySQL, and the pin fails if the emitted shape stops being Postgres-shaped.

Out-of-scope findings


Generated by Claude Code

`$contains` 族与 `$icontains` 的大小写答案此前取决于底层数据库的 `LIKE`
语义,两个方向都是**过匹配**——返回了 filter 排除的行,在 ADR-0021 的 RLS
读作用域上是越权而非「宽松过滤」(#3948):

| | `$contains` 族(契约要求大小写敏感,#4706 Q2=A) | `$icontains`(只折叠 ASCII,#4706 Q1=A) |
|:--|:--|:--|
| SQLite / turso / sqlite-wasm | ❌ `LIKE` 自带 ASCII 折叠 | ✅ `lower()` 只折 ASCII |
| Postgres | ✅ `LIKE` 恰好大小写精确 | ❌ `LOWER()` 折全 Unicode |
| MySQL | ❌ 随 collation | ❌ `LOWER()` 折全 Unicode |

横着读:**每个方言恰好在另一个方言犯错的那一半上是对的**——这就是两半都
无法从单一后端上被发现的原因。

下译改为按方言选构造,仍然只有一个发射点(转义与折叠是同一条代码路径,
未转义的通配符是 P0 —— #5567):

- **SQLite 族 → `GLOB`**。`LIKE` 的 ASCII 折叠无法按语句关闭
  (`PRAGMA case_sensitive_like` 是连接级全局开关),实测
  `CAST(col AS BLOB) LIKE ?` 一行都不匹配。`GLOB` 大小写精确,并自带转义类
  (`*` / `?` / `[` 写作自闭合字符类,因为 SQLite 语法不给 `GLOB` 任何
  `ESCAPE` 子句)。
- **Postgres → `LIKE` 不变**,只把折叠从 `LOWER()` 换成显式 `translate()`;
  live PG 16 实测 `LOWER('CAFÉ')` 就是 `'café'`。
- **MySQL → `CAST(… AS BINARY)` 上的 `LIKE`**,比较按字节,collation 不再
  参与;折叠同样按字节做,因 UTF-8 自同步故恰为 ASCII-only。
- 其余 client 保留改动前的 `LIKE`/`LOWER()` 形状,并记为残留而非静默。

turso 的 remote transport 自带一份编译器,同批移动;两个 transport 由新的
`turso-local-remote-text-parity.test.ts` 用共享 `FILTER_TEXT_CASES` 钉在同
一组行上。driver-sql / -sqlite-wasm / -turso 三格的 `FILTER_TEXT_CASES`
DEBT 随之清偿;driver-memory / driver-mongodb 属 #5499 冻结族,如实留 DEBT
并改指新填的后继单 #6682Fixes #6518

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AYvVzTaCWDCC32e9Le9zXn
@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 8, 2026 1:25pm

Request Review

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/driver-sql, @objectstack/driver-turso, @objectstack/service-analytics.

17 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/api/data-api.mdx (via @objectstack/service-analytics)
  • content/docs/api/index.mdx (via @objectstack/service-analytics)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-sql, @objectstack/driver-turso)
  • content/docs/deployment/cli.mdx (via @objectstack/driver-turso)
  • content/docs/deployment/environment-variables.mdx (via @objectstack/driver-turso)
  • content/docs/deployment/self-hosting.mdx (via @objectstack/driver-turso)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-sql, @objectstack/driver-turso)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-sql, @objectstack/service-analytics)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/service-analytics)
  • content/docs/plugins/anatomy.mdx (via @objectstack/driver-sql)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-sql, @objectstack/driver-turso, @objectstack/service-analytics)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/driver-sql)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/driver-sql)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-sql)
  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-sql, @objectstack/service-analytics)
  • content/docs/releases/v17.mdx (via @objectstack/service-analytics)
  • content/docs/releases/v9.mdx (via @objectstack/service-analytics)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Aug 8, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review August 8, 2026 13:48
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 8, 2026
Merged via the queue into main with commit 3172831 Aug 8, 2026
25 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-6518-contains-case-folding branch August 8, 2026 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

1 participant