Skip to content

fix(driver-sql,driver-memory,driver-mongodb): 越界过滤输入在门口拒收,而不是每个后端各答一个 (#5347, #5348) - #5368

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-5347-refuse-out-of-contract-filter-input
Aug 5, 2026
Merged

fix(driver-sql,driver-memory,driver-mongodb): 越界过滤输入在门口拒收,而不是每个后端各答一个 (#5347, #5348)#5368
os-zhuang merged 2 commits into
mainfrom
claude/issue-5347-refuse-out-of-contract-filter-input

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #5347
Fixes #5348

两条是同一形状的两面:契约声明之外的输入没有在门口被拦下,于是每个后端都给了它一个「答案」——而且互不相同。 #5348节点位置的越界 $op 被当成列名编译成静默空集;#5347字段位置的越界比较值被读成相反的意思。两条都改为 INVALID_FILTER / 400。

PM 已裁定 #5347A(两边都拒收非布尔比较值),本 PR 不重新论证 B/C。


一、现场核对(sql-driver.ts 今天已被改四次)

worktree 基于 origin/main@e18e3da77(含 60a7a2d37,即 #5349),逐条重核两个 issue 引用的行号:

issue 引用 现场 结论
#5347sql-driver.ts:6363case '$null': opValue === false ? whereNotNull : whereNull 现在在 :6363 附近#5243/#5296/#5327/#5329 之后仍在 applyFilterCondition 的发射器 switch 内),代码逐字未变 属实
#5347nullValueSatisfiesOperatorcase '$null': return value !== false 在 :866(#5296 引入),逐字未变 属实
#5347:driver-memory normalizeFieldOperatorsif (val === true) … else … memory-driver.ts:973#5349 大改后仍逐字未变 属实
#5348reduceFilterKey 对不认识的 $return 'clause',注释「A field key always contributes a predicate」 在 :832(#5327 在它上面加了 {field:{}} 闸门,注释也说明了为什么闸门放在校验遍历而非编译分支) 属实,#5327 的落点理由逐字适用于本单
#5348:driver-memory 文档级半边已由 #5349 补上 filter-refusal.tsassertFilterConditionShapeif (key.startsWith('$')) throw unknownLogicalOperatorError(...) 属实

四次 churn 对本单的实际影响:#5327 决定了两条闸门的落点。它把 {field:{}} 的拒收放在校验遍历 reduceFilterKey 而非编译分支,理由是编译分支会被布尔单位元短路。本单两条闸门落在同一层、同一位置,理由相同——PR 里有专门的用例证明这一点(见下「闸门落点」)。

一处与 issue 正文有出入,已按现场记录:#5348 说「$expr 之类同样静默」。实测 $expr / $elemMatch 在改前其实会抛错,但抛的是字段级的错,把它们当成了字段名:

{"$expr":{"$eq":["$stage","won"]}}  =>  THROW  Operator "$eq" on field "$expr" requires a single comparable value…
{"$elemMatch":{"stage":"won"}}      =>  THROW  Unsupported filter operator "stage" on field "$elemMatch"…

——报错信息把 $expr 当作字段名,是偶然因为它们的值恰好是带 $ 键的对象才撞上了字段级守卫。病因与 $where/$nor 完全相同,消息误导性更强。改后统一为节点级措辞。


二、四(五)个后端的实测

fixture:两行——{id:'1', stage:'won'}{id:'2', stage:null}加上 stage: null 的第二行是本单的关键#5347 的 fixture 只有一行非空值,而那正是让 driver-memory 两个面的分歧看不出来的原因。

#5347 { stage: { $null: 'yes' } } — 改前

后端 读成 返回
driver-sql IS NULL(除 false 外都算 true) ["2"]
driver-sqlite-wasm 同上(继承) ["2"]
cloud TursoDriver(local/replica,继承 SqlDriver) 同上 ["2"]
driver-memory find(mingo) IS NOT NULL(除 true 外都算 false) ["1"]
driver-mongodb translateFilter {"stage":{"$ne":null}} = 同上
driver-memory match(参考匹配器) 约束整条消失 ["1","2"]
cloud RemoteTransport(remote 模式) WHERE "stage" IS NULL

第三行是 issue 没能看到的新事实。 matcher 的 $null arm 写成两条 iftarget === true && … / target === false && …),第三种值两条都不满足,于是该算子不再约束任何东西——匹配全表。在只有非空值那一行的 fixture 上,「匹配全表」和「IS NOT NULL」答案相同,所以 #5347 记录成了「两个面一致」。加一行 stage: null 就分开了:driver-memory 包内部的两个面本身就不一致,而且 matcher 这一侧是放大方向——在 RLS read scope 上是权限绕过,不是降级过滤(#3948)。

这条把「必须拒收」的理由从「两个巧合挑一个」加强成「三个答案,其中一个是放大」。

#5348 节点位置越界 $op — 改前

driver-sql            {"$where":"return true"}    =>  RESOLVED []
driver-sql            {"$nor":[{"stage":"won"}]}  =>  RESOLVED []
driver-sql            {"$or":[{"$where":"x"}]}    =>  RESOLVED []
driver-sqlite-wasm    同上三条                     =>  RESOLVED []
TursoDriver (local)   同上三条                     =>  RESOLVED []
driver-memory         {"$where":…}                =>  THROW INVALID_FILTER 400   (#5349 已修)

SQLite 把解析不到列的双引号名降级成字符串字面量,所以查询编译了、跑了、返回零行——正是 #5041 记录的那个形状。其他方言会报「未知标识符」,症状不同、病因相同;本 PR 的闸门在编译之前,与方言无关。

改后(同一组输入)

SQL   $null:'yes' / 1 / 0 / null / undefined / {} / 'false'  =>  THROW INVALID_FILTER 400
SQL   $null:true  => ["2"]     $null:false => ["1"]          (逐条不变)
SQL   $where / $nor / $expr / $elemMatch(顶层与组合子内)      =>  THROW INVALID_FILTER 400
SQL   legit $and/$or/$not、$in、$between、eq、{}               (逐条不变)
MEMfind / MEMmatch  两个面对以上每条给出逐字相同的消息
MONGO $null:'yes' → THROW;$null:true → {"stage":{"$eq":null}}(不变)
WASM / TURSO-LOCAL  随基类,全部 THROW INVALID_FILTER 400

三、闸门落点与理由

两条都落在校验遍历上,不在发射器里:

理由与 #5327 逐字相同:发射器会被布尔单位元整段跳过。PR 里有两条用例专门证明这一点,改前它们都 RESOLVED:

{ $or: [ { stage: 'won' },  { $where: 'x' } ] }   // 可满足的兄弟分支
{ $or: [ {},                { $nor: []   } ] }   // TRUE 单位元兄弟分支

若把闸门放进发射器,这两条会「因兄弟节点不同而被拒收或被忽略」——reduceFilterNode 自己的注释称之为 "a gate conditional on evaluation order"。


四、守卫表(#5296 nullValueSatisfiesOperator)的收紧结论

$null 收紧,$exists 不动。

case '$null':   return value === true;   // was: value !== false
case '$exists': return value === false;  // unchanged, deliberately
  • $null:拒收之后到达这张表的只可能是 true / falsevalue !== falsevalue === true 等价。仍然收紧,因为等价只在上游闸门存在时成立;宽松写法在闸门被移动或删除后会静默恢复对没人裁定过的形状作答,而严格写法不会。这与拒收本身是同一个「declared = enforced」反射。
  • $exists没有配套的比较值闸门($null 的比较值不是布尔时,driver-sql 与 driver-memory 给出**完全相反**的答案(一个 IS NULL,一个 IS NOT NULL)—— 实测 #5347 只裁定了 $null),所以非布尔仍会到达这张表,守卫必须继续与发射器的 opValue === false 读法一致,否则两者会对同一行给出不同答案。单独收紧它才是制造分叉,不是修复分叉。 另行立单(见范围外清单)。

五、cloud / Turso 面(维护者新增常设规则:SQL 单一律连带排查)

两条路性质不同,实测结论也不同。本 PR 不改 cloud 代码(跨仓 PR 做不到),缺口已在 cloud 仓立单。

1. TursoDriver extends SqlDriver(local / replica)— 随基类自动生效,已实测

不是静态推断:把 cloud 的 probe worktree 的 @objectstack/driver-sql 软链指向本分支的 worktree,跑 TursoDriver({url:':memory:'}) 实测,改前改后各一遍,结果见上表——改前与 driver-sql 逐条相同(两个缺陷都在),改后两条拒收逐条生效,code=INVALID_FILTER status=400

关于 :670-678temporalFilterValue 透传表:确认不构成绕过。该表在 toRemoteFieldSpec 内,只被 toRemoteFiltertoRemoteQuery 调用,而 toRemoteQuery 只服务 remote 模式;local/replica 模式根本不经过它。且即便经过,$null 在那张表里是原样透传、不决定语义,而本 PR 的闸门读的是原始值、在编译之前——透传的值仍是被检查的那个值。

2. RemoteTransport.buildWhereSQL(remote 模式)— 独立的第五个 SQL 编译器,两条缺口都在

PM 的预判「#5348 那一半在 remote 侧可能已经关上了」经实测不成立。 default: arm 的「Declared = enforced」注释、:1003isFilterNode + uncompilableWhere:1356uncompilableSubFilter 都只覆盖顶层 where 的形状字段级算子,节点位置的 $ 键没有任何门:

{"$where":"return true"}    =>  SELECT * FROM "deal" WHERE "$where" = ?    args=["return true"]
{"$expr":{"$eq":1}}         =>  SELECT * FROM "deal" WHERE "$expr" = ?     args=[1]
{"$or":[{"$where":"x"}]}    =>  SELECT * FROM "deal" WHERE (("$where" = ?)) args=["x"]

——与 driver-sql 改前逐字同款:编成列名、语句真的发出去、静默空集。$nor / $elemMatch 只是偶然#1058/#1066 的比较值守卫拦下,且抛的是code、无 status 的裸 Error([RemoteTransport] 前缀也在)。

$null 侧:

$null:'yes' / 1 / 0 / null / {} / 'false'   =>  WHERE "stage" IS NULL

——正是 driver-sql 的 === false 约定。所以 A 落地后,remote Turso 成为非布尔 $null 上唯一的少数派(framework 四家全部拒收,remote Turso 仍编译 IS NULL)。

已在 cloud 仓立单(立单前已搜重复),关联本仓 #5347 / #5348

产出面计数修正:不是四个后端,是五个 SQL 产出面 —— driver-sql、driver-sqlite-wasm、TursoDriver(继承)、RemoteTransport(独立),加 driver-memory / driver-mongodb 两个非 SQL 后端。本 PR 覆盖 framework 侧全部四家;RemoteTransport 归 cloud 单。


六、测试

文件 覆盖
driver-sql/src/sql-driver-out-of-contract-filter-input.test.ts 22 例:#5348$where/$nor/$expr/$elemMatch × 顶层/$or/$and/$not;两条「闸门落点」用例;三个已声明组合子与布尔单位元不变;#5347 七种非布尔比较值 + 组合子内 + $not 内;true/false 逐条不变;$exists 明确不收紧;合法 filter 与 regex 族逐条不变
driver-memory/src/memory-null-comparand-refusal.test.ts 19 例:每条都在 findmatch 两个面各断言一次,且断言两个面消息逐字相同;路径名(filter.$or[1].stage.$null);可满足兄弟/TRUE 单位元不短路
driver-mongodb/src/mongodb-null-comparand-refusal.test.ts 11 例:七种非布尔 + 组合子路径 + true/false 不变 + 周边词表不变 + FilterArray 信封不回归
driver-sqlite-wasm/src/sqlite-wasm-out-of-contract-filter-input.test.ts 9 例:继承的交付证明(自定义 sql.js dialect 下 code/status 不被吞掉/重包)

driver-mongodb 为什么测 translateFilter 而不是真 mongod:该包的 live 套件依赖 mongodb-memory-server(~123MB 下载,本地被墙时 describe.skipIf 跳过,见 test-mongod.ts)——一个会被跳过的测试不是对裁定的测试。而 translateFilter 是纯函数、是本包唯一读 $null 的地方、其输出就是发给 MongoDB 的查询,分叉完整地落在这一层。mongodb-filter.test.ts 也正是在这一层钉 $null: true/false 的。

反向验证(stash 掉源码,新用例必须失败)

git stash push 五个源文件后重跑四个新套件,47 条失败,失败原文:

########## @objectstack/driver-sql ##########
⎯⎯⎯⎯⎯⎯ Failed Tests 17 ⎯⎯⎯⎯⎯⎯⎯
Error: expected the driver to refuse this filter, but it resolved
AssertionError: expected 'Operator "$eq" on field "$expr" requi…' to contain 'filter.$expr'
AssertionError: expected 'Unsupported filter operator "stage" o…' to contain 'filter.$elemMatch'

########## @objectstack/driver-memory ##########
⎯⎯⎯⎯⎯⎯ Failed Tests 16 ⎯⎯⎯⎯⎯⎯⎯
Error: expected the live query path to refuse this filter, but it resolved
Error: expected the reference matcher to refuse this filter, but it answered

########## @objectstack/driver-mongodb ##########
⎯⎯⎯⎯⎯⎯⎯ Failed Tests 8 ⎯⎯⎯⎯⎯⎯⎯
Error: expected the translator to refuse this filter, but it translated
 Tests  8 failed | 3 passed (11)

########## @objectstack/driver-sqlite-wasm ##########
⎯⎯⎯⎯⎯⎯⎯ Failed Tests 6 ⎯⎯⎯⎯⎯⎯⎯
Error: expected the driver to refuse this filter, but it resolved
 Tests  6 failed | 3 passed (9)

(mongodb 的 3 passed 与 wasm 的 3 passed 是「不变」类断言,本就该在改前也通过。)

全量与连带面

@objectstack/driver-sql          63 files passed | 4 skipped   851 tests passed | 44 skipped
@objectstack/driver-memory       16 files passed               424 passed
@objectstack/driver-mongodb       8 files passed | 5 skipped   150 passed | 121 skipped
@objectstack/driver-sqlite-wasm  18 files passed               246 passed
@objectstack/objectql           118 files passed              1908 passed
@objectstack/plugin-security     34 files passed               731 passed
@objectstack/runtime             90 files passed              1333 passed
@objectstack/service-analytics   45 files passed               695 passed
@objectstack/formula             16 files passed               357 passed
typecheck(以上受影响 + 连带包)  35 tasks successful
eslint(全部改动/新增文件)        0 problems,无 `as any`

本地跳过 / CI 才跑的盲区(如实核查)

grep -rnE 'skipIf|describe\.skip|it\.skip|test\.skip|\.todo\(|\.only\('

  • driver-sql 4 个文件 / 44 例:全部 describe.skipIf(!URL),门在活 Postgres / MySQL 连接串上(sql-driver-datetime-postgres-timezonesql-driver-datetime-mysql-storagesql-driver-time-live-dialectssql-driver-date-now-default-live)。全部是 temporal 存储形态,不触及过滤拒收。本 PR 的闸门在编译之前(校验遍历),任何方言都到不了 SQL 生成阶段,所以方言差异无法改变拒收结果;driver-sql 把**文档级**未声明 $op({$where:…}{$nor:[…]})当成列名编译,静默返回空结果集 —— #5324 的文档级一半在 SQL 侧还在 #5348症状是方言相关的(SQLite 降级为字符串字面量 vs 其他方言报未知标识符),修复不是
  • driver-mongodb 5 个文件 / 121 例:全部 describe.skipIf(!sharedMongod),门在 mongodb-memory-server 的 ~123MB 下载上,本容器下载不通所以本地跳过、CI 能下时会跑。逐一核查是否会被本改动顶红:其中 mongodb-filter-logic-conformance / mongodb-temporal-conformance / mongodb-pagination-conformance 消费 spec 的共享表 FILTER_LOGIC_CASES / TEMPORAL_CASESgrep '$null|$where|$nor|$expr|$elemMatch' packages/spec/src/data/filter-logic-conformance.ts 无非布尔 $null、无节点位置越界 $op(唯一命中是一句提到 $nor 的注释),故不受影响。mongodb-driver / mongodb-datetime-storage / mongodb-findone-query 同样不含这两类输入。新增的 mongodb 用例刻意放在 translateFilter 层,因此不进这个盲区、到处都跑。
  • .only( / .todo( / it.skip

七、合并 main

合并 origin/main@b4ad98435(无冲突)。重叠面:driver-sql 包新进了 adr0120-three-posture-conformance.test.ts。按 AGENTS.md §9/§10 已 pnpm install --frozen-lockfile + 重建 + 重跑四个 driver 包与 typecheck,全绿(上表即合并后的数字)。未触及 packages/spec,无生成物待重生(os-regen-pending 为空)。


八、范围外清单(均已按 Prime Directive #10 处理,本 PR 一行未改)

Changeset

.changeset/refuse-out-of-contract-filter-input.md,三个包 patch,正文写明两条可观察的行为变更:靠 truthy/falsy 巧合工作的非布尔 $null 调用方会拿到 400(含 "false" 字符串这个 truthy 陷阱的完整说明与 FROM → TO 修法),文档级越界 $op 从静默空集变为 400。


🤖 Generated with Claude Code

https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7


Generated by Claude Code

claude added 2 commits August 5, 2026 01:47
…filter input at the door (#5347, #5348)

Two shapes the Filter Protocol never declared reached the drivers, and every
driver ANSWERED them — with a different answer. Both are now refused with
INVALID_FILTER / 400, on the validating walk rather than in the emitter.

#5347 — `$null` with a non-boolean comparand. `FieldOperatorsSchema` declares
`$null: z.boolean()`. Measured against one row with `stage: 'won'` and one with
`stage: null`, `{ stage: { $null: 'yes' } }` returned the NULL row on
driver-sql / driver-sqlite-wasm / Turso local (IS NULL — anything but `false`),
the valued row on driver-memory's query path and driver-mongodb (IS NOT NULL —
anything but `true`), and BOTH rows through driver-memory's reference matcher,
whose two conditionals a third value satisfies neither of, so the constraint
vanished. Three readings of one declared operator; the third is new evidence the
issue's own fixture could not show. Refused on all four backends per the ruling.

#5348 — an undeclared `$op` in a node position. `FilterConditionSchema` declares
three `$`-keys at a node; driver-sql compiled the rest as COLUMNS, so
`{ $where: … }` / `{ $nor: … }` produced a predicate matching nothing and
reporting nothing. Its FIELD position had refused the same class of input since
#3948/#4436, so one driver answered two ways depending on depth.

Both gates sit in `reduceFilterKey` / `assertFilterConditionShape`, not in the
emitters, because the emitters are skipped wholesale by a boolean identity —
`{ $or: [ {}, { $where: … } ] }` would otherwise be refused or ignored depending
on its siblings. Same placement argument as #5240/#5327.

`nullValueSatisfiesOperator`'s `$null` arm is tightened from `value !== false`
to `value === true`: the two are equivalent only while the refusal holds, and
the lenient spelling would silently resume answering if the gate ever moved.
`$exists` keeps its lenient read deliberately — it has no comparand gate, so
tightening it alone would create the divergence rather than close one.

driver-sqlite-wasm and cloud's local/replica TursoDriver inherit both refusals
from SqlDriver; both verified by execution, not assumed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
@vercel

vercel Bot commented Aug 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 5, 2026 1:56am

Request Review

@github-actions github-actions Bot added size/xl documentation Improvements or additions to documentation tests tooling labels Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/driver-memory, @objectstack/driver-mongodb, @objectstack/driver-sql.

12 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-memory, @objectstack/driver-mongodb, @objectstack/driver-sql)
  • content/docs/deployment/vercel.mdx (via @objectstack/driver-memory)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-memory, @objectstack/driver-mongodb, @objectstack/driver-sql)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-memory, @objectstack/driver-mongodb, @objectstack/driver-sql)
  • content/docs/permissions/authentication.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/anatomy.mdx (via @objectstack/driver-sql)
  • content/docs/plugins/index.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-memory, @objectstack/driver-mongodb, @objectstack/driver-sql)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/driver-sql)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/driver-sql)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-memory, @objectstack/driver-mongodb, @objectstack/driver-sql)
  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-memory, @objectstack/driver-mongodb, @objectstack/driver-sql)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang marked this pull request as ready for review August 5, 2026 02:05
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 5, 2026
Merged via the queue into main with commit 9c5abf4 Aug 5, 2026
24 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-5347-refuse-out-of-contract-filter-input branch August 5, 2026 02:17
os-zhuang pushed a commit that referenced this pull request Aug 5, 2026
…ity-batch

Textual conflict: packages/plugins/driver-mongodb/src/mongodb-filter.ts —
both sides rewrote translateFilter/translateCondition (#5239 reduction vs
#5329 array-dialect deletion + #5368 $null gate/path threading). Resolution
keeps both: main's array refusal and path threading, this branch's
three-valued reduction and shape gates; the three helpers both sides defined
(unsupportedFilterError, describeFilterOperand, safeShapePreview) are
de-duplicated onto main's copies.

Semantic reconciliation the textual merge could not see (AGENTS.md s10):
main's #5347 $null comparand gate sat in the emitter, and this branch's
reduction makes emitters skippable by a boolean identity — { $or: [ {},
{ stage: { $null: 'yes' } } ] } would have translated to match-all while
driver-sql refuses it. The gate's load-bearing copy moved onto the
validating walk (reduceFilterKey), mirroring driver-sql's #5368 placement;
the emitter arm keeps its local check. Pinned in
mongodb-null-comparand-refusal.test.ts (three identity-sibling fixtures).

Fixture triage: the 'legacy array dialect is untouched' pin in
mongodb-filter-boolean-identity.test.ts pinned a dialect #5329 deleted —
replaced wholesale with the surviving boundary ([] = absent filter =
match-all, non-empty array refused before the reduction runs).

The reduceFilterKey field-arm comment on { field: {} } is recalibrated to
current main: #5327 gated the shape on the other four backends; this driver
remaining the one still answering it is now tracked by #5376.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

2 participants