Repository navigation
Commit 615cba8
Fixes #22646
Clause-②: no (narrowing)
## Summary
A field declared `internal: true` is withheld from every generic exit
(#21197): the engine omits it from every row. The ROW position honoured
that; the generic data door's EVALUATE positions did not, so a value the
platform withholds from rows could still be learned through a position
that answers by its stored value. Class ① product defect, `security` —
class, position and function level only.
Every position now judges the one flag reader
(`collectInternalReadFields` in `@objectstack/objectql`, or its
byte-identical `@objectstack/core` twin
`collectInternalWriteResponseFields` on the door, which sits below
objectql), and a position that reveals a stored value is refused with a
declared ADR-0112 envelope — `INVALID_FIELD` / 400. No new error code;
`Clause-②` stays `no`.
## Measurement (real stack, before)
Measured on `origin/main` as a seeded administrator and an ordinary
member, on the list and query data routes, over a platform object's
`internal: true` credential-digest column and a non-hidden `internal`
text column (the dev's report on #22646 holds the run; ⛔ no request
shapes here):
- **filter** (implicit, `filter=`, query `where`, nested `$or`,
cross-field comparand): served — the predicate reached the driver, a
confirmation oracle.
- **sort** (`$orderby`): served — the order leaked the comparative
stored value.
- **group-by** / **aggregate operand**: refused, but as an undeclared
`500 INTERNAL_ERROR` (`ObjectQL.rejectCredentialAggregation` threw a
bare `Error`).
- **per-aggregation filter** (`aggregations[].filter`) and **filter
beside an aggregation**: served — the same oracle at a second filter
position.
- **$search field list**: a non-hidden `internal` text column entered
the auto-default scan set and an explicit `$searchFields` naming one was
accepted.
- **$expand target field**: a one-level expand's own `where` / `orderBy`
on the target object's `internal` field reached the expansion sub-read —
the expanded record's presence is an oracle on the related row.
- **nested-relation condition** (a condition on a lookup's related
object, at the top level, under `$or`, and in the count total), measured
in patch round 1 on fixture objects: served — the parent row came back
only when the related object's `internal` value matched. The same
condition inside `aggregations[].filter` was already refused
(`INVALID_FILTER`), and the dotted array spelling already refused
(`INVALID_FIELD`).
- **dotted sort / group-by path through a lookup**: not reachable — the
ingress sort gate refuses every spelling (`INVALID_SORT`) and the
group-by gate refuses it as an unknown field (`INVALID_FIELD`); a dotted
sort inside an `$expand` entry is admitted but changes no answer. No
code for this position.
- **`$expand` deeper than one level**: served — a depth-2 entry's own
`where`, and a depth-1 entry whose `where` is a relation condition onto
the grand target, made the expanded record's presence depend on the
stored value.
- **row** position (control): already withheld (omit), unchanged.
## Fix, by location
- `packages/objectql/src/engine.ts`
- `rejectCredentialAggregation`: the refusal now carries the ADR-0112
envelope (`INVALID_FIELD` / 400, located at object + field) instead of
the bare `Error`. Covers the group-by and aggregate-operand positions
for every caller; the `secret` / `password` case that shared the bare
`Error` is upgraded with it.
- `expandSearchOnAst`: `internal` fields are dropped from the set the
search expansion resolves over (a withhold, like the auto-default's
`hidden` / credential-type exclusions), so `$search` never scans one —
for `find` / `findOne` / `aggregate`, and for `searchAll` through them.
- `packages/metadata-protocol/src/protocol.ts` (the generic data door,
`findData`)
- `assertNoInternalFieldEvaluated`: refuses an `internal` field in the
filter, sort, group-by, aggregate-operand and per-aggregation-filter
positions, after the field-existence gates and before the engine — the
shape, envelope and caller-independence of the stored-metadata body/hash
family one axis over.
- Every filter position runs through `refuseInternalInFilter`, which
judges the filter's own reads and then each nested-relation condition
(found by `relationConditionSites`) recursively against the RELATED
object's own flag reader; the refusal names the object that declares the
field.
- `refuseInternalInExpand` walks the whole `$expand` tree: every level's
`where` and `orderBy` against that level's target object.
- `assertSearchFieldsAreSearchable`: an explicit `$searchFields` naming
an `internal` field is refused `INVALID_FIELD` / 400 rather than
silently widened back to the default set.
The engine's privileged consumers (the credential verifier's lookup by
its stored digest under a system context; the share-link / SCIM /
approval-token lookups) call the engine directly and never pass the
generic data door, so authentication is untouched — exactly as #7823
kept the engine's write results whole while the generic-data-path
ingress stripped them. This is pinned by the `NEGATIVE` case in the
dogfood file and by `internal-fields.test.ts`'s "keeps the field usable
as a WHERE filter".
## Compile-surface checklist
No filter COMPILATION semantics change; the fix is a refusal at the
data-door ingress (and the engine aggregate-refusal envelope + the
search field-set withholding), before any filter compiler runs. Each
named surface is **out of scope**:
1. `driver-sql` `applyFilterCondition` (and `driver-sqlite-wasm` / local
`driver-turso` by inheritance) — out of scope: untouched; refusal
precedes the driver.
2. `driver-turso` RemoteTransport `buildWhereSQL` — out of scope:
untouched.
3. service-analytics `compileScopedFilterToSql` — out of scope:
untouched; the analytics door's own internal-field refusal landed in PR
#22645 (#22634).
4. service-analytics `lowerAnalyticsWhere` — out of scope: untouched.
5. `formula` `matchesFilterCondition` — out of scope: untouched.
- half-surface: objectql `applyHaving` / `matchesHaving` — out of scope:
untouched; `having` names the aggregated row's columns, where an
internal field cannot be named.
- `driver-mongodb` `translateFieldOperators` — out of scope: untouched.
## Tests
- New pins: `packages/objectql/src/internal-field-positions.test.ts`
(aggregate envelope + `$search` withhold, with controls),
`packages/metadata-protocol/src/protocol.data-door-internal-field-positions.test.ts`
(every evaluate position refused, controls),
`packages/qa/dogfood/test/internal-field-data-door-positions.dogfood.test.ts`
(member + admin on a real stack, plus the authentication-still-works
negative).
- Each negative pin is ablation-verified: the door refusal, the
aggregate envelope, the `$search` withhold and the
explicit-`$searchFields` arm were each mutated on disk
(`scripts/ablation-replace.mjs`), the pins observed to fail, and the
file restored byte-for-byte (`git diff HEAD` empty).
- `@objectstack/objectql` and `@objectstack/metadata-protocol` build and
typecheck clean. Affected-surface regression: objectql
aggregate/search/judge files (145 tests) and metadata-protocol
data-door/search files (159 tests) pass. Implicated gates run green
locally (changeset family, `error-status-conformance`,
`dispatcher-error-vocabulary`, `engine-double-contract`,
`test-source-alias`, `query-options-erasure`, `where-matcher`,
`filter-alias-parity`, `issue-citations`, `nul-bytes`, and others); the
full farm is CI's.
## Docs
The docs-drift list's hand-written pages were read, the data-door pages
in full. One correction: `content/docs/api/error-catalog.mdx`'s
`INVALID_FIELD` cause said only that a named field does not exist; it
now adds that the same code refuses an existing `internal: true` field
in an evaluate position, naming the positions. No other page claims an
`internal` field is usable in filter, sort, `$search` or `$expand`.
`content/docs/releases/` is untouched.
## Patch round 1 (seat review 6100268814)
New pins: 14 member + administrator refusal cases (relation condition in
four spellings, `$expand` depth-2 filter and sort, depth-1 relation
`where` onto the grand target) and 6 controls;
`protocol.data-door-internal-field-positions.test.ts` 41/41. Ablations
from a committed head, restored byte for byte: the relation-site
recursion emptied (10 of 41 red) and the expand-tree recursion removed
(4 of 41 red). The engine is unchanged this round.
## Scope
Target-OBJECT exposure through `$expand` is a separate axis (#22661).
The cross-object search sweep (`searchAll`) is #22640's region; it
inherits the engine's search-set withholding through `engine.find` with
no change to its own code.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01JfJfBUC3cQ6hhgm9MQK76T
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent c63028e commit 615cba8
8 files changed
Lines changed: 883 additions & 4 deletions
File tree
- .changeset
- content/docs/api
- packages
- metadata-protocol/src
- objectql/src
- qa/dogfood/test
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
147 | 147 | | |
148 | 148 | | |
149 | 149 | | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
150 | 157 | | |
151 | 158 | | |
152 | 159 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
0 commit comments