Skip to content

Commit 9ebb8ae

Browse files
committed
fix(objectql)!: the engine refuses evaluating an internal field for every caller but a system context
The `internal: true` evaluate refusal (filter, sort, group key, aggregation operand, aggregation filter, and the same positions inside expand) is now defined once, in the engine, and answers the data door's 400 INVALID_FIELD with the door's sentence. - objectql: find, findOne, count and aggregate apply it to every caller but a system context; ObjectQL#assertNoInternalFieldEvaluated answers it for any caller, with a per-hop hook a door hands its own exposure question. - spec: QueryEvaluationGate, InternalFieldEvaluationQuery and InternalFieldEvaluationOptions on the objectql slot contract. - metadata-protocol: the data door asks the engine's refusal instead of keeping its own copy; on an engine without the gate it still asks its relation-target exposure question at every hop. - runtime: the HTTP MCP data bridge asks the engine's refusal directly. - mcp: the stdio data bridge's copy is retired in favour of the engine's. Claude-Session: https://claude.ai/code/session_01FEkQ5nqYhjhRCqHrhVA59A Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 7fa9873 commit 9ebb8ae

24 files changed

Lines changed: 1276 additions & 492 deletions
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
'@objectstack/spec': patch
3+
'@objectstack/objectql': patch
4+
'@objectstack/metadata-protocol': patch
5+
'@objectstack/runtime': patch
6+
'@objectstack/mcp': patch
7+
---
8+
9+
fix(objectql)!: the engine refuses filtering, sorting, grouping and aggregating on an `internal: true` field for every caller but a system context
10+
11+
Clause-②: yes (narrowing)
12+
Metadata-protocol: none
13+
14+
<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and nothing an author writes is refused: the `internal: true` flag is read as before. The narrowing is a runtime one, on reads made as a user: a query that evaluates a flagged field answers 400 INVALID_FIELD from the engine itself, with the data door's sentence. The published type change adds a member to the `objectql` slot contract and is satisfied by the one implementer, ObjectQL. There is nothing for `objectstack migrate meta` to rewrite and no tombstone; the remedy is the refusal's own sentence (use another field), or running the server's own read as a system context. -->
15+
Skills-docs: updated here — `content/docs/permissions/system-context.mdx` (row 18e: the engine's read path) and `content/docs/api/error-catalog.mdx` (`INVALID_FIELD`) state who is refused and who is served; no file under `skills/**` teaches filtering on an `internal: true` field, so no skill line changes.
16+
17+
**BREAKING** for one shape only: server code that reads the engine as a user (not as a system context) and filters, sorts, groups or aggregates on a field declared `internal: true`.
18+
19+
- FROM: the engine ran such a read.
20+
- TO: the engine refuses it with the data door's `400 INVALID_FIELD`, naming the object, the field and the position, before anything is read. A read made as a system context (`isSystem: true`) is served as before, and the flagged column is still never returned on a row.
21+
22+
What to do: run a server-side lookup that matches on a flagged field as a system context, or match on another field.
23+
24+
- **`@objectstack/objectql`**: the `internal: true` evaluate refusal is defined once, in the engine. `find`, `findOne`, `count` and `aggregate` apply it to every caller but a system context, in every position the data door judges: a filter (including a relation hop and a `$field` comparand), a sort, a group key, an aggregation operand and an aggregation's own filter, and the same positions inside `expand`. `ObjectQL#assertNoInternalFieldEvaluated` answers the same question for any caller, for a door that judges before it reads.
25+
- **`@objectstack/spec`**: new types on the `objectql` slot contract: `QueryEvaluationGate`, `InternalFieldEvaluationQuery` and `InternalFieldEvaluationOptions`. `IObjectQLEngine` extends `QueryEvaluationGate`.
26+
- **`@objectstack/metadata-protocol`**: the data door asks the engine's refusal instead of keeping its own copy. Its answers are unchanged.
27+
- **`@objectstack/runtime`**: the HTTP MCP data bridge asks the engine's refusal directly. Its answers are the data door's.
28+
- **`@objectstack/mcp`**: the stdio data bridge's own copy of the refusal is retired in favour of the engine's. Its answers are the data door's.
29+
30+
The level is patch rather than the minor a new spec type would take: the changeset gate refuses `minor` beside `Metadata-protocol: none`, and nothing an author writes is refused.

‎content/docs/api/error-catalog.mdx‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,10 @@ when a read names it in a position that would evaluate its stored value: a filte
158158
aggregation's own `filter`), a sort, a `groupBy`, an `aggregations[].field`, a
159159
`searchFields` entry, or an `expand` entry's own filter or sort at any depth. Such a
160160
field is never returned on a generic read, so it cannot be queried by either — the
161-
refusal names the object that declares it. Query by another field.
161+
refusal names the object that declares it. Query by another field. The engine
162+
itself gives the same refusal on its own reads for every caller but a system
163+
context; server code that matches on such a field runs its lookup as a system
164+
context.
162165
A non-blank `search` is refused (`param: 'search'`), not answered with every row,
163166
when the object's resolved search set is empty: no `searchableFields` entry names a
164167
stored field, and the default search scans none (`internal: true` fields never

‎content/docs/permissions/system-context.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,7 @@ bugs — it is the protection or the side effect that silently does not happen.
123123
| 18b | Settings namespace read scope not applied | service-settings | Get: a read of `sys_setting`, `sys_setting_audit` or `sys_platform_setting` across every namespace, whatever each namespace's `readPermission` names — the settings service's own reads of its stores take this path. Lose: the per-namespace read gate the generic data API applies to every other caller, the same rule the settings routes enforce | `settingsReadDoorMiddleware` |
124124
| 18c | **Credential-column filter / sort refusal not applied** — a `where`, `orderBy`, per-aggregation `filter` or `expand` entry naming a `password` / `secret` column is evaluated | objectql | Get: the server's own match on a stored credential value, a lookup that never returns it. Every other caller, administrators included, is refused `400 INVALID_FIELD` for anything but a has-a-value check (ADR-0100). Lose: nothing a wire caller reaches — `isSystem` is never set on an inbound request | `rejectCredentialEvaluation` |
125125
| 18d | **Stored-metadata type read gate not applied** — a read of `sys_metadata` / `sys_metadata_history` is not narrowed by the capability its row's type requires | objectql | Get: every row of the family on `find` / `findOne` / `count` / `aggregate` — the metadata loader, the `/meta` doors' own store reads, boot (a read with no execution context is not narrowed either). Lose: the narrowing that serves a stored row of a capability-gated type (`datasource`, `external_catalog`) only to holders of the capability the `/meta` type gate names for it | `storedMetadataTypeReadFilter` |
126+
| 18e | **`internal: true` evaluate refusal not applied on the engine's read path** — a filter, sort, group key, aggregation operand, per-aggregation `filter`, nested-relation condition or `expand` entry naming a field declared `internal: true` is evaluated | objectql | Get: the server's own match on a flagged column (the API-key, share-link, SCIM-credential and approval-token lookups, the better-auth adapter), which still never returns the value. Every other caller of `find` / `findOne` / `count` / `aggregate`, administrators included, is refused `400 INVALID_FIELD`; the data door asks the same judgement for every caller, a system context included. Lose: nothing a wire caller reaches — `isSystem` is never set on an inbound request | `rejectInternalFieldEvaluation` |
126127

127128
### 2. Write pipeline and data integrity
128129

@@ -544,6 +545,7 @@ and CI refuses one that differs from what the tree produces.
544545
| `packages/objectql/src/engine.ts#insert` | 3 | 21, 22 |
545546
| `packages/objectql/src/engine.ts#isSystem` | 1 | 64 |
546547
| `packages/objectql/src/engine.ts#update` | 4 | 19, 20 |
548+
| `packages/objectql/src/internal-field-evaluation-guard.ts#rejectInternalFieldEvaluation` | 1 | 18e |
547549
| `packages/objectql/src/stored-metadata-type-read-gate.ts#storedMetadataTypeReadFilter` | 1 | 18d |
548550
| `packages/observability/src/perf-timing.ts#isPerfDisclosurePrincipal` | 1 | 13 |
549551
| `packages/plugins/organizations/src/organizations-plugin.ts#start` | 1 | 62 |

‎packages/core/src/security/second-object-read-exposure.pin.test.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,16 @@ const CALLERS: Record<string, Classification> = {
218218
"serves nothing of the target — the expanded read itself is the engine's `expandRelatedRecords`. Pinned by " +
219219
'`packages/objectql/src/credential-evaluation-guard.test.ts` (the expand refusals assert the driver was never asked)',
220220
},
221+
// The `internal: true` evaluate refusal's expand half (QueryEvaluationGate).
222+
'packages/objectql/src/internal-field-evaluation-guard.ts': {
223+
kind: 'not-a-served-read',
224+
reason:
225+
"names the object an `expand` entry the caller wrote targets, only to read that object's DECLARED `internal: true` " +
226+
'fields from the registry and refuse (`INVALID_FIELD` / 400) a filter or sort key on one; it reads no record and ' +
227+
"serves nothing of the target. A door's own exposure question runs first at each hop, through the gate's " +
228+
'`beforeRelationTarget` hook (the data door hands it `refuseUnservedRelationTarget`). Pinned by ' +
229+
'`packages/objectql/src/internal-field-evaluation-guard.test.ts` (the refusals assert the driver was never asked)',
230+
},
221231
// [#25522 cut 10] The write-path reference check, moved out of engine.ts.
222232
'packages/objectql/src/engine-referential-integrity.ts': {
223233
kind: 'not-a-served-read',

‎packages/mcp/src/stdio-data-bridge.internal-field-evaluation.test.ts‎

Lines changed: 55 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,20 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
22

33
/**
4-
* The stdio data bridge asks the data door's internal-field gate — the
5-
* `protocol` service's `assertNoInternalFieldEvaluated` — before its engine
6-
* reads anything, on `query` (the call's `where` and `orderBy`) and on
7-
* `aggregate` (`where`, `groupBy`, `aggregations`), and a refusal it throws
8-
* reaches the caller as thrown: the tool renders its `code`, `status` and
9-
* sentence, and the engine is never asked.
4+
* The stdio data bridge leaves the `internal: true` evaluate refusal to the
5+
* ENGINE: it hands the engine the caller's query under the caller's principal
6+
* on `query` (`where`, `orderBy`) and on `aggregate` (`where`, `groupBy`,
7+
* `aggregations`), never asks a `protocol` service for it (a host may register
8+
* none, or a protocol other than the shipped one), and a refusal the engine
9+
* throws reaches the caller as thrown: the tool renders its `code`, `status`
10+
* and sentence.
1011
*
11-
* The gate itself is the protocol's and is pinned there
12-
* (`@objectstack/metadata-protocol`, the public-face cases beside the data
13-
* door's position enumeration), and the whole path on a real stack in
14-
* `@objectstack/dogfood`. This file pins the bridge's half: that it asks, with
15-
* what, before what — and that a host whose protocol has no such face (or that
16-
* registers no protocol) is served exactly as before. Refusals are asserted on
17-
* the envelope, never on a bare throw.
12+
* The refusal itself is the engine's one definition and is pinned there
13+
* (`@objectstack/objectql`, `internal-field-evaluation-guard.test.ts`, which
14+
* also boots a kernel with no protocol service), and the whole path on a real
15+
* stack, with and without a protocol service, in `@objectstack/dogfood`. This
16+
* file pins the bridge's half. Refusals are asserted on the envelope, never on
17+
* a bare throw.
1818
*/
1919

2020
import { describe, it, expect, vi } from 'vitest';
@@ -23,40 +23,41 @@ import { Client } from '@modelcontextprotocol/sdk/client/index.js';
2323
import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js';
2424
import type { ExecutionContext } from '@objectstack/spec/kernel';
2525
import type { IDataEngine, IMetadataService } from '@objectstack/spec/contracts';
26-
import { createStdioDataBridge, type StdioProtocolObjectReads } from './stdio-data-bridge.js';
26+
import { createStdioDataBridge } from './stdio-data-bridge.js';
2727
import { registerObjectTools } from './mcp-http-tools.js';
2828

2929
const OBJECT = 'ifx_vault';
3030
const FLAGGED = 'pin';
3131
const DEF = { name: OBJECT, fields: { name: { type: 'text' }, [FLAGGED]: { type: 'text' } } };
3232

33-
/** The door's refusal as the protocol throws it (fields only; the sentence is the protocol's to pin). */
34-
function doorRefusal(param: string) {
35-
return Object.assign(new Error(`Cannot evaluate '${OBJECT}' by '${FLAGGED}' (${param}): the query was not run.`), {
33+
/** The engine's refusal (fields only; the sentence is the engine's to pin). */
34+
function engineRefusal(param: string) {
35+
return Object.assign(new Error(`Cannot filter '${OBJECT}' by '${FLAGGED}' (${param}): the query was not run.`), {
3636
code: 'INVALID_FIELD', status: 400, httpStatus: 400, object: OBJECT, field: FLAGGED, fields: [FLAGGED], param,
3737
});
3838
}
3939

40-
/** A stand-in for the protocol's gate: refuses any query whose JSON names the flagged field. */
41-
function gate() {
42-
return vi.fn((_object: string, query: Record<string, unknown>) => {
43-
if (JSON.stringify(query).includes(`"${FLAGGED}"`)) throw doorRefusal('filter');
44-
});
45-
}
40+
const PRINCIPAL = { userId: 'u1', isSystem: false } as unknown as ExecutionContext;
4641

47-
function makeBridge(protocol?: StdioProtocolObjectReads) {
42+
/** A stand-in engine that refuses any query whose JSON names the flagged field, as the real one does. */
43+
function makeBridge(withProtocolFace = false) {
44+
const refuses = (query: unknown) => {
45+
if (JSON.stringify(query).includes(`"${FLAGGED}"`)) throw engineRefusal('filter');
46+
};
4847
const engine = {
49-
find: vi.fn(async () => [{ id: 'r1', name: 'alpha' }]),
50-
aggregate: vi.fn(async () => [{ n: 1 }]),
48+
find: vi.fn(async (_object: string, query: unknown) => { refuses(query); return [{ id: 'r1', name: 'alpha' }]; }),
49+
aggregate: vi.fn(async (_object: string, query: unknown) => { refuses(query); return [{ n: 1 }]; }),
5150
};
5251
const metadataService = { getObject: vi.fn(async (name: string) => (name === OBJECT ? DEF : undefined)) };
52+
// A protocol that still carries a face of that name: the bridge must not ask it.
53+
const protocolFace = vi.fn();
5354
const bridge = createStdioDataBridge({
5455
engine: engine as unknown as IDataEngine,
5556
metadataService: metadataService as unknown as IMetadataService,
56-
...(protocol ? { protocol: () => protocol } : {}),
57-
resolvePrincipal: async () => ({ userId: 'u1', isSystem: false }) as unknown as ExecutionContext,
57+
...(withProtocolFace ? { protocol: () => ({ assertNoInternalFieldEvaluated: protocolFace }) as never } : {}),
58+
resolvePrincipal: async () => PRINCIPAL,
5859
});
59-
return { bridge, engine };
60+
return { bridge, engine, protocolFace };
6061
}
6162

6263
const refusalOf = async (run: () => Promise<unknown>) => {
@@ -66,24 +67,22 @@ const refusalOf = async (run: () => Promise<unknown>) => {
6667
};
6768
const REFUSED = { code: 'INVALID_FIELD', status: 400, field: FLAGGED, object: OBJECT };
6869

69-
describe('the stdio bridge asks the protocol\'s internal-field gate before the engine', () => {
70-
it('query: the gate is asked with the call\'s where and orderBy, and its refusal stops the read', async () => {
71-
const assertNoInternalFieldEvaluated = gate();
72-
const { bridge, engine } = makeBridge({ assertNoInternalFieldEvaluated });
70+
describe('the stdio bridge leaves the internal-field refusal to the engine', () => {
71+
it('query: the engine gets the call\'s where and orderBy under the principal, and its refusal reaches the caller', async () => {
72+
const { bridge, engine } = makeBridge();
7373
const where = { [FLAGGED]: { $startsWith: 'x' } };
7474
const orderBy = [{ field: 'name', order: 'asc' as const }];
7575
expect(await refusalOf(() => bridge.query(OBJECT, { where, orderBy, limit: 5 }))).toEqual(REFUSED);
76-
expect(assertNoInternalFieldEvaluated).toHaveBeenCalledWith(OBJECT, { where, orderBy });
77-
expect(engine.find).not.toHaveBeenCalled();
76+
expect(engine.find).toHaveBeenCalledTimes(1);
77+
expect(engine.find.mock.calls[0]![1]).toMatchObject({ where, orderBy });
78+
expect((engine.find.mock.calls[0] as unknown[])[2]).toEqual({ context: PRINCIPAL });
7879

7980
const sorted = [{ field: FLAGGED, order: 'desc' as const }];
8081
expect(await refusalOf(() => bridge.query(OBJECT, { orderBy: sorted }))).toEqual(REFUSED);
81-
expect(engine.find).not.toHaveBeenCalled();
8282
});
8383

84-
it('aggregate: the gate is asked with where, groupBy and aggregations, and its refusal stops the aggregate', async () => {
85-
const assertNoInternalFieldEvaluated = gate();
86-
const { bridge, engine } = makeBridge({ assertNoInternalFieldEvaluated });
84+
it('aggregate: the engine gets where, groupBy and aggregations under the principal, and its refusal reaches the caller', async () => {
85+
const { bridge, engine } = makeBridge();
8786
const bodies = [
8887
{ where: { [FLAGGED]: 'x' }, aggregations: [{ function: 'count' as const, alias: 'n' }] },
8988
{ groupBy: [FLAGGED], aggregations: [{ function: 'count' as const, alias: 'n' }] },
@@ -92,39 +91,31 @@ describe('the stdio bridge asks the protocol\'s internal-field gate before the e
9291
];
9392
for (const body of bodies) {
9493
expect(await refusalOf(() => bridge.aggregate!(OBJECT, body))).toEqual(REFUSED);
95-
expect(assertNoInternalFieldEvaluated).toHaveBeenLastCalledWith(OBJECT, {
96-
where: (body as { where?: unknown }).where,
97-
groupBy: (body as { groupBy?: unknown }).groupBy,
98-
aggregations: body.aggregations,
99-
});
94+
expect(engine.aggregate.mock.lastCall![1]).toMatchObject({ ...body, context: PRINCIPAL });
10095
}
101-
expect(engine.aggregate).not.toHaveBeenCalled();
10296
});
10397

104-
it('CONTROL — a query the gate passes reaches the engine unchanged, on both verbs', async () => {
105-
const assertNoInternalFieldEvaluated = gate();
106-
const { bridge, engine } = makeBridge({ assertNoInternalFieldEvaluated });
107-
const where = { name: 'alpha' };
108-
await bridge.query(OBJECT, { where, orderBy: [{ field: 'name', order: 'asc' }] });
98+
it('never asks a protocol service for it, even one carrying a face of that name', async () => {
99+
const { bridge, engine, protocolFace } = makeBridge(true);
100+
expect(await refusalOf(() => bridge.query(OBJECT, { where: { [FLAGGED]: 'x' } }))).toEqual(REFUSED);
101+
expect(await refusalOf(() => bridge.aggregate!(OBJECT, { groupBy: [FLAGGED], aggregations: [{ function: 'count', alias: 'n' }] }))).toEqual(REFUSED);
102+
expect(protocolFace).not.toHaveBeenCalled();
109103
expect(engine.find).toHaveBeenCalledTimes(1);
110-
expect((engine.find.mock.calls[0] as unknown[])[1]).toMatchObject({ where, orderBy: [{ field: 'name', order: 'asc' }] });
111-
await bridge.aggregate!(OBJECT, { where, groupBy: ['name'], aggregations: [{ function: 'count', alias: 'n' }] });
112104
expect(engine.aggregate).toHaveBeenCalledTimes(1);
113-
expect(assertNoInternalFieldEvaluated).toHaveBeenCalledTimes(2);
114105
});
115106

116-
it('CONTROL — no protocol, or one without the face: served as before (no door to agree with)', async () => {
117-
for (const protocol of [undefined, {} as StdioProtocolObjectReads]) {
118-
const { bridge, engine } = makeBridge(protocol);
119-
await bridge.query(OBJECT, { where: { [FLAGGED]: 'x' } });
120-
await bridge.aggregate!(OBJECT, { where: { [FLAGGED]: 'x' }, aggregations: [{ function: 'count', alias: 'n' }] });
121-
expect(engine.find).toHaveBeenCalledTimes(1);
122-
expect(engine.aggregate).toHaveBeenCalledTimes(1);
123-
}
107+
it('CONTROL — an ordinary-field query is served by the engine, on both verbs', async () => {
108+
const { bridge, engine } = makeBridge();
109+
const where = { name: 'alpha' };
110+
const rows = await bridge.query(OBJECT, { where, orderBy: [{ field: 'name', order: 'asc' }] });
111+
expect(engine.find).toHaveBeenCalledTimes(1);
112+
expect(JSON.stringify(rows)).toContain('alpha');
113+
await bridge.aggregate!(OBJECT, { where, groupBy: ['name'], aggregations: [{ function: 'count', alias: 'n' }] });
114+
expect(engine.aggregate).toHaveBeenCalledTimes(1);
124115
});
125116

126-
it('query_records and aggregate_records render the gate\'s refusal as its envelope', async () => {
127-
const { bridge, engine } = makeBridge({ assertNoInternalFieldEvaluated: gate() });
117+
it('query_records and aggregate_records render the engine\'s refusal as its envelope', async () => {
118+
const { bridge } = makeBridge();
128119
const server = new McpServer({ name: 'pin', version: '0' });
129120
registerObjectTools(server, bridge);
130121
const [clientSide, serverSide] = InMemoryTransport.createLinkedPair();
@@ -140,10 +131,8 @@ describe('the stdio bridge asks the protocol\'s internal-field gate before the e
140131
expect(res.isError, name).toBe(true);
141132
const { error } = JSON.parse(res.content[0]!.text) as { error: { code: unknown; status: unknown; message: string } };
142133
expect({ code: error.code, status: error.status }).toEqual({ code: 'INVALID_FIELD', status: 400 });
143-
expect(error.message).toBe(doorRefusal('filter').message);
134+
expect(error.message).toBe(engineRefusal('filter').message);
144135
}
145-
expect(engine.find).not.toHaveBeenCalled();
146-
expect(engine.aggregate).not.toHaveBeenCalled();
147136
} finally {
148137
await client.close();
149138
}

0 commit comments

Comments
 (0)