Repository navigation
fix(objectql): an update-mode validate() preview judges the stored row merged with the patch (#22445) - #22471
Conversation
…th the patch (wip) Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
… names Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
…de text Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
…date-preview-stored-row
…d-row pins Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
…d update Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
…he write's own prior-row read Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
… in the stored-row pins Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 3 package(s): 5 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 2 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 144 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin b0ca47d77975ed526c9adb5c2a49af05014824a5 && git checkout b0ca47d77975ed526c9adb5c2a49af05014824a5
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin f66c440de93c1733683a20b2a107ac8b9c98fc19 f874bb5600b0074d4670aca609699fc90afd519f && git checkout -B drift-repro f66c440de93c1733683a20b2a107ac8b9c98fc19 && git merge --no-ff f874bb5600b0074d4670aca609699fc90afd519f
node scripts/docs-audit/affected-docs.mjs --json f66c440de93c1733683a20b2a107ac8b9c98fc19
|
Contract reviewServed-tier: Inputs: card #22445 (body; triage 6077755047; claim 6078256149; os-dev-report 6079914508), PR #22471 (body, 9-file list, the one bot comment 6079883051, zero reviews), the net diff of the head against its merge base with Check-runs on the head, as read at 2026-10-09T11:38Z (33 runs): 26 ① Derived judgmentsAccept-set and public-surface changes the diff implies, each named right or wrong.
Reach of the widening, measured on Join: at my read the branch is 7 commits behind ② Semver level
③ Boundary flagsDev deviations (os-dev-report 6079914508), each answered:
PR-side flags: the Docs Drift Check (6079883051) named five hand-written docs via Implemented-by: VERDICT: FAIL — on ①-8 alone: the preview judges the raw stored value of a column the read door served partially masked, which is a predicate over a masked column (refused by the repo's own anti-filter-oracle rule) and contradicts the diff's stated invariant and the changeset's sentence; everything else judged above is right. A re-review is owed on the head that closes it (pin + mechanism + changeset text), and the landing also waits for the in-progress check-runs to conclude. Generated by Claude Code |
…date-preview-stored-row
Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
…d transformed as empty A partially masked column keeps its key in the read door's image, so a key-presence filter overlaid its raw stored value. Keep a stored column only where the read door served this caller that very value. Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
…s as its stored value Also states the served-as-stored column rule in the changeset. Claude-Session: https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: Inputs: card #22445 (body; triage 6077755047; claim 6078256149; round-0 report 6079914508; patch-round report 6081033502), PR #22471 (body with its "Patch round 1" section, the 10-file list, the bot comment 6079883051, the earlier record 6080151175 on head Check-runs on the head (42 runs, all ① Derived judgmentsAccept-set and public-surface changes the diff implies, each named right or wrong. Items 1–7 re-verified on this head's diff; 8–15 are this round's.
Reach, measured on Join: at my read the branch is 4 commits behind ② Semver level
Text: the patch round rewrote the "Read under the caller's access" paragraph to state the served-as-stored rule and the measured representation, and the round-0 sentence that was false of the mechanism ("discloses nothing about a row or column the caller could not read") is gone; what the paragraph now says is what the engine does (①-8, ①-9). One sentence remains overbroad: the "Unchanged" bullet's "A merge that really violates a rule is refused, as the write refuses it" — exact for a caller the read door serves in full, and for the merge the preview holds; for a caller served a withheld column the preview judges ③ Boundary flagsPatch-round deviations (os-dev-report 6081033502), each answered:
Round-0 deviations (6079914508) were answered in record 6080151175 and stand: the cross-lane
PR-side flags: the Docs Drift Check (6079883051) was recomputed on the new merge ref and lists the same five hand-written docs via Implemented-by: VERDICT: PASS — ①-8 of the earlier record is closed by a mechanism that keeps only values the caller already received in full, pinned on the masked shape with its precondition asserted and controlled on a real driver; every accept-set and surface change the diff implies is named right above, the one cosmetic wrong (①-13) and the one overbroad changeset sentence (②) move nothing; the level is right; every dev flag is answered and the two standing escalations are the seat's. Generated by Claude Code |
Fixes #22445
Clause-②: yes
Patch round 1 — contract review 6080151175, finding ①-8
The defect the review found. plugin-security's read mask (
field-masker.tsmaskRecord) deletes a hidden field, but a field under a partial-masking rule keeps its key and gets a masked value. The round-0 column filter kept every key the read door returned (hasOwnProperty). So for a partially masked column the preview judged the raw stored value, which gave any caller who can read the row a predicate over a value they only see masked.The mechanism now (
engine.ts,storedRowsblock). A stored column is kept only where the read door served this caller that very value. That is decided byservedAsStored(served, stored), a module-private structural comparison:Object.isdecides primitives.Datecompares by time; arrays and plain objects compare recursively.Every other column is judged as empty. That covers a hidden column (key deleted), a partially masked column (key kept, value replaced), a masked secret, an
internalcolumn, and a read hook's rewrite. So the verdict never depends on a value the caller could not have read in full.RAW_FILE_VALUES_CONTEXT_KEY, the spec-declared opt-out for a caller whose subject is the stored form. A readable file reference therefore compares as the id it stores, not as the read door's expanded object.maskingRulebut not whether this caller holds the unmask permission; plugin-security decides that. A declared-signal rule would judge every masking-rule field empty for every caller, unmasked ones included, and would miss any transformer that is not amaskingRule(a masked secret, a read hook). The comparison asks the read door's own answer for this caller.1f4ca1504b. A throwaway probe (not committed) wrote one row of every common type through a real engine, then compared the write's prior-row read with the read door's row under a user context.driver-sql(SQLite,better-sqlite3): 21 columns, 0 differences. That covers number, currency, percent, a 11-digit number, boolean true and false, date, datetime, JSON, multiselect, select, text, empty text, plusidand the seven injected system columns.driver-memory: 16 columns, 0 differences.registerWriteGateProbebecause it is SYSTEM-elevated. The stored-row read is not elevated: after this filter its image holds only values the read door served this caller unchanged, which the same caller can already read withfindOne. Gating it behind the write probe would protect nothing further. It would also make a preview's verdict depend on a probe that this read does not need.Pins added (
validate-update-stored-row.test.tsnow 18 cases; one new filepackages/rest/src/validate-update-stored-row-representation.test.ts, 2 cases).countrykey and replaces its value with its mask. The precondition asserts the read door servescountry: '**'. The verdict for{ province: 'zj', id }is deep-equal whethercnorusis stored:provinceinvalid_option, judged as empty.cn;provinceinvalid_optiononus.driver-sql/ SQLite, eightn_COLUMNfields carry arequiredWhenover number, boolean, date, datetime, JSON, multiselect, select and text. Clearing everyn_COLUMNgetsrequiredon all eight, from the preview and from the by-id update alike, so each rule judged the stored value. Control: clearing the columns too changes the answer (only the number rule faults:null > 10), so an empty column would have shown.memorequiredfrom both the preview and the write.Ablations through
scripts/ablation-replace.mjs. Each anchor hit once, and each restore was proven by blob == HEAD withgit diff HEADempty.servedAsStoredcondition removed (back to key presence only): 1 failed / 17 passed (d2).RAW_FILE_VALUES_CONTEXT_KEYremoved from the visibility read: 1 failed / 17 passed (the file pin).Sentences corrected.
storedRowsblock comment and thevalidate()docblock now state the served-as-stored rule.Clause-②: yesand theminorlevels are unchanged.protocol.zod.tssays nothing about masking, and its sentence ("a row the caller cannot read is judged on the supplied keys alone") stays true, so it is untouched.Base.
origin/mainwas merged (a6be68f2e6, 9 commits). The merge driver deferredcontent/docs/references/api/protocol.mdx. It was regenerated from the merged tree bycheck:generatedin its regenerate mode and committed in1f18eaddc0, which discharged the deferral. The joint diff carries thelabelrow from #22323 plus this PR'smodetext. The branch is now 3 commits behindorigin/main(f66c440de9). Those commits touch none of this PR's files, so they were not merged.Evidence on head
f874bb5600(builds and tests ran underos-verify-lock):pnpm --filter @objectstack/objectql test: 391 files, 7716 passed;test:repo5 passed.pnpm --filter @objectstack/core test: 85 files, 2261 passed;test:repo48 passed.check:test-typecheckOK.import-*.test.tsfiles plus the new representation file: 10 files, 115 passed. This ran after building@objectstack/rest^...anddriver-memory; the objectql dist carriesservedAsStored.eslint --no-inline-config --format jsonover the 8 changed.tsfiles gave 8 results, 0 errors, 0 warnings. The config enables no type-aware linting, so the diff cannot move a verdict on an untouched file.dispatch-gates --commandsderived 117 commands atf874bb5600. All 117 ran, with exit codes captured before any pipe, and all ended exit 0. Reconciliation with--ran:117 derived, 117 run, 0 NOT-MEASURED, 0 UNRUN.check:skill-examplesandcheck:dual-build-cjs-loads. Both lacked package dists in the fresh worktree. The battery's owncheck:type-check-debtbuild then supplied the dists, and both were re-run at the same head: exit 0.check:adr-anchorsexit 0 andcheck-nul-bytesexit 0.What this changes
An
update-modevalidate()preview now judges what the by-id update judges: the stored row merged with the patch. Before this change, the preview judged the patch alone.packages/objectql/src/engine.ts,validate()). A row that carries itsidis judged against the row that id names. Theidis the address every update door folds into the payload, and it is classified by the write's ownresolveEngineUpdateDispatch. The stored row becomes the rules'previous, and their record is that row merged with the patch, as on the by-id branch ofupdate(). A reference that a traversing rule reads is resolved from the same merge. That retires the named limit the old:13177comment stated.readUpdatePriorRow, and bothupdate()and the preview call it. It is the same driver read, with the samebuildDriverOptionsand the same declared-columns shaping. The preview therefore judges the row as stored, not the read door's served image.findOneunder the caller's own context) whether the caller can see the row. If the read door returns nothing, the preview judges the row on the patch alone. A hidden row therefore gets exactly the verdict a missing row gets. A stored column is kept only where the read door served this caller that very value. A column the read door hid, or served transformed (a partial mask keeps the key and replaces the value), is judged as empty. A refused read propagates as raised.packages/core/src/utils/import-runner.ts). The dry run of a matched row callsvalidateDatawith{ ...data, id: matched.id }. That is the same foldupdateDataapplies to the write. Nothing new crosses the protocol, andValidateDataRequestSchemagains no key.cel-fault.ts,rule-validator.ts). Sometimes a rule reads a column the object declares and the judged record does not hold it (a preview with no stored row). The refusal used to say "which this object does not declare". It now says the value was not supplied and no stored row was read.describeCelFaulttakes an optionalisDeclaredColumn, which the four rule-validator refusal builders answer from the object's fields: field rule, option gate, script and conditional. The answer is true only for a key the source reads directly offrecord/previous. An undeclared key keeps the old sentence. Hook conditions pass nothing and are unchanged.domain:spec). TheValidateDataRequest.modedescription said "updatejudges only the supplied keys, matching a PATCH". TheValidateDataResponsenote said the preview has no prior record. This change makes both partly false, so both are reworded. No key or type changes.content/docs/references/api/protocol.mdxwas regenerated withcheck:generated --fix.Premise, measured on
440bed63e7before the fixengine.validate('showcase_cascade', { province: 'zj' }, { mode: 'update' })refusesprovincewithrule_violation/unevaluable. That is the option gate since objectql:evaluateOptionVisibilitycontinues on a predicate fault, so a select option's server-side gate admits the write — fail open or fail closed, under ADR-0089 (the runtime half of #22394) #22402. The real by-id update{ province: 'zj' }over a storedcountry: 'cn'row is admitted.notecall. The shippedshowcase_cascadedeclares nonotefield, at440bed63e7and at the card's3054516ef1alike. So{ note: 'x' }throwsINVALID_FIELDon the shipped object. The card'snotemeasurement needs a fixture.notewhoserequiredWhenreadscountry. Both card calls refusenoteunevaluable, and{ province: 'zj' }also refusesprovince. The message was "The predicate reads 'country', which this object does not declare". Passingidchanged nothing, because the preview read no row. The by-id update admits.Pins, round 0 (
packages/objectql/src/validate-update-stored-row.test.ts, 15 cases then, 18 now;packages/core/src/utils/import-runner-update-preview-address.test.ts, 3 cases){ province: 'zj' }and{ note: 'x' }, are admitted over a storedcountry: 'cn'row, and the by-id update admits the same patches. Also, a formula column that the read door evaluates is judged the way the write judges it: preview and write agree.{ country: 'us' }getsnoterequired, and{ province: 'ca' }getsprovinceinvalid_option. Preview and write give the same findings, and the store is unchanged.requiredWhenand option refusals say "its value was not supplied" and never "which this object does not declare". Avalidations[]script rule and a conditional rule do the same. Control: an undeclared key (contry) keeps "does not declare".cnorus. A column masked from the caller is judged as empty, with the same verdict whichever value is stored, and the control caller who can read it gets the stored row's verdict.runImportthroughObjectStackProtocolImplementationover a kernel withObjectQLPlugin. The update-mode dry run of a matched row admitsprovinceandnoteand agrees with the committed import. Control:country: 'us'failsnoterequiredin both. The core test pins that the dry run carries the matched id, that the matched id wins over an id cell (as the write's fold makes it win), and that a create row gets no id.Round 0: red before the fix, and ablations (each from a committed state)
440bed63e7and core was rebuilt; the dist marker count fell from 1 to 0. The pin file then went 10 failed / 4 passed. Red: (a) ×2, (b)province: 'ca', (c) ×3, (d) owner and masked-column, (e) ×2. Green, as expected: (b)country: 'us', the (c) undeclared control, (d) hidden-row equality, and the (e) control. Restore leg: blobs equal HEAD,git diff HEADempty, core rebuilt, dist marker back to 1, 14/14 green. The core address test against the base runner: 2 failed / 1 passed (the control passed).scripts/ablation-replace.mjs. Each anchor hit once, and each restore was proven by blob == HEAD withgit diff HEADempty.Tests, round 0 (superseded by the patch round 1 section above)
All runs are on head
31ca6a891aunless a line says otherwise. Builds and tests went throughscripts/pm/os-verify-lock.sh.45c2877b8f. The commits after it change only the changeset and the two pin files, and the pin files were re-run on31ca6a891a.pnpm --filter @objectstack/objectql test: 391 files, 7713 passed.pnpm --filter @objectstack/objectql run test:repo: 1 file, 5 passed.pnpm --filter @objectstack/core test(run ond6fb574835): 84 files, 2235 passed.test:repo: 3 files, 48 passed.pnpm --filter @objectstack/spec test: 629 files, 18779 passed, 1 todo.validate-update-stored-row.test.ts15 passed.import-runner-update-preview-address.test.ts3 passed.pnpm --filter @objectstack/objectql run typecheckandpnpm --filter @objectstack/core run typecheckboth exit 0. Each includescheck:test-typecheckOK, with the new test files compiled under each package'stsconfig.test.json.@objectstack/rest^..., then ran the ninepackages/rest/src/import-*.test.tsfiles (dry-run parity, integration, job, dropped-fields, row report, schema drift, run-automations agreement, historical readonly insert, unique violation): 9 files, 113 passed. The objectql and core dists were confirmed to carry the change (readUpdatePriorRow: 4 hits; the...data, id: existing.idfold: 1 hit).pnpm --filter @objectstack/spec build, thencheck:generated --fix. That regenerated onlycontent/docs/references/api/protocol.mdx. A re-run reports all 15 artifacts up to date.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commandsderived 117 commands at31ca6a891a. All 117 ran with exit codes captured before any pipe, and all exit 0. Reconciliation with--ran:117 derived, 117 run, 0 NOT-MEASURED, 0 UNRUN. An earlier run of the same battery atf4fc729b08found two real reds, both in my test file, and fixed them:check:error-code-casing: a one-linecode: 'rule_violation'with nofield. The assertion now namesfield: '_record'.check:query-options-erasure: anas anyon afindOneoptions bag. The cast was removed.check:skill-examplesneededclient-reactbuilt, andcheck:dual-build-cjs-loadsneeded more dists. Both measured green on the final run.pnpm check:adr-anchorsexit 0, andnode scripts/check-nul-bytes.mjsexit 0.pnpm exec eslint --no-inline-config --format jsonover the 7 changed.tsfiles returned 7 results, 0 errors and 0 warnings. Those files are insideeslint.config.mjs'spackages/**/*.{ts,tsx,mts,cts}and**/*.{ts,...}populations. The config enables no type-aware linting (noparserOptions.project/projectServiceanywhere), so this diff cannot move a verdict on any file it does not touch. The repo-widepnpm lintis CI's.Acceptance notes
validate()binds no master-detailparentheader, in either mode. On a detail object whose field carriesrequiredWhen: "parent.status == 'sent'", the preview refuses withrule_violation/unevaluable(unboundparent), both for an insert under a draft header and for an update of a stored line. The real insert and the real by-id update admit both. The import dry run reaches this throughpreviewVerdict->validateData. Reported for the seat to file; not filed here.readonlyWhenor primary-key strip. It now holds the stored row those strips would judge, so closing this limit is possible, but it is out of this card's scope. TheValidateDataResponsenote keeps stating it.requiredWhenthat reads aformulacolumn (record.doubled > 100) faults with a null overload on every by-id update. The write's prior row is the stored row, which holds no formula value. The preview now agrees with the write here, and a pin holds that agreement. Whetheros validaterefuses such a predicate was not measured.ValidateDataIssueSchemadeclares{ field, code, message }. The engine's findings, whichvalidateDatarelays, also carryconstraint, and an option refusal carriesvalue. The pin file readsconstraintthrough a typed accessor for that reason.findOne, so it fires the object'sbeforeFind/afterFindhooks once per addressed row. The related-row read already did the same throughfind. No write hook runs.notecall. It needed a fixture field: the shippedshowcase_cascadedeclares nonote. The shipped object's ownprovincepick reproduces the defect.origin/mainwas merged once (b7c02e713f). Patch round 1 merged it again; see that section.packages/spec/src/api/protocol.zod.tsis edited: description text and a JSDoc note only, no key or type. The changeset bumps@objectstack/specat minor along with@objectstack/objectqland@objectstack/core.Generated by Claude Code