Repository navigation
feat(service-storage)!: retire the sys_file scope option public and rewrite stored public rows to user (#22443) - #22552
Conversation
…d its one-time rewrite to user The sys_file scope select no longer lists public: the value promised a public file and stored a private one, and the upload doors already refuse it. Rows a deployment already stored with it are rewritten to user by an operator-run sweep in the shape of the organization backfill (plan / apply / run, dry run first and by default, counts before it writes, a no-op at zero, idempotent). One column moves; the storage key and the bytes do not. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…or step and its rollback The test's engine reads now pass typed query options instead of erasing them. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…torage bundles Regenerated with check-i18n-bundles --write; no other key moves. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…py first Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…eady-registered storage-scope-public-retired) Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…s-file-public-scope-backfill Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…s-file-public-scope-backfill Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 2 package(s): 24 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 13 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 141 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin fd3a0e8bf1fcb25cb5fac36f58f63a94f84ea5a6 && git checkout fd3a0e8bf1fcb25cb5fac36f58f63a94f84ea5a6
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 6a3f82efa7d588267eb2e52c06a73c56315c636a 4d77d54a544d9f9c3d51a4e6b7ec96e34d259fbf && git checkout -B drift-repro 6a3f82efa7d588267eb2e52c06a73c56315c636a && git merge --no-ff 4d77d54a544d9f9c3d51a4e6b7ec96e34d259fbf
node scripts/docs-audit/affected-docs.mjs --json 6a3f82efa7d588267eb2e52c06a73c56315c636a
|
…s-file-public-scope-backfill
…e package entry plan / apply / run / format and their report types are exported beside backfillFileReferences. Until the sweep runs, a copy of a stored public row is refused, and a deployment consumes the published package, not a source checkout, so the operator step has to ship in the release that asks for it. The module docblock's usage section now imports from the package. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…ys_file option's retirement and its operator sweep Its reason said files already stored with scope public are not touched. The sys_file scope select now retires the option too, and the operator sweep @objectstack/service-storage exports rewrites those rows to user with no access change. The reason says so, and the acceptance criteria add the sweep's end state. The changeset names the package import for the operator step and gains a patch line for @objectstack/spec. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…edit pnpm --filter @objectstack/spec gen:migration-registry; the only change is the storage-scope-public-retired entry. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…enant-audit census node scripts/tenant-audit-census.mjs --write over the generated region and the counts ledger, plus the hand-written page counts the gate holds to it: 241 write call sites (from 240), 160 decidable, 105 of undecidable elevation, 53 named through a const. The new site is placed as the organization backfill's is: update, sys_file, tenancy enabled, context of undecidable elevation. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…the operator step itself Dropping one of the four exports from the entry left every test and every derived export gate green (measured), so this case imports the sweep and its report types from ./index.js, asserts the entry hands out the very functions this file drives, and runs plan, apply and a second run through the entry against a stored public row. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
…s-file-public-scope-backfill
Contract reviewServed-tier: PR #22552 (card #22443, ruling ① Derived judgments
② Semver level
③ Boundary flagsFrom
From
Nothing is escalated beyond the ② finding, which is the seat's to fix in a patch round; every other item is judged right at this head, so the next record is a check that the two declaration lines moved and nothing else did. Implemented-by: VERDICT: FAIL |
…d narrowing The diff adds four functions and four types to the service-storage package entry, a widening of its published surface, beside the narrowing of the sys_file scope accept set. The repo's reader of the line spells that case as yes (narrowing). The levels are unchanged: service-storage minor already meets what yes requires, spec stays patch. Claude-Session: https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: PR #22552 (card #22443, ruling The hop The net diff against Check-runs on the head at the time of this read: 34 in all; 28 success, 2 skipped (Console Pin Gate and Packed-tarball smoke: a path filter and an opt-in), 4 in progress, 0 failure; the combined commit status is success (1 context). In progress: ① Derived judgmentsEach of
② Semver level
③ Boundary flagsCarried from
New at this head:
Nothing is escalated. The one blocking finding of Implemented-by: VERDICT: PASS |
Fixes #22443
Clause-②: yes (narrowing)
The second half of #22443, as ruled (ruling
6081131776, letter B): thesys_file.scopeoptionpublicretires, and rows already stored with it are rewritten touserby a one-time operator sweep in the shape of thesys_fileorganization backfill. The first half (PR #22469, merged asee8751d41e) retiredpublicfromStorageScopeSchemaand refused it at both upload doors; it is not redone here.The ruling, verbatim:
What lands
sys_file.scope(packages/services/service-storage/src/objects/system-file.object.ts): thepublicoption is gone. The options areuser,tenant,private,temp,attachments. The engine now refuses a write ofpublicto the column:VALIDATION_FAILED,invalid_optiononscope.packages/services/service-storage/src/backfill-sys-file-public-scope.ts, new):planSysFilePublicScopeBackfill/applySysFilePublicScopeBackfill/runSysFilePublicScopeBackfill/formatSysFilePublicScopeBackfillReport. It scansscope = 'public'in id order, reads the whole population before writing, writes ONE column (scope: 'user') per row, and records a failed row without retrying it. It is a dry run by default, and it reports a scan it could not make, or a scan that stopped at its row ceiling, instead of reading as clean. Its four functions and their report types are exported from the package entry, besidebackfillFileReferences(patch round 1, seat answer Q2): a deployment runs the published package, not a source checkout, and until the sweep runs a copy of a storedpublicrow is refused.objects.generated.tsbundles lose theirpublicoption label (regenerated withcheck-i18n-bundles.mjs --write; no other key moves).storage-scope-public-retired(packages/spec/src/migrations/entries/semantic/18.storage-scope-public-retired.ts, plusregistry.tsregenerated withgen:migration-registry; seat answer Q3): its reason no longer says files already stored with scope public are not touched. It names thesys_fileoption's retirement and the operator sweep the package exports, which rewrites those records touserwith no access change, and its acceptance criteria add the sweep's end state (a dry run scans zero such records). Nospec-changes.json/ upgrade-guide regeneration is owed: since spec(changes): generate the per-major spec-changes section and the upgrade guide at publish; the pull request generates both in memory and renders the diff (#22449 B′, condition 1) #22533 both are generated in memory at check time and at publish.node scripts/tenant-audit-census.mjs --writeplaces the sweep's one write as it places the organization backfill's (update·sys_file· tenancy enabled · context of undecidable elevation), and the hand-written page counts the gate holds move with it: 241 write call sites (from 240), 160 decidable, 105 of undecidable elevation, 53 named through aconst..changeset/22443-sys-file-public-scope-rewrite.md:minoron@objectstack/service-storageandpatchon@objectstack/spec(the entry text), declared breaking (Clause-②: yes (narrowing): the entry export widens the package surface while the option retirement narrows the stored vocabulary), dispositionnot-required (already-registered storage-scope-public-retired). It states the data rewrite, the operator step (the package import) with the exact failure it prevents, the rollback, and the entry's correction.No governed surface is touched. 13 files, +757 / −40 (797 changed lines; numstat against merge base
4638625e07).Measurements: the four hypotheses
All on a real
ObjectQLover a realSqlDriver(sqlite:memory:), with the realSystemFile(itspublicoption removed) and the real field-reference hooks. A storedpublicrow is written through the DRIVER, because the engine on this branch refuses to write one, which is the state of a deployment upgrading from the previous release.H1, nothing reads
public. Holds. Re-measured atfaf6348508, acrosspackages/**and at the objectui pinf0268ad784. The only scope value any code compares against isattachments:storage-routes.ts:457, the download doors' gate (file.scope === 'attachments' || fieldOwned).attachment-lifecycle.ts:103, the tombstone hook (only attachments-scope committed files).attachment-lifecycle.ts:663, the reap guard (attachments-scope or not).stranded-orphan-inventory.ts:290and:303, the orphan inventory (where: { scope: 'attachments' }).verify-file-references.ts:289, the reference verifier (skips attachments-scope rows).file-reference-lifecycle.ts:439,copyOwnedFile. It does not branch on the value: it copies the source's scope onto the copy and uses it as the copy's key prefix, defaulting a missing one touser.Not readers: the local and S3 adapters read neither the scope nor the key prefix.
sys_upload_session.scopeis free text written at the chunked start and never read back. The upload gate atstorage-routes.ts:388reads the request, not a stored row. Outsideservice-storage, no package compares or filters on asys_filescope. At the objectui pin, every.scope ===comparison is about something else (settings, package manifests, CEL hints, SDUI props). Sopublic→userchanges no access behaviour.H2, the window between upgrade and the operator's run. Measured. The failure is loud only in the server log. On an un-swept row:
find,findOne, includingwhere: { scope: 'public' }) pass the row through;scopesucceeds, with a system context or none;publicfile another field already owns reachescopyOwnedFile, whose insert carriesscope: 'public'and is refusedinvalid_option. The hook throwsERR_FILE_REFERENCE_COPY(code, no status). Driven through@objectstack/rest's own classifiers (mapDataErrorandsendThrownError, a throwaway in-process probe), that is answered500{ error: 'Internal server error', code: 'INTERNAL_ERROR' }. The sentence (… Scope must be one of: user, tenant, private, temp, attachments) reaches only the server log.I judge the precedent's shape safe here and did not add a boot hook. The failure fails closed, nothing is lost or widened, and it is limited to deployments that stored
publicrows: no in-repo or objectui producer ever wrote one, and since PR #22469 no door can. The changeset names the operator step and this exact failure. The reachability gap, a sweep a published install could not import, is closed by seat answer Q2 below.H3, the inverse. Measured. After the sweep,
engine.update('sys_file', { id, scope: 'public' })is refusedVALIDATION_FAILED,invalid_optiononscope(pinned). A raw driver write ofpubliclands (measured in the probe, not pinned). So the rollback is the inverse on the ids the applied report lists (rows[], printed by the formatter), performed with a code rollback: on the previous release, through the engine there or as one raw driver statement againstsys_filefiltered to those ids. The module docblock and the changeset both say so.H4, counts first and idempotent. Holds.
scannedis the count taken before any write. A deployment with nopublicrows plans and writes nothing (zerosys_fileupdates reach the engine). A second run scans 0 and writes 0. Both are pinned.Pins (
backfill-sys-file-public-scope.test.ts, 10 cases)publicrow is refusedERR_FILE_REFERENCE_COPYand leaves no new row; the row still reads, and a scope-free update lands; a direct insert ofpublicis refusedVALIDATION_FAILED/invalid_optiononscope;copyOwnedFile, reached through the copy-on-claim hook). The copy isscope: 'user', keyeduser/…, and owned by the new slot. The bytes are read from the source's untouchedpublic/…key. The source row changed only its scope (ruling pin);sys_fileregistered) is reported, not read as clean;./index.ts, from which the publisheddistis built) are the very functions the file drives, and plan, apply and a second run through the entry rewrite a storedpublicrow and then write zero.The copy pin sits in this file rather than beside
file-reference-lifecycle.test.tsbecause it needs the rewritten row the sweep produces; it drives the realcopyOwnedFilethroughinstallFileReferenceHooks.Ablations (head
dec868f9f8, both throughscripts/ablation-replace.mjsin WRAP mode, restored with blob == HEAD,git diff HEADempty andgit status --porcelainempty, each checked on disk). The subject resolves through the test's relativesrcimports, not through a packageexports/dist, so no rebuild sits between mutation and run.Skip the rewrite (the apply loop's
engine.updatenever runs, but the row is still counted as written). Predicted and observed: 4 red, 5 green. The copy pin is red at the copy itself:Cannot copy file 'f_pub' for a second field reference … (Scope must be one of: user, tenant, private, temp, attachments). The second-run, ceiling and rollback pins are red too. The dry-run, no-row, pre-sweep, paging and unread-scan pins stay green.Restore the
publicoption. Predicted and observed: 2 red (the pre-sweep refusal pin and the inverse-refused pin), 7 green, the copy pin included. A first attempt was void: its replacement text contained the anchor, so the tool refused before running anything. The reading above is the rerun with a non-overlapping replacement.Drop one export from the entry, before the entry pin existed (patch round 1, head
15e5891c77;runSysFilePublicScopeBackfilldeleted fromsrc/index.ts, rebuilt, and read from the built artifact:typeofisundefinedindist/index.jsanddist/index.cjs, 0 hits indist/index.d.ts). Nothing went red: theservice-storagesuite 49 / 830 passed,typecheckexit 0, andcheck:dts-closure,check:published-files,check:lean-entry-closure,check:dual-build-cjs-loadsandcheck-undeclared-dep-importsall exit 0. So the entry pin above was added. Restored blob == HEAD,git diff HEADempty; the restore leg rebuilt and readfunctionback from both builds.The same deletion against the committed entry pin (head
b107953555). Value leg: 1 red / 9 green, the entry pin (expected undefined to be [AsyncFunction runSysFilePublicScopeBackfill]), andtypecheckred (TS2551, twice). Type leg (SysFilePublicScopeBackfillReportdeleted from the type exports): the suite stays 10 / 10, as predicted (types erase), andtypecheckis red (TS2724). Both restored blob == HEAD,git diff HEADempty,git status --porcelainempty. The test reads./index.jsthrough a relativesrcimport, so no rebuild sits between mutation and run.Unmutated control: 10 / 10 passed.
Tests and gates
origin/maintwice throughscripts/pm/os-regen-merge.sh:ebbb8f9c8dfirst (5551f8d8d5, which brings spec(changes): generate the per-major spec-changes section and the upgrade guide at publish; the pull request generates both in memory and renders the diff (#22449 B′, condition 1) #22533 into the branch), and4638625e07after PR fix(spec/automation): refuse a text-slot{{ $… }}hole whose root the flow engine does not bind #22499, which also writesregistry.ts, landed (df6619318c). After the second merge,gen:migration-registryon the merged tree wroteregistry.tsbyte-identical to the text merge (409 semantic entries), so no regeneration commit was owed. Noos-regenpath owed one either.service-storage, locked, at the final headdf6619318c, after a full rebuild (72 tasks):vitest run, 49 files / 831 tests passed;typecheck(tsc, the scripts tsconfig,check:test-typecheck) exit 0.@objectstack/spec, locked, atdf6619318c: the foursrc/migrationstests plusbuild-migration-registry-entry,projection-cliandrender-projection-diff, 7 files / 226 tests passed;typecheckexit 0.df6619318c(and before atb107953555, the same 116, all exit 0):dispatch-gates --commandsderives 116 commands (the spec families join because the entry andregistry.tsmoved). All 116 exit 0;--ranreads 116 run, 0 NOT-MEASURED, 0 UNRUN.check-tenant-audit-censusand its--self-testare green (241 sites, 23 prose figures held).check-changeset-no-major's clause-② level axis, which needs a pull-request payload, was driven offline with--eventfor both body spellings: exit 0 for each.eslint --no-inline-config --format jsonon the 6 touched.tsfiles gives 6 results, 0 errors and 0 warnings. The population iseslint.config.mjs's own TS glob. Invariance: that config enables no type-aware linting (parserOptionscarries onlyecmaVersionandsourceType), so an untouched file's verdict cannot move. The repo-widepnpm lintis CI's.Acceptance notes
publicvalue, since the option is gone. Not measured in the console..changeset/22443-storage-scope-public-retired.md(merged with PR feat(storage)!: retire the storage scope public from StorageScopeSchema and refuse it at the upload doors (#22443) #22469, unreleased) says files already stored with scopepublic"are not touched". After this PR, the sweep rewrites theirscopecolumn. That changeset is not edited here:check-empty-changesetrefuses a diff that modifies a changeset from the merge base, by design. This PR's changeset states the correction instead ("The step-18 entry says so too"), in the same release's CHANGELOG for both packages. Seat's disposition: no further action.Seat answers applied (
6090887049)registry.tsregenerated.Generated by Claude Code