diff --git a/docs/reference/native-compose-storage-witness.md b/docs/reference/native-compose-storage-witness.md index f4359fbe5..eeeeb9119 100644 --- a/docs/reference/native-compose-storage-witness.md +++ b/docs/reference/native-compose-storage-witness.md @@ -76,3 +76,13 @@ compiled CLI coverage and real engine acceptance remain required gates. No carri or helper download is implemented by this foundation. Store and source-CLI tests use synthetic observation ports and engine transports; they do not qualify Docker write or archive semantics. + +The separate [directory-xattr owner and helper source](native-compose-storage-xattr-carrier.md) +uses a create-only token without a PostgreSQL data-root directory entry. Required +version-three expectation/reference records and tagged artifact/journal-bound intent keep +it distinct from USTAR under the same version-three receipt owner. It adds +injectable cold provisioning and fresh read-only proof ports. The candidate Docker +port remains unactivated pending its persistent-volume qualification; no CLI caller +enrolls storage yet. Required finite carrier intent survives unknown +helper outcomes even after enrollment; saved recovery stops resources without +retiring that anchor. Its offline controls do not qualify a carrier. diff --git a/docs/reference/native-compose-storage-xattr-carrier.md b/docs/reference/native-compose-storage-xattr-carrier.md new file mode 100644 index 000000000..57b81ff98 --- /dev/null +++ b/docs/reference/native-compose-storage-xattr-carrier.md @@ -0,0 +1,253 @@ +# Native Compose directory-xattr carrier source + +This is an unactivated Linux helper source, injectable owner protocol and offline +controls. The [witness foundation](native-compose-storage-witness.md) accepts its +distinct xattr expectation/reference format under the same opaque generation +authority. A candidate Docker implementation of the carrier ports is unactivated; +the CLI does not enroll volume witnesses. Ordinary CLI instances retain the metadata +guard's same-metadata replacement limitation. + +The proposed directory xattr avoids adding a directory entry to PostgreSQL's +authored data root. It detects a replacement missing its enrolled random witness; +it does not prove database integrity or detect a clone containing that witness. +No `PGDATA` rewrite, root marker directory, package install or helper image pull +is part of this source. + +## Fixed helper contract + +`scripts/native-storage-witness-helper.ts` is the future helper entry. The CLI +does not invoke it. A qualified artifact must bundle this entry with its strict +codec and kernel adapter; an exact declared local Linux image, Bun executable, +libc and program digest must be checked before any carrier effect. The candidate +dependency is an explicitly selected local `oven/bun:1.4.2-slim` image. A tag +alone is not an artifact pin. Bun FFI remains experimental; source inspection and +mocked ports do not qualify its behavior. + +The input is one canonical JSON record, at most 4096 bytes, with required +`kind: "directory-xattr"` and `version: 1`. Output is at most 1024 bytes. Names +contain `user.hack.storage.` plus 256 random bits; values contain an independent +32-byte random token. Both travel only through bounded stdin/private stdout, +never command arguments, inherited environment, public errors or Docker logs. +Every failure emits only the fixed refused response and returns a failure code. +The enclosing carrier must set `logging=none` before start and privately capture +both streams. + +The only operations are metadata discovery, seed and verify. There is no +enumeration, replace, remove, repair or caller-selected path. Device and inode +facts use canonical unsigned 64-bit decimal strings; UID/GID are unsigned +32-bit integers. Unknown kinds/versions, alternate encodings, duplicate JSON +keys, trailing data and extra fields refuse. This format cannot be relabeled as +the foundation's USTAR proof or attached to its version-one completion reference. +Xattr expectations and references require version three, `kind: "directory-xattr"`, +exact image/platform/Bun/libc/helper and kernel-ABI pins, and a required private +carrier-journal token. Their version-three generation intent requires the carrier +tag, artifact pins and journal token even while Expected. Older strict owners +reject the new required fields; absence of a journal, artifact or tag cannot +downgrade to USTAR. The xattr helper wire format remains version one. + +The Linux adapter fixes the root at `/hack-storage-witness`. It checks Linux +arm64 or x64 open constants and fixed architecture-specific glibc paths, uses +64-bit `size_t`/`ssize_t`, and checks the directory descriptor's close-on-exec +flag. It opens with `O_RDONLY | O_DIRECTORY | O_NOFOLLOW`, checks descriptor +metadata and requires real/effective credentials to agree. `O_NOFOLLOW` covers +the leaf; trusted image parents and the daemon-side source chain remain separate +mount admission requirements. + +Seed requires the directory's exact UID/GID, calls `fsetxattr` with +`XATTR_CREATE` once, requires directory `fsync`, and rechecks root facts. An +existing attribute, including empty, wrong or matching contents, refuses without +overwrite. There is no write retry. A synchronization or close failure is an +uncertain attempt. A `seeded` response is not enrollment, durability proof or +generation readiness: the owner must obtain a separate fresh read-only proof +before publishing completion. + +Verify requires matching credentials and root facts, performs two exact bounded +`fgetxattr` reads with intervening/final root checks, and closes the descriptor. +It never writes or synchronizes storage. Absent, empty, oversized, wrong or +changing bytes refuse. These observations are finite fences, not an atomic +guard against later changes or a hostile daemon. UID/GID discovery only selects +an explicitly authorized reader; it is not witness proof. The reader must match +the same freshly observed root. Cold UID 0 to PostgreSQL UID 70 is covered by +mocked credential controls; real chown, mode 0700 and xattr access remain live +qualification cells. There is no capability or ownership-changing fallback. + +## Required cold provisioning and receipt order + +`startNativeComposeWorkloads` currently launches one Compose `up -d` child. +The injectable owner implements the following order; activation must supply a +qualified cold provisioning and carrier transport before that child, under the +same existing instance mutation and generation effect authority: + +1. Reacquire source, engine identity and exact owned inventory. Initial admission + requires no retained storage history or earlier witness intent, the selected + physical name absent, and no conflicting pending operation. Explicit adoption + instead pins an already selected stopped volume and its birth; it cannot claim + historical continuity. Existing v1/v2 or interrupted state never becomes cold + merely because its data volume is missing. +2. Publish required `Expected` and the immutable expectation before provisioning. + Cold intent contains the prospective name and no invented birth. The original + one-shot enrollment capability is consumed before its effect callback. +3. Provision only that admitted cold volume with exact owner/storage labels. + Check the create result, then freshly inspect and capture the actual birth, + local driver/options, current Mountpoint and holders before seed. Idempotent + `volume create` is not proof of original cold admission. No recovery invocation + gains provisioning or seed authority. The owner exposes exactly one captured + provisioning callback in the original consumed enrollment. A real engine + implementation must prove newness across that transaction; idempotent create + and synthetic observations do not qualify it. +4. Run one create-only seed while all selected workloads remain stopped and no + foreign or unaccounted holder exists. Require stopped, exact owned helper + absence before non-force helper removal. Obtain fresh read-only marker and + metadata proof, then let the owner publish its opaque completion proof and + required `Enrolled` reference. +5. Recheck witness/source/owner immediately before Compose workload admission and + after startup, as well as both final receipt boundaries. Read-only proofs may + coexist with only the exact admitted running generation's holders. They never + stop an application to obtain a proof or omit the existing final fences. + +The carrier handle synchronously captures its callbacks, immutable artifact, +AbortSignal and absolute deadline. Preparation captures that exact handle and +the source selection before its first await. The existing owner, private slot, +one-use enrollment and owner-issued publication proof serve both formats; there +is no second enrollment controller. Xattr completion additionally binds the +canonical kernel response hash and observed root. Fabricated references and +caller-supplied no-op verifiers cannot publish Enrolled. + +Any crash before enrollment keeps `Expected`, even if the token matches. Saved +recovery may stop exact owned resources but must preserve the intent and pending +anchor, report incomplete and skip dependent retirement. No automatic promotion, +reseed, journal rewrite or volume deletion is authorized by a successful helper +response. + +## Fresh selected-volume transport + +The candidate mounts the exact selected existing named volume with +`--mount type=volume,src=,dst=/hack-storage-witness,volume-nocopy` and +`readonly` for metadata and verification. Write access belongs only to the +original seed attempt. The helper program remains a separate nonrecursive, +read-only `rprivate` bind. The named-volume mount uses Docker's fixed private +propagation; the carrier does not bind the daemon's internal data directory or +broaden propagation. Configured `NoCopy`, selected name and read-only state must +match, with no driver, label or subpath overrides. Physical name, canonical +Mountpoint, local driver, read/write state and empty propagation must also match. + +Docker can implicitly create a missing named volume even with `volume-nocopy`. +Complete fresh metadata checks precede carrier creation and repeat after its +exact ID and birth are recorded, before start. Missing, foreign or rebound +storage refuses before helper execution, seed or workload. An external removal +racing the create API can leave an empty unqualified volume and carrier anchor; +those effects remain uncertain and cannot grant enrollment, repair or readiness. +This is an observed identity fence, not an atomic engine absence guarantee. + +Each proof must create a fresh carrier from a freshly checked current root. An +old carrier can retain a removed root even when the replacement repeats its +name, birth and Mountpoint string. Do not accept its token or reuse that pinned +mount across operations. Metadata discovery and the matching-UID reader must +each be bracketed by fresh engine/birth/Mountpoint/holder checks. After the token +read, a third fresh metadata carrier must still observe that exact root. This +detects a reader pinned to an old removed root across its await even when Docker +metadata repeats; it is a finite observation fence, not an atomic exclusion of +later swaps. Drift refuses; there is no UID retry loop or permission repair. + +The source port contract requires a complete selected-volume holder inventory, +the exact daemon and Compose owner, canonical local Mountpoint and empty local +driver options. Only current/pending owned generations may hold the volume; +enrollment forbids every running holder. Every invocation has a fresh nonce, +exact generation/pending context, artifact and UID/GID. Its response must match +those fields, contain one strict canonical helper result, and report exact +stopped helper identity followed by empty helper inventory. Replayed accepted +carrier IDs or invocation IDs refuse. These predicates validate an injectable +contract; they are not proof of actual engine identity or cleanup without the +qualified transport. + +The owner initializes a required private `carrier.json` in the anchored witness +slot before provisioning. Every invocation atomically publishes and synchronizes +a prospective intent before calling the captured transport. The transport must +publish the exact created carrier ID and birth through its one-use callback before +start. A response cannot substitute that pin or omit its publication. Only the +owner may publish idle after matching that pin with complete stopped and empty +cleanup observations. No marker values, helper output or commands enter the journal. +An exact canonical refused response with exit one may clear its proven absent +finite helper intent; it still rejects the witness and admits no workload or repair. +The journal changes independently of the generation receipt; the existing exact +material-binding comparison remains unchanged. + +An interrupted prospective or created intent, missing journal or changed anchor +blocks startup, run and normal down without another helper call. Explicit saved +recovery may stop saved engine resources, but remains incomplete with its pending +generation, witness reference and dependent claims retained. It cannot infer helper +absence from an unpublished ID, replay the invocation or reset the journal. Fresh +saved checks also precede both final receipt boundaries and material retirement. + +Exact image/program/ID labels, fixed mount correspondence, +no image-declared anonymous volumes, dropped capabilities, no network/ports/socket, +readonly root filesystem, finite CPU/memory/PID limits and absolute deadline/group +reaping are mandatory before implementing the engine owner. Cancellation or an +unknown create/start outcome keeps the anchor. Cleanup must prove the exact helper +stopped before non-force removal and never remove data, witness, material or +journals. The candidate Docker port now uses the shared bounded query and child +owners, captures canonical stdin privately, and records the created helper before +start. It inspects all selected-volume holders, mounts only the exact selected +named volume with `volume-nocopy`, and rechecks fresh metadata before start. +It never requests bind source creation. Known completion requires an exact stopped +helper, non-force removal and fresh empty helper inventory. Unknown disposition +retains the private journal and input. Journal bytes do not replace these actual +stop and absence observations. The CLI does not call this unactivated port. + +Holder selection includes normalized ancestor and descendant bind paths; similarly +named sibling roots stay independent. Command stdin and captures use held no-follow +descriptors, checked after the final admission await and held until owned group +absence. A fixed shell file-size quota bounds each capture to at most 8192 bytes +on the admitted shell platforms; acceptance is at most 4096 bytes per stream. +Unknown child disposition retains descriptors and the pending journal rather than +starting competing cleanup. No child redirects through a reopened named path. + +The first declared dependency is the fixed Linux arm64 Bun 1.4.2 image +`sha256:cb3bbbb08e13a4a2ff400f24c7a2a1d5efa83f6ef8544d52d95a519631e2fc61`. +The packaged helper pins its bytes and the qualified Bun/libc bytes separately. +Dependency/platform checks precede volume effects; every helper uses `--pull never`. +There is no image download, emulation fallback or platform substitution. Linux +amd64 and uncached first installation remain outside this qualification. +Maintain the packaged source with +`bun scripts/generate-native-storage-witness-helper.ts --check` or `--write`; +the generator refuses a changed artifact pin rather than approving new bytes. + +## Evidence limits and remaining gates + +Offline controls substitute synchronous syscall ports and synthetic engine/carrier +observations through the public generation store. They check strict input, +one create request, no overwrite/repair requests, missing/wrong witness under +unchanged metadata, root/credential drift, callback capture and uncertain fsync +or close; required Expected before cold provisioning; one seed; exact stopped +adoption; resumed missing/same-birth-empty/foreign-root refusals; artifact, +holder, context and cleanup drift; cancellation; retained interrupted state; and +unknown enrolled-helper refusal through explicit saved recovery. These controls +also reject an omitted created callback or changed carrier birth despite a +synthetic successful response. +They do not execute the Linux adapter or qualify kernel atomicity, +permissions, xattr persistence, filesystem crash durability or Docker semantics. + +Before activation: qualify exact source/artifact/image/Bun/libc ABI and filesystem; +fresh named-volume/current-root correspondence and missing/rebound refusal before +start; create-only existing/symlink/unsupported +refusals; unchanged PostgreSQL root initialization, real SQL retention, chown and +post-start proofs; same-metadata replacement including an old pinned carrier; +crash/timeout/cancellation with exact helper absence and retained uncertainty; +no daemon-log token persistence; full original engine/resource restoration. + +Count every helper process, container and query. The first straightforward proof +needs metadata discovery, a matching-UID read and a fresh metadata observation +after that read. Initial enrollment adds a discovery and write helper before its +separate three-invocation read-only proof. Required pre/post/final proofs +therefore have a real startup cost. Each finite invocation adds three synchronized +private journal publications (intent, created pin and idle), without weakening the +existing main receipt comparison. No performance improvement or production +activation is claimed, and no persistent helper is proposed. + +Primary contracts: Linux [fsetxattr](https://man7.org/linux/man-pages/man2/fsetxattr.2.html), +[fgetxattr](https://man7.org/linux/man-pages/man2/getxattr.2.html), +[fsync](https://man7.org/linux/man-pages/man2/fsync.2.html), +[arm64 open flags](https://github.com/torvalds/linux/blob/master/arch/arm64/include/uapi/asm/fcntl.h), +[generic open flags](https://github.com/torvalds/linux/blob/master/include/uapi/asm-generic/fcntl.h), +[Bun FFI](https://bun.sh/docs/runtime/ffi), Docker [volumes](https://docs.docker.com/engine/storage/volumes/), +and tagged Moby [mount fields](https://github.com/moby/moby/blob/v28.5.1/api/types/mount/mount.go). diff --git a/scripts/generate-native-storage-witness-helper.ts b/scripts/generate-native-storage-witness-helper.ts new file mode 100644 index 000000000..da464da14 --- /dev/null +++ b/scripts/generate-native-storage-witness-helper.ts @@ -0,0 +1,89 @@ +import { createHash } from "node:crypto"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { NATIVE_STORAGE_DOCKER_ARTIFACT } from "../src/lib/native-compose-storage-witness-docker-artifact.ts"; +import { NATIVE_STORAGE_WITNESS_HELPER } from "../src/lib/native-compose-storage-witness-helper-bundle.ts"; +import { run } from "../src/lib/shell.ts"; + +/** Explicit maintenance build only; never execute the helper or update its approved + * artifact pin automatically. Changed bytes require separate runtime qualification. */ +const root = await mkdtemp(join(tmpdir(), "hack-storage-helper-build-")); +let group = 0, + settled = false, + started = false; +try { + if ( + Bun.version !== "1.4.2" || + !["--check", "--write"].includes(process.argv[2] ?? "") + ) { + throw new Error("Pinned helper generation refused"); + } + const output = join(root, "helper.mjs"); + started = true; + const exit = await run( + [ + process.execPath, + "build", + "scripts/native-storage-witness-helper.ts", + "--target=bun", + "--outfile", + output, + ], + { + stdin: "ignore", + timeoutMs: 60_000, + stdout: "ignore", + stderr: "ignore", + onSpawn: (event) => { + group = event.ownsProcessGroup + ? (event.processGroupId ?? event.pid) + : 0; + return Promise.resolve(); + }, + } + ); + const deadline = Date.now() + 3000; + while (group > 1 && Date.now() <= deadline) { + try { + process.kill(-group, 0); + } catch (error: unknown) { + if ( + typeof error === "object" && + error !== null && + "code" in error && + error.code === "ESRCH" + ) { + settled = true; + break; + } + throw new Error("Helper build disposition uncertain"); + } + await new Promise((resolve) => setTimeout(resolve, 20)); + } + if (!settled) { + throw new Error("Helper build disposition uncertain"); + } + if (exit !== 0) { + throw new Error("Pinned helper build refused"); + } + const bytes = await readFile(output); + if ( + bytes.length > 128 * 1024 || + createHash("sha256").update(bytes).digest("hex") !== + NATIVE_STORAGE_DOCKER_ARTIFACT.helperHash + ) { + throw new Error("Helper artifact changed; separate qualification required"); + } + const path = "src/lib/native-compose-storage-witness-helper-bundle.ts"; + const source = `// Generated from scripts/native-storage-witness-helper.ts. Do not edit the payload.\nexport const NATIVE_STORAGE_WITNESS_HELPER =\n // biome-ignore lint/suspicious/noTemplateCurlyInString: Preserve qualified helper bytes without host interpolation.\n ${JSON.stringify(bytes.toString("utf8"))};\n`; + if (process.argv[2] === "--write") { + await writeFile(path, source); + } else if (NATIVE_STORAGE_WITNESS_HELPER !== bytes.toString("utf8")) { + throw new Error("Generated helper source differs"); + } +} finally { + if (settled || !started) { + await rm(root, { recursive: true, force: true }); + } +} diff --git a/scripts/native-storage-witness-helper.ts b/scripts/native-storage-witness-helper.ts new file mode 100644 index 000000000..e8c65ec27 --- /dev/null +++ b/scripts/native-storage-witness-helper.ts @@ -0,0 +1,81 @@ +import { + encodeNativeComposeStorageXattrResponse, + NATIVE_STORAGE_XATTR_INPUT_LIMIT, + NATIVE_STORAGE_XATTR_KIND, + NATIVE_STORAGE_XATTR_VERSION, + type NativeComposeStorageXattrResponse, + parseNativeComposeStorageXattrRequest, + refuseNativeComposeStorageXattr, +} from "../src/lib/native-compose-storage-witness-xattr-codec.ts"; +import { runNativeComposeStorageXattrHelper } from "../src/lib/native-compose-storage-witness-xattr-helper.ts"; +import { createNativeComposeStorageXattrLinuxKernel } from "../src/lib/native-compose-storage-witness-xattr-linux.ts"; + +async function input(): Promise { + const reader = Bun.stdin.stream().getReader(); + const chunks: Uint8Array[] = []; + let length = 0; + try { + while (true) { + const next = await reader.read(); + if (next.done) { + break; + } + length += next.value.byteLength; + if (length > NATIVE_STORAGE_XATTR_INPUT_LIMIT) { + await reader.cancel(); + return refuseNativeComposeStorageXattr(); + } + chunks.push(next.value); + } + } finally { + reader.releaseLock(); + } + return new TextDecoder("utf-8", { fatal: true }).decode( + Buffer.concat(chunks) + ); +} +function refused(): NativeComposeStorageXattrResponse { + return { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "refused", + }; +} +/** Bundling and exact artifact/ABI/engine qualification are prerequisites; this entry is never launched by the CLI. */ +export async function nativeStorageWitnessHelperMain(): Promise { + let response = refused(); + let close: (() => void) | null = null; + try { + const request = parseNativeComposeStorageXattrRequest(await input()); + const selected = await createNativeComposeStorageXattrLinuxKernel(); + close = selected.close; + response = runNativeComposeStorageXattrHelper({ + request, + kernel: selected.kernel, + }); + } catch { + response = refused(); + } finally { + if (close) { + try { + close(); + } catch { + response = refused(); + } + } + } + // The owner must force logging=none and capture this bounded response privately. + try { + await Bun.write( + Bun.stdout, + encodeNativeComposeStorageXattrResponse(response) + ); + } catch { + // A truncated private response is uncertain, not a successful helper proof. + return 1; + } + return response.outcome === "refused" ? 1 : 0; +} +if (import.meta.main) { + process.exitCode = await nativeStorageWitnessHelperMain(); +} diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index 4cbd1a6af..8d6d7aec9 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -631,6 +631,7 @@ async function savedCommand(opts: { }; if (options.operation === "down") { return await store.withMutation(async (mutation) => { + const recoveryState = await store.loadCurrent(); const saved = await savedRouteDocuments(store); const prepared = await prepareSavedStop({ store, @@ -651,7 +652,9 @@ async function savedCommand(opts: { .runEffect({ generation, operation: "down", - recoverPending: options.recover === true && pending !== null, + recoverPending: + options.recover === true && + (pending !== null || recoveryState.storageWitnessesPending), assertOwned, captureStorage: prepared.captureStorage, assertFresh: hooks?.assertFresh, @@ -777,7 +780,8 @@ async function savedCommand(opts: { data: { composeProject: generation.identity.composeProject, stopped: state.stopped, - pending: state.pending !== null, + pending: + state.pending !== null || state.storageWitnessesPending, beforeHooksPending: state.beforeHooksPending, hostHookPhase: state.hostHookPhase, services: observed.containers.map( @@ -799,6 +803,11 @@ async function savedCommand(opts: { "Native host hook completion remains uncertain; hook recovery is not supported in this slice.\n" ); } + if (state.storageWitnessesPending) { + process.stderr.write( + "Native storage continuity or helper completion remains uncertain; saved recovery retains its anchors. Values omitted.\n" + ); + } return await run([...composeArgs(generation), "ps"], { cwd: base.cwd, env: base.env, diff --git a/src/lib/native-compose-generation.ts b/src/lib/native-compose-generation.ts index 707061253..aeccf37e7 100644 --- a/src/lib/native-compose-generation.ts +++ b/src/lib/native-compose-generation.ts @@ -27,6 +27,7 @@ import { nativeComposeRetainedVolumesValid, } from "./native-compose-retained-storage.ts"; import { projectNativeComposeOneOff } from "./native-compose-run-projection.ts"; +import { nativeComposeStorageCarriersPending } from "./native-compose-storage-carrier-journal.ts"; import { createNativeComposeStorageWitnessReceiptProtocol } from "./native-compose-storage-witness-receipt.ts"; import { type NativeComposeStorageWitnessCompletionProof, @@ -36,6 +37,7 @@ import { nativeComposeStorageWitnessIntentValid, nativeComposeStorageWitnessStatesValid, } from "./native-compose-storage-witness-state.ts"; +import type { NativeComposeStorageXattrCarrier } from "./native-compose-storage-witness-xattr-carrier.ts"; // biome-ignore lint/performance/noBarrelFile: Preserve the generation store's public topology contract after the single-parser extraction. export { @@ -422,6 +424,15 @@ function witnessPending(state: Receipt): boolean { state.storageWitnesses?.some((entry) => entry.state === "expected") ?? false ); } +async function witnessWorkPending(state: Receipt): Promise { + return ( + witnessPending(state) || + (await nativeComposeStorageCarriersPending({ + identity: state.identity, + states: state.storageWitnesses, + })) + ); +} type FileIdentity = { readonly dev: number; readonly ino: number }; type CheckoutAnchor = FileIdentity & { readonly projectDirectory: FileIdentity; @@ -859,17 +870,24 @@ export type NativeComposeEffectOptions = { /** Reuse the exact preceding ownership observation; never perform extra probes here. */ readonly captureStorage?: () => readonly NativeComposeRetainedVolume[]; /** Required for previously enrolled v3 startup/run. Carrier remains unactivated in the CLI. */ - readonly storageWitnesses?: { - readonly engineId: string; - readonly observe: (selection: { - readonly name: string; - readonly storage: string; - readonly markerName: string; - }) => Promise<{ - readonly volume: NativeComposeRetainedVolume; - readonly archive: Uint8Array; - }>; - }; + readonly storageWitnesses?: + | { + readonly kind?: "file-ustar"; + readonly engineId: string; + readonly observe: (selection: { + readonly name: string; + readonly storage: string; + readonly markerName: string; + }) => Promise<{ + readonly volume: NativeComposeRetainedVolume; + readonly archive: Uint8Array; + }>; + } + | { + readonly kind: "directory-xattr"; + readonly engineId: string; + readonly carrier: NativeComposeStorageXattrCarrier; + }; readonly recoverPending?: boolean; /** Store-derived immutable one-off delivery, verified before/after run effects. */ readonly projection?: NativeComposeRunProjection; @@ -964,7 +982,8 @@ function hookComplete(result: { function admitEffect( input: NativeComposeEffectOptions, state: Receipt, - mode: "prepare" | "saved" | undefined + mode: "prepare" | "saved" | undefined, + admission: { readonly carrierRecovery: boolean } ): void { if (!["up", "restart", "run", "down"].includes(input.operation)) { refuse(); @@ -1007,7 +1026,11 @@ function admitEffect( ) { throw new NativeComposeGenerationError("E_NATIVE_COMPOSE_UNCERTAIN"); } - if (input.recoverPending && state.pending === null) { + if ( + input.recoverPending && + state.pending === null && + !admission.carrierRecovery + ) { refuse(); } } @@ -1513,7 +1536,7 @@ export async function openNativeComposeGenerationStore(opts: { await held.file.close(); } }; - const requireFinalPending = ( + const requireFinalPending = async ( latest: Receipt, pending: Receipt["pending"], operation: NativeComposeOperation, @@ -1523,7 +1546,7 @@ export async function openNativeComposeGenerationStore(opts: { JSON.stringify(latest.pending) !== JSON.stringify(pending) || ((operation !== "down" || hasDownHooks) && latest.beforeHooks !== null) || - witnessPending(latest) + (await witnessWorkPending(latest)) ) { refuse(); } @@ -1566,7 +1589,7 @@ export async function openNativeComposeGenerationStore(opts: { await rememberStorage(input); await assertWitnesses(); let latest = await receipt(); - requireFinalPending( + await requireFinalPending( latest, pending, input.operation, @@ -1585,7 +1608,7 @@ export async function openNativeComposeGenerationStore(opts: { await rememberStorage(input); await assertWitnesses(); latest = await receipt(); - requireFinalPending( + await requireFinalPending( latest, pending, input.operation, @@ -1624,7 +1647,7 @@ export async function openNativeComposeGenerationStore(opts: { storageWitnesses: state.storageWitnesses ? structuredClone(state.storageWitnesses) : null, - storageWitnessesPending: witnessPending(state), + storageWitnessesPending: await witnessWorkPending(state), }; Object.defineProperty(result, "retainedStorage", { enumerable: false }); Object.defineProperty(result, "storageWitnesses", { @@ -1674,6 +1697,16 @@ export async function openNativeComposeGenerationStore(opts: { refuse(); } }; + const matchesMutationReceipt = ( + read: Awaited> + ) => + mutationReceipt !== null && + sameFile(read.info, mutationReceipt) && + read.text === mutationReceipt.text; + const sameReceiptRead = ( + left: Awaited>, + right: Awaited> + ) => sameFile(left.info, right.info) && left.text === right.text; materialActions.set(materialAuthority, async (run) => { requireActive(); const work = run(); @@ -1755,11 +1788,7 @@ export async function openNativeComposeGenerationStore(opts: { requireActive(); await lease.assertHeld(); const read = await readPrivate(receiptPath, RECEIPT_LIMIT); - if ( - mutationReceipt === null || - !sameFile(read.info, mutationReceipt) || - read.text !== mutationReceipt.text - ) { + if (!matchesMutationReceipt(read)) { return refuse(); } const state = parseReceipt( @@ -1771,13 +1800,16 @@ export async function openNativeComposeGenerationStore(opts: { selection, state ); + if ( + selection.phase === "retire" && + (await witnessWorkPending(state)) + ) { + return refuse(); + } await lease.assertHeld(); const latest = await readPrivate(receiptPath, RECEIPT_LIMIT); requireActive(); - if ( - !sameFile(read.info, latest.info) || - read.text !== latest.text - ) { + if (!sameReceiptRead(read, latest)) { return refuse(); } requireMaterialEffectLive({ @@ -1929,7 +1961,7 @@ export async function openNativeComposeGenerationStore(opts: { ) => { if ( result.outcome !== "complete" || - witnessPending(await receipt()) + (await witnessWorkPending(await receipt())) ) { // Saved recovery may stop engine resources, but unresolved enrollment keeps its anchor. await rememberStorage(input); @@ -2243,13 +2275,45 @@ export async function openNativeComposeGenerationStore(opts: { }); return await runAction(async () => { requireActive(); + const admit = async (state: Receipt) => { + const unknown = await witnessWorkPending(state); + if ( + unknown && + !( + input.operation === "down" && + input.recoverPending === true + ) + ) { + throw new NativeComposeGenerationError( + "E_NATIVE_COMPOSE_UNCERTAIN" + ); + } + // A read-only preflight can leave required helper intent before a + // main workload intent exists. Only exact saved down recovery may + // arm its stop; all generic empty-pending recovery still refuses. + const carrierRecovery = + unknown && + state.pending === null && + !witnessPending(state) && + input.operation === "down" && + input.recoverPending === true && + state.current?.generationId === + input.generation.generationId && + (state.storageWitnesses?.some( + (entry) => + entry.state === "enrolled" && + entry.reference.version === 3 + ) ?? + false); + admitEffect(input, state, opts.mode, { carrierRecovery }); + }; let state = await receipt(); - admitEffect(input, state, opts.mode); + await admit(state); await checkEffect(input); // Ownership can publish newly observed births. Never overwrite them // with the pre-observation receipt when arming the effect intent. state = await receipt(); - admitEffect(input, state, opts.mode); + await admit(state); const anchor = knownAnchor(input.generation); const pending = effectPending(state, input, anchor); await save({ ...state, pending }); diff --git a/src/lib/native-compose-storage-carrier-journal.ts b/src/lib/native-compose-storage-carrier-journal.ts new file mode 100644 index 000000000..a059b2090 --- /dev/null +++ b/src/lib/native-compose-storage-carrier-journal.ts @@ -0,0 +1,345 @@ +import { createHash, randomBytes } from "node:crypto"; +import { rename } from "node:fs/promises"; +import { join } from "node:path"; +import { isRecord } from "./guards.ts"; +import type { NativeComposeIdentity } from "./native-compose-generation.ts"; +import { + type HeldDirectory, + holdDirectory, + keys, + parsePrivateJson, + readPrivate, + recheckDirectories, + sameFile, + writeExclusive, +} from "./native-compose-private-state.ts"; +import { nativeComposeRetainedVolumesValid } from "./native-compose-retained-storage.ts"; +import type { NativeComposeStorageWitnessState } from "./native-compose-storage-witness-state.ts"; +import { + type NativeComposeStorageXattrInvocation, + nativeComposeStorageXattrArtifactValid, +} from "./native-compose-storage-witness-xattr-carrier.ts"; +import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; + +const LIMIT = 16 * 1024; +const TOKEN = /^[a-f0-9]{32}$/; +const HASH = /^[a-f0-9]{64}$/; +const ENGINE = /^[a-zA-Z0-9][a-zA-Z0-9:-]{0,127}$/; +const RUNTIME = /^[a-z0-9][a-z0-9_-]{0,127}$/; +type Created = { readonly id: string; readonly createdAt: string }; +type Intent = { + readonly invocationId: string; + readonly engineId: string; + readonly runtimeIdentity: string; + readonly ownerToken: string; + readonly volume: NativeComposeStorageXattrInvocation["target"]["volume"]; + readonly artifact: NativeComposeStorageXattrInvocation["artifact"]; + readonly scope: NativeComposeStorageXattrInvocation["scope"]; + readonly operation: NativeComposeStorageXattrInvocation["request"]["operation"]; + readonly readonly: boolean; + readonly uid: number; + readonly gid: number; + readonly created: Created | null; +}; +type Journal = { + readonly version: 1; + readonly token: string; + readonly revision: number; + readonly intent: Intent | null; +}; +type Bound = { + readonly directory: HeldDirectory; + readonly token: string; + readonly check: () => Promise; +}; +function created(value: unknown): Created { + if ( + !( + isRecord(value) && + keys(value, "createdAt,id") && + typeof value.id === "string" && + HASH.test(value.id) && + nativeComposeRetainedVolumesValid([ + { name: "carrier", storage: "carrier", createdAt: value.createdAt }, + ]) + ) + ) { + return refuse(); + } + return Object.freeze({ id: value.id, createdAt: String(value.createdAt) }); +} +function scopeValid(value: unknown): boolean { + return ( + isRecord(value) && + keys( + value, + "currentGenerationId,generationId,pendingGenerationId,pendingToken" + ) && + typeof value.generationId === "string" && + TOKEN.test(value.generationId) && + [ + value.currentGenerationId, + value.pendingGenerationId, + value.pendingToken, + ].every( + (entry) => + entry === null || (typeof entry === "string" && TOKEN.test(entry)) + ) + ); +} +function intentValid(value: unknown): value is Intent { + return ( + isRecord(value) && + keys( + value, + "artifact,created,engineId,gid,invocationId,operation,ownerToken,readonly,runtimeIdentity,scope,uid,volume" + ) && + typeof value.invocationId === "string" && + TOKEN.test(value.invocationId) && + typeof value.engineId === "string" && + ENGINE.test(value.engineId) && + typeof value.runtimeIdentity === "string" && + RUNTIME.test(value.runtimeIdentity) && + typeof value.ownerToken === "string" && + TOKEN.test(value.ownerToken) && + nativeComposeRetainedVolumesValid([value.volume]) && + nativeComposeStorageXattrArtifactValid(value.artifact) && + scopeValid(value.scope) && + ["root", "verify", "seed"].includes(String(value.operation)) && + value.readonly === (value.operation !== "seed") && + [value.uid, value.gid].every( + (entry) => + typeof entry === "number" && + Number.isInteger(entry) && + entry >= 0 && + entry <= 4_294_967_295 && + !Object.is(entry, -0) + ) && + (value.created === null || Boolean(created(value.created))) + ); +} +function decode(text: string, token: string): Journal { + const value = parsePrivateJson(text); + if ( + !( + isRecord(value) && + keys(value, "intent,revision,token,version") && + value.version === 1 && + value.token === token && + TOKEN.test(token) && + typeof value.revision === "number" && + Number.isSafeInteger(value.revision) && + value.revision >= 0 && + (value.intent === null || intentValid(value.intent)) + ) + ) { + return refuse(); + } + return Object.freeze({ + version: 1, + token, + revision: value.revision, + intent: value.intent === null ? null : value.intent, + }); +} +async function read(bound: Bound) { + const value = await readPrivate( + join(bound.directory.path, "carrier.json"), + LIMIT + ); + await recheckDirectories([bound.directory]); + return { ...value, state: decode(value.text, bound.token) }; +} +async function save( + bound: Bound, + prior: Awaited>, + intent: Intent | null +): Promise { + if (prior.state.revision >= Number.MAX_SAFE_INTEGER) { + return refuse(); + } + const text = JSON.stringify({ + version: 1, + token: bound.token, + revision: prior.state.revision + 1, + intent, + }); + decode(text, bound.token); + const path = join(bound.directory.path, "carrier.json"); + const temporary = join( + bound.directory.path, + `.carrier-${randomBytes(16).toString("hex")}` + ); + const written = await writeExclusive(temporary, text); + const latest = await read(bound); + if (!sameFile(latest.info, prior.info) || latest.text !== prior.text) { + return refuse(); + } + await bound.check(); + await rename(temporary, path); + await bound.directory.file.sync(); + const published = await read(bound); + if (!sameFile(published.info, written) || published.text !== text) { + return refuse(); + } + await bound.check(); +} +/** Required before any finite carrier effects. A missing or changed journal is never initialized on resume. */ +export async function initializeNativeComposeStorageCarrierJournal( + opts: Bound +): Promise { + const bound = Object.freeze({ + directory: Object.freeze({ ...opts.directory }), + token: opts.token, + check: opts.check, + }); + if (!TOKEN.test(bound.token)) { + return refuse(); + } + await writeExclusive( + join(bound.directory.path, "carrier.json"), + JSON.stringify({ + version: 1, + token: bound.token, + revision: 0, + intent: null, + }) + ); + await bound.directory.file.sync(); + await read(bound); + await bound.check(); +} +/** The transport receives only created(), never the owner's completion authority. */ +export async function beginNativeComposeStorageCarrierIntent( + opts: Bound & { + readonly input: Omit; + } +) { + const input = structuredClone(opts.input); + const bound = Object.freeze({ + directory: Object.freeze({ ...opts.directory }), + token: opts.token, + check: opts.check, + }); + const prior = await read(bound); + if (prior.state.intent !== null || input.target.volume === null) { + return refuse(); + } + const intent: Intent = Object.freeze({ + invocationId: input.invocationId, + engineId: input.target.engineId, + runtimeIdentity: input.target.runtimeIdentity, + ownerToken: input.target.ownerToken, + volume: input.target.volume, + artifact: input.artifact, + scope: input.scope, + operation: input.request.operation, + readonly: input.readonly, + uid: input.uid, + gid: input.gid, + created: null, + }); + await save(bound, prior, intent); + let recorded: Created | null = null; + let recording = false; + let completing = false; + const requireIntent = async () => { + const latest = await read(bound); + if ( + JSON.stringify(latest.state.intent) !== + JSON.stringify({ ...intent, created: recorded }) + ) { + return refuse(); + } + return latest; + }; + return Object.freeze({ + async recordCreated(value: Created): Promise { + // Consume before awaiting; copies, duplicate publication and concurrent callbacks refuse. + if (recording || recorded) { + return refuse(); + } + recording = true; + const selected = created(structuredClone(value)); + await bound.check(); + const latest = await requireIntent(); + await save(bound, latest, { ...intent, created: selected }); + recorded = selected; + }, + async complete(value: Created): Promise { + if (completing) { + return refuse(); + } + completing = true; + const selected = created(structuredClone(value)); + if (!recorded || JSON.stringify(recorded) !== JSON.stringify(selected)) { + return refuse(); + } + await bound.check(); + const latest = await requireIntent(); + await save(bound, latest, null); + }, + }); +} + +/** Saved-only check. Broken anchors and unknown intent block admission/retirement but permit explicit saved stop. */ +export async function nativeComposeStorageCarriersPending(opts: { + readonly identity: NativeComposeIdentity; + readonly states: readonly NativeComposeStorageWitnessState[] | undefined; +}): Promise { + for (const state of opts.states ?? []) { + if (state.state === "expected") { + return true; + } + if (state.reference.version !== 3) { + continue; + } + const held: HeldDirectory[] = []; + try { + const parent = join( + opts.identity.checkoutRoot, + ".hack", + ".internal", + "native-compose", + opts.identity.instanceId + ); + held.push(await holdDirectory(parent, true)); + held.push(await holdDirectory(join(parent, "storage-witnesses"), true)); + held.push( + await holdDirectory( + join( + held[1]?.path ?? refuse(), + createHash("sha256").update(state.name).digest("hex") + ), + true + ) + ); + const root = held[1] ?? refuse(); + const directory = held[2] ?? refuse(); + if ( + !( + sameFile(root.info, state.reference.root) && + sameFile(directory.info, state.reference.directory) + ) + ) { + return true; + } + const current = await read({ + directory, + token: state.reference.carrierJournalToken, + check: async () => { + // Saved status has no effect authority; private anchors are checked below. + }, + }); + await recheckDirectories(held); + if (current.state.intent !== null) { + return true; + } + } catch { + return true; + } finally { + await Promise.allSettled(held.map((entry) => entry.file.close())); + } + } + return false; +} diff --git a/src/lib/native-compose-storage-witness-docker-artifact.ts b/src/lib/native-compose-storage-witness-docker-artifact.ts new file mode 100644 index 000000000..4e6418b21 --- /dev/null +++ b/src/lib/native-compose-storage-witness-docker-artifact.ts @@ -0,0 +1,30 @@ +import { createHash } from "node:crypto"; +import { NATIVE_STORAGE_WITNESS_HELPER } from "./native-compose-storage-witness-helper-bundle.ts"; +import type { NativeComposeStorageXattrArtifact } from "./native-compose-storage-witness-xattr-carrier.ts"; +import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; + +/** Explicit cached dependency; the owner never pulls or builds this image. Other + * platforms require their own artifact/ABI/volume qualification before admission. */ +export const NATIVE_STORAGE_DOCKER_ARTIFACT: NativeComposeStorageXattrArtifact = + Object.freeze({ + version: 1, + imageId: + "sha256:cb3bbbb08e13a4a2ff400f24c7a2a1d5efa83f6ef8544d52d95a519631e2fc61", + platform: "linux/arm64", + bunVersion: "1.4.2", + bunHash: "616f267a34278ff5ac282df37ffdfba1d7141f4f6926bca99af2cd6ef3ad32b1", + libcHash: + "9a325944115b36e74077786b29531963d96cd55cd6c82f11750a352cdb58fab0", + helperHash: + "9d9c322ee35f43e047a41fd4901926789673a072c1e0bbf9030a106e7050ba5f", + kernelAbi: 1, + }); +export function nativeComposeStorageDockerHelper(): string { + if ( + createHash("sha256").update(NATIVE_STORAGE_WITNESS_HELPER).digest("hex") !== + NATIVE_STORAGE_DOCKER_ARTIFACT.helperHash + ) { + return refuse(); + } + return NATIVE_STORAGE_WITNESS_HELPER; +} diff --git a/src/lib/native-compose-storage-witness-docker-inventory.ts b/src/lib/native-compose-storage-witness-docker-inventory.ts new file mode 100644 index 000000000..b5bdb7410 --- /dev/null +++ b/src/lib/native-compose-storage-witness-docker-inventory.ts @@ -0,0 +1,317 @@ +import { posix } from "node:path"; +import { isRecord } from "./guards.ts"; +import type { NativeComposeMaterialBinding } from "./native-compose-generation.ts"; +import type { createNativeComposeProbe } from "./native-compose-ownership.ts"; +import { nativeComposeVolumeCreatedAt } from "./native-compose-retained-storage.ts"; +import { + checkNativeComposeStorageXattrTarget, + type NativeComposeStorageXattrTarget, +} from "./native-compose-storage-witness-xattr-carrier.ts"; +import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; + +const ID = /^[a-f0-9]{64}$/; +const NAME = /^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,254}$/; +const STORAGE = /^[a-z0-9][a-z0-9._-]{0,62}$/; +const PREFIX = "io.hack.native-config"; +const TRAILING_SEPARATOR = /\/+$/; +const VOLUME = `{"name":{{json .Name}},"createdAt":{{json .CreatedAt}},"driver":{{json .Driver}},"options":{{json .Options}},"mountpoint":{{json .Mountpoint}},"project":{{json (index .Labels "com.docker.compose.project")}},"instance":{{json (index .Labels "${PREFIX}.instance")}},"owner":{{json (index .Labels "${PREFIX}.owner")}},"version":{{json (index .Labels "${PREFIX}.version")}},"storage":{{json (index .Labels "${PREFIX}.storage")}},"provision":{{json (index .Labels "${PREFIX}.storage-provision")}}}`; +const HOLDER = `{"id":{{json .Id}},"createdAt":{{json .Created}},"mounts":{{json .Mounts}},"running":{{json .State.Running}},"project":{{json (index .Config.Labels "com.docker.compose.project")}},"instance":{{json (index .Config.Labels "${PREFIX}.instance")}},"owner":{{json (index .Config.Labels "${PREFIX}.owner")}},"version":{{json (index .Config.Labels "${PREFIX}.version")}},"generation":{{json (index .Config.Labels "${PREFIX}.generation")}},"carrier":{{json (index .Config.Labels "io.hack.storage-witness.carrier")}},"carrierOwner":{{json (index .Config.Labels "io.hack.storage-witness.owner")}},"carrierGeneration":{{json (index .Config.Labels "io.hack.storage-witness.generation")}}}`; +type Probe = ReturnType; +type Selection = { readonly name: string; readonly storage: string }; +function captureSelection(value: Selection): Selection { + const { name, storage } = value; + if ( + !( + typeof name === "string" && + NAME.test(name) && + typeof storage === "string" && + STORAGE.test(storage) + ) + ) { + return refuse(); + } + return Object.freeze({ name, storage }); +} +/** A parent or child bind exposes the selected root too. Normalize complete paths, + * then compare separator boundaries so a similarly named sibling stays independent. */ +export function nativeComposeStorageDockerPathsOverlap( + left: string, + right: string +): boolean { + if (!(posix.isAbsolute(left) && posix.isAbsolute(right))) { + return false; + } + const a = posix.normalize(left).replace(TRAILING_SEPARATOR, "") || "/"; + const b = posix.normalize(right).replace(TRAILING_SEPARATOR, "") || "/"; + return ( + a === b || + a === "/" || + b === "/" || + a.startsWith(`${b}/`) || + b.startsWith(`${a}/`) + ); +} + +function rows(text: string): Record[] { + if (text.trim() === "") { + return []; + } + return text + .trim() + .split("\n") + .map((line) => { + const value: unknown = JSON.parse(line); + return isRecord(value) ? value : refuse(); + }); +} +function strings(text: string, pattern: RegExp): string[] { + const result = + text.trim() === "" + ? [] + : text + .trim() + .split("\n") + .map((line) => { + const value: unknown = JSON.parse(line); + return typeof value === "string" && pattern.test(value) + ? value + : refuse(); + }); + if (new Set(result).size !== result.length) { + return refuse(); + } + return result.sort(); +} +async function volumes(probe: Probe): Promise { + return strings( + await probe(["volume", "ls", "--format", "{{json .Name}}"]), + NAME + ); +} +async function containers(probe: Probe): Promise { + return strings( + await probe([ + "container", + "ls", + "-a", + "--no-trunc", + "--format", + "{{json .ID}}", + ]), + ID + ); +} +export async function inspectNativeComposeStorageDockerVolume(opts: { + readonly probe: Probe; + readonly selection: Selection; + readonly current: NativeComposeMaterialBinding; +}): Promise> { + const { probe, current } = opts; + const selection = captureSelection(opts.selection); + const values = rows( + await probe(["volume", "inspect", "--format", VOLUME, selection.name]) + ); + if (values.length !== 1) { + return refuse(); + } + const value = values[0] ?? refuse(); + if ( + !( + value.name === selection.name && + nativeComposeVolumeCreatedAt(value.createdAt) && + value.project === current.identity.composeProject && + value.instance === current.identity.composeProject && + value.owner === current.identity.ownerToken && + value.version === "1" && + value.storage === selection.storage && + value.driver === "local" && + value.mountpoint === `/var/lib/docker/volumes/${selection.name}/_data` && + (value.options === null || + (isRecord(value.options) && Object.keys(value.options).length === 0)) + ) + ) { + return refuse(); + } + return value; +} + +/** A named mount may recreate a missing name during container creation. Recheck + * its admitted metadata before start; a new footprint grants no seed or workload. */ +export async function assertNativeComposeStorageDockerCarrierVolume(opts: { + readonly probe: Probe; + readonly current: NativeComposeMaterialBinding; + readonly target: NativeComposeStorageXattrTarget; +}): Promise { + const { probe, current } = opts; + const selection = captureSelection(opts.target); + const createdAt = opts.target.volume?.createdAt; + const mountpoint = opts.target.mountpoint; + if ( + !nativeComposeVolumeCreatedAt(createdAt) || + typeof mountpoint !== "string" + ) { + return refuse(); + } + const volume = await inspectNativeComposeStorageDockerVolume({ + probe, + current, + selection, + }); + if (volume.createdAt !== createdAt || volume.mountpoint !== mountpoint) { + return refuse(); + } +} + +/** Complete holder reads include foreign containers. The two inventories and selected + * volume reread are observed fences; they do not claim an atomic daemon transaction. */ +export async function observeNativeComposeStorageDockerTarget(opts: { + readonly probe: Probe; + readonly selection: Selection; + readonly current: NativeComposeMaterialBinding; + readonly engineId: string; + readonly stopped: boolean; + /** Only the separately inspected exact created helper may be excluded. */ + readonly carrier?: { + readonly id: string; + readonly createdAt: string; + readonly invocationId: string; + }; +}): Promise { + const { probe, current, engineId, stopped } = opts; + const selection = captureSelection(opts.selection); + if ( + JSON.parse(await probe(["info", "--format", "{{json .ID}}"])) !== engineId + ) { + return refuse(); + } + const beforeVolumes = await volumes(probe); + const volume = beforeVolumes.includes(selection.name) + ? await inspectNativeComposeStorageDockerVolume({ + probe, + current, + selection, + }) + : null; + const beforeContainers = await containers(probe); + const holders: Record[] = []; + for (let offset = 0; offset < beforeContainers.length; offset += 128) { + const batch = beforeContainers.slice(offset, offset + 128); + const values = rows( + await probe(["container", "inspect", "--format", HOLDER, ...batch]) + ); + if ( + values.length !== batch.length || + new Set(values.map((value) => value.id)).size !== batch.length + ) { + return refuse(); + } + for (const value of values) { + if ( + !( + typeof value.id === "string" && + batch.includes(value.id) && + Array.isArray(value.mounts) + ) + ) { + return refuse(); + } + let selected = false; + for (const mount of value.mounts) { + if ( + !( + isRecord(mount) && + typeof mount.Type === "string" && + typeof mount.Source === "string" && + typeof mount.Destination === "string" && + typeof mount.RW === "boolean" + ) + ) { + return refuse(); + } + if ( + (mount.Type === "volume" && mount.Name === selection.name) || + nativeComposeStorageDockerPathsOverlap( + mount.Source, + `/var/lib/docker/volumes/${selection.name}/_data` + ) + ) { + selected = true; + } + } + if (!selected) { + continue; + } + if (opts.carrier && value.id === opts.carrier.id) { + if ( + !( + value.createdAt === opts.carrier.createdAt && + value.carrier === opts.carrier.invocationId && + value.carrierOwner === current.identity.ownerToken && + value.carrierGeneration === current.generationId && + value.running === false + ) + ) { + return refuse(); + } + continue; + } + if ( + !( + value.project === current.identity.composeProject && + value.instance === current.identity.composeProject && + value.owner === current.identity.ownerToken && + value.version === "1" + ) + ) { + return refuse(); + } + holders.push({ + id: value.id, + runtimeIdentity: value.project, + ownerToken: value.owner, + generationId: value.generation, + running: value.running, + }); + } + } + const repeated = volume + ? await inspectNativeComposeStorageDockerVolume({ + probe, + current, + selection, + }) + : null; + if ( + JSON.stringify(repeated) !== JSON.stringify(volume) || + JSON.stringify(await volumes(probe)) !== JSON.stringify(beforeVolumes) || + JSON.stringify(await containers(probe)) !== + JSON.stringify(beforeContainers) || + JSON.parse(await probe(["info", "--format", "{{json .ID}}"])) !== engineId + ) { + return refuse(); + } + return checkNativeComposeStorageXattrTarget({ + current, + engineId, + selection, + stopped, + value: { + engineId, + runtimeIdentity: current.identity.composeProject, + ownerToken: current.identity.ownerToken, + name: selection.name, + storage: selection.storage, + volume: volume + ? { + name: selection.name, + storage: selection.storage, + createdAt: volume.createdAt, + } + : null, + mountpoint: volume?.mountpoint ?? null, + driver: volume ? "local" : null, + options: volume ? {} : null, + holders, + }, + }); +} diff --git a/src/lib/native-compose-storage-witness-docker-io.ts b/src/lib/native-compose-storage-witness-docker-io.ts new file mode 100644 index 000000000..5fedbbda2 --- /dev/null +++ b/src/lib/native-compose-storage-witness-docker-io.ts @@ -0,0 +1,183 @@ +import { constants, fstatSync, lstatSync, readSync, type Stats } from "node:fs"; +import { type FileHandle, open } from "node:fs/promises"; +import { + type HeldDirectory, + sameFile, +} from "./native-compose-private-state.ts"; +import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; + +type Leaf = { readonly path: string; readonly info: Stats }; +type HeldLeaf = Leaf & { readonly file: FileHandle }; +// Unknown child disposition cannot authorize closing its inherited descriptors. +// Retention is not a reaper or recovery authority; the private journal stays pending. +const unsettledDescriptors = new Set(); + +function checkLeaf(leaf: HeldLeaf, limit: number): Stats { + try { + const info = fstatSync(leaf.file.fd), + named = lstatSync(leaf.path); + if ( + !( + info.isFile() && + named.isFile() && + sameFile(info, leaf.info) && + sameFile(info, named) && + info.nlink === 1 && + named.nlink === 1 && + info.uid === process.getuid?.() && + named.uid === info.uid && + (info.mode & 0o777) === 0o600 && + (named.mode & 0o777) === 0o600 && + info.size <= limit + ) + ) { + return refuse(); + } + return info; + } catch { + return refuse(); + } +} +function readLeaf(leaf: HeldLeaf, limit: number): string { + const before = checkLeaf(leaf, limit), + bytes = Buffer.alloc(before.size + 1); + const count = readSync(leaf.file.fd, bytes, 0, bytes.length, 0); + const after = checkLeaf(leaf, limit); + if ( + !( + count === before.size && + before.size === after.size && + before.mtimeMs === after.mtimeMs && + before.ctimeMs === after.ctimeMs + ) + ) { + return refuse(); + } + return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode( + bytes.subarray(0, count) + ); +} + +/** Empty command leaves are not receipts: the shared receipt writer deliberately + * requires nonempty bytes. This separate create-only leaf keeps that invariant. */ +export async function createNativeComposeStorageDockerEmptyLeaf( + path: string +): Promise { + const file = await open( + path, + constants.O_CREAT | + constants.O_EXCL | + constants.O_WRONLY | + constants.O_NOFOLLOW, + 0o600 + ); + try { + await file.sync(); + const info = await file.stat(); + return checkLeaf({ path, info, file }, 0); + } catch { + return refuse(); + } finally { + await file.close(); + } +} + +/** Holds exact no-follow leaves through shared run() and owned group settlement. + * Child writes are separately quota-limited by the fixed shell wrapper; reads + * accept at most limit bytes. No named path is reopened or truncated by the child. */ +export async function holdNativeComposeStorageDockerIo(opts: { + readonly directories: readonly HeldDirectory[]; + readonly input: Leaf & { readonly text: string }; + readonly stdout: Leaf; + readonly stderr: Leaf; + readonly limit: number; +}) { + const directories = [...opts.directories]; + const input = { ...opts.input }, + stdout = { ...opts.stdout }, + stderr = { ...opts.stderr }; + const limit = opts.limit; + if ( + !( + Number.isSafeInteger(limit) && + limit > 0 && + limit <= 4096 && + Buffer.byteLength(input.text) <= limit + ) + ) { + return refuse(); + } + const held: HeldLeaf[] = []; + try { + for (const [leaf, flags] of [ + [input, constants.O_RDONLY], + [stdout, constants.O_RDWR], + [stderr, constants.O_RDWR], + ] as const) { + const file = await open( + leaf.path, + flags | constants.O_NOFOLLOW | constants.O_NONBLOCK + ); + held.push({ ...leaf, file }); + } + const source = held[0] ?? refuse(), + out = held[1] ?? refuse(), + err = held[2] ?? refuse(); + const assertFresh = () => { + try { + for (const directory of directories) { + const info = fstatSync(directory.file.fd), + named = lstatSync(directory.path); + if ( + !( + info.isDirectory() && + named.isDirectory() && + sameFile(info, directory.info) && + sameFile(info, named) && + info.uid === process.getuid?.() && + named.uid === info.uid && + (info.mode & 0o022) === 0 && + (!directory.private || (info.mode & 0o777) === 0o700) + ) + ) { + return refuse(); + } + } + if ( + readLeaf(source, limit) !== input.text || + checkLeaf(out, limit).size !== 0 || + checkLeaf(err, limit).size !== 0 + ) { + return refuse(); + } + } catch { + return refuse(); + } + }; + assertFresh(); + return { + descriptors: { + stdin: source.file.fd, + stdout: out.file.fd, + stderr: err.file.fd, + }, + assertFresh, + read: () => { + try { + return { stdout: readLeaf(out, limit), stderr: readLeaf(err, limit) }; + } catch { + return refuse(); + } + }, + close: async () => { + await Promise.all(held.map((leaf) => leaf.file.close())); + }, + retainUnsettled: () => { + unsettledDescriptors.add(held); + }, + }; + } catch { + await Promise.allSettled(held.map((leaf) => leaf.file.close())); + return refuse(); + } +} diff --git a/src/lib/native-compose-storage-witness-docker-policy.ts b/src/lib/native-compose-storage-witness-docker-policy.ts new file mode 100644 index 000000000..24fe21a16 --- /dev/null +++ b/src/lib/native-compose-storage-witness-docker-policy.ts @@ -0,0 +1,253 @@ +import { isRecord } from "./guards.ts"; +import { nativeComposeVolumeCreatedAt } from "./native-compose-retained-storage.ts"; +import { NATIVE_STORAGE_DOCKER_ARTIFACT } from "./native-compose-storage-witness-docker-artifact.ts"; +import type { NativeComposeStorageXattrInvocation } from "./native-compose-storage-witness-xattr-carrier.ts"; +import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; + +const ID = /^[a-f0-9]{64}$/; +function configuredReadOnly( + mount: Record, + expected: boolean +): boolean { + // The Engine API omits the false default. A present value must stay exact. + return Object.hasOwn(mount, "ReadOnly") + ? mount.ReadOnly === expected + : expected === false; +} +export const NATIVE_STORAGE_CARRIER_LABEL = "io.hack.storage-witness.carrier"; +export const NATIVE_STORAGE_CARRIER_FORMAT = + '{"id":{{json .Id}},"createdAt":{{json .Created}},"image":{{json .Image}},"configImage":{{json .Config.Image}},"user":{{json .Config.User}},"entrypoint":{{json .Config.Entrypoint}},"cmd":{{json .Config.Cmd}},"openStdin":{{json .Config.OpenStdin}},"tty":{{json .Config.Tty}},"labels":{{json .Config.Labels}},"host":{{json .HostConfig}},"mounts":{{json .Mounts}},"state":{{json .State}},"execIds":{{json .ExecIDs}}}'; +export type NativeComposeStorageDockerCarrier = Record & { + readonly id: string; + readonly createdAt: string; + readonly state: Record; +}; +export function checkNativeComposeStorageDockerCarrier(opts: { + readonly value: unknown; + readonly input: NativeComposeStorageXattrInvocation; + readonly program: string; + readonly imageIds: readonly string[]; +}): NativeComposeStorageDockerCarrier { + const { value, input, program, imageIds } = opts; + if ( + !( + isRecord(value) && + typeof value.id === "string" && + ID.test(value.id) && + nativeComposeVolumeCreatedAt(value.createdAt) && + isRecord(value.host) && + isRecord(value.state) && + isRecord(value.labels) && + Array.isArray(value.mounts) && + value.configImage === input.artifact.imageId && + typeof value.image === "string" && + imageIds.includes(value.image) && + value.user === `${input.uid}:${input.gid}` && + value.openStdin === true && + value.tty === false && + JSON.stringify(value.entrypoint) === '["/usr/local/bin/bun"]' && + JSON.stringify(value.cmd) === + '["--no-env-file","/hack-storage-witness-helper.mjs"]' && + value.labels[NATIVE_STORAGE_CARRIER_LABEL] === input.invocationId && + value.labels["io.hack.storage-witness.owner"] === + input.target.ownerToken && + value.labels["io.hack.storage-witness.generation"] === + input.scope.generationId && + value.labels["io.hack.storage-witness.helper"] === + input.artifact.helperHash && + !Object.hasOwn(value.labels, "com.docker.compose.project") && + Object.hasOwn(value, "execIds") && + (value.execIds === null || + (Array.isArray(value.execIds) && value.execIds.length === 0)) + ) + ) { + return refuse(); + } + const host = value.host; + if ( + !( + host.Privileged === false && + host.ReadonlyRootfs === true && + host.NetworkMode === "none" && + host.Memory === 268_435_456 && + host.NanoCpus === 1_000_000_000 && + host.PidsLimit === 32 && + JSON.stringify(host.CapDrop) === '["ALL"]' && + (host.CapAdd === null || + (Array.isArray(host.CapAdd) && host.CapAdd.length === 0)) && + JSON.stringify(host.SecurityOpt) === '["no-new-privileges:true"]' && + isRecord(host.LogConfig) && + host.LogConfig.Type === "none" && + isRecord(host.LogConfig.Config) && + Object.keys(host.LogConfig.Config).length === 0 && + isRecord(host.RestartPolicy) && + host.RestartPolicy.Name === "no" && + (host.Binds === null || + (Array.isArray(host.Binds) && host.Binds.length === 0)) && + (!Object.hasOwn(host, "Tmpfs") || + host.Tmpfs === null || + (isRecord(host.Tmpfs) && Object.keys(host.Tmpfs).length === 0)) && + (host.Devices === null || + (Array.isArray(host.Devices) && host.Devices.length === 0)) && + (host.DeviceRequests === null || + (Array.isArray(host.DeviceRequests) && + host.DeviceRequests.length === 0)) && + isRecord(host.PortBindings) && + Object.keys(host.PortBindings).length === 0 && + Array.isArray(host.Mounts) && + host.Mounts.length === 2 && + value.mounts.length === 2 + ) + ) { + return refuse(); + } + for (const target of [ + "/hack-storage-witness-helper.mjs", + "/hack-storage-witness", + ] as const) { + const configured = host.Mounts.filter( + (mount) => isRecord(mount) && mount.Target === target + ); + const physical = value.mounts.filter( + (mount) => isRecord(mount) && mount.Destination === target + ); + if ( + !( + configured.length === 1 && + physical.length === 1 && + isRecord(configured[0]) && + isRecord(physical[0]) + ) + ) { + return refuse(); + } + const requested = configured[0], + observed = physical[0]; + if (target === "/hack-storage-witness-helper.mjs") { + if ( + !( + requested.Type === "bind" && + requested.Source === program && + configuredReadOnly(requested, true) && + isRecord(requested.BindOptions) && + requested.BindOptions.NonRecursive === true && + requested.BindOptions.Propagation === "rprivate" && + requested.BindOptions.CreateMountpoint !== true && + observed.Type === "bind" && + observed.Source === program && + observed.RW === false && + observed.Propagation === "rprivate" + ) + ) { + return refuse(); + } + } else if ( + !( + requested.Type === "volume" && + requested.Source === input.target.name && + configuredReadOnly(requested, input.readonly) && + !Object.hasOwn(requested, "BindOptions") && + isRecord(requested.VolumeOptions) && + Object.keys(requested.VolumeOptions).length === 1 && + requested.VolumeOptions.NoCopy === true && + observed.Type === "volume" && + observed.Name === input.target.name && + observed.Source === input.target.mountpoint && + observed.Driver === "local" && + observed.RW === !input.readonly && + observed.Propagation === "" + ) + ) { + return refuse(); + } + } + return value as NativeComposeStorageDockerCarrier; +} +/** Preserve every static field and complete mount row. Only the two observed closed + * optional defaults are normalized; no value, duplicate or projected field is dropped. */ +export function nativeComposeStorageDockerCarrierPolicy( + value: NativeComposeStorageDockerCarrier +): string { + if ( + !( + isRecord(value.host) && + Array.isArray(value.mounts) && + Object.hasOwn(value.host, "OomKillDisable") && + (value.host.OomKillDisable === null || + value.host.OomKillDisable === false) && + Object.hasOwn(value, "execIds") && + (value.execIds === null || + (Array.isArray(value.execIds) && value.execIds.length === 0)) + ) + ) { + return refuse(); + } + const { state: _, ...fixed } = value; + const mounts = [...value.mounts].sort((a, b) => + JSON.stringify(a).localeCompare(JSON.stringify(b)) + ); + return JSON.stringify({ + ...fixed, + host: { ...value.host, OomKillDisable: false }, + execIds: [], + mounts, + }); +} +export function nativeComposeStorageDockerCreateArgs(opts: { + readonly input: NativeComposeStorageXattrInvocation; + readonly program: string; +}): readonly string[] { + const { input, program } = opts; + if ( + input.artifact.imageId !== NATIVE_STORAGE_DOCKER_ARTIFACT.imageId || + !input.target.mountpoint + ) { + return refuse(); + } + return [ + "create", + "--interactive", + "--name", + `hack-storage-witness-${input.invocationId}`, + "--pull", + "never", + "--platform", + input.artifact.platform, + "--user", + `${input.uid}:${input.gid}`, + "--read-only", + "--network", + "none", + "--cap-drop", + "ALL", + "--security-opt", + "no-new-privileges:true", + "--memory", + "256m", + "--cpus", + "1", + "--pids-limit", + "32", + "--restart", + "no", + "--log-driver", + "none", + "--mount", + `type=bind,src=${program},dst=/hack-storage-witness-helper.mjs,readonly,bind-recursive=disabled,bind-propagation=rprivate`, + "--mount", + `type=volume,src=${input.target.name},dst=/hack-storage-witness,volume-nocopy${input.readonly ? ",readonly" : ""}`, + "--label", + `${NATIVE_STORAGE_CARRIER_LABEL}=${input.invocationId}`, + "--label", + `io.hack.storage-witness.owner=${input.target.ownerToken}`, + "--label", + `io.hack.storage-witness.generation=${input.scope.generationId}`, + "--label", + `io.hack.storage-witness.helper=${input.artifact.helperHash}`, + "--entrypoint", + "/usr/local/bin/bun", + input.artifact.imageId, + "--no-env-file", + "/hack-storage-witness-helper.mjs", + ]; +} diff --git a/src/lib/native-compose-storage-witness-docker.ts b/src/lib/native-compose-storage-witness-docker.ts new file mode 100644 index 000000000..a4a194f9e --- /dev/null +++ b/src/lib/native-compose-storage-witness-docker.ts @@ -0,0 +1,967 @@ +import { randomBytes } from "node:crypto"; +import { constants } from "node:fs"; +import { lstat, mkdir, open, readdir, rmdir, unlink } from "node:fs/promises"; +import { join } from "node:path"; +import { isRecord } from "./guards.ts"; +import { + assertNativeComposeMaterialAuthority, + type NativeComposeGenerationStore, + type NativeComposeMaterialAuthority, + type NativeComposeMaterialBinding, +} from "./native-compose-generation.ts"; +import { createNativeComposeProbe } from "./native-compose-ownership.ts"; +import { + type HeldDirectory, + holdDirectory, + privateDirectory, + recheckDirectories, + sameFile, + writeExclusive, +} from "./native-compose-private-state.ts"; +import { + NATIVE_STORAGE_DOCKER_ARTIFACT as artifact, + nativeComposeStorageDockerHelper, +} from "./native-compose-storage-witness-docker-artifact.ts"; +import { + assertNativeComposeStorageDockerCarrierVolume, + inspectNativeComposeStorageDockerVolume, + observeNativeComposeStorageDockerTarget, +} from "./native-compose-storage-witness-docker-inventory.ts"; +import { + createNativeComposeStorageDockerEmptyLeaf, + holdNativeComposeStorageDockerIo, +} from "./native-compose-storage-witness-docker-io.ts"; +import { + checkNativeComposeStorageDockerCarrier, + NATIVE_STORAGE_CARRIER_FORMAT, + NATIVE_STORAGE_CARRIER_LABEL, + type NativeComposeStorageDockerCarrier, + nativeComposeStorageDockerCarrierPolicy, + nativeComposeStorageDockerCreateArgs, +} from "./native-compose-storage-witness-docker-policy.ts"; +import { + captureNativeComposeStorageXattrCarrier, + type NativeComposeStorageXattrCarrier, + type NativeComposeStorageXattrInvocation, +} from "./native-compose-storage-witness-xattr-carrier.ts"; +import { + encodeNativeComposeStorageXattrRequest, + parseNativeComposeStorageXattrResponse, + refuseNativeComposeStorageXattr as refuse, +} from "./native-compose-storage-witness-xattr-codec.ts"; +import { findExecutableInPath, type RunExitEvent, run } from "./shell.ts"; + +const ID = /^[a-f0-9]{64}$/; +const IMAGE_IDS = [ + artifact.imageId, + "sha256:5c51cee225076d3c7db2150683141476298062489de4660f2d1729e522641f91", + "sha256:1cb8f81099813a0ec61f99b69348f6594e188cf1689c20a0a53a5ddd23b37708", +] as const; +const CAPTURE_LIMIT = 4096; +type Selection = { readonly name: string; readonly storage: string }; +type Context = { + readonly authority: NativeComposeMaterialAuthority; + readonly store: NativeComposeGenerationStore; + readonly engineId: string; + readonly signal: AbortSignal; + readonly deadline: number; +}; +type Files = { + readonly held: readonly HeldDirectory[]; + readonly path: string; + readonly program: string; + readonly input: string; + readonly programInfo: Awaited>; + readonly inputInfo: Awaited>; + readonly requestText: string; + readonly captures: { + readonly path: string; + readonly info: Awaited>; + }[]; +}; +function same(left: unknown, right: unknown): boolean { + return JSON.stringify(left) === JSON.stringify(right); +} +function lifetime(context: Context): void { + if (context.signal.aborted || Date.now() >= context.deadline) { + refuse(); + } +} +async function current( + context: Context, + selection?: Selection, + effect = false +): Promise { + const saved = await context.store.loadCurrent(); + const generation = + (await context.store.loadPending()) ?? saved.generation ?? refuse(); + if (selection) { + const document = await context.store.readGenerationDocument(generation); + const selected = + isRecord(document.volumes) && + Object.hasOwn(document.volumes, selection.storage) + ? document.volumes[selection.storage] + : null; + const declared = isRecord(selected) && selected.name === selection.name; + const retained = + saved.storageWitnesses?.some( + (entry) => + entry.name === selection.name && entry.storage === selection.storage + ) ?? false; + if (!(declared || (!effect && retained))) { + return refuse(); + } + } + return await assertNativeComposeMaterialAuthority({ + authority: context.authority, + generation, + phase: effect ? "effect" : "inspect", + }); +} +function engine( + probe: ReturnType, + context: Context +): Promise { + return probe(["info", "--format", "{{json .ID}}"]).then((text) => { + if (JSON.parse(text) !== context.engineId) { + refuse(); + } + }); +} +async function image( + probe: ReturnType +): Promise { + const value: unknown = JSON.parse( + await probe([ + "image", + "inspect", + "--format", + '{"id":{{json .Id}},"os":{{json .Os}},"arch":{{json .Architecture}},"volumes":{{json (index .Config "Volumes")}}}', + artifact.imageId, + ]) + ); + if ( + !( + isRecord(value) && + value.id === artifact.imageId && + value.os === "linux" && + value.arch === "arm64" && + value.volumes === null + ) + ) { + return refuse(); + } +} +export { image as assertNativeComposeStorageDockerImage }; +async function invocationFiles( + context: Context, + input: NativeComposeStorageXattrInvocation +): Promise { + const held: HeldDirectory[] = []; + try { + const owner = await holdDirectory( + join( + context.store.identity.checkoutRoot, + ".hack", + ".internal", + "native-compose", + context.store.identity.instanceId + ), + true + ); + held.push(owner); + const root = await privateDirectory(join(owner.path, "storage-carriers")); + held.push(root); + await owner.file.sync(); + const path = join(root.path, input.invocationId); + await mkdir(path, { mode: 0o700 }); + await root.file.sync(); + const directory = await holdDirectory(path, true); + held.push(directory); + const program = join(path, "helper.mjs"), + request = join(path, "request.json"); + const programInfo = await writeExclusive( + program, + nativeComposeStorageDockerHelper() + ); + const requestText = encodeNativeComposeStorageXattrRequest(input.request); + const inputInfo = await writeExclusive(request, requestText); + // read-only guest code, private owned parent. This leaf is not executable on the host. + const file = await open(program, constants.O_RDONLY | constants.O_NOFOLLOW); + try { + await file.chmod(0o444); + await file.sync(); + } finally { + await file.close(); + } + await directory.file.sync(); + await recheckDirectories(held); + return { + held, + path, + program, + input: request, + programInfo, + inputInfo, + requestText, + captures: [], + }; + } catch { + await Promise.allSettled(held.map((directory) => directory.file.close())); + return refuse(); + } +} +async function retireInvocationFiles(files: Files): Promise { + await checkFiles(files); + const owned = [ + { path: files.program, info: files.programInfo }, + { path: files.input, info: files.inputInfo }, + ...files.captures, + ]; + const names = owned + .map((file) => file.path.slice(files.path.length + 1)) + .sort(); + if (!same((await readdir(files.path)).sort(), names)) { + return refuse(); + } + for (const file of owned) { + await recheckDirectories(files.held); + const named = await lstat(file.path); + if ( + !( + named.isFile() && + named.nlink === 1 && + named.uid === process.getuid?.() && + sameFile(named, file.info) + ) + ) { + return refuse(); + } + await unlink(file.path); + } + await files.held.at(-1)?.file.sync(); + // The known helper and every owned host group are absent before this retirement. + await rmdir(files.path); + await files.held.at(-2)?.file.sync(); +} +async function checkFiles(files: Files): Promise { + await recheckDirectories(files.held); + for (const [path, original, text] of [ + [files.program, files.programInfo, nativeComposeStorageDockerHelper()], + [files.input, files.inputInfo, files.requestText], + ] as const) { + const file = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW); + try { + const before = await file.stat(); + if ( + !( + before.isFile() && + before.nlink === 1 && + before.uid === process.getuid?.() && + sameFile(before, original) && + before.size === Buffer.byteLength(text) && + (before.mode & 0o777) === (path === files.program ? 0o444 : 0o600) + ) + ) { + return refuse(); + } + const bytes = await file.readFile(); + const after = await file.stat(), + named = await lstat(path); + if ( + !( + sameFile(after, before) && + sameFile(named, before) && + bytes.toString("utf8") === text && + after.mtimeMs === before.mtimeMs && + after.ctimeMs === before.ctimeMs + ) + ) { + return refuse(); + } + } finally { + await file.close(); + } + } +} +async function groupAbsent(pid: number): Promise { + if (!(Number.isInteger(pid) && pid > 1)) { + return refuse(); + } + const deadline = Date.now() + 3000; + while (true) { + try { + process.kill(-pid, 0); + } catch (error: unknown) { + if (isRecord(error) && error.code === "ESRCH") { + return; + } + return refuse(); + } + if (Date.now() >= deadline) { + return refuse(); + } + await new Promise((resolve) => setTimeout(resolve, 20)); + } +} +/** No callback can reject before the shared owner awaits exit. A rejected owner or + * unproven group retains the journal; daemon effects are never inferred from exit. */ +export async function runNativeComposeStorageDockerCommand(supplied: { + readonly context: Pick; + readonly files: Pick< + Files, + "held" | "path" | "input" | "inputInfo" | "requestText" | "captures" + >; + readonly args: readonly string[]; + readonly beforeSpawn: () => void; + readonly assertAdmitted: () => Promise; + readonly timeoutMs: number; + readonly cleanup?: boolean; +}): Promise<{ readonly exitCode: number; readonly stdout: string }> { + const opts = { + ...supplied, + context: { ...supplied.context }, + args: [...supplied.args], + }; + const capture = randomBytes(16).toString("hex"); + const stdout = join(opts.files.path, `${capture}.stdout`), + stderr = join(opts.files.path, `${capture}.stderr`); + const out = await createNativeComposeStorageDockerEmptyLeaf(stdout), + err = await createNativeComposeStorageDockerEmptyLeaf(stderr); + opts.files.captures.push( + { path: stdout, info: out }, + { path: stderr, info: err } + ); + await opts.files.held.at(-1)?.file.sync(); + const io = await holdNativeComposeStorageDockerIo({ + directories: opts.files.held, + input: { + path: opts.files.input, + info: opts.files.inputInfo, + text: opts.files.requestText, + }, + stdout: { path: stdout, info: out }, + stderr: { path: stderr, info: err }, + limit: CAPTURE_LIMIT, + }); + let admitted = false, + settled = false, + pid = 0, + exit: RunExitEvent | null = null; + try { + await opts.assertAdmitted(); + const timeoutMs = opts.cleanup + ? opts.timeoutMs + : Math.min(opts.timeoutMs, opts.context.deadline - Date.now()); + if (!(timeoutMs > 0)) { + return refuse(); + } + // Fixed FSIZE quota bounds both inherited capture files, including daemon + // stderr. Acceptance remains 4096 bytes; no child reopens a named file. + const code = await run( + [ + "/bin/sh", + "-c", + 'ulimit -f 8; exec "$@"', + "storage-witness-command", + ...opts.args, + ], + { + privateIo: io.descriptors, + timeoutMs, + signal: opts.cleanup ? undefined : opts.context.signal, + beforeSpawn: () => { + io.assertFresh(); + opts.beforeSpawn(); + admitted = true; + }, + onSpawn: (event) => { + pid = event.ownsProcessGroup + ? (event.processGroupId ?? event.pid) + : 0; + return Promise.resolve(); + }, + onExit: (event) => { + exit = event; + return Promise.resolve(); + }, + } + ); + await groupAbsent(pid); + settled = true; + if ( + !( + exit && + !(exit as RunExitEvent).timedOut && + !(exit as RunExitEvent).cancelled + ) + ) { + return refuse(); + } + return { exitCode: code, stdout: io.read().stdout }; + } finally { + if (!admitted || settled) { + await io.close(); + } else { + io.retainUnsettled(); + } + } +} +function matchesScope( + input: NativeComposeStorageXattrInvocation, + binding: NativeComposeMaterialBinding +): boolean { + return ( + input.scope.generationId === binding.generationId && + input.scope.currentGenerationId === binding.currentGenerationId && + input.scope.pendingGenerationId === binding.pendingGenerationId && + input.scope.pendingToken === binding.pendingToken && + input.target.runtimeIdentity === binding.identity.composeProject && + input.target.ownerToken === binding.identity.ownerToken + ); +} + +/** SOURCE-only transport until its persistent-volume/PGDATA qualification and CLI + * admission are complete. No module initialization executes Docker or the helper. */ +export async function createNativeComposeDockerStorageXattrCarrier( + opts: Context +): Promise { + const context: Context = Object.freeze({ ...opts }); + const docker = findExecutableInPath("docker") ?? refuse(); + const selectors = [ + "DOCKER_HOST", + "DOCKER_CONTEXT", + "DOCKER_CONFIG", + "DOCKER_TLS_VERIFY", + "DOCKER_CERT_PATH", + ].map((key) => [key, process.env[key]] as const); + const tool = () => { + if ( + findExecutableInPath("docker") !== docker || + selectors.some(([key, value]) => process.env[key] !== value) + ) { + refuse(); + } + }; + const probe = (cleanup = false) => { + tool(); + if (!cleanup) { + lifetime(context); + } + return createNativeComposeProbe({ + signal: cleanup ? undefined : context.signal, + timeoutMs: cleanup + ? 10_000 + : Math.min(15_000, context.deadline - Date.now()), + }); + }; + // Resolve the declared dependency before Expected, volume creation or any helper + // footprint. This first platform remains explicit; emulation is not a fallback. + try { + const prerequisite = probe(); + const daemon: unknown = JSON.parse( + await prerequisite([ + "info", + "--format", + '{"id":{{json .ID}},"os":{{json .OSType}},"arch":{{json .Architecture}}}', + ]) + ); + if ( + !( + isRecord(daemon) && + daemon.id === context.engineId && + daemon.os === "linux" && + (daemon.arch === "arm64" || daemon.arch === "aarch64") + ) + ) { + refuse(); + } + await image(prerequisite); + nativeComposeStorageDockerHelper(); + lifetime(context); + } catch { + throw new Error( + "Native storage witnesses require a Linux arm64 daemon and the fixed cached Bun 1.4.2 helper image; no image is pulled automatically." + ); + } + const inspect = async (selection: Selection) => { + const before = await current(context, selection); + const target = await observeNativeComposeStorageDockerTarget({ + probe: probe(), + current: before, + selection, + engineId: context.engineId, + stopped: false, + }); + const after = await current(context, selection); + if (!same(before, after)) { + return refuse(); + } + lifetime(context); + return target; + }; + const provision = async ( + selection: Selection & { + readonly engineId: string; + readonly runtimeIdentity: string; + readonly ownerToken: string; + } + ) => { + const before = await current(context, selection, true); + if ( + !( + selection.engineId === context.engineId && + selection.runtimeIdentity === before.identity.composeProject && + selection.ownerToken === before.identity.ownerToken + ) + ) { + return refuse(); + } + const absent = await observeNativeComposeStorageDockerTarget({ + probe: probe(), + current: before, + selection, + engineId: context.engineId, + stopped: true, + }); + if (absent.volume !== null) { + return refuse(); + } + await image(probe()); + const nonce = randomBytes(16).toString("hex"); + const directory = await holdDirectory( + join( + context.store.identity.checkoutRoot, + ".hack", + ".internal", + "native-compose", + context.store.identity.instanceId + ), + true + ); + const held: HeldDirectory[] = [directory]; + let files: Files | null = null; + try { + const root = await privateDirectory( + join(directory.path, "storage-provisions") + ); + held.push(root); + const path = join(root.path, nonce); + await mkdir(path, { mode: 0o700 }); + await root.file.sync(); + const leaf = await holdDirectory(path, true); + held.push(leaf); + const request = join(path, "request.json"), + program = join(path, "intent.json"); + const inputInfo = + await createNativeComposeStorageDockerEmptyLeaf(request); + const programInfo = await writeExclusive( + program, + JSON.stringify({ ...selection, nonce }) + ); + await leaf.file.sync(); + files = { + held, + path, + input: request, + program, + programInfo, + inputInfo, + requestText: "", + captures: [], + }; + const read = probe(); + await engine(read, context); + const finalAbsent = await observeNativeComposeStorageDockerTarget({ + probe: read, + current: before, + selection, + engineId: context.engineId, + stopped: true, + }); + if (finalAbsent.volume !== null) { + return refuse(); + } + tool(); + await recheckDirectories(held); + const latest = await current(context, selection, true); + if (!same(before, latest)) { + return refuse(); + } + const result = await runNativeComposeStorageDockerCommand({ + context, + files, + args: [ + docker, + "volume", + "create", + "--driver", + "local", + "--label", + `com.docker.compose.project=${selection.runtimeIdentity}`, + "--label", + `com.docker.compose.volume=${selection.storage}`, + "--label", + "io.hack.native-config.version=1", + "--label", + `io.hack.native-config.instance=${selection.runtimeIdentity}`, + "--label", + `io.hack.native-config.owner=${selection.ownerToken}`, + "--label", + `io.hack.native-config.storage=${selection.storage}`, + "--label", + `io.hack.native-config.storage-provision=${nonce}`, + selection.name, + ], + timeoutMs: 15_000, + beforeSpawn: () => { + tool(); + lifetime(context); + }, + assertAdmitted: async () => { + await engine(probe(), context); + tool(); + if (!same(before, await current(context, selection, true))) { + refuse(); + } + }, + }); + if (!(result.exitCode === 0 && result.stdout.trim() === selection.name)) { + return refuse(); + } + const created = await inspectNativeComposeStorageDockerVolume({ + probe: probe(), + current: latest, + selection, + }); + if (created.provision !== nonce) { + return refuse(); + } + const after = await current(context, selection, true); + if (!same(before, after)) { + return refuse(); + } + } finally { + await Promise.allSettled(held.map((entry) => entry.file.close())); + } + }; + const invoke = async (input: NativeComposeStorageXattrInvocation) => { + const before = await current(context, input.target, !input.readonly); + if (!(matchesScope(input, before) && same(input.artifact, artifact))) { + return refuse(); + } + const target = await inspect(input.target); + if (!same(target, input.target)) { + return refuse(); + } + const files = await invocationFiles(context, input); + let known: { + readonly id: string; + readonly createdAt: string; + readonly policy: string; + } | null = null; + let completed = false; + try { + const read = probe(); + await engine(read, context); + await image(read); + await checkFiles(files); + tool(); + const admitted = await current(context, input.target, !input.readonly); + if (!same(before, admitted)) { + return refuse(); + } + const created = await runNativeComposeStorageDockerCommand({ + context, + files, + args: [ + docker, + ...nativeComposeStorageDockerCreateArgs({ + input, + program: files.program, + }), + ], + timeoutMs: 15_000, + beforeSpawn: () => { + tool(); + lifetime(context); + }, + assertAdmitted: async () => { + await engine(probe(), context); + tool(); + if ( + !same(before, await current(context, input.target, !input.readonly)) + ) { + refuse(); + } + }, + }); + const id = created.stdout.trim(); + if (!(created.exitCode === 0 && ID.test(id))) { + return refuse(); + } + const inspectCarrier = async ( + cleanup = false + ): Promise => { + const observed = probe(cleanup); + await engine(observed, context); + const value: unknown = JSON.parse( + await observed([ + "container", + "inspect", + "--format", + NATIVE_STORAGE_CARRIER_FORMAT, + id, + ]) + ); + return checkNativeComposeStorageDockerCarrier({ + value, + input, + program: files.program, + imageIds: IMAGE_IDS, + }); + }; + const carrier = await inspectCarrier(); + known = { + id, + createdAt: carrier.createdAt, + policy: nativeComposeStorageDockerCarrierPolicy(carrier), + }; + const owned = known; + if ( + !( + carrier.state.Running === false && + carrier.state.Pid === 0 && + carrier.state.Status === "created" + ) + ) { + return refuse(); + } + await input.recordCreated({ id, createdAt: known.createdAt }); + await checkFiles(files); + await image(probe()); + await assertNativeComposeStorageDockerCarrierVolume({ + probe: probe(), + current: before, + target: input.target, + }); + const holders = await observeNativeComposeStorageDockerTarget({ + probe: probe(), + current: before, + selection: input.target, + engineId: context.engineId, + stopped: !input.readonly, + carrier: { ...known, invocationId: input.invocationId }, + }); + if (!same(holders, input.target)) { + return refuse(); + } + const ready = await inspectCarrier(); + if ( + !( + ready.createdAt === known.createdAt && + nativeComposeStorageDockerCarrierPolicy(ready) === known.policy && + ready.state.Running === false && + ready.state.Pid === 0 && + ready.state.Status === "created" + ) + ) { + return refuse(); + } + tool(); + const final = await current(context, input.target, !input.readonly); + if (!same(before, final)) { + return refuse(); + } + const result = await runNativeComposeStorageDockerCommand({ + context, + files, + args: [docker, "start", "-ai", id], + timeoutMs: 15_000, + beforeSpawn: () => { + tool(); + lifetime(context); + }, + assertAdmitted: async () => { + const selected = await observeNativeComposeStorageDockerTarget({ + probe: probe(), + current: before, + selection: input.target, + engineId: context.engineId, + stopped: !input.readonly, + carrier: { ...owned, invocationId: input.invocationId }, + }); + if (!same(selected, input.target)) { + refuse(); + } + await checkFiles(files); + const ready = await inspectCarrier(); + if ( + !( + ready.createdAt === owned.createdAt && + nativeComposeStorageDockerCarrierPolicy(ready) === owned.policy && + ready.state.Running === false && + ready.state.Pid === 0 && + ready.state.Status === "created" + ) + ) { + refuse(); + } + await engine(probe(), context); + tool(); + if ( + !same(before, await current(context, input.target, !input.readonly)) + ) { + refuse(); + } + }, + }); + if (result.exitCode !== 0 && result.exitCode !== 1) { + return refuse(); + } + const response = parseNativeComposeStorageXattrResponse(result.stdout); + if ((response.outcome === "refused") !== (result.exitCode === 1)) { + return refuse(); + } + const stopped = await inspectCarrier(true); + if ( + !( + stopped.createdAt === known.createdAt && + nativeComposeStorageDockerCarrierPolicy(stopped) === known.policy && + stopped.state.Running === false && + stopped.state.Pid === 0 && + stopped.state.Status === "exited" && + stopped.state.ExitCode === result.exitCode && + stopped.state.Paused === false && + stopped.state.Restarting === false && + stopped.state.OOMKilled === false && + stopped.state.Dead === false && + stopped.state.Error === "" + ) + ) { + return refuse(); + } + const repeated = await inspectCarrier(true); + if ( + !( + repeated.createdAt === known.createdAt && + nativeComposeStorageDockerCarrierPolicy(repeated) === known.policy && + same(repeated.state, stopped.state) + ) + ) { + return refuse(); + } + await checkFiles(files); + tool(); + const cleanupAuthority = await current(context, input.target); + if (!same(before, cleanupAuthority)) { + return refuse(); + } + const removed = await runNativeComposeStorageDockerCommand({ + context, + files, + cleanup: true, + args: [docker, "rm", id], + timeoutMs: 10_000, + beforeSpawn: tool, + assertAdmitted: async () => { + const finalCarrier = await inspectCarrier(true); + if ( + !( + finalCarrier.createdAt === owned.createdAt && + nativeComposeStorageDockerCarrierPolicy(finalCarrier) === + owned.policy && + same(finalCarrier.state, stopped.state) + ) + ) { + refuse(); + } + await engine(probe(true), context); + tool(); + if (!same(before, await current(context, input.target))) { + refuse(); + } + }, + }); + if (removed.exitCode !== 0) { + return refuse(); + } + const absent = probe(true); + await engine(absent, context); + const inventory = ( + await absent([ + "container", + "ls", + "-a", + "--no-trunc", + "--format", + "{{json .ID}}", + ]) + ) + .trim() + .split("\n") + .filter(Boolean) + .map((line) => JSON.parse(line) as unknown); + if ( + inventory.some( + (value) => typeof value !== "string" || !ID.test(value) + ) || + inventory.includes(id) || + new Set(inventory).size !== inventory.length + ) { + return refuse(); + } + const selected = ( + await absent([ + "container", + "ls", + "-a", + "--no-trunc", + "--filter", + `label=${NATIVE_STORAGE_CARRIER_LABEL}=${input.invocationId}`, + "--format", + "{{json .ID}}", + ]) + ).trim(); + if (selected !== "") { + return refuse(); + } + const after = await inspect(input.target); + if (!same(after, input.target)) { + return refuse(); + } + await retireInvocationFiles(files); + completed = true; + return { + artifact, + carrierId: id, + carrierCreatedAt: known.createdAt, + containersAfterCleanup: [], + engineId: context.engineId, + invocationId: input.invocationId, + readonly: input.readonly, + uid: input.uid, + gid: input.gid, + scope: input.scope, + outcome: "complete", + exitCode: result.exitCode, + response: result.stdout, + target: after, + stopped: { id, running: false, pid: 0, exitCode: result.exitCode }, + }; + } finally { + // Failure never starts a competing stop/remove after unknown child disposition. + // The original prospective/created journal and private input remain authoritative. + await Promise.allSettled(files.held.map((entry) => entry.file.close())); + if (!completed && known) { + // Exact retained ID/birth are already in the existing private finite journal. + } + } + }; + return captureNativeComposeStorageXattrCarrier({ + artifact, + ports: { inspect, provision, invoke }, + signal: context.signal, + deadline: context.deadline, + }); +} diff --git a/src/lib/native-compose-storage-witness-helper-bundle.ts b/src/lib/native-compose-storage-witness-helper-bundle.ts new file mode 100644 index 000000000..ff3242635 --- /dev/null +++ b/src/lib/native-compose-storage-witness-helper-bundle.ts @@ -0,0 +1,4 @@ +// Generated from scripts/native-storage-witness-helper.ts. Do not edit the payload. +export const NATIVE_STORAGE_WITNESS_HELPER = + // biome-ignore lint/suspicious/noTemplateCurlyInString: Preserve qualified helper bytes without host interpolation. + '// @bun\n// src/lib/native-compose-storage-witness-xattr-codec.ts\nimport { randomBytes, timingSafeEqual } from "crypto";\n\n// src/lib/guards.ts\nfunction isRecord(value) {\n return typeof value === "object" && value !== null && !Array.isArray(value);\n}\n\n// src/lib/native-compose-storage-witness-xattr-codec.ts\nvar NATIVE_STORAGE_XATTR_KIND = "directory-xattr";\nvar NATIVE_STORAGE_XATTR_VERSION = 1;\nvar NATIVE_STORAGE_XATTR_BYTES = 32;\nvar NATIVE_STORAGE_XATTR_INPUT_LIMIT = 4096;\nvar NAME = /^user\\.hack\\.storage\\.[a-f0-9]{64}$/;\nvar VALUE = /^[a-f0-9]{64}$/;\nvar DECIMAL = /^(0|[1-9][0-9]{0,19})$/;\nvar UINT64_MAX = 18446744073709551615n;\nvar UINT32_MAX = 4294967295;\nfunction refuseNativeComposeStorageXattr() {\n throw new Error("Native storage xattr proof refused; values omitted.");\n}\nfunction keys(value, expected) {\n return Object.keys(value).sort().join(",") === expected;\n}\nfunction uint32(value) {\n return typeof value === "number" && Number.isInteger(value) && !Object.is(value, -0) && value >= 0 && value <= UINT32_MAX;\n}\nfunction uint64(value) {\n return typeof value === "string" && DECIMAL.test(value) && BigInt(value) <= UINT64_MAX;\n}\nfunction nativeComposeStorageXattrRootValid(value) {\n return isRecord(value) && keys(value, "device,gid,inode,uid") && uint64(value.device) && uint64(value.inode) && value.inode !== "0" && uint32(value.uid) && uint32(value.gid);\n}\nfunction sameNativeComposeStorageXattrRoot(left, right) {\n return left.device === right.device && left.inode === right.inode && left.uid === right.uid && left.gid === right.gid;\n}\nfunction markerValid(value) {\n return value.kind === NATIVE_STORAGE_XATTR_KIND && value.version === NATIVE_STORAGE_XATTR_VERSION && typeof value.name === "string" && NAME.test(value.name) && typeof value.valueHex === "string" && VALUE.test(value.valueHex);\n}\nfunction decodeNativeComposeStorageXattrRequest(value) {\n if (!isRecord(value)) {\n return refuseNativeComposeStorageXattr();\n }\n if (keys(value, "kind,operation,version") && value.kind === NATIVE_STORAGE_XATTR_KIND && value.version === NATIVE_STORAGE_XATTR_VERSION && value.operation === "root") {\n return Object.freeze({\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n operation: "root"\n });\n }\n if (!(keys(value, "kind,name,operation,root,valueHex,version") && markerValid(value) && (value.operation === "seed" || value.operation === "verify") && nativeComposeStorageXattrRootValid(value.root))) {\n return refuseNativeComposeStorageXattr();\n }\n return Object.freeze({\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n operation: value.operation,\n name: value.name,\n valueHex: value.valueHex,\n root: Object.freeze({ ...value.root })\n });\n}\nfunction decodeNativeComposeStorageXattrResponse(value) {\n if (!(isRecord(value) && value.kind === NATIVE_STORAGE_XATTR_KIND && value.version === NATIVE_STORAGE_XATTR_VERSION)) {\n return refuseNativeComposeStorageXattr();\n }\n if (value.outcome === "refused" && keys(value, "kind,outcome,version")) {\n return Object.freeze({\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n outcome: "refused"\n });\n }\n if (!nativeComposeStorageXattrRootValid(value.root)) {\n return refuseNativeComposeStorageXattr();\n }\n const root = Object.freeze({ ...value.root });\n if ((value.outcome === "root" || value.outcome === "seeded") && keys(value, "kind,outcome,root,version")) {\n return Object.freeze({\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n outcome: value.outcome,\n root\n });\n }\n if (value.outcome === "verified" && keys(value, "kind,outcome,root,valueHex,version") && typeof value.valueHex === "string" && VALUE.test(value.valueHex)) {\n return Object.freeze({\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n outcome: "verified",\n root,\n valueHex: value.valueHex\n });\n }\n return refuseNativeComposeStorageXattr();\n}\nfunction encodeNativeComposeStorageXattrResponse(value) {\n return `${JSON.stringify(decodeNativeComposeStorageXattrResponse(value))}\n`;\n}\nfunction parse(text, limit, decode) {\n if (Buffer.byteLength(text) > limit) {\n return refuseNativeComposeStorageXattr();\n }\n try {\n const value = decode(JSON.parse(text));\n if (text !== `${JSON.stringify(value)}\n`) {\n return refuseNativeComposeStorageXattr();\n }\n return value;\n } catch {\n return refuseNativeComposeStorageXattr();\n }\n}\nfunction parseNativeComposeStorageXattrRequest(text) {\n return parse(text, NATIVE_STORAGE_XATTR_INPUT_LIMIT, decodeNativeComposeStorageXattrRequest);\n}\nfunction nativeComposeStorageXattrValueMatches(actual, expectedHex) {\n return actual.byteLength === NATIVE_STORAGE_XATTR_BYTES && VALUE.test(expectedHex) && timingSafeEqual(actual, Buffer.from(expectedHex, "hex"));\n}\n\n// src/lib/native-compose-storage-witness-xattr-helper.ts\nfunction root(value) {\n if (!nativeComposeStorageXattrRootValid(value)) {\n return refuseNativeComposeStorageXattr();\n }\n return Object.freeze({ ...value });\n}\nfunction requireRoot(actual, expected) {\n if (!sameNativeComposeStorageXattrRoot(root(actual), expected)) {\n refuseNativeComposeStorageXattr();\n }\n}\nfunction runNativeComposeStorageXattrHelper(opts) {\n let fd = null;\n let capturedClose = null;\n try {\n const request = decodeNativeComposeStorageXattrRequest(opts.request);\n const {\n effectiveUid,\n effectiveGid,\n probeRoot,\n openRoot,\n statRoot,\n createXattr,\n syncRoot,\n readXattr,\n closeRoot\n } = opts.kernel;\n capturedClose = closeRoot;\n if (request.operation === "root") {\n const before = root(probeRoot());\n requireRoot(probeRoot(), before);\n return {\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n outcome: "root",\n root: before\n };\n }\n if (effectiveUid() !== request.root.uid || effectiveGid() !== request.root.gid) {\n return refuseNativeComposeStorageXattr();\n }\n fd = openRoot();\n if (!Number.isInteger(fd) || fd < 0) {\n return refuseNativeComposeStorageXattr();\n }\n requireRoot(statRoot(fd), request.root);\n let response;\n if (request.operation === "seed") {\n createXattr(fd, request.name, Buffer.from(request.valueHex, "hex"));\n syncRoot(fd);\n requireRoot(statRoot(fd), request.root);\n response = {\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n outcome: "seeded",\n root: request.root\n };\n } else {\n const first = readXattr(fd, request.name, NATIVE_STORAGE_XATTR_BYTES);\n if (!nativeComposeStorageXattrValueMatches(first, request.valueHex)) {\n return refuseNativeComposeStorageXattr();\n }\n requireRoot(statRoot(fd), request.root);\n const latest = readXattr(fd, request.name, NATIVE_STORAGE_XATTR_BYTES);\n if (!nativeComposeStorageXattrValueMatches(latest, request.valueHex)) {\n return refuseNativeComposeStorageXattr();\n }\n requireRoot(statRoot(fd), request.root);\n response = {\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n outcome: "verified",\n root: request.root,\n valueHex: Buffer.from(latest).toString("hex")\n };\n }\n const selected = fd;\n fd = null;\n closeRoot(selected);\n return response;\n } catch {\n return {\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n outcome: "refused"\n };\n } finally {\n if (fd !== null && Number.isInteger(fd) && fd >= 0 && capturedClose) {\n try {\n capturedClose(fd);\n } catch {}\n }\n }\n}\n\n// src/lib/native-compose-storage-witness-xattr-linux.ts\nimport {\n closeSync,\n constants,\n fstatSync,\n fsyncSync,\n lstatSync,\n openSync\n} from "fs";\nvar XATTR_NAME = /^user\\.hack\\.storage\\.[a-f0-9]{64}$/;\nvar NATIVE_STORAGE_XATTR_LINUX_ABI = Object.freeze({\n version: 1,\n root: "/hack-storage-witness",\n xattrCreate: 1,\n openCloseExec: 1 << 19,\n fGetFd: 1,\n fdCloseExec: 1,\n pointerBits: 64,\n sizeTBits: 64,\n signedSizeBits: 64,\n arm64: Object.freeze({\n libc: "/lib/aarch64-linux-gnu/libc.so.6",\n directory: 1 << 14,\n noFollow: 1 << 15\n }),\n x64: Object.freeze({\n libc: "/lib/x86_64-linux-gnu/libc.so.6",\n directory: 1 << 16,\n noFollow: 1 << 17\n })\n});\nfunction directory(info) {\n if (!info.isDirectory() || info.isSymbolicLink()) {\n return refuseNativeComposeStorageXattr();\n }\n const value = {\n device: info.dev.toString(),\n inode: info.ino.toString(),\n uid: Number(info.uid),\n gid: Number(info.gid)\n };\n if (!nativeComposeStorageXattrRootValid(value)) {\n return refuseNativeComposeStorageXattr();\n }\n return Object.freeze(value);\n}\nfunction libc() {\n let abi = null;\n if (process.arch === "arm64") {\n abi = NATIVE_STORAGE_XATTR_LINUX_ABI.arm64;\n } else if (process.arch === "x64") {\n abi = NATIVE_STORAGE_XATTR_LINUX_ABI.x64;\n }\n if (process.platform !== "linux" || !abi || constants.O_DIRECTORY !== abi.directory || constants.O_NOFOLLOW !== abi.noFollow || constants.O_RDONLY !== 0 || !process.geteuid || !process.getegid || !process.getuid || !process.getgid || process.getuid() !== process.geteuid() || process.getgid() !== process.getegid()) {\n return refuseNativeComposeStorageXattr();\n }\n return abi.libc;\n}\nasync function createNativeComposeStorageXattrLinuxKernel() {\n const libraryPath = libc();\n const { dlopen, FFIType, ptr } = await import("bun:ffi");\n const library = dlopen(libraryPath, {\n fsetxattr: {\n args: [FFIType.i32, FFIType.ptr, FFIType.ptr, FFIType.u64, FFIType.i32],\n returns: FFIType.i32\n },\n fgetxattr: {\n args: [FFIType.i32, FFIType.ptr, FFIType.ptr, FFIType.u64],\n returns: FFIType.i64\n },\n fcntl: {\n args: [FFIType.i32, FFIType.i32, FFIType.i32],\n returns: FFIType.i32\n }\n });\n const { fsetxattr, fgetxattr, fcntl } = library.symbols;\n const held = new Set;\n let closed = false;\n const active = () => {\n if (closed) {\n refuseNativeComposeStorageXattr();\n }\n };\n const fdOwned = (fd) => {\n active();\n if (!held.has(fd)) {\n refuseNativeComposeStorageXattr();\n }\n };\n const nameBytes = (name) => {\n if (!XATTR_NAME.test(name)) {\n return refuseNativeComposeStorageXattr();\n }\n return Buffer.from(`${name}\\x00`, "ascii");\n };\n const kernel = Object.freeze({\n effectiveUid: () => process.geteuid?.() ?? refuseNativeComposeStorageXattr(),\n effectiveGid: () => process.getegid?.() ?? refuseNativeComposeStorageXattr(),\n probeRoot() {\n active();\n return directory(lstatSync(NATIVE_STORAGE_XATTR_LINUX_ABI.root, {\n bigint: true\n }));\n },\n openRoot() {\n active();\n const fd = openSync(NATIVE_STORAGE_XATTR_LINUX_ABI.root, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW | NATIVE_STORAGE_XATTR_LINUX_ABI.openCloseExec);\n held.add(fd);\n const flags = fcntl(fd, NATIVE_STORAGE_XATTR_LINUX_ABI.fGetFd, 0);\n if (flags < 0 || (flags & NATIVE_STORAGE_XATTR_LINUX_ABI.fdCloseExec) === 0) {\n held.delete(fd);\n closeSync(fd);\n return refuseNativeComposeStorageXattr();\n }\n return fd;\n },\n statRoot(fd) {\n fdOwned(fd);\n return directory(fstatSync(fd, { bigint: true }));\n },\n createXattr(fd, name, value) {\n fdOwned(fd);\n if (value.byteLength !== NATIVE_STORAGE_XATTR_BYTES) {\n return refuseNativeComposeStorageXattr();\n }\n const selectedName = nameBytes(name);\n const bytes = Buffer.from(value);\n if (fsetxattr(fd, ptr(selectedName), ptr(bytes), BigInt(bytes.length), NATIVE_STORAGE_XATTR_LINUX_ABI.xattrCreate) !== 0) {\n return refuseNativeComposeStorageXattr();\n }\n },\n syncRoot(fd) {\n fdOwned(fd);\n fsyncSync(fd);\n },\n readXattr(fd, name, bytes) {\n fdOwned(fd);\n if (bytes !== NATIVE_STORAGE_XATTR_BYTES) {\n return refuseNativeComposeStorageXattr();\n }\n const selectedName = nameBytes(name);\n const value = Buffer.alloc(bytes);\n if (fgetxattr(fd, ptr(selectedName), ptr(value), BigInt(bytes)) !== BigInt(bytes)) {\n return refuseNativeComposeStorageXattr();\n }\n return new Uint8Array(value);\n },\n closeRoot(fd) {\n fdOwned(fd);\n held.delete(fd);\n closeSync(fd);\n }\n });\n return Object.freeze({\n kernel,\n close() {\n active();\n closed = true;\n try {\n for (const fd of held) {\n closeSync(fd);\n }\n } finally {\n held.clear();\n library.close();\n }\n }\n });\n}\n\n// scripts/native-storage-witness-helper.ts\nasync function input() {\n const reader = Bun.stdin.stream().getReader();\n const chunks = [];\n let length = 0;\n try {\n while (true) {\n const next = await reader.read();\n if (next.done) {\n break;\n }\n length += next.value.byteLength;\n if (length > NATIVE_STORAGE_XATTR_INPUT_LIMIT) {\n await reader.cancel();\n return refuseNativeComposeStorageXattr();\n }\n chunks.push(next.value);\n }\n } finally {\n reader.releaseLock();\n }\n return new TextDecoder("utf-8", { fatal: true }).decode(Buffer.concat(chunks));\n}\nfunction refused() {\n return {\n kind: NATIVE_STORAGE_XATTR_KIND,\n version: NATIVE_STORAGE_XATTR_VERSION,\n outcome: "refused"\n };\n}\nasync function nativeStorageWitnessHelperMain() {\n let response = refused();\n let close = null;\n try {\n const request = parseNativeComposeStorageXattrRequest(await input());\n const selected = await createNativeComposeStorageXattrLinuxKernel();\n close = selected.close;\n response = runNativeComposeStorageXattrHelper({\n request,\n kernel: selected.kernel\n });\n } catch {\n response = refused();\n } finally {\n if (close) {\n try {\n close();\n } catch {\n response = refused();\n }\n }\n }\n try {\n await Bun.write(Bun.stdout, encodeNativeComposeStorageXattrResponse(response));\n } catch {\n return 1;\n }\n return response.outcome === "refused" ? 1 : 0;\n}\nif (import.meta.main) {\n process.exitCode = await nativeStorageWitnessHelperMain();\n}\nexport {\n nativeStorageWitnessHelperMain\n};\n'; diff --git a/src/lib/native-compose-storage-witness-receipt.ts b/src/lib/native-compose-storage-witness-receipt.ts index 80f116421..e766e9e95 100644 --- a/src/lib/native-compose-storage-witness-receipt.ts +++ b/src/lib/native-compose-storage-witness-receipt.ts @@ -152,6 +152,22 @@ export function createNativeComposeStorageWitnessReceiptProtocol< const { verifyNativeComposeStorageWitness } = await import( "./native-compose-storage-witness.ts" ); + if (entry.reference.version === 3) { + if (carrier.kind !== "directory-xattr") { + return refuse(); + } + await verifyNativeComposeStorageWitness({ + authority: opts.authority, + generation, + engineId: carrier.engineId, + reference: entry.reference, + xattrCarrier: carrier.carrier, + }); + return; + } + if (carrier.kind === "directory-xattr") { + return refuse(); + } await verifyNativeComposeStorageWitness({ authority: opts.authority, generation, diff --git a/src/lib/native-compose-storage-witness-state.ts b/src/lib/native-compose-storage-witness-state.ts index f92347369..6f18f5501 100644 --- a/src/lib/native-compose-storage-witness-state.ts +++ b/src/lib/native-compose-storage-witness-state.ts @@ -4,6 +4,11 @@ import { type NativeComposeRetainedVolume, nativeComposeRetainedVolumesValid, } from "./native-compose-retained-storage.ts"; +import { + captureNativeComposeStorageXattrArtifact, + type NativeComposeStorageXattrArtifact, + nativeComposeStorageXattrArtifactValid, +} from "./native-compose-storage-witness-xattr-carrier.ts"; const HASH = /^[a-f0-9]{64}$/; const TOKEN = /^[a-f0-9]{32}$/; @@ -11,19 +16,28 @@ const ENGINE = /^[a-zA-Z0-9][a-zA-Z0-9:-]{0,127}$/; type DirectoryAnchor = { readonly dev: number; readonly ino: number }; type Anchor = DirectoryAnchor & { readonly hash: string }; /** Private immutable extension references. Never include in a public DTO. */ -export type NativeComposeStorageWitnessReference = { - readonly version: 1; +type ReferenceAnchors = { readonly volume: NativeComposeRetainedVolume; readonly root: DirectoryAnchor; readonly directory: DirectoryAnchor; readonly expectation: Anchor; readonly completion: Anchor; }; +export type NativeComposeStorageWitnessReference = ReferenceAnchors & + ( + | { readonly version: 1 } + | { + readonly version: 3; + readonly kind: "directory-xattr"; + readonly artifact: NativeComposeStorageXattrArtifact; + readonly carrierJournalToken: string; + } + ); /** Issued privately by the witness owner. An object copy cannot authorize receipt publication. */ export type NativeComposeStorageWitnessCompletionProof = Readonly< Record >; -export type NativeComposeStorageWitnessIntent = { +type IntentBinding = { readonly name: string; readonly storage: string; readonly engineId: string; @@ -32,6 +46,15 @@ export type NativeComposeStorageWitnessIntent = { readonly admission: "initial-create" | "explicit-adoption"; readonly originalVolume: NativeComposeRetainedVolume | null; }; +export type NativeComposeStorageWitnessIntent = IntentBinding & + ( + | { readonly carrier?: never; readonly artifact?: never } + | { + readonly carrier: "directory-xattr"; + readonly artifact: NativeComposeStorageXattrArtifact; + readonly carrierJournalToken: string; + } + ); /** Required v3 state: Expected survives every interruption; only this invocation can enroll. */ export type NativeComposeStorageWitnessState = NativeComposeStorageWitnessIntent & @@ -70,8 +93,17 @@ export function nativeComposeStorageWitnessReferenceValid( ): value is NativeComposeStorageWitnessReference { return ( isRecord(value) && - keys(value, "completion,directory,expectation,root,version,volume") && - value.version === 1 && + ((value.version === 1 && + keys(value, "completion,directory,expectation,root,version,volume")) || + (value.version === 3 && + keys( + value, + "artifact,carrierJournalToken,completion,directory,expectation,kind,root,version,volume" + ) && + value.kind === "directory-xattr" && + typeof value.carrierJournalToken === "string" && + TOKEN.test(value.carrierJournalToken) && + nativeComposeStorageXattrArtifactValid(value.artifact))) && nativeComposeRetainedVolumesValid([value.volume]) && directoryValid(value.root) && directoryValid(value.directory) && @@ -85,10 +117,19 @@ export function nativeComposeStorageWitnessIntentValid( if ( !( isRecord(value) && - keys( - value, - "admission,engineId,generationId,name,originalVolume,pendingToken,storage" - ) && + (Object.hasOwn(value, "carrier") + ? keys( + value, + "admission,artifact,carrier,carrierJournalToken,engineId,generationId,name,originalVolume,pendingToken,storage" + ) && + value.carrier === "directory-xattr" && + typeof value.carrierJournalToken === "string" && + TOKEN.test(value.carrierJournalToken) && + nativeComposeStorageXattrArtifactValid(value.artifact) + : keys( + value, + "admission,engineId,generationId,name,originalVolume,pendingToken,storage" + )) && nativeComposeRetainedVolumesValid([ { name: value.name, @@ -117,6 +158,32 @@ export function nativeComposeStorageWitnessIntentValid( value.originalVolume.storage === value.storage ); } +function enrolledReferenceMatches( + intent: NativeComposeStorageWitnessIntent, + reference: unknown +): boolean { + if (!nativeComposeStorageWitnessReferenceValid(reference)) { + return false; + } + const carrierMatches = + intent.carrier === "directory-xattr" + ? reference.version === 3 && + reference.carrierJournalToken === intent.carrierJournalToken && + JSON.stringify( + captureNativeComposeStorageXattrArtifact(reference.artifact) + ) === + JSON.stringify( + captureNativeComposeStorageXattrArtifact(intent.artifact) + ) + : reference.version === 1; + return ( + carrierMatches && + reference.volume.name === intent.name && + reference.volume.storage === intent.storage && + (intent.originalVolume === null || + reference.volume.createdAt === intent.originalVolume.createdAt) + ); +} export function nativeComposeStorageWitnessStatesValid( value: unknown ): value is readonly NativeComposeStorageWitnessState[] { @@ -132,9 +199,22 @@ export function nativeComposeStorageWitnessStatesValid( isRecord(entry) && keys( entry, - entry.state === "expected" - ? "admission,engineId,generationId,name,originalVolume,pendingToken,state,storage" - : "admission,engineId,generationId,name,originalVolume,pendingToken,reference,state,storage" + [ + "admission", + "engineId", + "generationId", + "name", + "originalVolume", + "pendingToken", + "state", + "storage", + ...(entry.state === "expected" ? [] : ["reference"]), + ...(Object.hasOwn(entry, "carrier") + ? ["artifact", "carrier", "carrierJournalToken"] + : []), + ] + .sort() + .join(",") ) ) ) { @@ -154,14 +234,7 @@ export function nativeComposeStorageWitnessStatesValid( engineId = intent.engineId; if ( state !== "expected" && - !( - state === "enrolled" && - nativeComposeStorageWitnessReferenceValid(reference) && - reference.volume.name === intent.name && - reference.volume.storage === intent.storage && - (intent.originalVolume === null || - reference.volume.createdAt === intent.originalVolume.createdAt) - ) + !(state === "enrolled" && enrolledReferenceMatches(intent, reference)) ) { return false; } diff --git a/src/lib/native-compose-storage-witness-xattr-carrier.ts b/src/lib/native-compose-storage-witness-xattr-carrier.ts new file mode 100644 index 000000000..a43efba03 --- /dev/null +++ b/src/lib/native-compose-storage-witness-xattr-carrier.ts @@ -0,0 +1,392 @@ +import { createHash } from "node:crypto"; +import { isRecord } from "./guards.ts"; +import type { NativeComposeMaterialBinding } from "./native-compose-generation.ts"; +import { keys } from "./native-compose-private-state.ts"; +import { + type NativeComposeRetainedVolume, + nativeComposeRetainedVolumesValid, +} from "./native-compose-retained-storage.ts"; +import { + type NativeComposeStorageXattrRequest, + type NativeComposeStorageXattrResponse, + parseNativeComposeStorageXattrResponse, + refuseNativeComposeStorageXattr as refuse, +} from "./native-compose-storage-witness-xattr-codec.ts"; + +const HASH = /^[a-f0-9]{64}$/; +const TOKEN = /^[a-f0-9]{32}$/; +const IMAGE = /^sha256:[a-f0-9]{64}$/; +const ENGINE = /^[a-zA-Z0-9][a-zA-Z0-9:-]{0,127}$/; + +/** Required private pins. A well-shaped pin is not an artifact/runtime qualification. */ +export type NativeComposeStorageXattrArtifact = { + readonly version: 1; + readonly imageId: string; + readonly platform: "linux/arm64" | "linux/amd64"; + readonly bunVersion: "1.4.2"; + readonly bunHash: string; + readonly libcHash: string; + readonly helperHash: string; + readonly kernelAbi: 1; +}; +export function nativeComposeStorageXattrArtifactValid( + value: unknown +): value is NativeComposeStorageXattrArtifact { + return ( + isRecord(value) && + keys( + value, + "bunHash,bunVersion,helperHash,imageId,kernelAbi,libcHash,platform,version" + ) && + value.version === 1 && + value.kernelAbi === 1 && + value.bunVersion === "1.4.2" && + (value.platform === "linux/arm64" || value.platform === "linux/amd64") && + typeof value.imageId === "string" && + IMAGE.test(value.imageId) && + typeof value.bunHash === "string" && + HASH.test(value.bunHash) && + typeof value.libcHash === "string" && + HASH.test(value.libcHash) && + typeof value.helperHash === "string" && + HASH.test(value.helperHash) + ); +} +export function captureNativeComposeStorageXattrArtifact( + value: unknown +): NativeComposeStorageXattrArtifact { + if (!nativeComposeStorageXattrArtifactValid(value)) { + return refuse(); + } + return Object.freeze({ + version: 1, + imageId: value.imageId, + platform: value.platform, + bunVersion: "1.4.2", + bunHash: value.bunHash, + libcHash: value.libcHash, + helperHash: value.helperHash, + kernelAbi: 1, + }); +} +type Holder = { + readonly id: string; + readonly runtimeIdentity: string; + readonly ownerToken: string; + readonly generationId: string; + readonly running: boolean; +}; +/** A complete selected-volume holder inventory, not merely project-label filtered containers. */ +export type NativeComposeStorageXattrTarget = { + readonly engineId: string; + readonly runtimeIdentity: string; + readonly ownerToken: string; + readonly name: string; + readonly storage: string; + readonly volume: NativeComposeRetainedVolume | null; + readonly mountpoint: string | null; + readonly driver: "local" | null; + readonly options: Readonly> | null; + readonly holders: readonly Holder[]; +}; +export type NativeComposeStorageXattrInvocation = { + /** Durably pin the exact created carrier before starting it; invocation intent already exists. */ + readonly recordCreated: (value: { + readonly id: string; + readonly createdAt: string; + }) => Promise; + readonly invocationId: string; + readonly artifact: NativeComposeStorageXattrArtifact; + readonly target: NativeComposeStorageXattrTarget; + readonly readonly: boolean; + readonly uid: number; + readonly gid: number; + readonly request: NativeComposeStorageXattrRequest; + readonly scope: { + readonly generationId: string; + readonly currentGenerationId: string | null; + readonly pendingGenerationId: string | null; + readonly pendingToken: string | null; + }; +}; +/** Only a future qualified engine owner may implement these ports; the CLI supplies none. */ +export type NativeComposeStorageXattrPorts = { + readonly inspect: (selection: { + readonly name: string; + readonly storage: string; + }) => Promise; + /** Exactly one original cold attempt. Idempotent create alone cannot establish admission. */ + readonly provision?: (selection: { + readonly name: string; + readonly storage: string; + readonly engineId: string; + readonly runtimeIdentity: string; + readonly ownerToken: string; + }) => Promise; + /** Fresh selected-root identity fence; durable intent and exact cleanup. + * An implicit empty footprint cannot grant helper start, seed or enrollment. */ + readonly invoke: ( + input: NativeComposeStorageXattrInvocation + ) => Promise; +}; +export type NativeComposeStorageXattrCarrier = Readonly>; +type Captured = { + readonly artifact: NativeComposeStorageXattrArtifact; + readonly ports: NativeComposeStorageXattrPorts; + readonly invocationIds: Set; + readonly carrierIds: Set; + readonly signal: AbortSignal; + readonly deadline: number; +}; +const carriers = new WeakMap(); +/** Synchronous capture prevents later callback, scalar or artifact retargeting. It grants no effect authority. */ +export function captureNativeComposeStorageXattrCarrier(opts: { + readonly artifact: NativeComposeStorageXattrArtifact; + readonly ports: NativeComposeStorageXattrPorts; + readonly signal: AbortSignal; + readonly deadline: number; +}): NativeComposeStorageXattrCarrier { + const artifact = captureNativeComposeStorageXattrArtifact(opts.artifact); + const { signal, deadline } = opts; + if ( + !(signal instanceof AbortSignal && Number.isSafeInteger(deadline)) || + deadline <= Date.now() + ) { + return refuse(); + } + const { inspect, provision, invoke } = opts.ports; + if ( + typeof inspect !== "function" || + typeof invoke !== "function" || + (provision !== undefined && typeof provision !== "function") + ) { + return refuse(); + } + const handle = Object.freeze({}); + carriers.set(handle, { + artifact, + ports: Object.freeze({ inspect, provision, invoke }), + invocationIds: new Set(), + carrierIds: new Set(), + signal, + deadline, + }); + return handle; +} +function parts(handle: NativeComposeStorageXattrCarrier): Captured { + return carriers.get(handle) ?? refuse(); +} +export function nativeComposeStorageXattrCarrierArtifact( + handle: NativeComposeStorageXattrCarrier +): NativeComposeStorageXattrArtifact { + return parts(handle).artifact; +} +export function nativeComposeStorageXattrCarrierPorts( + handle: NativeComposeStorageXattrCarrier +): NativeComposeStorageXattrPorts { + return parts(handle).ports; +} +export function checkNativeComposeStorageXattrCarrierLifetime( + handle: NativeComposeStorageXattrCarrier +): void { + const { signal, deadline } = parts(handle); + if (signal.aborted || deadline <= Date.now()) { + refuse(); + } +} +export function checkNativeComposeStorageXattrTarget(opts: { + readonly value: unknown; + readonly current: NativeComposeMaterialBinding; + readonly engineId: string; + readonly selection: { readonly name: string; readonly storage: string }; + readonly stopped: boolean; +}): NativeComposeStorageXattrTarget { + const { value, current, engineId, selection, stopped } = opts; + if ( + !( + isRecord(value) && + keys( + value, + "driver,engineId,holders,mountpoint,name,options,ownerToken,runtimeIdentity,storage,volume" + ) && + typeof value.engineId === "string" && + ENGINE.test(value.engineId) && + value.engineId === engineId && + value.runtimeIdentity === current.identity.composeProject && + value.ownerToken === current.identity.ownerToken && + value.name === selection.name && + value.storage === selection.storage && + Array.isArray(value.holders) && + value.holders.length <= 256 + ) + ) { + return refuse(); + } + let volume: NativeComposeRetainedVolume | null = null; + if (value.volume === null) { + if ( + value.mountpoint !== null || + value.driver !== null || + value.options !== null || + value.holders.length !== 0 + ) { + return refuse(); + } + } else { + if ( + !( + nativeComposeRetainedVolumesValid([value.volume]) && + isRecord(value.volume) && + value.volume.name === selection.name && + value.volume.storage === selection.storage && + value.driver === "local" && + isRecord(value.options) && + keys(value.options, "") && + value.mountpoint === `/var/lib/docker/volumes/${selection.name}/_data` + ) + ) { + return refuse(); + } + volume = Object.freeze({ + name: selection.name, + storage: selection.storage, + createdAt: String(value.volume.createdAt), + }); + } + const ids = new Set(); + const holders = value.holders + .map((holder: unknown): Holder => { + if ( + !( + isRecord(holder) && + keys(holder, "generationId,id,ownerToken,running,runtimeIdentity") && + typeof holder.id === "string" && + HASH.test(holder.id) && + !ids.has(holder.id) && + holder.runtimeIdentity === current.identity.composeProject && + holder.ownerToken === current.identity.ownerToken && + typeof holder.generationId === "string" && + TOKEN.test(holder.generationId) && + (holder.generationId === current.currentGenerationId || + holder.generationId === current.pendingGenerationId) && + typeof holder.running === "boolean" && + !(stopped && holder.running) + ) + ) { + return refuse(); + } + ids.add(holder.id); + return Object.freeze({ + id: holder.id, + runtimeIdentity: current.identity.composeProject, + ownerToken: current.identity.ownerToken, + generationId: holder.generationId, + running: holder.running, + }); + }) + .sort((a, b) => a.id.localeCompare(b.id)); + return Object.freeze({ + engineId, + runtimeIdentity: current.identity.composeProject, + ownerToken: current.identity.ownerToken, + name: selection.name, + storage: selection.storage, + volume, + mountpoint: volume + ? `/var/lib/docker/volumes/${selection.name}/_data` + : null, + driver: volume ? "local" : null, + options: volume ? Object.freeze({}) : null, + holders: Object.freeze(holders), + }); +} +/** Validate the exact captured artifact, fresh invocation/current-root target and finite cleanup observations. */ +export function checkNativeComposeStorageXattrResult(opts: { + readonly value: unknown; + readonly input: NativeComposeStorageXattrInvocation; + readonly carrier: NativeComposeStorageXattrCarrier; + readonly current: NativeComposeMaterialBinding; +}): { + readonly response: NativeComposeStorageXattrResponse; + readonly responseHash: string; + readonly created: { readonly id: string; readonly createdAt: string }; +} { + const { value, input } = opts; + const captured = parts(opts.carrier); + if ( + !( + isRecord(value) && + keys( + value, + "artifact,carrierCreatedAt,carrierId,containersAfterCleanup,engineId,exitCode,gid,invocationId,outcome,readonly,response,scope,stopped,target,uid" + ) && + value.outcome === "complete" && + (value.exitCode === 0 || value.exitCode === 1) && + value.invocationId === input.invocationId && + TOKEN.test(input.invocationId) && + !captured.invocationIds.has(input.invocationId) && + typeof value.carrierId === "string" && + HASH.test(value.carrierId) && + !captured.carrierIds.has(value.carrierId) && + value.engineId === input.target.engineId && + value.readonly === input.readonly && + value.uid === input.uid && + value.gid === input.gid && + isRecord(value.scope) && + keys( + value.scope, + "currentGenerationId,generationId,pendingGenerationId,pendingToken" + ) && + value.scope.generationId === input.scope.generationId && + value.scope.currentGenerationId === input.scope.currentGenerationId && + value.scope.pendingGenerationId === input.scope.pendingGenerationId && + value.scope.pendingToken === input.scope.pendingToken && + nativeComposeStorageXattrArtifactValid(value.artifact) && + JSON.stringify( + captureNativeComposeStorageXattrArtifact(value.artifact) + ) === JSON.stringify(captured.artifact) && + isRecord(value.stopped) && + keys(value.stopped, "exitCode,id,pid,running") && + value.stopped.id === value.carrierId && + value.stopped.running === false && + value.stopped.pid === 0 && + value.stopped.exitCode === value.exitCode && + nativeComposeRetainedVolumesValid([ + { + name: "carrier", + storage: "carrier", + createdAt: value.carrierCreatedAt, + }, + ]) && + Array.isArray(value.containersAfterCleanup) && + value.containersAfterCleanup.length === 0 && + typeof value.response === "string" + ) + ) { + return refuse(); + } + const target = checkNativeComposeStorageXattrTarget({ + value: value.target, + current: opts.current, + engineId: input.target.engineId, + selection: input.target, + stopped: !input.readonly, + }); + if (JSON.stringify(target) !== JSON.stringify(input.target)) { + return refuse(); + } + const response = parseNativeComposeStorageXattrResponse(value.response); + if ((response.outcome === "refused") !== (value.exitCode === 1)) { + return refuse(); + } + captured.invocationIds.add(input.invocationId); + captured.carrierIds.add(value.carrierId); + return Object.freeze({ + response, + created: Object.freeze({ + id: value.carrierId, + createdAt: String(value.carrierCreatedAt), + }), + responseHash: createHash("sha256").update(value.response).digest("hex"), + }); +} diff --git a/src/lib/native-compose-storage-witness-xattr-codec.ts b/src/lib/native-compose-storage-witness-xattr-codec.ts new file mode 100644 index 000000000..5443d280b --- /dev/null +++ b/src/lib/native-compose-storage-witness-xattr-codec.ts @@ -0,0 +1,267 @@ +import { randomBytes, timingSafeEqual } from "node:crypto"; +import { isRecord } from "./guards.ts"; + +/** Distinct from the regular-file/USTAR carrier; never reinterpret one as the other. */ +export const NATIVE_STORAGE_XATTR_KIND = "directory-xattr" as const; +export const NATIVE_STORAGE_XATTR_VERSION = 1 as const; +export const NATIVE_STORAGE_XATTR_BYTES = 32; +export const NATIVE_STORAGE_XATTR_INPUT_LIMIT = 4096; +export const NATIVE_STORAGE_XATTR_OUTPUT_LIMIT = 1024; +const NAME = /^user\.hack\.storage\.[a-f0-9]{64}$/; +const VALUE = /^[a-f0-9]{64}$/; +const DECIMAL = /^(0|[1-9][0-9]{0,19})$/; +const UINT64_MAX = 18_446_744_073_709_551_615n; +const UINT32_MAX = 4_294_967_295; + +export type NativeComposeStorageXattrRoot = { + readonly device: string; + readonly inode: string; + readonly uid: number; + readonly gid: number; +}; +export type NativeComposeStorageXattrMarker = { + readonly kind: typeof NATIVE_STORAGE_XATTR_KIND; + readonly version: typeof NATIVE_STORAGE_XATTR_VERSION; + readonly name: string; + readonly valueHex: string; +}; +export type NativeComposeStorageXattrRequest = + | { + readonly kind: typeof NATIVE_STORAGE_XATTR_KIND; + readonly version: typeof NATIVE_STORAGE_XATTR_VERSION; + readonly operation: "root"; + } + | (NativeComposeStorageXattrMarker & { + readonly operation: "seed" | "verify"; + readonly root: NativeComposeStorageXattrRoot; + }); +export type NativeComposeStorageXattrResponse = + | { + readonly kind: typeof NATIVE_STORAGE_XATTR_KIND; + readonly version: typeof NATIVE_STORAGE_XATTR_VERSION; + readonly outcome: "refused"; + } + | { + readonly kind: typeof NATIVE_STORAGE_XATTR_KIND; + readonly version: typeof NATIVE_STORAGE_XATTR_VERSION; + readonly outcome: "root" | "seeded"; + readonly root: NativeComposeStorageXattrRoot; + } + | { + readonly kind: typeof NATIVE_STORAGE_XATTR_KIND; + readonly version: typeof NATIVE_STORAGE_XATTR_VERSION; + readonly outcome: "verified"; + readonly root: NativeComposeStorageXattrRoot; + readonly valueHex: string; + }; + +export function refuseNativeComposeStorageXattr(): never { + throw new Error("Native storage xattr proof refused; values omitted."); +} +function keys(value: Record, expected: string): boolean { + return Object.keys(value).sort().join(",") === expected; +} +function uint32(value: unknown): value is number { + return ( + typeof value === "number" && + Number.isInteger(value) && + !Object.is(value, -0) && + value >= 0 && + value <= UINT32_MAX + ); +} +function uint64(value: unknown): value is string { + return ( + typeof value === "string" && + DECIMAL.test(value) && + BigInt(value) <= UINT64_MAX + ); +} +export function nativeComposeStorageXattrRootValid( + value: unknown +): value is NativeComposeStorageXattrRoot { + return ( + isRecord(value) && + keys(value, "device,gid,inode,uid") && + uint64(value.device) && + uint64(value.inode) && + value.inode !== "0" && + uint32(value.uid) && + uint32(value.gid) + ); +} +export function sameNativeComposeStorageXattrRoot( + left: NativeComposeStorageXattrRoot, + right: NativeComposeStorageXattrRoot +): boolean { + return ( + left.device === right.device && + left.inode === right.inode && + left.uid === right.uid && + left.gid === right.gid + ); +} +function markerValid( + value: Record +): value is Record & NativeComposeStorageXattrMarker { + return ( + value.kind === NATIVE_STORAGE_XATTR_KIND && + value.version === NATIVE_STORAGE_XATTR_VERSION && + typeof value.name === "string" && + NAME.test(value.name) && + typeof value.valueHex === "string" && + VALUE.test(value.valueHex) + ); +} +export function createNativeComposeStorageXattrMarker(): NativeComposeStorageXattrMarker { + return Object.freeze({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + name: `user.hack.storage.${randomBytes(32).toString("hex")}`, + valueHex: randomBytes(NATIVE_STORAGE_XATTR_BYTES).toString("hex"), + }); +} +export function decodeNativeComposeStorageXattrRequest( + value: unknown +): NativeComposeStorageXattrRequest { + if (!isRecord(value)) { + return refuseNativeComposeStorageXattr(); + } + if ( + keys(value, "kind,operation,version") && + value.kind === NATIVE_STORAGE_XATTR_KIND && + value.version === NATIVE_STORAGE_XATTR_VERSION && + value.operation === "root" + ) { + return Object.freeze({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + operation: "root", + }); + } + if ( + !( + keys(value, "kind,name,operation,root,valueHex,version") && + markerValid(value) && + (value.operation === "seed" || value.operation === "verify") && + nativeComposeStorageXattrRootValid(value.root) + ) + ) { + return refuseNativeComposeStorageXattr(); + } + return Object.freeze({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + operation: value.operation, + name: value.name, + valueHex: value.valueHex, + root: Object.freeze({ ...value.root }), + }); +} +export function decodeNativeComposeStorageXattrResponse( + value: unknown +): NativeComposeStorageXattrResponse { + if ( + !( + isRecord(value) && + value.kind === NATIVE_STORAGE_XATTR_KIND && + value.version === NATIVE_STORAGE_XATTR_VERSION + ) + ) { + return refuseNativeComposeStorageXattr(); + } + if (value.outcome === "refused" && keys(value, "kind,outcome,version")) { + return Object.freeze({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "refused", + }); + } + if (!nativeComposeStorageXattrRootValid(value.root)) { + return refuseNativeComposeStorageXattr(); + } + const root = Object.freeze({ ...value.root }); + if ( + (value.outcome === "root" || value.outcome === "seeded") && + keys(value, "kind,outcome,root,version") + ) { + return Object.freeze({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: value.outcome, + root, + }); + } + if ( + value.outcome === "verified" && + keys(value, "kind,outcome,root,valueHex,version") && + typeof value.valueHex === "string" && + VALUE.test(value.valueHex) + ) { + return Object.freeze({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "verified", + root, + valueHex: value.valueHex, + }); + } + return refuseNativeComposeStorageXattr(); +} +export function encodeNativeComposeStorageXattrRequest( + value: NativeComposeStorageXattrRequest +): string { + return `${JSON.stringify(decodeNativeComposeStorageXattrRequest(value))}\n`; +} +export function encodeNativeComposeStorageXattrResponse( + value: NativeComposeStorageXattrResponse +): string { + return `${JSON.stringify(decodeNativeComposeStorageXattrResponse(value))}\n`; +} +function parse( + text: string, + limit: number, + decode: (value: unknown) => T +): T { + if (Buffer.byteLength(text) > limit) { + return refuseNativeComposeStorageXattr(); + } + try { + const value = decode(JSON.parse(text)); + // One canonical member rejects duplicate keys, suffixes and alternate encodings. + if (text !== `${JSON.stringify(value)}\n`) { + return refuseNativeComposeStorageXattr(); + } + return value; + } catch { + return refuseNativeComposeStorageXattr(); + } +} +export function parseNativeComposeStorageXattrRequest( + text: string +): NativeComposeStorageXattrRequest { + return parse( + text, + NATIVE_STORAGE_XATTR_INPUT_LIMIT, + decodeNativeComposeStorageXattrRequest + ); +} +export function parseNativeComposeStorageXattrResponse( + text: string +): NativeComposeStorageXattrResponse { + return parse( + text, + NATIVE_STORAGE_XATTR_OUTPUT_LIMIT, + decodeNativeComposeStorageXattrResponse + ); +} +export function nativeComposeStorageXattrValueMatches( + actual: Uint8Array, + expectedHex: string +): boolean { + return ( + actual.byteLength === NATIVE_STORAGE_XATTR_BYTES && + VALUE.test(expectedHex) && + timingSafeEqual(actual, Buffer.from(expectedHex, "hex")) + ); +} diff --git a/src/lib/native-compose-storage-witness-xattr-helper.ts b/src/lib/native-compose-storage-witness-xattr-helper.ts new file mode 100644 index 000000000..149918c42 --- /dev/null +++ b/src/lib/native-compose-storage-witness-xattr-helper.ts @@ -0,0 +1,133 @@ +import { + decodeNativeComposeStorageXattrRequest, + NATIVE_STORAGE_XATTR_BYTES, + NATIVE_STORAGE_XATTR_KIND, + NATIVE_STORAGE_XATTR_VERSION, + type NativeComposeStorageXattrResponse, + type NativeComposeStorageXattrRoot, + nativeComposeStorageXattrRootValid, + nativeComposeStorageXattrValueMatches, + refuseNativeComposeStorageXattr as refuse, + sameNativeComposeStorageXattrRoot, +} from "./native-compose-storage-witness-xattr-codec.ts"; + +/** Synchronous fixed-root syscall boundary. Offline tests substitute ports; no engine transport lives here. */ +export type NativeComposeStorageXattrKernel = { + readonly effectiveUid: () => number; + readonly effectiveGid: () => number; + readonly probeRoot: () => NativeComposeStorageXattrRoot; + readonly openRoot: () => number; + readonly statRoot: (fd: number) => NativeComposeStorageXattrRoot; + readonly createXattr: (fd: number, name: string, value: Uint8Array) => void; + readonly syncRoot: (fd: number) => void; + readonly readXattr: (fd: number, name: string, bytes: number) => Uint8Array; + readonly closeRoot: (fd: number) => void; +}; + +function root(value: unknown): NativeComposeStorageXattrRoot { + if (!nativeComposeStorageXattrRootValid(value)) { + return refuse(); + } + return Object.freeze({ ...value }); +} +function requireRoot( + actual: NativeComposeStorageXattrRoot, + expected: NativeComposeStorageXattrRoot +): void { + if (!sameNativeComposeStorageXattrRoot(root(actual), expected)) { + refuse(); + } +} +/** Never reports enrollment or generation readiness. Only the enclosing owner can publish either. */ +export function runNativeComposeStorageXattrHelper(opts: { + readonly request: unknown; + readonly kernel: NativeComposeStorageXattrKernel; +}): NativeComposeStorageXattrResponse { + let fd: number | null = null; + let capturedClose: ((fd: number) => void) | null = null; + try { + const request = decodeNativeComposeStorageXattrRequest(opts.request); + const { + effectiveUid, + effectiveGid, + probeRoot, + openRoot, + statRoot, + createXattr, + syncRoot, + readXattr, + closeRoot, + } = opts.kernel; + capturedClose = closeRoot; + if (request.operation === "root") { + const before = root(probeRoot()); + requireRoot(probeRoot(), before); + return { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "root", + root: before, + }; + } + if ( + effectiveUid() !== request.root.uid || + effectiveGid() !== request.root.gid + ) { + return refuse(); + } + fd = openRoot(); + if (!Number.isInteger(fd) || fd < 0) { + return refuse(); + } + requireRoot(statRoot(fd), request.root); + let response: NativeComposeStorageXattrResponse; + if (request.operation === "seed") { + createXattr(fd, request.name, Buffer.from(request.valueHex, "hex")); + syncRoot(fd); + requireRoot(statRoot(fd), request.root); + response = { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "seeded", + root: request.root, + }; + } else { + const first = readXattr(fd, request.name, NATIVE_STORAGE_XATTR_BYTES); + if (!nativeComposeStorageXattrValueMatches(first, request.valueHex)) { + return refuse(); + } + requireRoot(statRoot(fd), request.root); + const latest = readXattr(fd, request.name, NATIVE_STORAGE_XATTR_BYTES); + if (!nativeComposeStorageXattrValueMatches(latest, request.valueHex)) { + return refuse(); + } + requireRoot(statRoot(fd), request.root); + response = { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "verified", + root: request.root, + valueHex: Buffer.from(latest).toString("hex"), + }; + } + // A failed close is not a completed helper proof, and must never be hidden. + const selected = fd; + fd = null; + closeRoot(selected); + return response; + } catch { + return { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "refused", + }; + } finally { + if (fd !== null && Number.isInteger(fd) && fd >= 0 && capturedClose) { + try { + capturedClose(fd); + } catch { + // The fixed refusal already represents uncertain completion; no error text escapes. + } + } + } +} diff --git a/src/lib/native-compose-storage-witness-xattr-linux.ts b/src/lib/native-compose-storage-witness-xattr-linux.ts new file mode 100644 index 000000000..c5a7ddca0 --- /dev/null +++ b/src/lib/native-compose-storage-witness-xattr-linux.ts @@ -0,0 +1,224 @@ +import { + type BigIntStats, + closeSync, + constants, + fstatSync, + fsyncSync, + lstatSync, + openSync, +} from "node:fs"; +import { + NATIVE_STORAGE_XATTR_BYTES, + type NativeComposeStorageXattrRoot, + nativeComposeStorageXattrRootValid, + refuseNativeComposeStorageXattr as refuse, +} from "./native-compose-storage-witness-xattr-codec.ts"; +import type { NativeComposeStorageXattrKernel } from "./native-compose-storage-witness-xattr-helper.ts"; + +const XATTR_NAME = /^user\.hack\.storage\.[a-f0-9]{64}$/; + +/** Source contract only. Image/Bun/libc/program hashes and engine mounts require external qualification. */ +export const NATIVE_STORAGE_XATTR_LINUX_ABI = Object.freeze({ + version: 1, + root: "/hack-storage-witness", + xattrCreate: 1, + openCloseExec: 1 << 19, + fGetFd: 1, + fdCloseExec: 1, + pointerBits: 64, + sizeTBits: 64, + signedSizeBits: 64, + arm64: Object.freeze({ + libc: "/lib/aarch64-linux-gnu/libc.so.6", + directory: 1 << 14, + noFollow: 1 << 15, + }), + x64: Object.freeze({ + libc: "/lib/x86_64-linux-gnu/libc.so.6", + directory: 1 << 16, + noFollow: 1 << 17, + }), +}); + +function directory(info: BigIntStats): NativeComposeStorageXattrRoot { + if (!info.isDirectory() || info.isSymbolicLink()) { + return refuse(); + } + const value = { + device: info.dev.toString(), + inode: info.ino.toString(), + uid: Number(info.uid), + gid: Number(info.gid), + }; + if (!nativeComposeStorageXattrRootValid(value)) { + return refuse(); + } + return Object.freeze(value); +} +function libc(): string { + let abi: { + readonly libc: string; + readonly directory: number; + readonly noFollow: number; + } | null = null; + if (process.arch === "arm64") { + abi = NATIVE_STORAGE_XATTR_LINUX_ABI.arm64; + } else if (process.arch === "x64") { + abi = NATIVE_STORAGE_XATTR_LINUX_ABI.x64; + } + if ( + process.platform !== "linux" || + !abi || + constants.O_DIRECTORY !== abi.directory || + constants.O_NOFOLLOW !== abi.noFollow || + constants.O_RDONLY !== 0 || + !process.geteuid || + !process.getegid || + !process.getuid || + !process.getgid || + process.getuid() !== process.geteuid() || + process.getgid() !== process.getegid() + ) { + return refuse(); + } + return abi.libc; +} + +/** Never called by the CLI or offline controls. A qualified Linux helper is its only intended caller. */ +export async function createNativeComposeStorageXattrLinuxKernel(): Promise<{ + readonly kernel: NativeComposeStorageXattrKernel; + readonly close: () => void; +}> { + const libraryPath = libc(); + const { dlopen, FFIType, ptr } = await import("bun:ffi"); + const library = dlopen(libraryPath, { + fsetxattr: { + args: [FFIType.i32, FFIType.ptr, FFIType.ptr, FFIType.u64, FFIType.i32], + returns: FFIType.i32, + }, + fgetxattr: { + args: [FFIType.i32, FFIType.ptr, FFIType.ptr, FFIType.u64], + returns: FFIType.i64, + }, + fcntl: { + args: [FFIType.i32, FFIType.i32, FFIType.i32], + returns: FFIType.i32, + }, + }); + const { fsetxattr, fgetxattr, fcntl } = library.symbols; + const held = new Set(); + let closed = false; + const active = () => { + if (closed) { + refuse(); + } + }; + const fdOwned = (fd: number) => { + active(); + if (!held.has(fd)) { + refuse(); + } + }; + const nameBytes = (name: string) => { + if (!XATTR_NAME.test(name)) { + return refuse(); + } + return Buffer.from(`${name}\0`, "ascii"); + }; + const kernel: NativeComposeStorageXattrKernel = Object.freeze({ + effectiveUid: () => process.geteuid?.() ?? refuse(), + effectiveGid: () => process.getegid?.() ?? refuse(), + probeRoot() { + active(); + return directory( + lstatSync(NATIVE_STORAGE_XATTR_LINUX_ABI.root, { + bigint: true, + }) + ); + }, + openRoot() { + active(); + const fd = openSync( + NATIVE_STORAGE_XATTR_LINUX_ABI.root, + constants.O_RDONLY | + constants.O_DIRECTORY | + constants.O_NOFOLLOW | + NATIVE_STORAGE_XATTR_LINUX_ABI.openCloseExec + ); + held.add(fd); + const flags = fcntl(fd, NATIVE_STORAGE_XATTR_LINUX_ABI.fGetFd, 0); + if ( + flags < 0 || + (flags & NATIVE_STORAGE_XATTR_LINUX_ABI.fdCloseExec) === 0 + ) { + held.delete(fd); + closeSync(fd); + return refuse(); + } + return fd; + }, + statRoot(fd) { + fdOwned(fd); + return directory(fstatSync(fd, { bigint: true })); + }, + createXattr(fd, name, value) { + fdOwned(fd); + if (value.byteLength !== NATIVE_STORAGE_XATTR_BYTES) { + return refuse(); + } + const selectedName = nameBytes(name); + const bytes = Buffer.from(value); + if ( + fsetxattr( + fd, + ptr(selectedName), + ptr(bytes), + BigInt(bytes.length), + NATIVE_STORAGE_XATTR_LINUX_ABI.xattrCreate + ) !== 0 + ) { + return refuse(); + } + }, + syncRoot(fd) { + fdOwned(fd); + // Metadata synchronization is mandatory; never substitute fdatasync. + fsyncSync(fd); + }, + readXattr(fd, name, bytes) { + fdOwned(fd); + if (bytes !== NATIVE_STORAGE_XATTR_BYTES) { + return refuse(); + } + const selectedName = nameBytes(name); + const value = Buffer.alloc(bytes); + if ( + fgetxattr(fd, ptr(selectedName), ptr(value), BigInt(bytes)) !== + BigInt(bytes) + ) { + return refuse(); + } + return new Uint8Array(value); + }, + closeRoot(fd) { + fdOwned(fd); + held.delete(fd); + closeSync(fd); + }, + }); + return Object.freeze({ + kernel, + close() { + active(); + closed = true; + try { + for (const fd of held) { + closeSync(fd); + } + } finally { + held.clear(); + library.close(); + } + }, + }); +} diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index a3b5fa3ef..a89149a85 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -26,6 +26,10 @@ import { type NativeComposeRetainedVolume, nativeComposeRetainedVolumesValid, } from "./native-compose-retained-storage.ts"; +import { + beginNativeComposeStorageCarrierIntent, + initializeNativeComposeStorageCarrierJournal, +} from "./native-compose-storage-carrier-journal.ts"; import { encodeNativeComposeStorageWitnessArchive, type NativeComposeStorageWitnessMarker, @@ -39,6 +43,27 @@ import { type NativeComposeStorageWitnessReference, nativeComposeStorageWitnessReferenceValid, } from "./native-compose-storage-witness-state.ts"; +import { + captureNativeComposeStorageXattrArtifact, + checkNativeComposeStorageXattrCarrierLifetime, + checkNativeComposeStorageXattrResult, + checkNativeComposeStorageXattrTarget, + type NativeComposeStorageXattrArtifact, + type NativeComposeStorageXattrCarrier, + type NativeComposeStorageXattrInvocation, + type NativeComposeStorageXattrTarget, + nativeComposeStorageXattrCarrierArtifact, + nativeComposeStorageXattrCarrierPorts, +} from "./native-compose-storage-witness-xattr-carrier.ts"; +import { + createNativeComposeStorageXattrMarker, + decodeNativeComposeStorageXattrRequest, + type NativeComposeStorageXattrMarker, + type NativeComposeStorageXattrRoot, + nativeComposeStorageXattrRootValid, + nativeComposeStorageXattrValueMatches, + sameNativeComposeStorageXattrRoot, +} from "./native-compose-storage-witness-xattr-codec.ts"; export type { NativeComposeStorageWitnessReference } from "./native-compose-storage-witness-state.ts"; @@ -61,18 +86,43 @@ type Binding = Selection & { readonly generationId: string; readonly pendingToken: string; }; -type Expectation = { - readonly version: 1; +type ExpectationBinding = { readonly binding: Binding; readonly admission: "initial-create" | "explicit-adoption"; readonly originalVolume: NativeComposeRetainedVolume | null; - readonly marker: NativeComposeStorageWitnessMarker; }; -type Completion = { - readonly version: 1; +type XattrExpectation = ExpectationBinding & { + readonly version: 3; + readonly kind: "directory-xattr"; + readonly artifact: NativeComposeStorageXattrArtifact; + readonly carrierJournalToken: string; + readonly marker: NativeComposeStorageXattrMarker; +}; +type Expectation = ExpectationBinding & + ( + | { + readonly version: 1; + readonly marker: NativeComposeStorageWitnessMarker; + } + | XattrExpectation + ); +type CompletionBinding = { readonly expectationHash: string; readonly volume: NativeComposeRetainedVolume; }; +type KernelProof = { + readonly root: NativeComposeStorageXattrRoot; + readonly responseHash: string; +}; +type Completion = CompletionBinding & + ( + | { readonly version: 1 } + | { + readonly version: 3; + readonly kind: "directory-xattr"; + readonly kernelProof: KernelProof; + } + ); export type NativeComposeStorageWitnessEnrollment = Readonly< Record >; @@ -84,6 +134,7 @@ type Enrollment = { readonly root: DirectoryAnchor; readonly directory: DirectoryAnchor; readonly assertAdmission: () => Promise; + readonly carrier?: NativeComposeStorageXattrCarrier; consumed: boolean; }; const enrollments = new WeakMap< @@ -163,28 +214,43 @@ function bindingValid(value: unknown): value is Binding { TOKEN.test(value.pendingToken) ); } +function expectationRecordValid(value: unknown): value is Record< + string, + unknown +> & { + readonly binding: Binding; + readonly marker: Record; +} { + return ( + isRecord(value) && + ((value.version === 1 && + keys(value, "admission,binding,marker,originalVolume,version")) || + (value.version === 3 && + value.kind === "directory-xattr" && + keys( + value, + "admission,artifact,binding,carrierJournalToken,kind,marker,originalVolume,version" + ))) && + bindingValid(value.binding) && + isRecord(value.marker) && + (value.version === 1 + ? keys(value.marker, "name,token") && + typeof value.marker.name === "string" && + typeof value.marker.token === "string" && + nativeComposeStorageWitnessMarkerValid({ + name: value.marker.name, + token: value.marker.token, + }) + : keys(value.marker, "kind,name,valueHex,version")) && + (value.admission === "initial-create" + ? value.originalVolume === null + : value.admission === "explicit-adoption" && + nativeComposeRetainedVolumesValid([value.originalVolume])) + ); +} function expectation(text: string): Expectation { const value = parsePrivateJson(text); - if ( - !( - isRecord(value) && - keys(value, "admission,binding,marker,originalVolume,version") && - value.version === 1 && - bindingValid(value.binding) && - isRecord(value.marker) && - keys(value.marker, "name,token") && - typeof value.marker.name === "string" && - typeof value.marker.token === "string" && - nativeComposeStorageWitnessMarkerValid({ - name: value.marker.name, - token: value.marker.token, - }) && - (value.admission === "initial-create" - ? value.originalVolume === null - : value.admission === "explicit-adoption" && - nativeComposeRetainedVolumesValid([value.originalVolume])) - ) - ) { + if (!expectationRecordValid(value)) { return refuse(); } const originalVolume = @@ -196,15 +262,51 @@ function expectation(text: string): Expectation { ) { return refuse(); } - return { - version: 1, + const common: ExpectationBinding = { binding: value.binding, admission: value.admission === "initial-create" ? "initial-create" : "explicit-adoption", originalVolume, - marker: { name: value.marker.name, token: value.marker.token }, + }; + if (value.version === 1) { + if ( + typeof value.marker.name !== "string" || + typeof value.marker.token !== "string" + ) { + return refuse(); + } + return { + ...common, + version: 1, + marker: { name: value.marker.name, token: value.marker.token }, + }; + } + const marker = decodeNativeComposeStorageXattrRequest({ + ...value.marker, + operation: "verify", + root: { device: "1", inode: "1", uid: 0, gid: 0 }, + }); + if (marker.operation === "root") { + return refuse(); + } + return { + ...common, + version: 3, + kind: "directory-xattr", + artifact: captureNativeComposeStorageXattrArtifact(value.artifact), + carrierJournalToken: + typeof value.carrierJournalToken === "string" && + TOKEN.test(value.carrierJournalToken) + ? value.carrierJournalToken + : refuse(), + marker: { + kind: marker.kind, + version: marker.version, + name: marker.name, + valueHex: marker.valueHex, + }, }; } function parseVolume(value: unknown): NativeComposeRetainedVolume { @@ -212,26 +314,53 @@ function parseVolume(value: unknown): NativeComposeRetainedVolume { if (!nativeComposeRetainedVolumesValid(entries)) { return refuse(); } - return entries[0] ?? refuse(); + return Object.freeze({ ...(entries[0] ?? refuse()) }); } function completion(text: string): Completion { const value = parsePrivateJson(text); if ( !( isRecord(value) && - keys(value, "expectationHash,version,volume") && - value.version === 1 && + ((value.version === 1 && keys(value, "expectationHash,version,volume")) || + (value.version === 3 && + value.kind === "directory-xattr" && + keys(value, "expectationHash,kernelProof,kind,version,volume") && + isRecord(value.kernelProof) && + keys(value.kernelProof, "responseHash,root") && + nativeComposeStorageXattrRootValid(value.kernelProof.root) && + typeof value.kernelProof.responseHash === "string" && + HASH.test(value.kernelProof.responseHash))) && typeof value.expectationHash === "string" && HASH.test(value.expectationHash) ) ) { return refuse(); } - return { - version: 1, + const common: CompletionBinding = { expectationHash: value.expectationHash, volume: parseVolume(value.volume), }; + if (value.version === 1) { + return { ...common, version: 1 }; + } + if ( + !( + isRecord(value.kernelProof) && + nativeComposeStorageXattrRootValid(value.kernelProof.root) + ) || + typeof value.kernelProof.responseHash !== "string" + ) { + return refuse(); + } + return { + ...common, + version: 3, + kind: "directory-xattr", + kernelProof: { + root: { ...value.kernelProof.root }, + responseHash: value.kernelProof.responseHash, + }, + }; } function sameVolume( left: NativeComposeRetainedVolume, @@ -265,6 +394,260 @@ function matchesAdmission( current.pendingGenerationId === saved.generationId ); } +type XattrCheck = () => Promise; +function checkXattrArtifact( + saved: XattrExpectation, + carrier: NativeComposeStorageXattrCarrier +): void { + checkNativeComposeStorageXattrCarrierLifetime(carrier); + if ( + JSON.stringify(saved.artifact) !== + JSON.stringify(nativeComposeStorageXattrCarrierArtifact(carrier)) + ) { + refuse(); + } +} +async function inspectXattr(opts: { + readonly carrier: NativeComposeStorageXattrCarrier; + readonly saved: XattrExpectation; + readonly stopped: boolean; + readonly check: XattrCheck; +}): Promise { + checkXattrArtifact(opts.saved, opts.carrier); + const before = await opts.check(); + checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); + const value = await nativeComposeStorageXattrCarrierPorts( + opts.carrier + ).inspect( + Object.freeze({ + name: opts.saved.binding.name, + storage: opts.saved.binding.storage, + }) + ); + const current = await opts.check(); + checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); + if (JSON.stringify(before) !== JSON.stringify(current)) { + return refuse(); + } + return checkNativeComposeStorageXattrTarget({ + value, + current, + engineId: opts.saved.binding.engineId, + selection: opts.saved.binding, + stopped: opts.stopped, + }); +} +function requireXattrAdmission( + saved: XattrExpectation, + target: NativeComposeStorageXattrTarget +): void { + if (saved.admission === "initial-create") { + if (target.volume !== null) { + refuse(); + } + } else if ( + !( + target.volume && + saved.originalVolume && + sameVolume(target.volume, saved.originalVolume) + ) + ) { + refuse(); + } +} +async function invokeXattr(opts: { + readonly carrier: NativeComposeStorageXattrCarrier; + readonly saved: XattrExpectation; + readonly volume: NativeComposeRetainedVolume; + readonly stopped: boolean; + readonly check: XattrCheck; + readonly journal: HeldDirectory; + readonly request: Parameters< + typeof decodeNativeComposeStorageXattrRequest + >[0]; +}) { + const request = decodeNativeComposeStorageXattrRequest(opts.request); + const target = await inspectXattr(opts); + if (!(target.volume && sameVolume(target.volume, opts.volume))) { + return refuse(); + } + const current = await opts.check(); + checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); + const captured = Object.freeze({ + invocationId: randomBytes(16).toString("hex"), + artifact: opts.saved.artifact, + target, + readonly: request.operation !== "seed", + uid: request.operation === "root" ? 0 : request.root.uid, + gid: request.operation === "root" ? 0 : request.root.gid, + request, + scope: Object.freeze({ + generationId: current.generationId, + currentGenerationId: current.currentGenerationId, + pendingGenerationId: current.pendingGenerationId, + pendingToken: current.pendingToken, + }), + }); + const journal = await beginNativeComposeStorageCarrierIntent({ + directory: opts.journal, + token: opts.saved.carrierJournalToken, + check: opts.check, + input: captured, + }); + const beforeInvoke = await opts.check(); + checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); + if (JSON.stringify(current) !== JSON.stringify(beforeInvoke)) { + return refuse(); + } + const input: NativeComposeStorageXattrInvocation = Object.freeze({ + ...captured, + recordCreated: journal.recordCreated, + }); + const value = await nativeComposeStorageXattrCarrierPorts( + opts.carrier + ).invoke(input); + const latest = await opts.check(); + checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); + if (JSON.stringify(current) !== JSON.stringify(latest)) { + return refuse(); + } + const result = checkNativeComposeStorageXattrResult({ + value, + input, + carrier: opts.carrier, + current: latest, + }); + const after = await inspectXattr(opts); + if (JSON.stringify(after) !== JSON.stringify(target)) { + return refuse(); + } + await journal.complete(result.created); + await opts.check(); + checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); + const expectedOutcome = ( + { root: "root", seed: "seeded", verify: "verified" } as const + )[request.operation]; + if (result.response.outcome !== expectedOutcome) { + return refuse(); + } + if ( + request.operation !== "root" && + !sameNativeComposeStorageXattrRoot(result.response.root, request.root) + ) { + return refuse(); + } + return result; +} +async function observeXattr(opts: { + readonly carrier: NativeComposeStorageXattrCarrier; + readonly saved: XattrExpectation; + readonly volume: NativeComposeRetainedVolume; + readonly stopped: boolean; + readonly check: XattrCheck; + readonly journal: HeldDirectory; +}): Promise<{ + readonly volume: NativeComposeRetainedVolume; + readonly kernelProof: KernelProof; +}> { + const discovered = await invokeXattr({ + ...opts, + request: { kind: "directory-xattr", version: 1, operation: "root" }, + }); + if (discovered.response.outcome !== "root") { + return refuse(); + } + const proof = await invokeXattr({ + ...opts, + request: { + ...opts.saved.marker, + operation: "verify", + root: discovered.response.root, + }, + }); + if ( + proof.response.outcome !== "verified" || + !nativeComposeStorageXattrValueMatches( + Buffer.from(proof.response.valueHex, "hex"), + opts.saved.marker.valueHex + ) + ) { + return refuse(); + } + // A bind reader may retain an unlinked root across its await. Observe the current + // namespace with another fresh non-creating carrier; Docker metadata alone cannot + // detect a replacement which repeats both name and birth. This remains a finite fence. + const after = await invokeXattr({ + ...opts, + request: { kind: "directory-xattr", version: 1, operation: "root" }, + }); + if ( + after.response.outcome !== "root" || + !sameNativeComposeStorageXattrRoot(after.response.root, proof.response.root) + ) { + return refuse(); + } + return { + volume: opts.volume, + kernelProof: { + root: proof.response.root, + responseHash: proof.responseHash, + }, + }; +} +async function seedXattr(opts: { + readonly carrier: NativeComposeStorageXattrCarrier; + readonly saved: XattrExpectation; + readonly check: XattrCheck; + readonly journal: HeldDirectory; +}): Promise<{ + readonly volume: NativeComposeRetainedVolume; + readonly kernelProof: KernelProof; +}> { + const initial = await inspectXattr({ ...opts, stopped: true }); + requireXattrAdmission(opts.saved, initial); + if (opts.saved.admission === "initial-create") { + const provision = + nativeComposeStorageXattrCarrierPorts(opts.carrier).provision ?? refuse(); + const current = await opts.check(); + checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); + await provision( + Object.freeze({ + name: opts.saved.binding.name, + storage: opts.saved.binding.storage, + engineId: opts.saved.binding.engineId, + runtimeIdentity: current.identity.composeProject, + ownerToken: opts.saved.binding.ownerToken, + }) + ); + await opts.check(); + checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); + } + const created = await inspectXattr({ ...opts, stopped: true }); + const volume = created.volume ?? refuse(); + if ( + opts.saved.originalVolume && + !sameVolume(volume, opts.saved.originalVolume) + ) { + return refuse(); + } + const stopped = { ...opts, stopped: true, volume }; + const discovered = await invokeXattr({ + ...stopped, + request: { kind: "directory-xattr", version: 1, operation: "root" }, + }); + if (discovered.response.outcome !== "root") { + return refuse(); + } + await invokeXattr({ + ...stopped, + request: { + ...opts.saved.marker, + operation: "seed", + root: discovered.response.root, + }, + }); + return await observeXattr(stopped); +} async function directories(opts: { readonly binding: NativeComposeMaterialBinding; readonly name: string; @@ -364,6 +747,77 @@ async function checkedExpectation( return expectation(read.text); } +type EnrollmentObservation = { + readonly volume: NativeComposeRetainedVolume; + readonly archive?: Uint8Array; + readonly kernelProof?: KernelProof; +}; +async function observeEnrollment(opts: { + readonly saved: Expectation; + readonly carrier?: NativeComposeStorageXattrCarrier; + readonly seed?: (archive: Uint8Array) => Promise; + readonly observe?: () => Promise<{ + readonly volume: NativeComposeRetainedVolume; + readonly archive: Uint8Array; + }>; + readonly check: XattrCheck; + readonly journal: HeldDirectory; +}): Promise { + const { saved, carrier, seed, observe, check } = opts; + if (saved.version === 3) { + if (seed || observe || !carrier) { + return refuse(); + } + return await seedXattr({ carrier, saved, check, journal: opts.journal }); + } + if (!(seed && observe) || carrier) { + return refuse(); + } + await seed(encodeNativeComposeStorageWitnessArchive(saved.marker)); + const observed = await observe(); + verifyNativeComposeStorageWitnessArchive({ + marker: saved.marker, + archive: observed.archive, + }); + return observed; +} +function completionMatches(opts: { + readonly record: Completion; + readonly reference: NativeComposeStorageWitnessReference; + readonly saved: Expectation; +}): boolean { + const { record, reference, saved } = opts; + return ( + record.version === reference.version && + record.expectationHash === reference.expectation.hash && + sameVolume(record.volume, reference.volume) && + (saved.originalVolume === null || + sameVolume(saved.originalVolume, record.volume)) && + record.volume.name === saved.binding.name && + record.volume.storage === saved.binding.storage + ); +} +function expectationMatchesReference(opts: { + readonly saved: Expectation; + readonly reference: NativeComposeStorageWitnessReference; +}): boolean { + const { saved, reference } = opts; + if (saved.version !== reference.version) { + return false; + } + if (saved.version === 1) { + return true; + } + return ( + reference.version === 3 && + saved.carrierJournalToken === reference.carrierJournalToken && + JSON.stringify(saved.artifact) === + JSON.stringify( + captureNativeComposeStorageXattrArtifact(reference.artifact) + ) + ); +} + /** * Internal foundation: the CLI refuses witness-bearing workload admission until its carrier is qualified. * The owning generation mutation must prove a genuinely absent new volume or @@ -379,6 +833,8 @@ export async function prepareNativeComposeStorageWitness(opts: { readonly admission: "initial-create" | "explicit-adoption"; readonly originalVolume?: NativeComposeRetainedVolume; readonly assertAdmission: () => Promise; + /** Explicit internal xattr path; no CLI caller supplies this unactivated carrier. */ + readonly xattrCarrier?: NativeComposeStorageXattrCarrier; }): Promise { const input = Object.freeze({ ...opts, @@ -387,6 +843,12 @@ export async function prepareNativeComposeStorageWitness(opts: { ? Object.freeze({ ...opts.originalVolume }) : undefined, }); + const artifact = input.xattrCarrier + ? nativeComposeStorageXattrCarrierArtifact(input.xattrCarrier) + : null; + if (input.xattrCarrier) { + checkNativeComposeStorageXattrCarrierLifetime(input.xattrCarrier); + } return await runNativeComposeMaterialAction({ authority: input.authority, run: async () => { @@ -403,34 +865,74 @@ export async function prepareNativeComposeStorageWitness(opts: { generationId: current.generationId, pendingToken: current.pendingToken ?? refuse(), }; - const record: Expectation = { - version: 1, + const common: ExpectationBinding = { binding, admission: input.admission, originalVolume: input.originalVolume ?? null, - marker: { - name: `.hack-storage-${randomBytes(32).toString("hex")}.witness`, - token: randomBytes(32).toString("hex"), - }, }; + const record: Expectation = artifact + ? { + ...common, + version: 3, + kind: "directory-xattr", + artifact, + carrierJournalToken: randomBytes(16).toString("hex"), + marker: createNativeComposeStorageXattrMarker(), + } + : { + ...common, + version: 1, + marker: { + name: `.hack-storage-${randomBytes(32).toString("hex")}.witness`, + token: randomBytes(32).toString("hex"), + }, + }; const text = JSON.stringify(record); expectation(text); if (!matchesAdmission(binding, current)) { return refuse(); } await input.assertAdmission(); + if (input.xattrCarrier && record.version === 3) { + const target = await inspectXattr({ + carrier: input.xattrCarrier, + saved: record, + stopped: true, + check: async () => { + const latest = await assertNativeComposeMaterialAuthority({ + authority: input.authority, + generation: input.generation, + phase: "effect", + }); + if (!matchesAdmission(binding, latest)) { + return refuse(); + } + return latest; + }, + }); + requireXattrAdmission(record, target); + } + const intent = { + name: binding.name, + storage: binding.storage, + engineId: binding.engineId, + generationId: binding.generationId, + pendingToken: binding.pendingToken, + admission: record.admission, + originalVolume: record.originalVolume, + }; await armNativeComposeStorageWitnessIntent({ authority: input.authority, generation: input.generation, - intent: { - name: binding.name, - storage: binding.storage, - engineId: binding.engineId, - generationId: binding.generationId, - pendingToken: binding.pendingToken, - admission: record.admission, - originalVolume: record.originalVolume, - }, + intent: + record.version === 3 + ? { + ...intent, + carrier: record.kind, + artifact: record.artifact, + carrierJournalToken: record.carrierJournalToken, + } + : intent, }); const held = await directories({ binding: current, @@ -452,6 +954,31 @@ export async function prepareNativeComposeStorageWitness(opts: { if (!matchesAdmission(binding, latest)) { return refuse(); } + if (record.version === 3) { + await initializeNativeComposeStorageCarrierJournal({ + directory: last(held), + token: record.carrierJournalToken, + check: async () => { + await checkedExpectation(held, anchor({ info, text })); + const live = await assertNativeComposeMaterialAuthority({ + authority: input.authority, + generation: input.generation, + phase: "effect", + }); + if (!matchesAdmission(binding, live)) { + return refuse(); + } + if (input.xattrCarrier) { + checkNativeComposeStorageXattrCarrierLifetime( + input.xattrCarrier + ); + } + }, + }); + } + if (input.xattrCarrier) { + checkNativeComposeStorageXattrCarrierLifetime(input.xattrCarrier); + } const capability = Object.freeze({}); enrollments.set(capability, { authority: input.authority, @@ -461,6 +988,7 @@ export async function prepareNativeComposeStorageWitness(opts: { root: directoryAnchor(held[1] ?? refuse()), directory: directoryAnchor(last(held)), assertAdmission: input.assertAdmission, + carrier: input.xattrCarrier, consumed: false, }); return capability; @@ -479,8 +1007,8 @@ export async function prepareNativeComposeStorageWitness(opts: { */ export async function enrollNativeComposeStorageWitness(opts: { readonly enrollment: NativeComposeStorageWitnessEnrollment; - readonly seed: (archive: Uint8Array) => Promise; - readonly observe: () => Promise<{ + readonly seed?: (archive: Uint8Array) => Promise; + readonly observe?: () => Promise<{ readonly volume: NativeComposeRetainedVolume; readonly archive: Uint8Array; }>; @@ -503,6 +1031,9 @@ export async function enrollNativeComposeStorageWitness(opts: { if (!matchesAdmission(selected.expectation.binding, binding)) { return refuse(); } + if (selected.carrier) { + checkNativeComposeStorageXattrCarrierLifetime(selected.carrier); + } return binding; }; const held = await directories({ @@ -516,8 +1047,18 @@ export async function enrollNativeComposeStorageWitness(opts: { await selected.assertAdmission(); await checkedExpectation(held, selected.anchor); await check(); - await seed(encodeNativeComposeStorageWitnessArchive(saved.marker)); - const observed = await observe(); + const live = async () => { + await checkedExpectation(held, selected.anchor); + return await check(); + }; + const observed = await observeEnrollment({ + saved, + carrier: selected.carrier, + seed, + observe, + check: live, + journal: last(held), + }); const volume = parseVolume(observed.volume); if ( volume.name !== saved.binding.name || @@ -526,17 +1067,19 @@ export async function enrollNativeComposeStorageWitness(opts: { ) { return refuse(); } - verifyNativeComposeStorageWitnessArchive({ - marker: saved.marker, - archive: observed.archive, - }); await checkedExpectation(held, selected.anchor); await check(); - const text = JSON.stringify({ - version: 1, - expectationHash: selected.anchor.hash, - volume, - } satisfies Completion); + const completedRecord: Completion = + saved.version === 1 + ? { version: 1, expectationHash: selected.anchor.hash, volume } + : { + version: 3, + kind: "directory-xattr", + expectationHash: selected.anchor.hash, + volume, + kernelProof: observed.kernelProof ?? refuse(), + }; + const text = JSON.stringify(completedRecord); const info = await writeExclusive( join(last(held).path, "enrolled.json"), text @@ -554,14 +1097,24 @@ export async function enrollNativeComposeStorageWitness(opts: { } await recheckDirectories(held); await check(); - const reference = Object.freeze({ - version: 1 as const, + const anchors = { volume: Object.freeze({ ...volume }), root: selected.root, directory: selected.directory, expectation: selected.anchor, completion: anchor({ info, text }), - }); + }; + const reference: NativeComposeStorageWitnessReference = Object.freeze( + saved.version === 1 + ? { ...anchors, version: 1 as const } + : { + ...anchors, + version: 3 as const, + kind: "directory-xattr" as const, + artifact: saved.artifact, + carrierJournalToken: saved.carrierJournalToken, + } + ); const proof = Object.freeze({}); completionProofs.set(proof, { authority: selected.authority, @@ -574,7 +1127,9 @@ export async function enrollNativeComposeStorageWitness(opts: { generation: selected.generation, engineId: saved.binding.engineId, reference, - observe: async () => await observe(), + ...(saved.version === 1 && observe + ? { observe: async () => await observe() } + : { xattrCarrier: selected.carrier }), }); }, }); @@ -597,16 +1152,36 @@ export async function verifyNativeComposeStorageWitness(opts: { readonly generation: NativeComposeGeneration; readonly engineId: string; readonly reference: NativeComposeStorageWitnessReference; - readonly observe: (markerName: string) => Promise<{ + readonly observe?: (markerName: string) => Promise<{ readonly volume: NativeComposeRetainedVolume; readonly archive: Uint8Array; }>; + readonly xattrCarrier?: NativeComposeStorageXattrCarrier; }): Promise { - const { authority, generation, engineId, observe } = opts; + const { authority, generation, engineId, observe, xattrCarrier } = opts; if (!nativeComposeStorageWitnessReferenceValid(opts.reference)) { return refuse(); } const reference = structuredClone(opts.reference); + if ( + reference.version === 1 + ? !observe || xattrCarrier !== undefined + : !xattrCarrier || observe !== undefined + ) { + return refuse(); + } + if ( + reference.version === 3 && + xattrCarrier && + JSON.stringify( + captureNativeComposeStorageXattrArtifact(reference.artifact) + ) !== JSON.stringify(nativeComposeStorageXattrCarrierArtifact(xattrCarrier)) + ) { + return refuse(); + } + if (xattrCarrier) { + checkNativeComposeStorageXattrCarrierLifetime(xattrCarrier); + } return await runNativeComposeMaterialAction({ authority, run: async () => { @@ -623,6 +1198,9 @@ export async function verifyNativeComposeStorageWitness(opts: { try { checkDirectoryAnchors(held, reference); const saved = await checkedExpectation(held, reference.expectation); + if (!expectationMatchesReference({ saved, reference })) { + return refuse(); + } const readCompletion = async () => { const read = await readPrivate( join(last(held).path, "enrolled.json"), @@ -632,12 +1210,7 @@ export async function verifyNativeComposeStorageWitness(opts: { if ( !sameFile(read.info, reference.completion) || hash(read.text) !== reference.completion.hash || - record.expectationHash !== reference.expectation.hash || - !sameVolume(record.volume, reference.volume) || - (saved.originalVolume !== null && - !sameVolume(saved.originalVolume, record.volume)) || - record.volume.name !== saved.binding.name || - record.volume.storage !== saved.binding.storage + !completionMatches({ record, reference, saved }) ) { return refuse(); } @@ -646,14 +1219,44 @@ export async function verifyNativeComposeStorageWitness(opts: { if (!matchesBinding(saved.binding, current, engineId)) { return refuse(); } - const observed = await observe(saved.marker.name); - if (!sameVolume(parseVolume(observed.volume), reference.volume)) { - return refuse(); + if (saved.version === 1) { + if (!observe) { + return refuse(); + } + const observed = await observe(saved.marker.name); + if (!sameVolume(parseVolume(observed.volume), reference.volume)) { + return refuse(); + } + verifyNativeComposeStorageWitnessArchive({ + marker: saved.marker, + archive: observed.archive, + }); + } else { + if (!xattrCarrier) { + return refuse(); + } + await observeXattr({ + carrier: xattrCarrier, + saved, + volume: reference.volume, + stopped: false, + journal: last(held), + check: async () => { + await checkedExpectation(held, reference.expectation); + await readCompletion(); + await recheckDirectories(held); + const latest = await assertNativeComposeMaterialAuthority({ + authority, + generation, + phase: "inspect", + }); + if (!matchesBinding(saved.binding, latest, engineId)) { + return refuse(); + } + return latest; + }, + }); } - verifyNativeComposeStorageWitnessArchive({ - marker: saved.marker, - archive: observed.archive, - }); await checkedExpectation(held, reference.expectation); await readCompletion(); await recheckDirectories(held); @@ -665,9 +1268,46 @@ export async function verifyNativeComposeStorageWitness(opts: { if (!matchesBinding(saved.binding, latest, engineId)) { refuse(); } + if (xattrCarrier) { + checkNativeComposeStorageXattrCarrierLifetime(xattrCarrier); + } } finally { await close(held); } }, }).catch(() => refuse()); } + +/** Explicitly tagged, unactivated source-only xattr enrollment path. */ +export async function prepareNativeComposeStorageXattrWitness( + opts: Omit< + Parameters[0], + "xattrCarrier" + > & { + readonly carrier: NativeComposeStorageXattrCarrier; + } +): Promise { + const { carrier, ...input } = opts; + return await prepareNativeComposeStorageWitness({ + ...input, + xattrCarrier: carrier, + }); +} +export async function enrollNativeComposeStorageXattrWitness(opts: { + readonly enrollment: NativeComposeStorageWitnessEnrollment; +}): Promise { + return await enrollNativeComposeStorageWitness({ + enrollment: opts.enrollment, + }); +} +export async function verifyNativeComposeStorageXattrWitness( + opts: Omit< + Parameters[0], + "observe" | "xattrCarrier" + > & { + readonly carrier: NativeComposeStorageXattrCarrier; + } +): Promise { + const { carrier, ...input } = opts; + await verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }); +} diff --git a/src/lib/shell.ts b/src/lib/shell.ts index 0243eee3a..de61f7026 100644 --- a/src/lib/shell.ts +++ b/src/lib/shell.ts @@ -86,6 +86,13 @@ export interface RunOptions { */ readonly stdout?: "inherit" | "stderr" | "ignore"; readonly stderr?: "inherit" | "ignore"; + /** Non-TTY owner-supplied descriptors. The caller holds them through settlement; + * run captures numbers synchronously and never reopens names or closes the FDs. */ + readonly privateIo?: { + readonly stdin: number; + readonly stdout: number; + readonly stderr: number; + }; readonly timeoutMs?: number; /** Forward cancellation to an owned command process group, preserving TTY input. */ readonly forwardSignals?: boolean; @@ -121,8 +128,27 @@ export async function run( ...opts, env: opts.env ? { ...opts.env } : undefined, unsetEnvKeys: opts.unsetEnvKeys ? [...opts.unsetEnvKeys] : undefined, + privateIo: opts.privateIo ? { ...opts.privateIo } : undefined, }; const command = [...cmd]; + if ( + options.privateIo && + (options.forwardSignals === true || + !( + typeof options.timeoutMs === "number" && + Number.isFinite(options.timeoutMs) && + options.timeoutMs > 0 + ) || + ![ + options.privateIo.stdin, + options.privateIo.stdout, + options.privateIo.stderr, + ].every((fd) => Number.isInteger(fd) && fd >= 0 && fd <= 2_147_483_647)) + ) { + throw new Error( + "Private subprocess descriptors require a non-TTY owned invocation." + ); + } const signal = options.signal; if (signal?.aborted) { return 143; @@ -158,9 +184,11 @@ export async function run( const proc = Bun.spawn(command, { cwd: options.cwd, env: buildSpawnEnv(options.env, options.unsetEnvKeys), - stdin: options.stdin ?? "inherit", - stdout: options.stdout === "stderr" ? 2 : (options.stdout ?? "inherit"), - stderr: options.stderr ?? "inherit", + stdin: options.privateIo?.stdin ?? options.stdin ?? "inherit", + stdout: + options.privateIo?.stdout ?? + (options.stdout === "stderr" ? 2 : (options.stdout ?? "inherit")), + stderr: options.privateIo?.stderr ?? options.stderr ?? "inherit", detached: ownsProcessGroup, }); const observeCpu = beginNativeCpuChild( diff --git a/src/lib/worktree-local-config.ts b/src/lib/worktree-local-config.ts index 24a9e1799..3b5bda8c7 100644 --- a/src/lib/worktree-local-config.ts +++ b/src/lib/worktree-local-config.ts @@ -428,13 +428,19 @@ async function readGitCheckoutIdentity(opts: { }; } -/** Ignore caller Git redirection and never surface repository paths or child output. */ -async function readGitInspection(opts: { +/** Bounded read-only Git acquisition, separate from engine command file quotas. + * Ignore caller Git redirection and never surface repository paths or child output. + * An owner may supply a synchronous final admission and observe the owned child. */ +export async function readGitInspection(opts: { readonly projectRoot: string; readonly args: readonly string[]; readonly signal?: AbortSignal; + readonly beforeSpawn?: () => undefined; + readonly onSpawn?: (pid: number) => undefined; }): Promise { - throwIfCancelled(opts.signal); + const { projectRoot, signal, beforeSpawn, onSpawn } = opts; + const args = [...opts.args]; + throwIfCancelled(signal); const env: Record = { GIT_OPTIONAL_LOCKS: "0", GIT_CONFIG_NOSYSTEM: "1", @@ -457,7 +463,12 @@ async function readGitInspection(opts: { } } try { - const child = Bun.spawn(["git", "-C", opts.projectRoot, ...opts.args], { + const admitted: unknown = beforeSpawn?.(); + if (admitted !== undefined) { + throw worktreeVerificationError(); + } + throwIfCancelled(signal); + const child = Bun.spawn(["git", "-C", projectRoot, ...args], { env, stdin: "ignore", stdout: "pipe", @@ -481,29 +492,34 @@ async function readGitInspection(opts: { timedOut = true; kill(); }, 10_000); - opts.signal?.addEventListener("abort", cancel, { once: true }); - if (opts.signal?.aborted) { + signal?.addEventListener("abort", cancel, { once: true }); + if (signal?.aborted) { cancel(); } try { - const [output, code] = await Promise.all([ - readGitOutput(child.stdout), - child.exited, - ]); + const reading = readGitOutput(child.stdout); + let observationFailed = false; + try { + const observed: unknown = onSpawn?.(child.pid); + observationFailed = observed !== undefined; + } catch { + observationFailed = true; + } + const [output, code] = await Promise.all([reading, child.exited]); if (cancelled) { - throwIfCancelled(opts.signal); + throwIfCancelled(signal); } - return code === 0 && !timedOut ? output : null; + return code === 0 && !timedOut && !observationFailed ? output : null; } finally { clearTimeout(timer); - opts.signal?.removeEventListener("abort", cancel); + signal?.removeEventListener("abort", cancel); if (child.exitCode === null) { kill(); } await child.exited; } } catch (error: unknown) { - throwIfCancelled(opts.signal); + throwIfCancelled(signal); if (error instanceof NativeConfigCompilerError) { throw error; } diff --git a/tests/models/tla/README.md b/tests/models/tla/README.md index f9d8e6694..fb31907e9 100644 --- a/tests/models/tla/README.md +++ b/tests/models/tla/README.md @@ -63,12 +63,12 @@ fail the runner. | Model action/state | Intended foundation mapping and separate implementation obligation | | --- | --- | | `ReserveIntent` / `intent` | `prepareNativeComposeStorageWitness` first calls `armNativeComposeStorageWitnessIntent` to publish a required v3 `Expected` entry under the existing generation mutation, then exclusively creates and synchronizes the private per-volume slot before writing the expectation. An empty or absent interrupted slot retains the receipt intent and cannot be recreated. The model's slot intent abstracts that refusal; it does not model the enclosing generation receipt. | -| `PublishExpected` / `expected` | `prepareNativeComposeStorageWitness` in the separate `native-compose-storage-witness.ts` foundation: validate active pending-generation authority, exclusively publish/sync an immutable random 32-byte expectation and exact engine/instance/owner/physical-volume/logical-volume/generation/pending binding, then return an opaque live capability. A failed publication must not grant one. | -| `ArmSeed` / `PublishMarker` | `enrollNativeComposeStorageWitness` consumes the original capability before awaiting its single seed callback. Callback transport, create-only behavior, child ownership/reaping and no writes to an existing witness require implementation and carrier tests. `CreateVolume` is an abstract cold-creation boundary, not a claim that the foundation creates Docker volumes. | -| `ObserveEnrollment` / `CommitEnrollment` | The owner verifies an observed archive through `verifyNativeComposeStorageWitnessArchive`, exclusively publishes/syncs/re-reads the completion file, and rechecks owner/expectation anchors. It issues a private one-use proof bound to the exact live authority and generation; `publishNativeComposeStorageWitnessEnrollment` consumes only that issued proof and obtains its reference and fresh verifier from the owner. `createNativeComposeStorageWitnessReceiptProtocol` verifies again before attaching `Enrolled` and rechecks live authority after the final filesystem await before receipt publication. The model additionally requires the current exact witness at commit; qualifying that predicate across actual carrier awaits remains a separate engine-boundary obligation. Its completion is the enrollment record, not a generation's ready receipt. | -| `Down` / `BeginResume` / `ReadWitness` | `verifyNativeComposeStorageWitness` validates the saved reference, anchored files/directories, fresh carrier observation and exact archive, then re-reads and rechecks ownership without writing. The receipt protocol requires read-only proof before/after startup/run and at both completion receipt boundaries. The CLI refuses witness-bearing startup/run/exec while carrier authority is withheld; actual down/up activation remains unqualified. | -| `Crash` / `Recover` / `RecoveryDisposition` | Persisted `Expected` without confirmed completion retains its original bytes and refuses inspection-to-enrollment promotion. Reopened records cannot recover an original seed capability. Explicit saved `down --recover` can stop engine resources but returns incomplete with `Expected` and pending generation retained, skipping dependent ownership retirement. An explicit read-only reconciliation interface requires its own source review and tests; neither this model nor the foundation implements it. | -| `EmptyReplacement` / `WrongMarker` / `ForeignMetadata` | Owner/codec tests must distinguish absent/wrong bytes, fresh foreign binding, a same-metadata empty replacement and a copied witness. `encodeNativeComposeStorageWitnessArchive` is the proposed carrier codec; decoding bounds, archive path validation and read-only engine transport are outside this model. | +| `PublishExpected` / `expected` | `prepareNativeComposeStorageWitness` validates active pending-generation authority, exclusively publishes/syncs an immutable random 32-byte expectation and exact engine/instance/owner/physical-volume/logical-volume/generation/pending binding, then returns an opaque live capability. Its explicit `prepareNativeComposeStorageXattrWitness` path additionally requires captured artifact pins and a distinct version-three directory-xattr expectation; the v3 intent requires the carrier tag, artifact and private journal token before slot creation. A failed publication must not grant a capability. | +| `ArmSeed` / `PublishMarker` | `enrollNativeComposeStorageWitness` consumes the original capability before awaiting its single seed callback. The explicit xattr path instead permits one captured cold-provision callback and one seed invocation, then separate read-only proof. `runNativeComposeStorageXattrHelper` requests `XATTR_CREATE` once through syscall ports, without retry/repair. Actual cold newness, non-creating current-root transport, child ownership/reaping, Linux ABI and kernel durability remain unqualified. `CreateVolume` is an abstract boundary, not evidence that this source creates Docker volumes. | +| `ObserveEnrollment` / `CommitEnrollment` | The owner verifies the distinct archive or xattr result, exclusively publishes/syncs/re-reads the completion file, and rechecks owner/expectation anchors. The xattr path binds exact artifact/context/volume/root and canonical kernel response hash; fresh root discovery, token read and post-read root discovery are separate injectable invocations. It issues a private one-use proof bound to the exact live authority and generation; `publishNativeComposeStorageWitnessEnrollment` consumes only that issued proof and obtains its reference and fresh verifier from the owner. The receipt protocol verifies again before attaching `Enrolled` and rechecks live authority after the final filesystem await before publication. The model's current exact witness predicate is stronger than finite observations across real engine awaits; that remains a separate transport obligation. Its completion is the enrollment record, not a generation's ready receipt. | +| `Down` / `BeginResume` / `ReadWitness` | `verifyNativeComposeStorageWitness` validates the saved reference, anchored files/directories, fresh observation and exact archive or xattr value, then re-reads and rechecks ownership without writing storage. Xattr verification publishes/synchronizes private finite helper intent while keeping the volume read-only; it never provisions, seeds or reenrolls, and brackets each fresh invocation with exact daemon/birth/current-Mountpoint/holder and authority checks. The receipt protocol requires read-only storage proof before/after startup/run and at both completion receipt boundaries. The CLI refuses witness-bearing startup/run/exec; actual down/up activation remains unqualified. | +| `Crash` / `Recover` / `RecoveryDisposition` | Persisted `Expected` without confirmed completion retains its original bytes and refuses inspection-to-enrollment promotion. Reopened records cannot recover an original seed capability. Required finite xattr carrier intent also survives unknown outcomes after enrollment; saved checks block replay and retirement, including unpublished helper IDs. Explicit saved `down --recover` can stop engine resources but returns incomplete with unresolved intent and pending generation retained, skipping dependent ownership retirement. An explicit read-only reconciliation interface requires its own source review and tests; neither this model nor the foundation implements it. The model does not represent finite helper intent/publication/cleanup. | +| `EmptyReplacement` / `WrongMarker` / `ForeignMetadata` | Owner/codec controls distinguish absent/wrong bytes, fresh foreign binding and same-metadata empty replacement. Xattr owner controls also refuse an old pinned-root result and same-birth root exchange during final proof. `encodeNativeComposeStorageWitnessArchive` and the tagged xattr codec are distinct; byte decoding, namespace transport and read-only engine behavior are outside this model. A copied matching witness with otherwise identical metadata is outside the empty-replacement guarantee. | | `DriftAuthority` | Existing opaque generation authority must reject changed pending selection or revoked lease before seed/completion, and recheck after asynchronous proofs. The model summarizes those identities by one version; it cannot qualify real lease/token/inode checks. | | `ReplaceExpectation` / `ReenrollReadOnly` | Immutable anchored records must reject replacement and read-only reopen must not append enrollment, even with matching marker bytes. Filesystem substitution and exact write counts require separate source regressions. | diff --git a/tests/native-compose-storage-witness-command.test.ts b/tests/native-compose-storage-witness-command.test.ts index e32a75c8d..e68823317 100644 --- a/tests/native-compose-storage-witness-command.test.ts +++ b/tests/native-compose-storage-witness-command.test.ts @@ -4,8 +4,14 @@ import { openNativeComposeGenerationStore } from "../src/lib/native-compose-gene import { nativeComposeRetainedVolumesValid } from "../src/lib/native-compose-retained-storage.ts"; import { enrollNativeComposeStorageWitness, + enrollNativeComposeStorageXattrWitness, prepareNativeComposeStorageWitness, + prepareNativeComposeStorageXattrWitness, + verifyNativeComposeStorageXattrWitness, } from "../src/lib/native-compose-storage-witness.ts"; +import { captureNativeComposeStorageXattrCarrier } from "../src/lib/native-compose-storage-witness-xattr-carrier.ts"; +import { encodeNativeComposeStorageXattrResponse } from "../src/lib/native-compose-storage-witness-xattr-codec.ts"; +import { runNativeComposeStorageXattrHelper } from "../src/lib/native-compose-storage-witness-xattr-helper.ts"; import { fixture, invoke } from "./helpers/native-compose-command.ts"; async function prepared(enroll: boolean) { @@ -134,3 +140,191 @@ test("source-unavailable explicit saved recovery stops engine but retains Expect expect(current.retainedStorage).toHaveLength(1); expect(JSON.stringify(current)).not.toContain(".hack-storage-"); }, 30_000); + +test("source CLI reports and explicitly stops saved resources for unknown enrolled preflight helper with no main pending", async () => { + const root = await fixture("", false, { + noHooks: true, + storage: { data: { kind: "persistent", scope: "worktree" } }, + }); + expect((await invoke(root)).code).toBe(0); + expect((await invoke(root, ["down", "--json"])).code).toBe(0); + const store = await openNativeComposeGenerationStore({ + projectRoot: root, + instance: null, + mode: "prepare", + }); + let generationId = ""; + let witnesses: Awaited< + ReturnType + >["storageWitnesses"] = null; + const marker = new Map(); + let unknown = false; + let helperId = 1; + try { + const current = await store.loadCurrent(); + const generation = current.generation; + const volume = current.retainedStorage?.[0]; + if (!(generation && volume)) { + throw new Error("Missing saved synthetic volume"); + } + generationId = generation.generationId; + const carrier = captureNativeComposeStorageXattrCarrier({ + artifact: { + version: 1, + imageId: `sha256:${"a".repeat(64)}`, + platform: "linux/arm64", + bunVersion: "1.4.2", + bunHash: "b".repeat(64), + libcHash: "c".repeat(64), + helperHash: "e".repeat(64), + kernelAbi: 1, + }, + signal: new AbortController().signal, + deadline: Date.now() + 30_000, + ports: { + inspect: async () => ({ + engineId: "d".repeat(64), + runtimeIdentity: store.identity.composeProject, + ownerToken: store.identity.ownerToken, + name: volume.name, + storage: volume.storage, + volume, + mountpoint: `/var/lib/docker/volumes/${volume.name}/_data`, + driver: "local", + options: {}, + holders: [], + }), + invoke: async (input) => { + if (unknown) { + throw new Error( + "Synthetic create outcome unknown; no actual helper or Docker" + ); + } + const id = (helperId++).toString(16).padStart(64, "0"); + await input.recordCreated({ id, createdAt: volume.createdAt }); + const rootFacts = { device: "1", inode: "42", uid: 0, gid: 0 }; + const response = runNativeComposeStorageXattrHelper({ + request: input.request, + kernel: { + effectiveUid: () => input.uid, + effectiveGid: () => input.gid, + probeRoot: () => rootFacts, + openRoot: () => 42, + statRoot: () => rootFacts, + createXattr: (_, name, bytes) => { + if (input.readonly || marker.has(name)) { + throw new Error("EEXIST/EROFS"); + } + marker.set(name, new Uint8Array(bytes)); + }, + syncRoot: () => {}, + readXattr: (_, name) => marker.get(name) ?? new Uint8Array(), + closeRoot: () => {}, + }, + }); + const exitCode = response.outcome === "refused" ? 1 : 0; + return { + artifact: input.artifact, + carrierId: id, + carrierCreatedAt: volume.createdAt, + engineId: input.target.engineId, + invocationId: input.invocationId, + readonly: input.readonly, + uid: input.uid, + gid: input.gid, + target: input.target, + scope: input.scope, + response: encodeNativeComposeStorageXattrResponse(response), + outcome: "complete", + exitCode, + stopped: { id, running: false, pid: 0, exitCode }, + containersAfterCleanup: [], + }; + }, + }, + }); + await store.withMutation(async (mutation) => { + expect( + await mutation.runEffect({ + generation, + operation: "up", + assertFresh: async () => {}, + assertOwned: async () => {}, + effect: async () => { + const enrollment = await prepareNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation, + engineId: "d".repeat(64), + volume: { name: volume.name, storage: volume.storage }, + admission: "explicit-adoption", + originalVolume: volume, + assertAdmission: async () => {}, + carrier, + }); + await enrollNativeComposeStorageXattrWitness({ enrollment }); + return { value: 0, outcome: "complete" }; + }, + }) + ).toEqual({ value: 0, outcome: "complete" }); + }); + const enrolled = (await store.loadCurrent()).storageWitnesses?.[0]; + if ( + !enrolled || + enrolled.state !== "enrolled" || + enrolled.reference.version !== 3 + ) { + throw new Error("Missing issued xattr enrollment"); + } + unknown = true; + await store.withMutation(async (mutation) => { + await expect( + verifyNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation, + engineId: "d".repeat(64), + reference: enrolled.reference, + carrier, + }) + ).rejects.toThrow("values omitted"); + }); + const uncertain = await store.loadCurrent(); + expect(uncertain.pending).toBeNull(); + expect(uncertain.storageWitnessesPending).toBe(true); + witnesses = uncertain.storageWitnesses; + } finally { + await store.close(); + } + const ps = await invoke(root, ["ps", "--json"]); + expect(ps.code).toBe(0); + expect(JSON.parse(ps.stdout)).toMatchObject({ + ok: true, + data: { pending: true }, + }); + expect(ps.stdout).not.toContain("carrierJournalToken"); + expect(ps.stdout).not.toContain("user.hack.storage."); + const requests = await Bun.file(join(root, "requests")).text(); + const compiler = await Bun.file(join(root, "compiler-requests")).text(); + await Bun.write( + join(root, ".hack/hack.project.json"), + "malformed authored source" + ); + await Bun.write( + join(root, ".hack/hack.env.default.yaml"), + "malformed private env" + ); + expect((await invoke(root, ["down", "--json"])).code).toBe(1); + expect(await Bun.file(join(root, "requests")).text()).toBe(requests); + const recovered = await invoke(root, ["down", "--recover", "--json"]); + expect(recovered.code).toBe(1); + expect(JSON.parse(recovered.stdout)).toMatchObject({ + ok: false, + error: { code: "E_COMPOSE_FAILED" }, + }); + expect(await Bun.file(join(root, "engine")).exists()).toBe(false); + expect(await Bun.file(join(root, "compiler-requests")).text()).toBe(compiler); + const retained = await saved(root); + expect(retained.pending?.generationId).toBe(generationId); + expect(retained.storageWitnesses).toEqual(witnesses); + expect(retained.storageWitnessesPending).toBe(true); + expect(retained.retainedStorage).toHaveLength(1); +}, 30_000); diff --git a/tests/native-compose-storage-witness-docker-io.test.ts b/tests/native-compose-storage-witness-docker-io.test.ts new file mode 100644 index 000000000..d7da3cd22 --- /dev/null +++ b/tests/native-compose-storage-witness-docker-io.test.ts @@ -0,0 +1,244 @@ +import { afterEach, expect, test } from "bun:test"; +import { fstatSync } from "node:fs"; +import { + link, + mkdtemp, + readFile, + realpath, + rename, + rm, + symlink, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + holdDirectory, + writeExclusive, +} from "../src/lib/native-compose-private-state.ts"; +import { runNativeComposeStorageDockerCommand } from "../src/lib/native-compose-storage-witness-docker.ts"; +import { + createNativeComposeStorageDockerEmptyLeaf, + holdNativeComposeStorageDockerIo, +} from "../src/lib/native-compose-storage-witness-docker-io.ts"; +import { run } from "../src/lib/shell.ts"; + +const roots: string[] = []; +afterEach(async () => { + await Promise.all( + roots.splice(0).map((root) => rm(root, { recursive: true, force: true })) + ); +}); +async function fixture() { + const root = await realpath( + await mkdtemp(join(tmpdir(), "native-storage-held-io-")) + ); + roots.push(root); + const directory = await holdDirectory(root, true); + const input = join(root, "input"), + stdout = join(root, "stdout"), + stderr = join(root, "stderr"); + const text = "synthetic private input"; + const inputInfo = await writeExclusive(input, text), + outInfo = await createNativeComposeStorageDockerEmptyLeaf(stdout), + errInfo = await createNativeComposeStorageDockerEmptyLeaf(stderr); + const io = await holdNativeComposeStorageDockerIo({ + directories: [directory], + input: { path: input, info: inputInfo, text }, + stdout: { path: stdout, info: outInfo }, + stderr: { path: stderr, info: errInfo }, + limit: 4096, + }); + return { root, directory, input, inputInfo, stdout, stderr, text, io }; +} +test("held IO passes exact stdin and preserves caller descriptors through completion", async () => { + const f = await fixture(); + try { + let exited = false; + const code = await run(["/bin/sh", "-c", "ulimit -f 8; exec cat"], { + privateIo: f.io.descriptors, + timeoutMs: 3000, + beforeSpawn: f.io.assertFresh, + onExit: async () => { + exited = true; + expect(fstatSync(f.io.descriptors.stdout).isFile()).toBe(true); + }, + }); + expect(code).toBe(0); + expect(exited).toBe(true); + expect(f.io.read()).toEqual({ stdout: f.text, stderr: "" }); + expect(fstatSync(f.io.descriptors.stdin).isFile()).toBe(true); + } finally { + await f.io.close(); + await f.directory.file.close(); + } +}); +test("separate empty command leaf never overwrites existing bytes or follows a symlink", async () => { + const f = await fixture(); + try { + await expect( + createNativeComposeStorageDockerEmptyLeaf(f.input) + ).rejects.toThrow(); + const alias = join(f.root, "alias"); + await symlink(f.input, alias); + await expect( + createNativeComposeStorageDockerEmptyLeaf(alias) + ).rejects.toThrow(); + expect(await readFile(f.input, "utf8")).toBe(f.text); + await expect( + writeExclusive(join(f.root, "receipt-empty"), "") + ).rejects.toThrow("values omitted"); + } finally { + await f.io.close(); + await f.directory.file.close(); + } +}); +test.each([ + "input-replace", + "input-symlink", + "stdout-replace", + "stderr-replace", + "input-hardlink", + "input-change", + "stdout-hardlink", + "stdout-symlink", +] as const)("last admission await %s refuses before spawn and preserves the ungranted file", async (failure) => { + const f = await fixture(); + const foreign = join(f.root, "ungranted"); + await writeFile(foreign, "preserved foreign bytes", { mode: 0o600 }); + let admitted = false; + const files = { + held: [f.directory], + path: f.root, + input: f.input, + inputInfo: f.inputInfo, + requestText: f.text, + captures: [] as { path: string; info: typeof f.inputInfo }[], + }; + try { + await expect( + runNativeComposeStorageDockerCommand({ + files, + context: { + signal: new AbortController().signal, + deadline: Date.now() + 3000, + }, + args: ["/bin/cat"], + timeoutMs: 3000, + beforeSpawn: () => { + admitted = true; + }, + assertAdmitted: async () => { + const selected = failure.startsWith("input") + ? f.input + : (files.captures[failure.startsWith("stdout") ? 0 : 1]?.path ?? + ""); + if (failure.endsWith("replace")) { + await rename(selected, `${selected}.original`); + await link(foreign, selected); + } else if (failure.endsWith("symlink")) { + await rename(selected, `${selected}.original`); + await symlink(foreign, selected); + } else if (failure.endsWith("hardlink")) { + await link(selected, `${selected}.alias`); + } else { + await writeFile(selected, "substituted input"); + } + }, + }) + ).rejects.toThrow("values omitted"); + expect(admitted).toBe(false); + expect(await readFile(foreign, "utf8")).toBe("preserved foreign bytes"); + } finally { + await f.io.close(); + await f.directory.file.close(); + } +}); +test("cancellation at the final admission leaves captures empty and spawns no child", async () => { + const f = await fixture(), + controller = new AbortController(); + let spawned = false; + try { + const code = await run(["/bin/sh", "-c", "ulimit -f 8; exec cat"], { + privateIo: f.io.descriptors, + timeoutMs: 3000, + signal: controller.signal, + beforeSpawn: () => { + f.io.assertFresh(); + controller.abort(); + }, + onSpawn: async () => { + spawned = true; + }, + }); + expect(code).toBe(143); + expect(spawned).toBe(false); + expect(f.io.read()).toEqual({ stdout: "", stderr: "" }); + } finally { + await f.io.close(); + await f.directory.file.close(); + } +}); +test("actual transport refuses an abort during its last admission await without child effects", async () => { + const f = await fixture(), + controller = new AbortController(); + const files = { + held: [f.directory], + path: f.root, + input: f.input, + inputInfo: f.inputInfo, + requestText: f.text, + captures: [] as { path: string; info: typeof f.inputInfo }[], + }; + let admitted = false; + try { + await expect( + runNativeComposeStorageDockerCommand({ + files, + context: { signal: controller.signal, deadline: Date.now() + 3000 }, + args: ["/bin/cat"], + timeoutMs: 3000, + beforeSpawn: () => { + admitted = true; + }, + assertAdmitted: async () => { + await Promise.resolve(); + controller.abort(); + }, + }) + ).rejects.toThrow("values omitted"); + expect(admitted).toBe(false); + for (const capture of files.captures) { + expect(await readFile(capture.path, "utf8")).toBe(""); + } + } finally { + await f.io.close(); + await f.directory.file.close(); + } +}); +test("fixed wrapper quota bounds capture growth and oversized output cannot be accepted", async () => { + const f = await fixture(); + try { + const code = await run( + [ + "/bin/sh", + "-c", + 'ulimit -f 8; exec /bin/sh -c "while :; do printf 0123456789; done"', + ], + { + privateIo: f.io.descriptors, + timeoutMs: 3000, + beforeSpawn: f.io.assertFresh, + } + ); + expect(code).not.toBe(0); + expect(code).not.toBe(124); + expect(fstatSync(f.io.descriptors.stdout).size).toBeLessThanOrEqual(8192); + if (fstatSync(f.io.descriptors.stdout).size > 4096) { + expect(() => f.io.read()).toThrow("values omitted"); + } + } finally { + await f.io.close(); + await f.directory.file.close(); + } +}); diff --git a/tests/native-compose-storage-witness-docker.test.ts b/tests/native-compose-storage-witness-docker.test.ts new file mode 100644 index 000000000..d9e655a38 --- /dev/null +++ b/tests/native-compose-storage-witness-docker.test.ts @@ -0,0 +1,838 @@ +import { afterEach, expect, test } from "bun:test"; +import { createHash } from "node:crypto"; +import { mkdir, mkdtemp, realpath, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + assertNativeComposeMaterialAuthority, + type NativeComposeGenerationStore, + type NativeComposeMaterialBinding, + openNativeComposeGenerationStore, +} from "../src/lib/native-compose-generation.ts"; +import { assertNativeComposeStorageDockerImage } from "../src/lib/native-compose-storage-witness-docker.ts"; +import { + NATIVE_STORAGE_DOCKER_ARTIFACT, + nativeComposeStorageDockerHelper, +} from "../src/lib/native-compose-storage-witness-docker-artifact.ts"; +import { + assertNativeComposeStorageDockerCarrierVolume, + nativeComposeStorageDockerPathsOverlap, + observeNativeComposeStorageDockerTarget, +} from "../src/lib/native-compose-storage-witness-docker-inventory.ts"; +import { + checkNativeComposeStorageDockerCarrier, + NATIVE_STORAGE_CARRIER_LABEL, + nativeComposeStorageDockerCarrierPolicy, + nativeComposeStorageDockerCreateArgs, +} from "../src/lib/native-compose-storage-witness-docker-policy.ts"; +import type { NativeComposeStorageXattrInvocation } from "../src/lib/native-compose-storage-witness-xattr-carrier.ts"; + +const roots: string[] = [], + stores: NativeComposeGenerationStore[] = []; +const engineId = "engine-fixture"; +const selection = { name: "owned_data", storage: "data" }; +const createdAt = "2026-10-08T12:00:00Z"; +const SPARSE_IMAGE_FIELD = /{{json ([^{}]+)}}/g; + +test("cached image prerequisite emits complete sparse JSON before carrier effects", async () => { + const fields: Readonly> = { + ".Id": NATIVE_STORAGE_DOCKER_ARTIFACT.imageId, + ".Os": "linux", + ".Architecture": "arm64", + '(index .Config "Volumes")': null, + }; + const requests: string[][] = []; + await assertNativeComposeStorageDockerImage(async (args) => { + requests.push([...args]); + expect(args).toHaveLength(5); + expect(args.slice(0, 3)).toEqual(["image", "inspect", "--format"]); + expect(args[4]).toBe(NATIVE_STORAGE_DOCKER_ARTIFACT.imageId); + const format = args[3]; + if (typeof format !== "string") { + throw new Error("Missing fixed sparse image format"); + } + const rendered = format.replace(SPARSE_IMAGE_FIELD, (_, field: string) => { + if (!Object.hasOwn(fields, field)) { + throw new Error("Unexpected sparse image field"); + } + const encoded = JSON.stringify(fields[field]); + if (typeof encoded !== "string") { + throw new Error("Invalid fixed sparse image field"); + } + return encoded; + }); + const output = `${rendered}\n`; + expect(Buffer.byteLength(output)).toBe(124); + return output; + }); + expect(requests).toHaveLength(1); +}); + +test("cached image prerequisite refuses the faithful truncated sparse output", async () => { + const truncated = `{"id":"${NATIVE_STORAGE_DOCKER_ARTIFACT.imageId}","os":"linux","arch":"arm64","volumes":null\n`; + expect(Buffer.byteLength(truncated)).toBe(123); + expect(createHash("sha256").update(truncated).digest("hex")).toBe( + "aea01511888599e3e1cacfc66c146dadb3ffdec207e1a620fe0136c2bca72ad8" + ); + await expect( + assertNativeComposeStorageDockerImage(async () => truncated) + ).rejects.toBeInstanceOf(SyntaxError); +}); + +afterEach(async () => { + await Promise.all(stores.splice(0).map((store) => store.close())); + await Promise.all( + roots.splice(0).map((root) => rm(root, { recursive: true, force: true })) + ); +}); +async function withBinding( + run: (value: NativeComposeMaterialBinding) => Promise +) { + const root = await realpath( + await mkdtemp(join(tmpdir(), "native-storage-docker-")) + ); + roots.push(root); + await mkdir(join(root, ".hack")); + await Bun.write( + join(root, ".hack/hack.project.json"), + JSON.stringify({ schema_version: 1, name: "storage" }) + ); + const store = await openNativeComposeGenerationStore({ + projectRoot: root, + instance: null, + }); + stores.push(store); + await store.withMutation(async (mutation) => { + const reservation = mutation.reserveGeneration(); + const labels = { + "io.hack.native-config.version": "1", + "io.hack.native-config.owner": store.identity.ownerToken, + "io.hack.native-config.instance": store.identity.composeProject, + }; + const generation = await mutation.publish({ + reservation, + profiles: [], + inputRevision: "a".repeat(64), + assertFresh: async () => {}, + composeJson: JSON.stringify({ + name: store.identity.composeProject, + services: { + app: { + image: "synthetic:1", + labels: { + ...labels, + "io.hack.native-config.generation": reservation.generationId, + "io.hack.native-config.workload": "service", + }, + }, + }, + volumes: { + data: { + name: selection.name, + labels: { + ...labels, + "io.hack.native-config.storage": selection.storage, + }, + }, + }, + }), + }); + await mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + effect: async () => { + await run( + await assertNativeComposeMaterialAuthority({ + authority: mutation.materialAuthority, + generation, + phase: "effect", + }) + ); + return { outcome: "complete", value: 0 }; + }, + }); + }); +} +function inventory(binding: NativeComposeMaterialBinding) { + const state = { + present: true, + drift: false, + foreign: false, + running: false, + mounts: true, + version: "1", + volumeReads: 0, + volumeOwner: binding.identity.ownerToken, + mountType: "volume", + mountName: selection.name, + mountSource: `/var/lib/docker/volumes/${selection.name}/_data`, + }; + const id = "c".repeat(64); + const mutableCalls: string[][] = []; + const probe = async (args: readonly string[]): Promise => { + mutableCalls.push([...args]); + if (args[0] === "info") { + return JSON.stringify(engineId); + } + if (args[0] === "volume" && args[1] === "ls") { + return state.present ? JSON.stringify(selection.name) : ""; + } + if (args[0] === "volume" && args[1] === "inspect") { + state.volumeReads++; + if (!state.present) { + return ""; + } + return JSON.stringify({ + ...selection, + createdAt: + state.drift && state.volumeReads > 1 + ? "2026-10-08T13:00:00Z" + : createdAt, + driver: "local", + options: null, + mountpoint: `/var/lib/docker/volumes/${selection.name}/_data`, + project: binding.identity.composeProject, + instance: binding.identity.composeProject, + owner: state.volumeOwner, + version: state.version, + provision: null, + }); + } + if (args[0] === "container" && args[1] === "ls") { + return state.mounts ? JSON.stringify(id) : ""; + } + if (args[0] === "container" && args[1] === "inspect") { + return JSON.stringify({ + id, + createdAt, + mounts: [ + { + Type: state.mountType, + Name: state.mountName, + Source: state.mountSource, + Destination: "/data", + RW: true, + }, + ], + running: state.running, + project: binding.identity.composeProject, + instance: binding.identity.composeProject, + owner: state.foreign ? "f".repeat(32) : binding.identity.ownerToken, + version: state.version, + generation: binding.generationId, + carrier: null, + carrierOwner: null, + carrierGeneration: null, + }); + } + throw new Error("Unexpected synthetic Docker request"); + }; + return { state, probe, calls: mutableCalls }; +} +test("full holder inventory admits stopped and exact owned running readers and preserves strict sparse reads", async () => { + await withBinding(async (current) => { + const fake = inventory(current); + const target = await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }); + expect(target.volume).toEqual({ ...selection, createdAt }); + expect(target.holders).toHaveLength(1); + fake.state.running = true; + expect( + ( + await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: false, + probe: fake.probe, + }) + ).holders[0]?.running + ).toBe(true); + expect( + fake.calls.every( + (args) => + !( + args.includes("create") || + args.includes("start") || + args.includes("rm") || + args.join(" ").includes("Config.Env") + ) + ) + ).toBe(true); + }); +}); +test("full carrier target is projected into exact retained volume facts", async () => { + await withBinding(async (current) => { + const fake = inventory(current); + const target = await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }); + expect(Object.keys(target)).toHaveLength(10); + const repeated = await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection: target, + stopped: true, + probe: fake.probe, + }); + expect(repeated).toEqual(target); + expect(Object.keys(repeated.volume ?? {}).sort()).toEqual([ + "createdAt", + "name", + "storage", + ]); + expect(fake.calls.every((args) => !args.includes("create"))).toBe(true); + }); +}); + +test("holder observation captures selection before the first probe await", async () => { + await withBinding(async (current) => { + const fake = inventory(current); + const mutable = { ...selection }; + let changed = false; + const target = await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection: mutable, + stopped: true, + probe: async (args) => { + if (!changed) { + changed = true; + mutable.name = "substituted_data"; + mutable.storage = "substituted"; + } + return await fake.probe(args); + }, + }); + expect(changed).toBe(true); + expect(target.name).toBe(selection.name); + expect(target.storage).toBe(selection.storage); + expect(target.volume).toEqual({ ...selection, createdAt }); + expect(fake.calls.some((args) => args.includes(mutable.name))).toBe(false); + }); +}); + +test.each([ + "missing", + "rebirth", + "foreign", +] as const)("post-create %s storage cannot admit helper start", async (failure) => { + await withBinding(async (current) => { + const fake = inventory(current); + const target = await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }); + if (failure === "missing") { + fake.state.present = false; + } + if (failure === "rebirth") { + fake.state.drift = true; + } + if (failure === "foreign") { + fake.state.volumeOwner = "f".repeat(32); + } + let starts = 0; + await expect( + (async () => { + await assertNativeComposeStorageDockerCarrierVolume({ + current, + target, + probe: fake.probe, + }); + starts++; + })() + ).rejects.toThrow("values omitted"); + expect(starts).toBe(0); + }); +}); +test("post-create storage proof preserves the full target snapshot across its read", async () => { + await withBinding(async (current) => { + const fake = inventory(current); + const observed = await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }); + const target = { + ...observed, + volume: observed.volume ? { ...observed.volume } : null, + }; + await assertNativeComposeStorageDockerCarrierVolume({ + current, + target, + probe: async (args) => { + target.name = "substituted_data"; + target.storage = "substituted"; + target.mountpoint = "/var/lib/docker/volumes/substituted_data/_data"; + if (target.volume) { + target.volume.createdAt = "2026-10-08T13:00:00Z"; + } + return await fake.probe(args); + }, + }); + expect(fake.calls.at(-1)?.at(-1)).toBe(selection.name); + }); +}); + +test.each([ + "foreign", + "running", + "drift", + "version", +] as const)("selected holder or volume %s refuses before any effects", async (failure) => { + await withBinding(async (current) => { + const fake = inventory(current); + if (failure === "version") { + fake.state.version = "2"; + } else { + fake.state[failure] = true; + } + await expect( + observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }) + ).rejects.toThrow("values omitted"); + expect( + fake.calls.every( + (args) => + !( + args.includes("create") || + args.includes("start") || + args.includes("rm") + ) + ) + ).toBe(true); + }); +}); +test("genuinely absent cold storage needs an empty complete holder selection", async () => { + await withBinding(async (current) => { + const fake = inventory(current); + fake.state.present = false; + fake.state.mounts = false; + expect( + ( + await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }) + ).volume + ).toBeNull(); + fake.state.mounts = true; + await expect( + observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }) + ).rejects.toThrow(); + }); +}); +test.each([ + "/var/lib/docker/volumes", + "/var/lib/docker/volumes/owned_data/", + "/var/lib/docker/volumes/owned_data/_data/", + "/var/lib/docker/volumes/owned_data/_data/child", + "/var/lib/docker/volumes/other/../owned_data/_data", +] as const)("foreign overlapping bind %s refuses rather than disappearing from holder selection", async (source) => { + await withBinding(async (current) => { + const fake = inventory(current); + fake.state.foreign = true; + fake.state.mountType = "bind"; + fake.state.mountName = ""; + fake.state.mountSource = source; + await expect( + observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }) + ).rejects.toThrow("values omitted"); + expect( + fake.calls.every( + (args) => + !( + args.includes("create") || + args.includes("start") || + args.includes("rm") + ) + ) + ).toBe(true); + }); +}); +test("complete holder inventory keeps a similarly named foreign sibling independent", async () => { + await withBinding(async (current) => { + const fake = inventory(current); + fake.state.foreign = true; + fake.state.mountType = "bind"; + fake.state.mountName = ""; + fake.state.mountSource = "/var/lib/docker/volumes/owned_data_other/_data"; + expect( + ( + await observeNativeComposeStorageDockerTarget({ + current, + engineId, + selection, + stopped: true, + probe: fake.probe, + }) + ).holders + ).toEqual([]); + }); + expect( + nativeComposeStorageDockerPathsOverlap( + "relative/path", + "/var/lib/docker/volumes/owned_data/_data" + ) + ).toBe(false); +}); +function input(): NativeComposeStorageXattrInvocation { + return { + recordCreated: async () => {}, + invocationId: "a".repeat(32), + artifact: NATIVE_STORAGE_DOCKER_ARTIFACT, + target: { + ...selection, + engineId, + runtimeIdentity: "owned", + ownerToken: "b".repeat(32), + volume: { ...selection, createdAt }, + mountpoint: `/var/lib/docker/volumes/${selection.name}/_data`, + driver: "local", + options: {}, + holders: [], + }, + readonly: true, + uid: 70, + gid: 70, + request: { kind: "directory-xattr", version: 1, operation: "root" }, + scope: { + generationId: "d".repeat(32), + currentGenerationId: "d".repeat(32), + pendingGenerationId: null, + pendingToken: null, + }, + }; +} +function carrier(input: NativeComposeStorageXattrInvocation) { + const program = "/private-owned/helper.mjs"; + const rootMount = { + Type: "volume", + Name: input.target.name, + Source: input.target.mountpoint, + Destination: "/hack-storage-witness", + Driver: "local", + Mode: "", + RW: !input.readonly, + Propagation: "", + }; + const volumeOptions: Record = { NoCopy: true }; + const rootConfig: Record = { + Type: "volume", + Source: input.target.name, + Target: "/hack-storage-witness", + ...(input.readonly ? { ReadOnly: true } : {}), + VolumeOptions: volumeOptions, + }; + const mounts: Record[] = [ + { + Type: "bind", + Source: program, + Destination: "/hack-storage-witness-helper.mjs", + RW: false, + Propagation: "rprivate", + }, + rootMount, + ]; + const value = { + id: "e".repeat(64), + createdAt, + image: input.artifact.imageId, + configImage: input.artifact.imageId, + user: "70:70", + entrypoint: ["/usr/local/bin/bun"], + cmd: ["--no-env-file", "/hack-storage-witness-helper.mjs"], + openStdin: true, + tty: false, + labels: { + [NATIVE_STORAGE_CARRIER_LABEL]: input.invocationId, + "io.hack.storage-witness.owner": input.target.ownerToken, + "io.hack.storage-witness.generation": input.scope.generationId, + "io.hack.storage-witness.helper": input.artifact.helperHash, + }, + host: { + Privileged: false, + ReadonlyRootfs: true, + NetworkMode: "none", + Memory: 268_435_456, + NanoCpus: 1_000_000_000, + PidsLimit: 32, + CapDrop: ["ALL"], + CapAdd: null, + SecurityOpt: ["no-new-privileges:true"], + LogConfig: { Type: "none", Config: {} }, + RestartPolicy: { Name: "no" }, + Binds: null, + Tmpfs: null, + Devices: [], + DeviceRequests: null, + PortBindings: {}, + OomKillDisable: false as false | null | true, + Mounts: [ + { + Type: "bind", + Source: program, + Target: "/hack-storage-witness-helper.mjs", + ReadOnly: true, + BindOptions: { + NonRecursive: true, + Propagation: "rprivate", + CreateMountpoint: false, + }, + }, + rootConfig, + ], + }, + mounts, + state: {}, + execIds: null as null | string[], + }; + return { program, value, rootMount, rootConfig, volumeOptions }; +} +test.each([ + "omitted", + "null", + "empty", +] as const)("Engine empty Tmpfs representation %s preserves the exact carrier projection", (representation) => { + const selected = { ...input(), uid: 0, gid: 0 }; + const fake = carrier(selected); + fake.value.user = "0:0"; + const host: Record = fake.value.host; + if (representation === "omitted") { + Reflect.deleteProperty(host, "Tmpfs"); + } else { + host.Tmpfs = representation === "null" ? null : {}; + } + // Exercise the serialized Engine shape, including both unchanged mount rows. + const value: unknown = JSON.parse(JSON.stringify(fake.value)); + const checked = checkNativeComposeStorageDockerCarrier({ + value, + input: selected, + program: fake.program, + imageIds: [selected.artifact.imageId], + }); + expect(checked.id).toBe(fake.value.id); + const policy = JSON.parse(nativeComposeStorageDockerCarrierPolicy(checked)); + expect(Object.hasOwn(policy.host, "Tmpfs")).toBe( + representation !== "omitted" + ); + expect(policy.mounts).toHaveLength(2); + expect(policy.host.Mounts).toHaveLength(2); +}); +test.each([ + { label: "mount", value: { "/extra": "rw" } }, + { label: "array", value: [] }, + { label: "string", value: "" }, + { label: "number", value: 0 }, + { label: "boolean", value: false }, + { label: "own undefined", value: undefined }, +])("Tmpfs $label cannot qualify a helper", ({ value }) => { + const selected = input(), + fake = carrier(selected); + const host: Record = fake.value.host; + host.Tmpfs = value; + expect(() => + checkNativeComposeStorageDockerCarrier({ + value: fake.value, + input: selected, + program: fake.program, + imageIds: [selected.artifact.imageId], + }) + ).toThrow("values omitted"); +}); + +test("helper args require the cached dependency, unchanged program bind and exact nocopy volume", () => { + const selected = input(), + fake = carrier(selected); + expect(nativeComposeStorageDockerHelper().length).toBeGreaterThan(1000); + const args = nativeComposeStorageDockerCreateArgs({ + input: selected, + program: fake.program, + }); + expect(args).toContain("never"); + expect(args).toContain("none"); + expect( + args + .filter((value) => value.startsWith("type=bind,")) + .every((value) => value.includes("readonly,bind-recursive=disabled")) + ).toBe(true); + expect(args.filter((value) => value.startsWith("type=bind,"))).toHaveLength( + 1 + ); + expect(args.filter((value) => value.startsWith("type=volume,"))).toEqual([ + `type=volume,src=${selected.target.name},dst=/hack-storage-witness,volume-nocopy,readonly`, + ]); + expect(args.some((value) => value.includes("bind-create-src"))).toBe(false); + checkNativeComposeStorageDockerCarrier({ + value: fake.value, + input: selected, + program: fake.program, + imageIds: [selected.artifact.imageId], + }); +}); +test.each([ + "empty", + "subset", + "duplicate", + "writable", + "copying", + "logging", +] as const)("physical/configured projection %s cannot qualify a helper", (failure) => { + const selected = input(), + fake = carrier(selected); + if (failure === "empty") { + fake.value.mounts = []; + } + if (failure === "subset") { + fake.value.mounts.pop(); + } + if (failure === "duplicate") { + fake.value.mounts[1] = fake.value.mounts[0]!; + } + if (failure === "writable") { + fake.value.mounts[1]!.RW = true; + } + if (failure === "copying") { + fake.volumeOptions.NoCopy = false; + } + if (failure === "logging") { + fake.value.host.LogConfig.Type = "json-file"; + } + expect(() => + checkNativeComposeStorageDockerCarrier({ + value: fake.value, + input: selected, + program: fake.program, + imageIds: [selected.artifact.imageId], + }) + ).toThrow("values omitted"); +}); +test("cold seed accepts only the Engine's omitted or explicit false writable default", () => { + const selected = { ...input(), readonly: false }; + const fake = carrier(selected); + const check = () => + checkNativeComposeStorageDockerCarrier({ + value: fake.value, + input: selected, + program: fake.program, + imageIds: [selected.artifact.imageId], + }); + expect(check().id).toBe(fake.value.id); + fake.rootConfig.ReadOnly = false; + expect(check().id).toBe(fake.value.id); + fake.rootConfig.ReadOnly = null; + expect(check).toThrow("values omitted"); + expect( + nativeComposeStorageDockerCreateArgs({ + input: selected, + program: fake.program, + }) + ).toContain( + `type=volume,src=${selected.target.name},dst=/hack-storage-witness,volume-nocopy` + ); +}); +test.each([ + "bind", + "name", + "source", + "driver", + "propagation", + "subpath", + "driver-options", + "missing-nocopy", + "missing-ro", +] as const)("named-volume projection %s cannot substitute the selected root", (failure) => { + const selected = input(), + fake = carrier(selected); + if (failure === "bind") { + fake.rootConfig.Type = "bind"; + fake.rootMount.Type = "bind"; + } + if (failure === "name") { + fake.rootMount.Name = "foreign_data"; + } + if (failure === "source") { + fake.rootMount.Source = "/var/lib/docker/volumes/foreign_data/_data"; + } + if (failure === "driver") { + fake.rootMount.Driver = "foreign"; + } + if (failure === "propagation") { + fake.rootMount.Propagation = "rslave"; + } + if (failure === "subpath") { + fake.volumeOptions.Subpath = "data"; + } + if (failure === "driver-options") { + fake.volumeOptions.DriverConfig = { Name: "local", Options: {} }; + } + if (failure === "missing-nocopy") { + Reflect.deleteProperty(fake.volumeOptions, "NoCopy"); + } + if (failure === "missing-ro") { + Reflect.deleteProperty(fake.rootConfig, "ReadOnly"); + } + expect(() => + checkNativeComposeStorageDockerCarrier({ + value: fake.value, + input: selected, + program: fake.program, + imageIds: [selected.artifact.imageId], + }) + ).toThrow("values omitted"); +}); +test("policy comparison preserves all physical fields/duplicates and closes only observed optional defaults", () => { + const selected = input(), + fake = carrier(selected); + const checked = () => + checkNativeComposeStorageDockerCarrier({ + value: fake.value, + input: selected, + program: fake.program, + imageIds: [selected.artifact.imageId], + }); + const before = nativeComposeStorageDockerCarrierPolicy(checked()); + fake.value.mounts.reverse(); + fake.value.host.OomKillDisable = null; + fake.value.execIds = []; + expect(nativeComposeStorageDockerCarrierPolicy(checked())).toBe(before); + fake.value.host.OomKillDisable = true; + expect(() => nativeComposeStorageDockerCarrierPolicy(checked())).toThrow(); +}); diff --git a/tests/native-compose-storage-witness-xattr-owner.test.ts b/tests/native-compose-storage-witness-xattr-owner.test.ts new file mode 100644 index 000000000..f25c23c32 --- /dev/null +++ b/tests/native-compose-storage-witness-xattr-owner.test.ts @@ -0,0 +1,1133 @@ +import { afterEach, expect, spyOn, test } from "bun:test"; +import { + mkdir, + mkdtemp, + readFile, + realpath, + rename, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + assertNativeComposeMaterialAuthority, + type NativeComposeGeneration, + type NativeComposeGenerationStore, + type NativeComposeMutation, + openNativeComposeGenerationStore, +} from "../src/lib/native-compose-generation.ts"; +import * as carrierJournal from "../src/lib/native-compose-storage-carrier-journal.ts"; +import { + enrollNativeComposeStorageXattrWitness, + type NativeComposeStorageWitnessEnrollment, + prepareNativeComposeStorageXattrWitness, + verifyNativeComposeStorageXattrWitness, +} from "../src/lib/native-compose-storage-witness.ts"; +import { + nativeComposeStorageWitnessReferenceValid, + nativeComposeStorageWitnessStatesValid, +} from "../src/lib/native-compose-storage-witness-state.ts"; +import { + captureNativeComposeStorageXattrCarrier, + type NativeComposeStorageXattrArtifact, + type NativeComposeStorageXattrInvocation, + type NativeComposeStorageXattrPorts, + type NativeComposeStorageXattrTarget, +} from "../src/lib/native-compose-storage-witness-xattr-carrier.ts"; +import { encodeNativeComposeStorageXattrResponse } from "../src/lib/native-compose-storage-witness-xattr-codec.ts"; +import { runNativeComposeStorageXattrHelper } from "../src/lib/native-compose-storage-witness-xattr-helper.ts"; + +const engineId = "d".repeat(64); +const volume = { + name: "owned_data", + storage: "data", + createdAt: "2026-10-08T12:00:00Z", +}; +const selection = { name: volume.name, storage: volume.storage }; +const artifact: NativeComposeStorageXattrArtifact = { + version: 1, + imageId: `sha256:${"a".repeat(64)}`, + platform: "linux/arm64", + bunVersion: "1.4.2", + bunHash: "b".repeat(64), + libcHash: "c".repeat(64), + helperHash: "e".repeat(64), + kernelAbi: 1, +}; +const roots: string[] = []; +const stores: NativeComposeGenerationStore[] = []; +afterEach(async () => { + await Promise.all(stores.splice(0).map((store) => store.close())); + await Promise.all( + roots.splice(0).map((path) => rm(path, { recursive: true, force: true })) + ); +}); +async function fixture() { + const root = await realpath( + await mkdtemp(join(tmpdir(), "native-xattr-owner-")) + ); + roots.push(root); + await mkdir(join(root, ".hack")); + await Bun.write( + join(root, ".hack", "hack.project.json"), + JSON.stringify({ schema_version: 1, name: "xattr" }) + ); + const store = await openNativeComposeGenerationStore({ + projectRoot: root, + instance: null, + }); + stores.push(store); + return store; +} +async function publish(mutation: NativeComposeMutation) { + const reservation = mutation.reserveGeneration(); + const labels = { + "io.hack.native-config.version": "1", + "io.hack.native-config.instance": reservation.identity.composeProject, + "io.hack.native-config.owner": reservation.identity.ownerToken, + }; + return await mutation.publish({ + reservation, + profiles: [], + inputRevision: "a".repeat(64), + assertFresh: async () => {}, + composeJson: JSON.stringify({ + name: reservation.identity.composeProject, + services: { + app: { + image: "synthetic:1", + labels: { + ...labels, + "io.hack.native-config.generation": reservation.generationId, + "io.hack.native-config.workload": "service", + }, + }, + }, + volumes: { + data: { + name: volume.name, + labels: { ...labels, "io.hack.native-config.storage": "data" }, + }, + }, + }), + }); +} +function fake(store: NativeComposeGenerationStore) { + const calls: string[] = []; + const attributes = new Map(); + const state = { + volume: null as typeof volume | null, + root: { device: "1", inode: "42", uid: 0, gid: 0 }, + holders: [] as NativeComposeStorageXattrTarget["holders"], + failure: null as + | "provision" + | "seed" + | "verify" + | "cleanup" + | "artifact" + | "pinned-root" + | "scope" + | "readonly" + | "uid" + | "replay" + | "no-created" + | "changed-created" + | "created-uncertain" + | null, + generation: null as NativeComposeGeneration | null, + afterInvoke: null as + | ((input: NativeComposeStorageXattrInvocation) => void) + | null, + }; + let nextId = 1; + const ports: NativeComposeStorageXattrPorts = { + async inspect(selection) { + calls.push("inspect"); + return { + engineId, + runtimeIdentity: store.identity.composeProject, + ownerToken: store.identity.ownerToken, + ...selection, + volume: state.volume, + mountpoint: state.volume + ? `/var/lib/docker/volumes/${selection.name}/_data` + : null, + driver: state.volume ? "local" : null, + options: state.volume ? {} : null, + holders: state.holders, + }; + }, + async provision() { + calls.push("provision"); + const saved = await store.loadCurrent(); + expect(saved.storageWitnessesPending).toBe(true); + expect(saved.storageWitnesses?.[0]).toMatchObject({ + state: "expected", + carrier: "directory-xattr", + artifact, + originalVolume: null, + }); + expect(saved.pending?.generationId).toBe(state.generation?.generationId); + if (state.failure === "provision") { + throw new Error("private create uncertainty"); + } + state.volume = { ...volume }; + }, + async invoke(input: NativeComposeStorageXattrInvocation) { + calls.push(input.request.operation); + if (state.failure === input.request.operation) { + throw new Error("private child uncertainty"); + } + const carrierId = (state.failure === "replay" ? 1 : nextId++) + .toString(16) + .padStart(64, "0"); + expect((await store.loadCurrent()).storageWitnessesPending).toBe(true); + if (state.failure !== "no-created") { + await input.recordCreated({ + id: carrierId, + createdAt: volume.createdAt, + }); + } + if (state.failure === "created-uncertain") { + throw new Error( + "private start uncertainty after exact create publication" + ); + } + const selectedRoot = + state.failure === "pinned-root" && input.request.operation === "verify" + ? { ...state.root, inode: "41" } + : state.root; + const response = runNativeComposeStorageXattrHelper({ + request: input.request, + kernel: { + effectiveUid: () => input.uid, + effectiveGid: () => input.gid, + probeRoot: () => selectedRoot, + openRoot: () => 42, + statRoot: () => selectedRoot, + createXattr: (_, name, bytes) => { + if (input.readonly || attributes.has(name)) { + throw new Error("EEXIST/EROFS"); + } + attributes.set(name, new Uint8Array(bytes)); + }, + syncRoot: () => {}, + readXattr: (_, name) => attributes.get(name) ?? new Uint8Array(), + closeRoot: () => {}, + }, + }); + state.afterInvoke?.(input); + return { + artifact: + state.failure === "artifact" + ? { ...artifact, helperHash: "f".repeat(64) } + : input.artifact, + carrierId, + carrierCreatedAt: + state.failure === "changed-created" + ? "2026-10-08T12:01:00Z" + : volume.createdAt, + engineId, + invocationId: input.invocationId, + readonly: + state.failure === "readonly" ? !input.readonly : input.readonly, + uid: state.failure === "uid" ? input.uid + 1 : input.uid, + gid: input.gid, + target: input.target, + scope: + state.failure === "scope" + ? { ...input.scope, pendingToken: "f".repeat(32) } + : input.scope, + response: encodeNativeComposeStorageXattrResponse(response), + outcome: "complete", + exitCode: response.outcome === "refused" ? 1 : 0, + stopped: { + id: carrierId, + running: false, + pid: 0, + exitCode: response.outcome === "refused" ? 1 : 0, + }, + containersAfterCleanup: state.failure === "cleanup" ? [carrierId] : [], + }; + }, + }; + const controller = new AbortController(); + const carrier = captureNativeComposeStorageXattrCarrier({ + artifact, + ports, + signal: controller.signal, + deadline: Date.now() + 60_000, + }); + return { state, ports, carrier, calls, attributes, controller }; +} +async function active() { + const store = await fixture(); + const transport = fake(store); + const { generation, reference } = await store.withMutation( + async (mutation) => { + const published = await publish(mutation); + transport.state.generation = published; + const result = await mutation.runEffect({ + generation: published, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + effect: async () => { + const enrollment = await prepareNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation: published, + engineId, + volume: selection, + admission: "initial-create", + assertAdmission: async () => {}, + carrier: transport.carrier, + }); + const enrolled = await enrollNativeComposeStorageXattrWitness({ + enrollment, + }); + transport.calls.push("workload"); + return { value: enrolled, outcome: "complete" }; + }, + }); + expect(result.outcome).toBe("complete"); + return { generation: published, reference: result.value }; + } + ); + return { store, transport, generation, reference }; +} + +test("durable tagged Expected precedes one cold provision/seed and fresh proof before workload", async () => { + const { store, transport, reference } = await active(); + expect(reference).toMatchObject({ + version: 3, + kind: "directory-xattr", + artifact, + volume, + }); + expect(transport.calls.filter((call) => call === "provision")).toHaveLength( + 1 + ); + expect(transport.calls.filter((call) => call === "seed")).toHaveLength(1); + expect(transport.calls.indexOf("provision")).toBeLessThan( + transport.calls.indexOf("seed") + ); + expect(transport.calls.indexOf("seed")).toBeLessThan( + transport.calls.indexOf("verify") + ); + expect(transport.calls.indexOf("verify")).toBeLessThan( + transport.calls.indexOf("workload") + ); + expect((await store.loadCurrent()).storageWitnesses?.[0]?.state).toBe( + "enrolled" + ); + expect((await store.loadCurrent()).pending).toBeNull(); + expect(JSON.stringify(await store.loadCurrent())).not.toContain( + "user.hack.storage." + ); +}); + +test("required xattr artifact/kind cannot be removed or relabeled as a legacy USTAR reference", async () => { + const { store, reference } = await active(); + expect(nativeComposeStorageWitnessReferenceValid(reference)).toBe(true); + if (reference.version !== 3) { + throw new Error("Missing xattr reference"); + } + const { artifact: ignoredArtifact, ...missingArtifact } = reference; + expect(nativeComposeStorageWitnessReferenceValid(missingArtifact)).toBe( + false + ); + expect( + nativeComposeStorageWitnessReferenceValid({ + ...reference, + kind: "file-ustar", + }) + ).toBe(false); + expect( + nativeComposeStorageWitnessReferenceValid({ ...reference, version: 1 }) + ).toBe(false); + expect( + nativeComposeStorageWitnessReferenceValid({ ...reference, version: 2 }) + ).toBe(false); + const { carrierJournalToken: ignoredJournal, ...missingJournal } = reference; + expect(nativeComposeStorageWitnessReferenceValid(missingJournal)).toBe(false); + expect(ignoredJournal).toMatch(/^[a-f0-9]{32}$/); + const states = (await store.loadCurrent()).storageWitnesses; + expect(nativeComposeStorageWitnessStatesValid(states)).toBe(true); + const tagged = states?.[0]; + if (!tagged || tagged.carrier !== "directory-xattr") { + throw new Error("Missing tagged state"); + } + const { + carrier: ignoredCarrier, + artifact: ignoredPins, + ...downgraded + } = tagged; + expect(nativeComposeStorageWitnessStatesValid([downgraded])).toBe(false); + expect( + nativeComposeStorageWitnessStatesValid([ + { ...tagged, artifact: { ...artifact, kernelAbi: 2 } }, + ]) + ).toBe(false); + // Keep the destructuring intentional: these fields are the adversarial deletion. + expect(ignoredArtifact).toEqual(artifact); + expect(ignoredCarrier).toBe("directory-xattr"); + expect(ignoredPins).toEqual(artifact); +}); + +test.each([ + "missing", + "birth", + "same-birth-empty", + "pinned-root", + "artifact", + "scope", + "cleanup", + "readonly", + "uid", + "replay", +] as const)("resume refuses %s before workload and never provisions/seeds", async (change) => { + const { store, transport, generation } = await active(); + transport.calls.length = 0; + if (change === "missing") { + transport.state.volume = null; + } else if (change === "birth") { + transport.state.volume = { ...volume, createdAt: "2026-10-08T12:00:01Z" }; + } else if (change === "same-birth-empty") { + transport.attributes.clear(); + } else { + transport.state.failure = change; + } + let workloads = 0; + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + effect: async () => { + workloads++; + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toThrow("values omitted"); + }); + expect(workloads).toBe(0); + expect(transport.calls).not.toContain("provision"); + expect(transport.calls).not.toContain("seed"); + expect((await store.loadCurrent()).storageWitnesses?.[0]?.state).toBe( + "enrolled" + ); +}); + +test("fresh readonly proofs tolerate observed UID change and exact owned running holders through run and final fences", async () => { + const { store, transport, generation, reference } = await active(); + transport.state.root.uid = 70; + transport.state.root.gid = 70; + transport.state.holders = [ + { + id: "f".repeat(64), + runtimeIdentity: store.identity.composeProject, + ownerToken: store.identity.ownerToken, + generationId: generation.generationId, + running: true, + }, + ]; + transport.calls.length = 0; + await store.withMutation(async (mutation) => { + await verifyNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation, + engineId, + reference, + carrier: transport.carrier, + }); + await mutation.runEffect({ + generation, + operation: "run", + assertOwned: async () => {}, + assertFresh: async () => {}, + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + beforeComplete: async () => { + transport.calls.push("finalizer"); + }, + effect: async () => { + transport.calls.push("workload"); + return { value: 0, outcome: "complete" }; + }, + }); + }); + expect( + transport.calls.filter((call) => call === "verify").length + ).toBeGreaterThanOrEqual(4); + expect(transport.calls.indexOf("verify")).toBeLessThan( + transport.calls.indexOf("workload") + ); + expect(transport.calls.lastIndexOf("verify")).toBeGreaterThan( + transport.calls.indexOf("finalizer") + ); + expect(transport.calls).not.toContain("provision"); + expect(transport.calls).not.toContain("seed"); +}); + +test.each([ + "provision", + "seed", + "verify", + "cleanup", +] as const)("cold %s uncertainty retains required Expected; saved stop never reenrolls", async (failure) => { + const store = await fixture(); + const transport = fake(store); + transport.state.failure = failure; + let enrollment: NativeComposeStorageWitnessEnrollment | null = null; + const generation = await store.withMutation(async (mutation) => { + const published = await publish(mutation); + transport.state.generation = published; + await expect( + mutation.runEffect({ + generation: published, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + effect: async () => { + enrollment = await prepareNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation: published, + engineId, + volume: selection, + admission: "initial-create", + assertAdmission: async () => {}, + carrier: transport.carrier, + }); + await enrollNativeComposeStorageXattrWitness({ enrollment }); + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toMatchObject({ code: "E_NATIVE_COMPOSE_UNCERTAIN" }); + return published; + }); + expect((await store.loadCurrent()).storageWitnesses?.[0]).toMatchObject({ + state: "expected", + carrier: "directory-xattr", + artifact, + }); + const attempts = transport.calls.filter( + (call) => call === "seed" || call === "provision" + ); + await expect( + enrollNativeComposeStorageXattrWitness({ enrollment: enrollment ?? {} }) + ).rejects.toThrow("values omitted"); + await store.withMutation(async (mutation) => { + expect( + await mutation.runEffect({ + generation, + operation: "down", + recoverPending: true, + assertOwned: async () => {}, + assertFresh: async () => {}, + effect: async () => ({ value: 0, outcome: "complete" }), + beforeComplete: async () => { + throw new Error("Must not retire"); + }, + }) + ).toEqual({ value: 0, outcome: "uncertain" }); + }); + expect( + transport.calls.filter((call) => call === "seed" || call === "provision") + ).toEqual(attempts); + expect((await store.loadCurrent()).storageWitnessesPending).toBe(true); + expect((await store.loadCurrent()).pending?.generationId).toBe( + generation.generationId + ); +}); + +test("post-workload marker loss refuses completion, retaining exact pending generation", async () => { + const { store, transport, generation } = await active(); + transport.calls.length = 0; + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + effect: async () => { + transport.calls.push("workload"); + transport.attributes.clear(); + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toMatchObject({ code: "E_NATIVE_COMPOSE_UNCERTAIN" }); + }); + expect(transport.calls).toContain("workload"); + expect((await store.loadCurrent()).pending?.generationId).toBe( + generation.generationId + ); + expect((await store.loadCurrent()).storageWitnesses?.[0]?.state).toBe( + "enrolled" + ); + expect(transport.calls).not.toContain("seed"); +}); + +test("same-birth root replacement while awaiting the final reader cannot publish ready from its pinned old directory", async () => { + const { store, transport, generation } = await active(); + let exchanged = false; + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + beforeComplete: async () => { + transport.state.afterInvoke = (input) => { + if (!exchanged && input.request.operation === "verify") { + exchanged = true; + transport.state.root = { ...transport.state.root, inode: "43" }; + transport.attributes.clear(); + } + }; + }, + effect: async () => ({ value: 0, outcome: "complete" }), + }) + ).rejects.toMatchObject({ code: "E_NATIVE_COMPOSE_UNCERTAIN" }); + }); + expect(exchanged).toBe(true); + expect(transport.state.volume).toEqual(volume); + expect((await store.loadCurrent()).pending?.generationId).toBe( + generation.generationId + ); + expect((await store.loadCurrent()).storageWitnesses?.[0]?.state).toBe( + "enrolled" + ); +}); + +test("known finite refused marker read clears only its proven absent helper intent and never repairs the witness", async () => { + const { store, transport, generation, reference } = await active(); + transport.attributes.clear(); + transport.calls.length = 0; + let workload = 0; + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "up", + assertFresh: async () => {}, + assertOwned: async () => {}, + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + effect: async () => { + workload++; + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toThrow("values omitted"); + }); + expect(workload).toBe(0); + expect(transport.attributes.size).toBe(0); + expect(transport.calls).not.toContain("seed"); + expect(transport.calls).not.toContain("provision"); + const failed = await store.loadCurrent(); + expect(failed.storageWitnessesPending).toBe(false); + expect(failed.pending).toBeNull(); + expect(failed.storageWitnesses?.[0]).toMatchObject({ + state: "enrolled", + reference, + }); + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "down", + recoverPending: true, + assertOwned: async () => {}, + effect: async () => { + workload++; + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toMatchObject({ code: "E_NATIVE_COMPOSE_STATE" }); + expect( + await mutation.runEffect({ + generation, + operation: "down", + assertFresh: async () => {}, + assertOwned: async () => {}, + effect: async () => ({ value: 0, outcome: "complete" }), + }) + ).toEqual({ value: 0, outcome: "complete" }); + }); + const stopped = await store.loadCurrent(); + expect(stopped.stopped).toBe(true); + expect(stopped.pending).toBeNull(); + expect(stopped.storageWitnesses?.[0]).toMatchObject({ + state: "enrolled", + reference, + }); +}); + +test.each([ + "no-created", + "changed-created", + "created-uncertain", + "cleanup", +] as const)("unknown enrolled helper %s blocks replay and saved retirement without changing the main material receipt", async (failure) => { + const { store, transport, generation, reference } = await active(); + transport.state.failure = failure; + transport.calls.length = 0; + let workload = 0; + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "run", + assertFresh: async () => {}, + assertOwned: async () => {}, + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + effect: async () => { + workload++; + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toThrow("values omitted"); + }); + expect(workload).toBe(0); + const unknown = await store.loadCurrent(); + expect(unknown.generation?.generationId).toBe(generation.generationId); + expect(unknown.pending).toBeNull(); + expect(unknown.storageWitnessesPending).toBe(true); + expect(unknown.storageWitnesses?.[0]).toMatchObject({ + state: "enrolled", + reference, + }); + const observed = [...transport.calls]; + await store.withMutation(async (mutation) => { + if (failure === "no-created") { + const foreignGeneration = await publish(mutation); + await expect( + mutation.runEffect({ + generation: foreignGeneration, + operation: "down", + recoverPending: true, + assertOwned: async () => {}, + effect: async () => { + workload++; + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toMatchObject({ code: "E_NATIVE_COMPOSE_STATE" }); + } + for (const operation of ["up", "down"] as const) { + await expect( + mutation.runEffect({ + generation, + operation, + assertFresh: async () => {}, + assertOwned: async () => {}, + effect: async () => { + workload++; + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toMatchObject({ code: "E_NATIVE_COMPOSE_UNCERTAIN" }); + } + const stopped = await mutation.runEffect({ + generation, + operation: "down", + recoverPending: true, + assertFresh: async () => {}, + assertOwned: async () => {}, + effect: async () => { + workload++; + return { value: 0, outcome: "complete" }; + }, + beforeComplete: async () => { + throw new Error("Unknown helper cannot retire claims"); + }, + }); + expect(stopped).toEqual({ value: 0, outcome: "uncertain" }); + }); + expect(workload).toBe(1); + expect(transport.calls).toEqual(observed); + const retained = await store.loadCurrent(); + expect(retained.pending?.generationId).toBe(generation.generationId); + expect(retained.storageWitnessesPending).toBe(true); + expect(retained.storageWitnesses?.[0]).toMatchObject({ + state: "enrolled", + reference, + }); +}); + +test("retirement rereads the original receipt after its final carrier-journal await", async () => { + const { store, generation } = await active(); + const original = carrierJournal.nativeComposeStorageCarriersPending; + let pause = false; + let enter = () => {}; + let release = () => {}; + const entered = new Promise((resolve) => { + enter = resolve; + }); + const resumed = new Promise((resolve) => { + release = resolve; + }); + const probe = spyOn( + carrierJournal, + "nativeComposeStorageCarriersPending" + ).mockImplementation(async (opts) => { + if (pause) { + pause = false; + enter(); + await resumed; + } + return await original(opts); + }); + try { + await store.withMutation(async (mutation) => { + expect( + await mutation.runEffect({ + generation, + operation: "down", + assertOwned: async () => {}, + effect: async () => { + pause = true; + const retirement = assertNativeComposeMaterialAuthority({ + authority: mutation.materialAuthority, + generation, + phase: "retire", + }); + await entered; + // Adversarial substitution only in this isolated synthetic store: same + // complete bytes, different inode while the final journal probe awaits. + const path = join( + store.identity.checkoutRoot, + ".hack", + ".internal", + "native-compose", + store.identity.instanceId, + "receipt.json" + ); + const bytes = await readFile(path); + await writeFile(`${path}.replacement`, bytes, { + mode: 0o600, + flag: "wx", + }); + await rename(`${path}.replacement`, path); + release(); + await expect(retirement).rejects.toMatchObject({ + code: "E_NATIVE_COMPOSE_STATE", + }); + return { value: 0, outcome: "uncertain" }; + }, + }) + ).toEqual({ value: 0, outcome: "uncertain" }); + }); + } finally { + release(); + probe.mockRestore(); + } +}); + +test("captured artifacts/ports cannot be substituted and cancelled lifetime stops before engine callbacks", async () => { + const store = await fixture(); + const transport = fake(store); + const originalInspect = transport.ports.inspect; + let substituted = 0; + const mutable = { + artifact: { ...artifact }, + ports: { ...transport.ports }, + signal: transport.controller.signal, + deadline: Date.now() + 60_000, + }; + const carrier = captureNativeComposeStorageXattrCarrier(mutable); + mutable.artifact.helperHash = "f".repeat(64); + mutable.ports.inspect = async () => { + substituted++; + return null; + }; + mutable.ports.invoke = async () => { + substituted++; + return null; + }; + await store.withMutation(async (mutation) => { + const generation = await publish(mutation); + transport.state.generation = generation; + await mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + effect: async () => { + const enrollment = await prepareNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation, + engineId, + volume: selection, + admission: "initial-create", + assertAdmission: async () => {}, + carrier, + }); + await enrollNativeComposeStorageXattrWitness({ enrollment }); + return { value: 0, outcome: "complete" }; + }, + }); + }); + expect(substituted).toBe(0); + expect(transport.ports.inspect).toBe(originalInspect); + transport.controller.abort(); + transport.calls.length = 0; + const current = await store.loadCurrent(); + const generation = current.generation; + if (!generation) { + throw new Error("Missing generation"); + } + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + storageWitnesses: { kind: "directory-xattr", engineId, carrier }, + effect: async () => ({ value: 0, outcome: "complete" }), + }) + ).rejects.toThrow("values omitted"); + }); + expect(transport.calls).toEqual([]); +}); + +test("prepare captures original carrier, selection and admission callback before its first await", async () => { + const store = await fixture(); + const transport = fake(store); + let substituted = 0; + const poison = captureNativeComposeStorageXattrCarrier({ + artifact, + signal: transport.controller.signal, + deadline: Date.now() + 60_000, + ports: { + inspect: async () => { + substituted++; + return null; + }, + invoke: async () => { + substituted++; + return null; + }, + }, + }); + await store.withMutation(async (mutation) => { + const generation = await publish(mutation); + transport.state.generation = generation; + await mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + effect: async () => { + const input = { + authority: mutation.materialAuthority, + generation, + engineId, + volume: { ...selection }, + admission: "initial-create" as const, + carrier: transport.carrier, + assertAdmission: async () => { + input.carrier = poison; + input.engineId = "wrong-engine"; + input.volume.name = "wrong_name"; + input.assertAdmission = async () => { + substituted++; + }; + }, + }; + const enrollment = await prepareNativeComposeStorageXattrWitness(input); + await enrollNativeComposeStorageXattrWitness({ enrollment }); + return { value: 0, outcome: "complete" }; + }, + }); + }); + expect(substituted).toBe(0); + expect((await store.loadCurrent()).storageWitnesses?.[0]).toMatchObject({ + name: volume.name, + engineId, + state: "enrolled", + carrier: "directory-xattr", + }); +}); + +test("explicit stopped adoption pins the original birth and seeds once without cold provisioning", async () => { + const store = await fixture(); + const transport = fake(store); + transport.state.volume = { ...volume }; + await store.withMutation(async (mutation) => { + const generation = await publish(mutation); + transport.state.generation = generation; + await mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + effect: async () => { + const enrollment = await prepareNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation, + engineId, + volume: selection, + admission: "explicit-adoption", + originalVolume: volume, + assertAdmission: async () => {}, + carrier: transport.carrier, + }); + await enrollNativeComposeStorageXattrWitness({ enrollment }); + return { value: 0, outcome: "complete" }; + }, + }); + }); + expect(transport.calls).not.toContain("provision"); + expect(transport.calls.filter((call) => call === "seed")).toHaveLength(1); + expect((await store.loadCurrent()).storageWitnesses?.[0]).toMatchObject({ + admission: "explicit-adoption", + originalVolume: volume, + state: "enrolled", + }); +}); + +test("initial admission refuses a preexisting volume before required intent, provisioning or seed", async () => { + const store = await fixture(); + const transport = fake(store); + transport.state.volume = { ...volume }; + await store.withMutation(async (mutation) => { + const generation = await publish(mutation); + transport.state.generation = generation; + await expect( + mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + effect: async () => { + await prepareNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation, + engineId, + volume: selection, + admission: "initial-create", + assertAdmission: async () => {}, + carrier: transport.carrier, + }); + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toMatchObject({ code: "E_NATIVE_COMPOSE_UNCERTAIN" }); + }); + expect(transport.calls).toEqual(["inspect"]); + expect((await store.loadCurrent()).storageWitnesses).toBeNull(); +}); + +test.each([ + "generation", + "owner", + "runtime", +] as const)("readonly admission refuses a foreign %s holder before helper or workload", async (change) => { + const { store, transport, generation } = await active(); + const holder = { + id: "f".repeat(64), + runtimeIdentity: store.identity.composeProject, + ownerToken: store.identity.ownerToken, + generationId: generation.generationId, + running: true, + }; + if (change === "generation") { + holder.generationId = "f".repeat(32); + } + if (change === "owner") { + holder.ownerToken = "f".repeat(32); + } + if (change === "runtime") { + holder.runtimeIdentity = "foreign"; + } + transport.state.holders = [holder]; + transport.calls.length = 0; + let workloads = 0; + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "up", + assertOwned: async () => {}, + assertFresh: async () => {}, + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + effect: async () => { + workloads++; + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toThrow("values omitted"); + }); + expect(workloads).toBe(0); + expect(transport.calls).toEqual(["inspect"]); +}); + +test("cancellation during readonly carrier await refuses before workload without seed or repair", async () => { + const { store, transport, generation } = await active(); + transport.calls.length = 0; + transport.state.afterInvoke = (input) => { + if (input.request.operation === "verify") { + transport.controller.abort(); + } + }; + let workloads = 0; + await store.withMutation(async (mutation) => { + await expect( + mutation.runEffect({ + generation, + operation: "run", + assertOwned: async () => {}, + assertFresh: async () => {}, + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + effect: async () => { + workloads++; + return { value: 0, outcome: "complete" }; + }, + }) + ).rejects.toThrow("values omitted"); + }); + expect(workloads).toBe(0); + expect(transport.calls.filter((call) => call === "verify")).toHaveLength(1); + expect(transport.calls).not.toContain("seed"); + expect(transport.calls).not.toContain("provision"); +}); diff --git a/tests/native-compose-storage-witness-xattr.test.ts b/tests/native-compose-storage-witness-xattr.test.ts new file mode 100644 index 000000000..aa0501fd4 --- /dev/null +++ b/tests/native-compose-storage-witness-xattr.test.ts @@ -0,0 +1,531 @@ +import { describe, expect, test } from "bun:test"; +import { + createNativeComposeStorageXattrMarker, + decodeNativeComposeStorageXattrRequest, + decodeNativeComposeStorageXattrResponse, + encodeNativeComposeStorageXattrRequest, + encodeNativeComposeStorageXattrResponse, + NATIVE_STORAGE_XATTR_KIND, + NATIVE_STORAGE_XATTR_VERSION, + type NativeComposeStorageXattrRequest, + type NativeComposeStorageXattrRoot, + parseNativeComposeStorageXattrRequest, + parseNativeComposeStorageXattrResponse, +} from "../src/lib/native-compose-storage-witness-xattr-codec.ts"; +import { + type NativeComposeStorageXattrKernel, + runNativeComposeStorageXattrHelper, +} from "../src/lib/native-compose-storage-witness-xattr-helper.ts"; +import { NATIVE_STORAGE_XATTR_LINUX_ABI } from "../src/lib/native-compose-storage-witness-xattr-linux.ts"; + +const root: NativeComposeStorageXattrRoot = { + device: "1234567890123456789", + inode: "9876543210123456789", + uid: 0, + gid: 0, +}; +const marker = { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + name: `user.hack.storage.${"a".repeat(64)}`, + valueHex: "b".repeat(64), +}; +function request( + operation: "seed" | "verify" +): NativeComposeStorageXattrRequest { + return { ...marker, operation, root }; +} +function fake() { + const calls: string[] = []; + const attributes = new Map(); + const state = { + root: { ...root }, + uid: 0, + gid: 0, + readonly: false, + }; + const kernel: NativeComposeStorageXattrKernel = { + effectiveUid: () => state.uid, + effectiveGid: () => state.gid, + probeRoot() { + calls.push("probe"); + return state.root; + }, + openRoot() { + calls.push("open"); + return 42; + }, + statRoot(fd) { + expect(fd).toBe(42); + calls.push("stat"); + return state.root; + }, + createXattr(fd, name, value) { + expect(fd).toBe(42); + calls.push("create"); + if (state.readonly || attributes.has(name)) { + throw new Error("EEXIST/EROFS private diagnostic must not escape"); + } + attributes.set(name, new Uint8Array(value)); + }, + syncRoot(fd) { + expect(fd).toBe(42); + calls.push("sync"); + }, + readXattr(fd, name, bytes) { + expect(fd).toBe(42); + expect(bytes).toBe(32); + calls.push("read"); + const value = attributes.get(name); + if (!value) { + throw new Error(`ENODATA ${name} private diagnostic`); + } + return new Uint8Array(value); + }, + closeRoot(fd) { + expect(fd).toBe(42); + calls.push("close"); + }, + }; + return { kernel, calls, attributes, state }; +} +function refused(value: unknown) { + expect(value).toEqual({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "refused", + }); + expect(JSON.stringify(value)).not.toContain(marker.name); + expect(JSON.stringify(value)).not.toContain(marker.valueHex); +} + +describe("distinct bounded directory-xattr codec", () => { + test("Linux open flags retain the separate arm64 and x64 UAPI contracts without loading libc", () => { + // Linux open does not set FD_CLOEXEC unless the initial flags request it. + expect(NATIVE_STORAGE_XATTR_LINUX_ABI.openCloseExec).toBe(524_288); + expect(NATIVE_STORAGE_XATTR_LINUX_ABI.fGetFd).toBe(1); + expect(NATIVE_STORAGE_XATTR_LINUX_ABI.fdCloseExec).toBe(1); + expect(NATIVE_STORAGE_XATTR_LINUX_ABI.arm64).toEqual({ + libc: "/lib/aarch64-linux-gnu/libc.so.6", + directory: 16_384, + noFollow: 32_768, + }); + expect(NATIVE_STORAGE_XATTR_LINUX_ABI.x64).toEqual({ + libc: "/lib/x86_64-linux-gnu/libc.so.6", + directory: 65_536, + noFollow: 131_072, + }); + }); + test("canonical requests and responses retain 64-bit root facts without number rounding", () => { + const selected = request("verify"); + expect( + parseNativeComposeStorageXattrRequest( + encodeNativeComposeStorageXattrRequest(selected) + ) + ).toEqual(selected); + const response = { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "verified" as const, + root, + valueHex: marker.valueHex, + }; + expect( + parseNativeComposeStorageXattrResponse( + encodeNativeComposeStorageXattrResponse(response) + ) + ).toEqual(response); + }); + test("metadata discovery contains neither marker bytes nor enrollment authority", () => { + const discovery = { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + operation: "root" as const, + }; + expect( + parseNativeComposeStorageXattrRequest( + encodeNativeComposeStorageXattrRequest(discovery) + ) + ).toEqual(discovery); + const selected = fake(); + const result = runNativeComposeStorageXattrHelper({ + request: discovery, + kernel: selected.kernel, + }); + expect(result).toEqual({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "root", + root, + }); + expect(selected.calls).toEqual(["probe", "probe"]); + }); + test("random names and values are independent opaque 256-bit selections", () => { + const first = createNativeComposeStorageXattrMarker(); + const second = createNativeComposeStorageXattrMarker(); + expect(first.name).not.toBe(second.name); + expect(first.valueHex).not.toBe(second.valueHex); + expect(first.name.endsWith(first.valueHex)).toBe(false); + expect(Object.isFrozen(first)).toBe(true); + }); + test.each([ + ["file kind", { ...request("seed"), kind: "regular-file" }], + [ + "USTAR leaf", + { ...request("seed"), name: `.hack-storage-${"a".repeat(64)}.witness` }, + ], + ["unknown version", { ...request("seed"), version: 2 }], + ["repair operation", { ...request("seed"), operation: "replace" }], + ["extra path", { ...request("seed"), path: "/etc/passwd" }], + ["name NUL", { ...request("seed"), name: `${marker.name}\0suffix` }], + ["short value", { ...request("seed"), valueHex: "a".repeat(62) }], + ["uppercase value", { ...request("seed"), valueHex: "B".repeat(64) }], + ["negative uid", { ...request("seed"), root: { ...root, uid: -1 } }], + ["negative zero uid", { ...request("seed"), root: { ...root, uid: -0 } }], + [ + "oversized uid", + { ...request("seed"), root: { ...root, uid: 4_294_967_296 } }, + ], + ["float uid", { ...request("seed"), root: { ...root, uid: 1.5 } }], + ["number inode", { ...request("seed"), root: { ...root, inode: 42 } }], + ["zero inode", { ...request("seed"), root: { ...root, inode: "0" } }], + [ + "rounded inode", + { ...request("seed"), root: { ...root, inode: "18446744073709551616" } }, + ], + ["leading zero", { ...request("seed"), root: { ...root, device: "001" } }], + [ + "extra root field", + { ...request("seed"), root: { ...root, path: "/private" } }, + ], + ])("refuses %s before any syscall", (_, value) => { + expect(() => decodeNativeComposeStorageXattrRequest(value)).toThrow( + "values omitted" + ); + const selected = fake(); + refused( + runNativeComposeStorageXattrHelper({ + request: value, + kernel: selected.kernel, + }) + ); + expect(selected.calls).toEqual([]); + }); + test.each([ + `{"kind":"directory-xattr","version":1,"operation":"root","operation":"root"}\n`, + `{"kind":"directory-xattr","version":1,"operation":"root"}\n\n`, + `${encodeNativeComposeStorageXattrRequest(request("verify"))}suffix`, + " ".repeat(4097), + ])("refuses noncanonical input without printing it", (text) => { + expect(() => parseNativeComposeStorageXattrRequest(text)).toThrow( + "values omitted" + ); + }); + test("unknown responses cannot turn metadata discovery into marker proof", () => { + expect(() => + decodeNativeComposeStorageXattrResponse({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "verified", + root, + }) + ).toThrow("values omitted"); + expect(() => + decodeNativeComposeStorageXattrResponse({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "refused", + error: marker.valueHex, + }) + ).toThrow("values omitted"); + }); +}); + +describe("offline syscall boundary, no Linux helper execution", () => { + test("seed performs one exclusive creation, synchronization and root recheck", () => { + const selected = fake(); + const result = runNativeComposeStorageXattrHelper({ + request: request("seed"), + kernel: selected.kernel, + }); + expect(result.outcome).toBe("seeded"); + expect(selected.calls).toEqual([ + "open", + "stat", + "create", + "sync", + "stat", + "close", + ]); + expect(selected.attributes.get(marker.name)).toEqual( + Buffer.from(marker.valueHex, "hex") + ); + expect(result).not.toHaveProperty("valueHex"); + }); + test.each([ + new Uint8Array(), + Buffer.from("c".repeat(64), "hex"), + Buffer.from(marker.valueHex, "hex"), + ])("existing empty, wrong or matching attribute is never overwritten", (existing) => { + const selected = fake(); + selected.attributes.set(marker.name, new Uint8Array(existing)); + refused( + runNativeComposeStorageXattrHelper({ + request: request("seed"), + kernel: selected.kernel, + }) + ); + expect(selected.attributes.get(marker.name)).toEqual(existing); + expect(selected.calls).toEqual(["open", "stat", "create", "close"]); + }); + test("read-only verification never creates, synchronizes or repairs storage", () => { + const selected = fake(); + selected.state.readonly = true; + selected.attributes.set(marker.name, Buffer.from(marker.valueHex, "hex")); + const result = runNativeComposeStorageXattrHelper({ + request: request("verify"), + kernel: selected.kernel, + }); + expect(result).toEqual({ + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + outcome: "verified", + root, + valueHex: marker.valueHex, + }); + expect(selected.calls).toEqual([ + "open", + "stat", + "read", + "stat", + "read", + "stat", + "close", + ]); + }); + test("an enrolled-token stand-in refuses an empty replacement with every observed metadata field unchanged", () => { + const metadata = Object.freeze({ + name: "owned-data", + storage: "data", + createdAt: "2026-10-08T12:00:00Z", + owner: "a".repeat(32), + root: { ...root }, + }); + const selected = fake(); + expect( + runNativeComposeStorageXattrHelper({ + request: request("seed"), + kernel: selected.kernel, + }).outcome + ).toBe("seeded"); + expect( + runNativeComposeStorageXattrHelper({ + request: request("verify"), + kernel: selected.kernel, + }).outcome + ).toBe("verified"); + const replacementMetadata = structuredClone(metadata); + selected.attributes.clear(); + selected.state.readonly = true; + selected.calls.length = 0; + expect(replacementMetadata).toEqual(metadata); + refused( + runNativeComposeStorageXattrHelper({ + request: request("verify"), + kernel: selected.kernel, + }) + ); + expect(selected.calls).toEqual(["open", "stat", "read", "close"]); + expect(selected.attributes.size).toBe(0); + }); + test.each([ + undefined, + new Uint8Array(), + Buffer.from("c".repeat(64), "hex"), + new Uint8Array(33), + ])("same name/labels/birth with absent, empty, wrong or oversized bytes refuses without seed", (replacement) => { + const selected = fake(); + selected.state.readonly = true; + if (replacement) { + selected.attributes.set(marker.name, replacement); + } + refused( + runNativeComposeStorageXattrHelper({ + request: request("verify"), + kernel: selected.kernel, + }) + ); + expect(selected.calls).not.toContain("create"); + expect(selected.calls).not.toContain("sync"); + expect(selected.calls.at(-1)).toBe("close"); + }); + test("a post-read marker change is detected by a second bounded read", () => { + const selected = fake(); + selected.attributes.set(marker.name, Buffer.from(marker.valueHex, "hex")); + const original = selected.kernel.readXattr; + let reads = 0; + const kernel = { + ...selected.kernel, + readXattr(fd: number, name: string, bytes: number) { + const value = original(fd, name, bytes); + if (++reads === 1) { + selected.attributes.set(name, Buffer.from("c".repeat(64), "hex")); + } + return value; + }, + }; + refused( + runNativeComposeStorageXattrHelper({ request: request("verify"), kernel }) + ); + expect(reads).toBe(2); + expect(selected.calls).not.toContain("create"); + }); + test("replacement behind a pinned root and UID drift refuse", () => { + const selected = fake(); + selected.attributes.set(marker.name, Buffer.from(marker.valueHex, "hex")); + const original = selected.kernel.readXattr; + const kernel = { + ...selected.kernel, + readXattr(fd: number, name: string, bytes: number) { + const value = original(fd, name, bytes); + selected.state.root.inode = "54321"; + selected.state.root.uid = 70; + return value; + }, + }; + refused( + runNativeComposeStorageXattrHelper({ request: request("verify"), kernel }) + ); + expect(selected.calls).toEqual(["open", "stat", "read", "stat", "close"]); + }); + test("cold UID0 then Postgres70 requires explicit matching credentials; actual permissions are unqualified", () => { + const selected = fake(); + expect( + runNativeComposeStorageXattrHelper({ + request: request("seed"), + kernel: selected.kernel, + }).outcome + ).toBe("seeded"); + selected.state.root.uid = 70; + selected.state.root.gid = 70; + const postStart = { + ...request("verify"), + root: { ...selected.state.root }, + }; + selected.calls.length = 0; + refused( + runNativeComposeStorageXattrHelper({ + request: postStart, + kernel: selected.kernel, + }) + ); + expect(selected.calls).toEqual([]); + selected.state.uid = 70; + selected.state.gid = 70; + expect( + runNativeComposeStorageXattrHelper({ + request: postStart, + kernel: selected.kernel, + }).outcome + ).toBe("verified"); + expect(selected.calls).not.toContain("create"); + }); + test("synchronization failure retains the attempted value without a success or repair", () => { + const selected = fake(); + const kernel = { + ...selected.kernel, + syncRoot() { + throw new Error(marker.valueHex); + }, + }; + refused( + runNativeComposeStorageXattrHelper({ request: request("seed"), kernel }) + ); + expect(selected.attributes.get(marker.name)).toEqual( + Buffer.from(marker.valueHex, "hex") + ); + expect(selected.calls).toEqual(["open", "stat", "create", "close"]); + }); + test("kernel callback substitution cannot replace captured seed authority", () => { + const selected = fake(); + let substituted = 0; + const kernel = { + ...selected.kernel, + effectiveUid() { + kernel.createXattr = () => { + substituted++; + }; + return 0; + }, + }; + expect( + runNativeComposeStorageXattrHelper({ request: request("seed"), kernel }) + .outcome + ).toBe("seeded"); + expect(substituted).toBe(0); + expect(selected.attributes.has(marker.name)).toBe(true); + }); + test("root drift before seed prevents creation and closes the descriptor", () => { + const selected = fake(); + selected.state.root.inode = "12345"; + refused( + runNativeComposeStorageXattrHelper({ + request: request("seed"), + kernel: selected.kernel, + }) + ); + expect(selected.calls).toEqual(["open", "stat", "close"]); + expect(selected.attributes.size).toBe(0); + }); + test("read-only mount refuses a seed attempt without synchronizing or repairing", () => { + const selected = fake(); + selected.state.readonly = true; + refused( + runNativeComposeStorageXattrHelper({ + request: request("seed"), + kernel: selected.kernel, + }) + ); + expect(selected.calls).toEqual(["open", "stat", "create", "close"]); + expect(selected.attributes.size).toBe(0); + }); + test("close failure does not publish a successful token proof", () => { + const selected = fake(); + selected.attributes.set(marker.name, Buffer.from(marker.valueHex, "hex")); + const kernel = { + ...selected.kernel, + closeRoot() { + throw new Error(marker.valueHex); + }, + }; + refused( + runNativeComposeStorageXattrHelper({ request: request("verify"), kernel }) + ); + expect(selected.calls).not.toContain("create"); + }); + test("metadata discovery drift is a refusal, never an unverified UID switch", () => { + const selected = fake(); + let probes = 0; + const kernel = { + ...selected.kernel, + probeRoot() { + if (++probes === 2) { + selected.state.root.uid = 70; + } + return selected.state.root; + }, + }; + refused( + runNativeComposeStorageXattrHelper({ + request: { + kind: NATIVE_STORAGE_XATTR_KIND, + version: NATIVE_STORAGE_XATTR_VERSION, + operation: "root", + }, + kernel, + }) + ); + expect(probes).toBe(2); + expect(selected.attributes.size).toBe(0); + }); +}); diff --git a/tests/shell-private-io.test.ts b/tests/shell-private-io.test.ts new file mode 100644 index 000000000..2b27c3a16 --- /dev/null +++ b/tests/shell-private-io.test.ts @@ -0,0 +1,71 @@ +import { expect, test } from "bun:test"; +import { constants, fstatSync } from "node:fs"; +import { mkdtemp, open, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { run } from "../src/lib/shell.ts"; + +test("private descriptor capture precedes spawn admission and run does not close caller FDs", async () => { + const root = await mkdtemp(join(tmpdir(), "shell-held-io-")); + const inputPath = join(root, "input"), + outPath = join(root, "output"), + errPath = join(root, "error"), + otherPath = join(root, "other"); + await Promise.all([ + writeFile(inputPath, "captured input"), + writeFile(outPath, ""), + writeFile(errPath, ""), + writeFile(otherPath, ""), + ]); + const input = await open(inputPath, constants.O_RDONLY), + output = await open(outPath, constants.O_RDWR), + error = await open(errPath, constants.O_RDWR), + other = await open(otherPath, constants.O_RDWR); + const privateIo = { stdin: input.fd, stdout: output.fd, stderr: error.fd }; + try { + expect( + await run(["/bin/cat"], { + privateIo, + timeoutMs: 3000, + beforeSpawn: () => { + privateIo.stdout = other.fd; + }, + }) + ).toBe(0); + expect(await readFile(outPath, "utf8")).toBe("captured input"); + expect(await readFile(otherPath, "utf8")).toBe(""); + expect(fstatSync(input.fd).isFile()).toBe(true); + expect(fstatSync(output.fd).isFile()).toBe(true); + } finally { + await Promise.all([ + input.close(), + output.close(), + error.close(), + other.close(), + ]); + await rm(root, { recursive: true, force: true }); + } +}); +test.each([ + "negative", + "fractional", + "unbounded", + "tty", +] as const)("private IO %s refuses before admission", async (failure) => { + let admitted = false; + await expect( + run(["/bin/cat"], { + privateIo: { + stdin: failure === "negative" ? -1 : failure === "fractional" ? 1.5 : 0, + stdout: 1, + stderr: 2, + }, + timeoutMs: failure === "unbounded" ? undefined : 3000, + forwardSignals: failure === "tty", + beforeSpawn: () => { + admitted = true; + }, + }) + ).rejects.toThrow("Private subprocess descriptors"); + expect(admitted).toBe(false); +}); diff --git a/tests/worktree-git-inspection.test.ts b/tests/worktree-git-inspection.test.ts new file mode 100644 index 000000000..8c2a77829 --- /dev/null +++ b/tests/worktree-git-inspection.test.ts @@ -0,0 +1,116 @@ +import { expect, spyOn, test } from "bun:test"; +import { readGitInspection } from "../src/lib/worktree-local-config.ts"; + +test("Git inspection refuses changed final source admission before spawning", async () => { + const child = spyOn(Bun, "spawn").mockImplementation(() => { + throw new Error("Unexpected child"); + }); + try { + const output = await readGitInspection({ + projectRoot: "/synthetic-owned-source", + args: ["rev-parse", "HEAD"], + beforeSpawn: () => { + throw new Error("Source changed"); + }, + }); + expect(output).toBeNull(); + expect(child).not.toHaveBeenCalled(); + } finally { + child.mockRestore(); + } +}); + +test("Git inspection observes cancellation at the final source admission", async () => { + const controller = new AbortController(); + const child = spyOn(Bun, "spawn").mockImplementation(() => { + throw new Error("Unexpected child"); + }); + try { + await expect( + readGitInspection({ + projectRoot: "/synthetic-owned-source", + args: ["rev-parse", "HEAD"], + signal: controller.signal, + beforeSpawn: () => { + controller.abort(); + }, + }) + ).rejects.toMatchObject({ code: "E_COMPILER_CANCELLED" }); + expect(child).not.toHaveBeenCalled(); + } finally { + child.mockRestore(); + } +}); + +test("Git inspection drains peer-held stdout after its spawn observer throws", async () => { + let readerAcquired = false; + let eofObserved = false; + let released = false; + let readCount = 0; + let releasePeer: () => void = () => {}; + const peerClosed = new Promise((resolve) => { + releasePeer = resolve; + }); + const stdout = { + getReader: () => { + readerAcquired = true; + return { + read: async () => { + readCount++; + if (readCount === 1) { + return { + done: false, + value: new TextEncoder().encode("owned-head"), + }; + } + await peerClosed; + eofObserved = true; + return { done: true, value: undefined }; + }, + releaseLock: () => { + released = true; + }, + }; + }, + }; + const child = spyOn(Bun, "spawn").mockReturnValue({ + pid: 123, + stdout, + exitCode: 0, + exited: Promise.resolve(0), + } as unknown as ReturnType); + try { + const output = await readGitInspection({ + projectRoot: "/synthetic-owned-source", + args: ["rev-parse", "HEAD"], + onSpawn: () => { + expect(readerAcquired).toBe(true); + queueMicrotask(releasePeer); + throw new Error("Observation refused"); + }, + }); + expect(output).toBeNull(); + expect(eofObserved).toBe(true); + expect(released).toBe(true); + expect(child).toHaveBeenCalledTimes(1); + } finally { + child.mockRestore(); + } +}); + +test("Git inspection refuses a Promise from final admission before spawning", async () => { + const child = spyOn(Bun, "spawn").mockImplementation(() => { + throw new Error("Unexpected child"); + }); + try { + const output = await readGitInspection({ + projectRoot: "/synthetic-owned-source", + args: ["rev-parse", "HEAD"], + beforeSpawn: (() => Promise.resolve()) as unknown as () => undefined, + }); + expect(output).toBeNull(); + expect(child).not.toHaveBeenCalled(); + } finally { + child.mockRestore(); + } +});