diff --git a/packages/runtime-core/README.md b/packages/runtime-core/README.md index 2ed03adad..269cee130 100644 --- a/packages/runtime-core/README.md +++ b/packages/runtime-core/README.md @@ -8,7 +8,7 @@ provider, image acquisition or receipt effects. This feature's compiler path dependency requires the repository's pinned Rust 1.97.1. The default runtime package keeps its declared Rust 1.85 minimum; enabling the adapter requires Rust 1.97.1. -The adapter accepts at most 32 selected image-only services/jobs, exec readiness, +The adapter accepts at most 32 selected services/jobs with immutable images, exec readiness, explicit exec/shell commands, entrypoint clearing, init, exact shutdown intent and working directories. Jobs require successful completion; services with readiness require health, and other services require startup. Workload names, including dots, @@ -23,7 +23,8 @@ The adapter remaps those values into a separate destination-keyed map. Public literals/defaults remain separate; private values never affect portable identities. Outputs deliberately lack `Debug` and `Serialize`. Encoded private values are bounded to 32 KiB and 256 destination keys per workload; process argv is bounded -to 4096 arguments and 64 KiB. Build/acquisition policy, mounts, storage, routing, +to 4096 arguments and 64 KiB. Persistent worktree storage mounts are a separate +receipt-v4 path described below. Build/acquisition policy, source/file mounts, routing, endpoints, host effects, HTTP/TCP readiness and automatic restart explicitly refuse. `project::native::review` validates the same subset without acquiring private values. @@ -54,7 +55,7 @@ namespace, image/source/provider admission and effect-time deadline, integrate a native runs into common capacity/inventory, and implement tagged native ownership and recovery. These preparation artifacts carry no resource ownership or replay authority. -`provider::graph::native::configuration` lowers freshly prepared image-only input +`provider::graph::native::configuration` lowers freshly prepared native input into public container configuration using the existing bounded container isolation. It preserves exact process/exec-readiness values and compiler job/dependency goals, requires immutable image IDs and the default source root, and adds distinct native @@ -63,7 +64,8 @@ service/job DNS aliases, including dotted names, through the shared network lowe Omitted image process/environment defaults remain omitted; managed values remain in separate pending handles. Whole-second shutdown grace up to 30 seconds is represented exactly; fractional seconds refuse. This pure -lowerer checks owner shape and deadline, never real guest ownership, image presence, +lowerer retains logical storage mount intent without fabricating provider volume IDs. +It checks owner shape and deadline, never real guest ownership, image presence, combined capacity, private staging or engine effects. `provider::graph::native::selection` selects only the exact absolute native project @@ -79,7 +81,7 @@ preserves the owning compiler's checkout-local semantics. This is read-only inpu selection and private preparation, with no durable enrollment or runtime ownership. `provider::graph::native::run` is an explicit library consumer for the bounded -image-only subset. It retains the development guest mutation lease, requires an +image/process subset and the separate persistent-storage path. It retains the development guest mutation lease, requires an admitted Internet or explicitly restricted outbound pool, verifies existing immutable images and shared graph/allocation capacity, and reserves a distinct v2 `native-graph-runtime` journal in `run/native-graphs` before effects. Create/start @@ -1389,3 +1391,144 @@ machine name and both disks' current identities and declared sizes. Every sample and the acquired startup lease recheck the selected owner; changed ownership refuses, and a reserve-qualified request cannot enter VM create or boot. Stopped, missing or unproved capacity retains fresh-allocation requirements or refuses. + +### Native persistent-data identity codec + +`provider::graph::native::persistent_data` defines one closed, private-candidate +version-1 owner record and a pure comparison. Its persistent binding contains a +64-hex project namespace, logical storage name, 32-hex owner, exact guest owner, +boot UUID and backing-disk identity, and the default local-volume policy. An +enrolled record retains the exact volume name, UTC `CreatedAt` bytes and directory +device/inode. Runtime run, plan and generation IDs are absent, so independent +compute generations can compare the same explicitly selected data identity. + +Pending intent is distinct from enrolled observation and always refuses the +comparison, even when supplied an exact volume observation. Missing observations, +unknown/duplicate fields, malformed identities, unsupported versions/policies, +and scope, guest, birth or directory changes refuse. Copied labels do not make a +replacement volume match its original birth. Guest boot/disk rollover remains a +refusal requiring a future separately owned handoff; no migration is inferred. + +This codec alone is data-only. Decoding an enrolled assertion does +not prove a durable enrollment commit, fresh observation, contents or effect +authority. It is separate from dependency-cache provenance and does not qualify +persistent databases, initializer replay, SQL retention or the full NC05 corpus. +The enrollment and adapter owners establish their separate commit and observation +boundaries. Their presence does not qualify the real SQL/application gates below. + +### Persistent-data enrollment owner + +`persistent_data::enrollment` owns the private-filesystem lifecycle. The owner uses a stable namespace/storage slot, +independent of compute generations, under an explicitly supplied existing private +host directory outside application data. An exclusive slot/lock and pending +record are synchronized before the sole original-create attempt. Promotion +requires that attempt's captured volume identity, matching fresh observations, +and unchanged guest, root, lock and original record identities/bytes. Existing +volume names, slots, incomplete staging and pending attempts refuse; no later +invocation recreates or promotes an interrupted attempt. + +The sealed transport requires exclusive creation relative to every supported writer +under an existing guest effect owner. An absence probe followed by Docker's +idempotent volume create API alone does not satisfy this contract. The native +adapter supplies the continuously held common provider lease described below; +unserialized direct guest/socket writers remain outside that authority. No +adoption, repair, deletion or global default is introduced. Existing-only retained reads acquire the existing lock, compare the +exact binding/birth/directory, and leave record bytes and generation references +unchanged. Pending or missing/foreign state cannot become enrollment by reading. + +Before rename, failures retain pending or incomplete staging. After rename, a +publication/directory-sync failure is uncertain: the enrolled pathname may exist, +but the operation returns no successful durable commit. A later retained read is +a fresh data-only comparison and cannot retroactively prove that failed operation +completed, or authorize runtime effects by itself. The lock coordinates these +writers; unsynchronized same-user external mutations are not atomically frozen. +Local synchronous filesystem I/O and trusted transport deadline obligations are +not process cancellation or crash-durability proof. Real private-filesystem and +stand-in tests qualify sequencing/refusal; persistent SQL and full NC05 runtime +retention remain open. + +### Native persistent-storage adapter and receipt v4 + +Ordinary persistent startup is explicitly gated before provider connection or +graph/data owner publication. The adapter and receipt v4 below are inactive +groundwork until a durable root-continuity witness and its transport are qualified. +Name, labels, `CreatedAt` and directory device/inode can all alias after an empty +volume replacement; matching that tuple is insufficient. No metadata-only pass +can enable storage or establish unique physical continuity. + +The inactive source implementation connects `persistent_data::engine` to the existing +native graph Engine. Receipt v4 carries stable data references and exact workload +mounts outside its run-owned container/network inventory. A persistent identity has +no run, plan or generation ID. Graph2 image-only receipts retain their old binding +bytes; graph3 is reserved for the separate source-bearing contract. Persistent +graph4 is not eligible for the dead-publication recovery selector: its explicit +qualification remains open. Ordinary authenticated compute teardown uses the +original graph owner and never deletes, recreates or replays data or jobs. + +All supported mutation paths retain `OwnedGuest` and its original provider +`operation.lock`. The adapter checks the held descriptor against the canonical +private pathname and root incarnation, in addition to the existing process, +guest boot and backing-disk checks. It holds that lease from authoritative absence +through one volume POST and final private owner commit. Supported native workloads +cannot mount a Docker socket or invoke arbitrary guest control; source/file mounts +remain refused, and private-delivery binds combined with storage also refuse in +this first slice. Existing provider Engine/guest writers use the same common lease. +External same-user Docker/guest writers are not serialized by this cooperative +contract. Docker's idempotent POST is not an exclusive creation primitive. + +The private graph reservation and synchronized data pending intent precede the +sole original create. The returned name, local driver/default policy, exact labels, +`CreatedAt` and guest directory device/inode are captured and freshly compared +before enrollment. A timeout, lost lock, failed observation or changed identity +returns uncertainty without adopting a later matching row, retrying POST or deleting +data. A visible enrolled file after failed final sync is not successful completion. +An earlier graph's unconfirmed reserved data reference blocks another compute +attempt even after its compute-only retirement. Missing owner state with an +existing volume for that logical namespace also refuses; copied labels or empty +contents do not authorize enrollment. + +Retained startup validates the existing owner and physical volume under the same +lease without rewriting its stable binding. Previous compute consumers must have +completed exact retirement. Explicit mounts use `NoCopy`; every image-declared +volume must be covered by exactly one admitted persistent mount, so no anonymous +volume is created. Physical mount membership is exact. Startup/final readiness, +inspection, shared active-run admission and final teardown publication recheck data. +Startup uses the original ingress deadline; read-only data observation and cleanup +validation use a bounded 40-second budget per observation group, without renewing +it per volume; final teardown validation begins a separate group after compute stops. +Compute teardown stops/deletes only recorded containers and their empty bridge. +Persistent volume deletion, pending enrollment recovery and guest/disk rebinding +are not implemented. + +`tests/fixtures/native-persistent-sqlite.json` pins the Bun/SQLite programs adapted +from the existing graph SQL fixture. It separates initializer attempts from an +`INSERT OR IGNORE` seed marker/nonce, adds an app-to-database HTTP write/readback, +and keeps `CapDrop: ALL` and `no-new-privileges`. The compiler/lowerer/scheduler +regression uses synthetic observations; it does not execute SQL or prove Engine +behavior. The first real native acceptance must use a source-pinned cached Bun +image ID and two independent compute generations of this same logical data slot: + +- First startup records the exact data owner/birth/directory and seed nonce, requires + a fresh successful initializer attempt, and proves the app HTTP SQL write/readback. +- Authenticated down retires only its exact compute inventory. The volume identity, + SQL marker, seed nonce and written value must survive unchanged. +- Fresh up uses new compute IDs and the same data identity. The initializer process + runs again and increments the attempt table once; the seed nonce stays unchanged. + SQL initialization is idempotent, not a cached completed-job result. +- Known initializer failure prevents dependent starts. Pending or missing owner + state, lock loss, a copied-label replacement with changed birth/directory and + changed guest identity refuse before dependent effects; uncertainty retains + receipts/data without an automatic create/adoption/delete retry. + +The root-witness continuation must reuse the reviewed directory-xattr contract: +an independently random name/value is persisted by the owner, then exclusively +created only during the original enrollment with `XATTR_CREATE` on a retained +nofollow root descriptor, synchronized, and independently reread with exact +descriptor/path/root checks. Retained startup only reads; missing/changed witness +refuses without repair. Whole-root/xattr copying remains outside the claimed +guarantee. This native helper/transport is not implemented or qualified here. + +These runtime/SQL gates and replacement safety are not yet qualified by this source implementation. Stock +PostgreSQL parity remains a separate NC05 gate requiring authored ownership/user +or a specifically qualified capability policy. This slice guesses no UID/GID, +changes no volume permissions, and adds no capability to make a stock image work. diff --git a/packages/runtime-core/src/project/native.rs b/packages/runtime-core/src/project/native.rs index 06ae8ecb9..6444df9f6 100644 --- a/packages/runtime-core/src/project/native.rs +++ b/packages/runtime-core/src/project/native.rs @@ -6,8 +6,8 @@ use hack_config_compiler::{ environment::{EnvironmentBinding, EnvironmentPlan, PlanResult}, local::LocalResolution, model::{ - Command, Dependency, EnvironmentValue, Plan, Readiness, ServiceCondition, Source, Workload, - WorktreePolicy, + Access, Command, Dependency, EnvironmentValue, Mount, Plan, Readiness, ServiceCondition, + Source, Workload, WorktreePolicy, }, process::{Entrypoint, Restart, ShutdownSignal}, }; @@ -39,6 +39,8 @@ pub struct NativeInputs { pub selected_profiles: Vec, pub graph: Graph, pub workloads: BTreeMap, + /// Selected persistent/worktree logical storage names; no provider volume identity. + pub storage: BTreeSet, /// Destination keys only; values cannot enter public engine configuration or receipts. pub managed_environment: ManagedValues, } @@ -98,6 +100,15 @@ pub struct WorkloadInputs { pub working_directory: Option, pub environment: BTreeMap, pub readiness: Option, + pub mounts: Vec, +} + +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct StorageMount { + pub storage: String, + pub target: String, + pub read_only: bool, } /// Millisecond precision is retained; a future backend must qualify signal/timing delivery. @@ -117,7 +128,7 @@ pub struct ExecReadiness { fn refused() -> CandidateError { CandidateError::new( "native_graph_subset", - "Native graph adapter requires image-only workloads, exec readiness and no acquisition, mounts, storage, custom networks, routing, endpoints, host effects or automatic restart; values omitted.", + "Native graph adapter requires image-only workloads, exec readiness and only persistent worktree storage mounts; acquisition, source/file mounts, custom networks, routing, endpoints, host effects and automatic restart remain refused. Values omitted.", ) } @@ -178,7 +189,6 @@ fn entrypoint(value: Entrypoint) -> Result, CandidateError> { fn workload(value: Workload, kind: WorkloadKind) -> Result { if value.build.is_some() || value.pull_policy.is_some() - || !value.mounts.is_empty() || (value.entrypoint.is_some() && value.command.is_none()) || value .restart @@ -187,6 +197,32 @@ fn workload(value: Workload, kind: WorkloadKind) -> Result 0 && matches!(byte, b'-' | b'_' | b'.')) + }) + && target != "/" => + { + Ok(StorageMount { + storage, + target, + read_only: matches!(access, Access::ReadOnly), + }) + } + _ => Err(refused()), + }) + .collect::, _>>()?; let readiness = value .readiness .map(|check| match check { @@ -228,6 +264,7 @@ fn workload(value: Workload, kind: WorkloadKind) -> Result Result { + let mut stream = connect_until(path, deadline)?; + let remaining = deadline + .checked_duration_since(Instant::now()) + .ok_or_else(|| failure("Guest observation deadline expired."))?; + let milliseconds = remaining.as_millis().min(40_000); + if milliseconds == 0 { + return Err(failure("Guest observation deadline expired.")); + } + let mut command = vec!["/bin/sh", "-c", script, "hack-local"]; + command.extend_from_slice(arguments); + let body = json!({"method":"vm_exec","command":command,"env":[["PATH","/opt/hack-engine:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"]],"workdir":"/","timeout_ms":milliseconds,"interactive":false,"tty":false,"background":false,"stdin_data":null}); + let bytes = encode(&body)?; + let timeout = deadline + .checked_duration_since(Instant::now()) + .ok_or_else(|| failure("Guest observation deadline expired."))?; + let response = exchange(&mut stream, &bytes, timeout)?; + if Instant::now() >= deadline { + return Err(failure("Guest observation deadline expired.")); + } + decode_execution(response) +} + pub fn exec_input( path: &Path, script: &str, diff --git a/packages/runtime-core/src/provider/engine.rs b/packages/runtime-core/src/provider/engine.rs index 89011def2..1f2943fc3 100644 --- a/packages/runtime-core/src/provider/engine.rs +++ b/packages/runtime-core/src/provider/engine.rs @@ -74,6 +74,16 @@ impl Transport { method: Method, path: &str, body: Option<&Value>, + ) -> Result, CandidateError> { + self.request_bytes_until(method, path, body, None) + } + + fn request_bytes_until( + &self, + method: Method, + path: &str, + body: Option<&Value>, + deadline: Option, ) -> Result, CandidateError> { if !path.starts_with('/') || path.starts_with("//") @@ -100,15 +110,32 @@ impl Transport { // before dispatch. A connection wait cannot extend the original budget. // Cleanup observations and deletion remain possible after expiry; owned // stop has its separate cleanup-only batch transport and deadline. - if requires_admission && let Some(deadline) = self.admission_deadline { - let remaining = super::managed_environment::remaining_until(deadline)?; - request = request.timeout(remaining.min(self.timeout)); - } + request = request.timeout(request_timeout( + self.timeout, + requires_admission + .then_some(self.admission_deadline) + .flatten(), + deadline, + )?); let response = request.send().map_err(|_| failure("Engine request failed or timed out; its effect may be uncertain. No request was replayed."))?; response_bytes(response) } } +// One minimum budget reaches dispatch. A later explicit deadline cannot replace +// an earlier mutation-admission deadline; cleanup admission exemptions stay above. +fn request_timeout( + default: Duration, + admission: Option, + explicit: Option, +) -> Result { + let mut timeout = default; + for deadline in admission.into_iter().chain(explicit) { + timeout = timeout.min(super::managed_environment::remaining_until(deadline)?); + } + Ok(timeout) +} + fn response_bytes(mut response: reqwest::blocking::Response) -> Result, CandidateError> { let status = response.status(); if response.content_length().is_some_and(|n| n > MAX_BODY) { @@ -185,6 +212,39 @@ pub(super) struct Engine<'a> { } impl<'a> Engine<'a> { + #[cfg(feature = "native-config-plan")] + pub(in crate::provider) fn request_until( + &self, + method: Method, + path: &str, + body: Option<&Value>, + deadline: Instant, + ) -> Result { + if self.cleanup_only && ![Method::GET, Method::HEAD, Method::DELETE].contains(&method) { + return Err(failure( + "A cleanup connection cannot allocate or start resources.", + )); + } + self.guest.verify()?; + if ![Method::GET, Method::HEAD, Method::DELETE].contains(&method) { + self.guest.before_effect()?; + } + let bytes = self + .transport + .request_bytes_until(method, path, body, Some(deadline)); + self.guest.verify()?; + if Instant::now() >= deadline { + return Err(failure( + "Private engine deadline expired; outcome remains uncertain.", + )); + } + let bytes = bytes?; + if bytes.is_empty() { + return Ok(Value::Null); + } + serde_json::from_slice(&bytes) + .map_err(|_| failure("Malformed engine response; values omitted.")) + } pub(super) fn load_image_archive(&self, archive: Vec) -> Result<(), CandidateError> { if self.cleanup_only { return Err(failure("A cleanup connection cannot load images.")); @@ -586,6 +646,25 @@ mod tests { use std::io::Write; use std::os::unix::net::UnixListener; + #[test] + fn explicit_request_budget_never_extends_mutation_admission() { + let now = Instant::now(); + let early = now + Duration::from_secs(1); + let late = now + Duration::from_secs(20); + let default = Duration::from_secs(40); + assert!( + request_timeout(default, Some(early), Some(late)).unwrap() <= Duration::from_secs(1) + ); + assert!( + request_timeout(default, Some(late), Some(early)).unwrap() <= Duration::from_secs(1) + ); + assert_eq!(request_timeout(default, None, None).unwrap(), default); + assert!(request_timeout(default, Some(now), Some(late)).is_err()); + assert!(request_timeout(default, Some(late), Some(now)).is_err()); + // GET/HEAD/DELETE omit admission but still honor an explicit cleanup budget. + assert!(request_timeout(default, None, Some(late)).unwrap() > Duration::from_secs(1)); + } + #[cfg(target_os = "macos")] #[test] fn bounded_acquisition_cancellation_and_expiry_do_not_initialize_state() { diff --git a/packages/runtime-core/src/provider/graph/native/journal.rs b/packages/runtime-core/src/provider/graph/native/journal.rs index a04c49d5b..6a3436fb9 100644 --- a/packages/runtime-core/src/provider/graph/native/journal.rs +++ b/packages/runtime-core/src/provider/graph/native/journal.rs @@ -54,7 +54,7 @@ fn decode_failure_observation<'de, D: serde::Deserializer<'de>>( /// Hash-only native provenance plus value-free resource ownership. No replay authority, /// compiler request, argv, environment values or renewable timestamp is persisted. #[derive(Clone, Debug, Serialize, Deserialize)] -#[serde(deny_unknown_fields)] +#[serde(try_from = "ReceiptWire")] pub struct Receipt { version: u32, kind: InputKind, @@ -64,11 +64,67 @@ pub struct Receipt { pub(super) phase: Phase, pub(super) readiness: BTreeMap, pub(super) resources: BTreeMap, + #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] + pub(super) data: BTreeMap, + #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] + pub(super) data_mounts: BTreeMap>, #[serde(default, skip_serializing_if = "Option::is_none")] failure: Option, #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] pub(super) terminal: BTreeMap, } + +// Presence is a wire-version boundary: an explicit empty storage field must not +// become indistinguishable from an absent graph2 field. Present null also refuses. +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct ReceiptWire { + version: u32, + kind: InputKind, + owner: String, + boot: String, + review: native_input::Review, + phase: Phase, + readiness: BTreeMap, + resources: BTreeMap, + #[serde(default, deserialize_with = "present_map")] + data: Option>, + #[serde(default, deserialize_with = "present_map")] + data_mounts: Option>>, + #[serde(default)] + failure: Option, + #[serde(default)] + terminal: BTreeMap, +} +fn present_map<'de, D: serde::Deserializer<'de>, T: Deserialize<'de>>( + reader: D, +) -> Result, D::Error> { + T::deserialize(reader).map(Some) +} +impl TryFrom for Receipt { + type Error = &'static str; + fn try_from(wire: ReceiptWire) -> Result { + if (wire.version == 2 && (wire.data.is_some() || wire.data_mounts.is_some())) + || (wire.version == 4 && (wire.data.is_none() || wire.data_mounts.is_none())) + { + return Err("Native receipt storage fields do not match its wire version."); + } + Ok(Self { + version: wire.version, + kind: wire.kind, + owner: wire.owner, + boot: wire.boot, + review: wire.review, + phase: wire.phase, + readiness: wire.readiness, + resources: wire.resources, + data: wire.data.unwrap_or_default(), + data_mounts: wire.data_mounts.unwrap_or_default(), + failure: wire.failure, + terminal: wire.terminal, + }) + } +} impl Receipt { /// Mutable phases and terminal evidence may advance; admitted identity cannot. pub(super) fn check_binding(&self, expected: &Self) -> Result<(), CandidateError> { @@ -76,6 +132,8 @@ impl Receipt { if self.review != expected.review || self.boot != expected.boot || self.readiness != expected.readiness + || self.data != expected.data + || self.data_mounts != expected.data_mounts || self.resources.keys().ne(expected.resources.keys()) || self.resources.iter().any(|(key, resource)| { let prior = &expected.resources[key]; @@ -111,7 +169,8 @@ impl Receipt { #[cfg(target_os = "macos")] pub(super) fn require_recovery_ready(&self) -> Result<(), CandidateError> { self.validate(self.review.scope().run, &self.owner)?; - if self.phase != Phase::ReadyObserved + if self.version != 2 + || self.phase != Phase::ReadyObserved || self.failure.is_some() || self .resources @@ -128,7 +187,7 @@ impl Receipt { boot: &str, ) -> Result { let receipt = Self { - version: 2, + version: if config.storage.is_empty() { 2 } else { 4 }, kind: InputKind::NativeGraphRuntime, owner: owner.into(), boot: boot.into(), @@ -141,6 +200,8 @@ impl Receipt { .map(|(name, service)| (name.clone(), service.ready)) .collect(), resources: config.resources.clone(), + data: config.data.clone(), + data_mounts: config.data_mounts.clone(), failure: None, terminal: BTreeMap::new(), }; @@ -156,7 +217,7 @@ impl Receipt { pub(super) fn validate(&self, run: &str, owner: &str) -> Result<(), CandidateError> { let scope = self.review.scope(); self.review.validate(scope).map_err(|_| refused())?; - if self.version != 2 + if ![2, 4].contains(&self.version) || !hex(run, 32) || run != scope.run || !hex(owner, 32) @@ -171,6 +232,7 @@ impl Receipt { { return Err(refused()); } + self.validate_data()?; let network = self.resources.get("network:default").ok_or_else(refused)?; if network.kind != Kind::Network || network.key != "default" @@ -283,6 +345,63 @@ impl Receipt { } Ok(()) } + + fn validate_data(&self) -> Result<(), CandidateError> { + if self.version == 2 { + return if self.data.is_empty() && self.data_mounts.is_empty() { + Ok(()) + } else { + Err(refused()) + }; + } + if self.data.is_empty() || self.data_mounts.is_empty() { + return Err(refused()); + } + let mut used = BTreeSet::new(); + for (service, mounts) in &self.data_mounts { + if !self.readiness.contains_key(service) || mounts.is_empty() { + return Err(refused()); + } + let mut targets = BTreeSet::new(); + for mount in mounts { + if !self.data.contains_key(&mount.storage) + || !targets.insert(&mount.target) + || !mount.target.starts_with('/') + || mount.target.contains(['\0', '\\']) + || mount + .target + .split('/') + .skip(1) + .any(|part| part == "." || part == ".." || part.is_empty()) + { + return Err(refused()); + } + used.insert(&mount.storage); + } + } + if used.into_iter().ne(self.data.keys()) { + return Err(refused()); + } + for (logical, data) in &self.data { + data.validate( + self.review.scope().namespace, + logical, + &self.owner, + &self.boot, + ) + .map_err(|_| refused())?; + if (self.phase == Phase::ReadyObserved + || self + .resources + .values() + .any(|resource| resource.kind == Kind::Container && resource.id.is_some())) + && !data.enrolled() + { + return Err(refused()); + } + } + Ok(()) + } } fn refused() -> CandidateError { error( diff --git a/packages/runtime-core/src/provider/graph/native/mod.rs b/packages/runtime-core/src/provider/graph/native/mod.rs index 76d7cf022..fe096ce7e 100644 --- a/packages/runtime-core/src/provider/graph/native/mod.rs +++ b/packages/runtime-core/src/provider/graph/native/mod.rs @@ -1,9 +1,10 @@ -//! Native image-only lowering; runtime ownership and effects are separately admitted. +//! Native image/process and persistent-mount lowering; effects are separately admitted. use super::*; use crate::{project::native::NativeInputs, provider::native_input}; #[cfg(target_os = "macos")] pub mod foreground; mod journal; +pub mod persistent_data; mod runtime; pub mod selection; pub use journal::{Phase, Receipt}; @@ -45,6 +46,9 @@ pub struct Configuration { review: native_input::Review, configs: BTreeMap, resources: BTreeMap, + storage: BTreeSet, + data_mounts: BTreeMap>, + data: BTreeMap, } impl Configuration { pub fn graph(&self) -> &execution::Graph { @@ -64,7 +68,7 @@ impl Configuration { fn refused() -> CandidateError { error( "native_graph_admission", - "Native image-only consumption requires its exact compiler review, immutable images and bounded process/readiness; values omitted.", + "Native consumption requires its exact compiler review, immutable images, bounded process/readiness and separately enrolled persistent storage; values omitted.", ) } @@ -193,6 +197,14 @@ pub fn configuration( review: review.clone(), configs, resources, + storage: inputs.storage.clone(), + data_mounts: inputs + .workloads + .iter() + .filter(|(_, workload)| !workload.mounts.is_empty()) + .map(|(name, workload)| (name.clone(), workload.mounts.clone())) + .collect(), + data: BTreeMap::new(), }) } diff --git a/packages/runtime-core/src/provider/graph/native/persistent_data.rs b/packages/runtime-core/src/provider/graph/native/persistent_data.rs new file mode 100644 index 000000000..4d8f14607 --- /dev/null +++ b/packages/runtime-core/src/provider/graph/native/persistent_data.rs @@ -0,0 +1,263 @@ +//! Data-only persistent-volume identity codec for the private native candidate. +//! +//! A decoded enrolled record is an assertion to compare, not proof of durable enrollment, +//! freshness, or permission to create, adopt, start, repair or delete anything. No runtime +//! generation, graph run, plan, dependency-cache completion or application data is encoded. +//! Enrollment and the engine adapter separately own their commit/effect boundaries and +//! obtain original identities themselves; parsing cannot replace those owners. + +use crate::CandidateError; +use serde::{Deserialize, Serialize}; + +pub use crate::provider::identity::DiskIdentity; + +const LIMIT: usize = 4096; + +fn refused() -> CandidateError { + CandidateError::new( + "native_persistent_data_identity", + "Persistent data enrollment is incomplete, malformed or changed; no data effects were authorized.", + ) +} + +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct Scope { + pub namespace: String, + pub storage: String, + pub owner: String, +} + +/// Conservative same-guest fence. A different boot or backing disk requires a future +/// separately owned handoff; this codec never infers that two guests share the same data. +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct GuestIdentity { + pub owner: String, + pub boot_id: String, + pub storage: DiskIdentity, +} + +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum Local { + Local, +} +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct NoOptions {} + +/// Only the existing default local-volume policy is representable. Unknown driver, +/// scope or options refuse rather than importing provider/cache semantics. +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct Policy { + pub driver: Local, + pub scope: Local, + pub options: NoOptions, +} + +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct Binding { + pub scope: Scope, + pub guest: GuestIdentity, + pub policy: Policy, +} +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct DirectoryIdentity { + pub device: u64, + pub inode: u64, +} +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct VolumeIdentity { + pub name: String, + pub created_at: String, + pub directory: DirectoryIdentity, +} + +/// Complete already-acquired observation. Constructing or deserializing this object +/// does not establish that an engine or guest actually supplied these facts. +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct Observation { + pub binding: Binding, + pub volume: VolumeIdentity, +} + +#[derive(Clone, Serialize, Deserialize)] +#[serde(rename_all = "kebab-case")] +enum Kind { + NativePersistentDataOwner, +} +#[derive(Clone, Serialize, Deserialize)] +#[serde(tag = "status", rename_all = "snake_case", deny_unknown_fields)] +enum Enrollment { + Pending { intent: String, volume_name: String }, + Enrolled { volume: VolumeIdentity }, +} +#[derive(Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct Record { + version: u8, + kind: Kind, + binding: Binding, + enrollment: Enrollment, +} + +/// Closed validated record; no public constructor/promoter or effect capability. +#[derive(Clone, Serialize)] +#[serde(transparent)] +pub struct Owner(Record); + +/// Decode only, with the byte bound applied before deserialization allocations. +/// Duplicate/unknown fields, missing/null values and unsupported versions refuse. +pub fn decode(bytes: &[u8]) -> Result { + if bytes.is_empty() || bytes.len() > LIMIT { + return Err(refused()); + } + let record: Record = serde_json::from_slice(bytes).map_err(|_| refused())?; + let enrollment_valid = match &record.enrollment { + Enrollment::Pending { + intent, + volume_name, + } => super::super::hex(intent, 32) && volume_name_valid(volume_name), + Enrollment::Enrolled { volume } => volume_valid(volume), + }; + if record.version != 1 || !binding_valid(&record.binding) || !enrollment_valid { + return Err(refused()); + } + Ok(Owner(record)) +} + +pub struct CompareOptions<'a> { + pub record: &'a Owner, + pub expected: &'a Binding, + pub observed: Option<&'a Observation>, +} + +/// Pure equality fence independent of any caller's runtime-generation reference. +/// Pending enrollment never matches, including an otherwise exact current observation. +/// Success is not durable-record, observation freshness, lease or execution authority. +pub fn compare(options: CompareOptions<'_>) -> Result<(), CandidateError> { + let Enrollment::Enrolled { volume } = &options.record.0.enrollment else { + return Err(refused()); + }; + let observed = options.observed.ok_or_else(refused)?; + if !binding_valid(options.expected) + || !binding_valid(&observed.binding) + || !volume_valid(&observed.volume) + || options.record.0.binding != *options.expected + || observed.binding != *options.expected + || observed.volume != *volume + { + return Err(refused()); + } + Ok(()) +} + +fn binding_valid(binding: &Binding) -> bool { + let guest = &binding.guest; + let disk = &guest.storage; + super::super::hex(&binding.scope.namespace, 64) + && super::super::hex(&binding.scope.owner, 32) + && logical_name(&binding.scope.storage) + && super::super::hex(&guest.owner, 32) + && uuid(&guest.boot_id) + && disk.inode > 0 + && disk.bytes > 0 + && uuid(&disk.uuid) +} +fn logical_name(value: &str) -> bool { + !value.is_empty() + && value.len() <= 63 + && value.bytes().enumerate().all(|(index, byte)| { + byte.is_ascii_lowercase() + || byte.is_ascii_digit() + || (index > 0 && matches!(byte, b'-' | b'_' | b'.')) + }) +} +fn volume_name_valid(value: &str) -> bool { + !value.is_empty() + && value.len() <= 255 + && value.bytes().enumerate().all(|(index, byte)| { + byte.is_ascii_alphanumeric() || (index > 0 && matches!(byte, b'-' | b'_' | b'.')) + }) +} +fn uuid(value: &str) -> bool { + value.len() == 36 + && value != "00000000-0000-0000-0000-000000000000" + && value.bytes().enumerate().all(|(index, byte)| { + if [8, 13, 18, 23].contains(&index) { + byte == b'-' + } else { + byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte) + } + }) +} +fn volume_valid(volume: &VolumeIdentity) -> bool { + volume_name_valid(&volume.name) && volume.directory.inode > 0 && timestamp(&volume.created_at) +} + +// Closed UTC RFC3339Nano spelling. Equality retains the supplied engine bytes; +// alternate spellings are not silently normalized into a replacement's birth. +fn timestamp(value: &str) -> bool { + let bytes = value.as_bytes(); + if !(20..=30).contains(&bytes.len()) + || bytes.last() != Some(&b'Z') + || [4, 7].iter().any(|index| bytes[*index] != b'-') + || bytes[10] != b'T' + || [13, 16].iter().any(|index| bytes[*index] != b':') + || !(bytes.len() == 20 || (bytes.len() >= 22 && bytes[19] == b'.')) + || bytes.iter().enumerate().any(|(index, byte)| { + ![4, 7, 10, 13, 16, bytes.len() - 1].contains(&index) + && !(index == 19 && bytes.len() > 20) + && !byte.is_ascii_digit() + }) + { + return false; + } + let number = |start: usize, end: usize| { + bytes[start..end] + .iter() + .fold(0_u32, |n, b| n * 10 + u32::from(*b - b'0')) + }; + let year = number(0, 4); + let month = number(5, 7); + let day = number(8, 10); + let leap = year % 4 == 0 && (year % 100 != 0 || year % 400 == 0); + let days = match month { + 2 => { + if leap { + 29 + } else { + 28 + } + } + 4 | 6 | 9 | 11 => 30, + 1 | 3 | 5 | 7 | 8 | 10 | 12 => 31, + _ => 0, + }; + let zero_birth = year == 1 + && month == 1 + && day == 1 + && number(11, 13) == 0 + && number(14, 16) == 0 + && number(17, 19) == 0 + && (bytes.len() == 20 || bytes[20..bytes.len() - 1].iter().all(|b| *b == b'0')); + year > 0 + && !zero_birth + && day > 0 + && day <= days + && number(11, 13) < 24 + && number(14, 16) < 60 + && number(17, 19) < 60 +} + +#[cfg(test)] +mod tests; + +pub(super) mod engine; +pub mod enrollment; diff --git a/packages/runtime-core/src/provider/graph/native/persistent_data/engine.rs b/packages/runtime-core/src/provider/graph/native/persistent_data/engine.rs new file mode 100644 index 000000000..808559d8d --- /dev/null +++ b/packages/runtime-core/src/provider/graph/native/persistent_data/engine.rs @@ -0,0 +1,449 @@ +//! Production adapter under the existing provider mutation lease. This is cooperative +//! exclusive creation, not an exclusive Docker endpoint. Every supported Engine creator +//! holds the same original lock; direct same-user socket/guest writers are not serialized. +//! No volume deletion, adoption, pending replay or guest rollover is implemented here. + +use super::{enrollment, *}; +use crate::{Candidate, provider::engine::Engine}; +use reqwest::Method; +use serde_json::{Value, json}; +use std::{collections::BTreeMap, io::Read, sync::atomic::AtomicBool, time::Instant}; + +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "status", rename_all = "snake_case", deny_unknown_fields)] +enum State { + Reserved { intent: String }, + Enrolled { volume: VolumeIdentity }, +} +/// Journal assertion only. The private durable owner and fresh guest/volume proof are +/// independently required before use. Stable storage never becomes a run-owned Resource. +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(in crate::provider::graph::native) struct Reference { + binding: Binding, + state: State, +} +impl std::fmt::Debug for Reference { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter.write_str("PersistentDataReference") + } +} +impl Reference { + pub(in crate::provider::graph::native) fn name(&self) -> String { + enrollment::volume_name(&self.binding) + } + pub(in crate::provider::graph::native) fn enrolled(&self) -> bool { + matches!(self.state, State::Enrolled { .. }) + } + pub(in crate::provider::graph::native) fn validate( + &self, + namespace: &str, + logical: &str, + owner: &str, + boot: &str, + ) -> Result<(), CandidateError> { + if !binding_valid(&self.binding) + || self.binding.scope.namespace != namespace + || self.binding.scope.storage != logical + || self.binding.guest.owner != owner + || self.binding.guest.boot_id != boot + { + return Err(refused()); + } + match &self.state { + State::Reserved { intent } if super::super::super::hex(intent, 32) => Ok(()), + State::Enrolled { volume } if volume_valid(volume) && volume.name == self.name() => { + Ok(()) + } + _ => Err(refused()), + } + } + pub(in crate::provider::graph::native) fn mountpoint(&self) -> String { + format!("/var/lib/docker/volumes/{}/_data", self.name()) + } +} +fn nonce() -> Result { + let mut bytes = [0_u8; 16]; + std::fs::File::open("/dev/urandom") + .and_then(|mut file| file.read_exact(&mut bytes)) + .map_err(|_| refused())?; + Ok(bytes.iter().map(|byte| format!("{byte:02x}")).collect()) +} + +pub(in crate::provider::graph::native) struct Adapter<'a, 'guest> { + engine: &'a Engine<'guest>, + fresh: &'a dyn Fn() -> Result<(), CandidateError>, +} +impl<'a, 'guest> Adapter<'a, 'guest> { + fn new(engine: &'a Engine<'guest>, fresh: &'a dyn Fn() -> Result<(), CandidateError>) -> Self { + Self { engine, fresh } + } + fn check(&self, deadline: Instant) -> Result<(), CandidateError> { + (self.fresh)()?; + self.engine.guest().verify()?; + if Instant::now() >= deadline { + return Err(refused()); + } + Ok(()) + } + fn request( + &self, + method: Method, + path: &str, + body: Option<&Value>, + deadline: Instant, + ) -> Result { + self.check(deadline)?; + let result = self.engine.request_until(method, path, body, deadline); + self.check(deadline)?; + result + } + fn observe( + &self, + binding: &Binding, + name: &str, + value: &Value, + deadline: Instant, + ) -> Result { + let mountpoint = format!("/var/lib/docker/volumes/{name}/_data"); + if value["Name"] != name + || value["Driver"] != "local" + || value["Scope"] != "local" + || !(value["Options"].is_null() + || value["Options"] + .as_object() + .is_some_and(|options| options.is_empty())) + || value["Mountpoint"] != mountpoint + || value["Labels"] != labels(binding) + { + return Err(refused()); + } + let created_at = value["CreatedAt"].as_str().ok_or_else(refused)?.to_owned(); + self.check(deadline)?; + let identity = self + .engine + .guest() + .execute_until(DIRECTORY, &[&mountpoint], deadline) + .map_err(|_| refused()); + self.check(deadline)?; + let identity = identity?; + let (device, inode) = identity + .strip_suffix('\n') + .and_then(|value| value.split_once(':')) + .ok_or_else(refused)?; + let number = |value: &str| { + if value.is_empty() || !value.bytes().all(|byte| byte.is_ascii_digit()) { + return Err(refused()); + } + value.parse::().map_err(|_| refused()) + }; + let volume = VolumeIdentity { + name: name.into(), + created_at, + directory: DirectoryIdentity { + device: number(device)?, + inode: number(inode)?, + }, + }; + if !volume_valid(&volume) { + return Err(refused()); + } + Ok(Observation { + binding: binding.clone(), + volume, + }) + } +} +impl enrollment::sealed::Transport for Adapter<'_, '_> {} +impl enrollment::Transport for Adapter<'_, '_> { + fn verify(&mut self, expected: &Binding, deadline: Instant) -> Result<(), CandidateError> { + self.check(deadline)?; + if !binding_valid(expected) || self.engine.guest().persistent_identity()? != expected.guest + { + return Err(refused()); + } + self.check(deadline) + } + fn inspect( + &mut self, + name: &str, + deadline: Instant, + ) -> Result, CandidateError> { + // Name carries no authority. Only the closed persistent labels select a binding, + // subsequently compared to the independent expected durable owner by enrollment. + if !volume_name_valid(name) { + return Err(refused()); + } + let value = match self.request( + Method::GET, + &format!("/v1.53/volumes/{name}"), + None, + deadline, + ) { + Err(error) if error.code == "engine_not_found" => return Ok(None), + result => result?, + }; + let labels = value["Labels"].as_object().ok_or_else(refused)?; + let string = |key: &str| { + labels + .get(key) + .and_then(Value::as_str) + .map(str::to_owned) + .ok_or_else(refused) + }; + let binding = Binding { + scope: Scope { + namespace: string("io.hack-local.namespace")?, + storage: string("io.hack-local.storage")?, + owner: string("io.hack-local.data-owner")?, + }, + guest: self.engine.guest().persistent_identity()?, + policy: Policy { + driver: Local::Local, + scope: Local::Local, + options: NoOptions {}, + }, + }; + if !binding_valid(&binding) || enrollment::volume_name(&binding) != name { + return Err(refused()); + } + let observed = self.observe(&binding, name, &value, deadline)?; + let after = self.request( + Method::GET, + &format!("/v1.53/volumes/{name}"), + None, + deadline, + )?; + let confirmed = self.observe(&binding, name, &after, deadline)?; + if confirmed != observed { + return Err(refused()); + } + Ok(Some(observed)) + } + fn create_new( + &mut self, + request: &enrollment::CreateRequest, + deadline: Instant, + ) -> Result { + create_original(self, request, deadline) + } +} +trait Creation { + fn fence(&mut self, binding: &Binding, deadline: Instant) -> Result<(), CandidateError>; + fn find( + &mut self, + name: &str, + deadline: Instant, + ) -> Result, CandidateError>; + fn post( + &mut self, + request: &enrollment::CreateRequest, + deadline: Instant, + ) -> Result; +} +impl Creation for Adapter<'_, '_> { + fn fence(&mut self, binding: &Binding, deadline: Instant) -> Result<(), CandidateError> { + enrollment::Transport::verify(self, binding, deadline) + } + fn find( + &mut self, + name: &str, + deadline: Instant, + ) -> Result, CandidateError> { + enrollment::Transport::inspect(self, name, deadline) + } + fn post( + &mut self, + request: &enrollment::CreateRequest, + deadline: Instant, + ) -> Result { + let value = self.request(Method::POST, "/v1.53/volumes/create", Some(&json!({"Name":request.name(),"Driver":"local","DriverOpts":{},"Labels":labels(request.binding())})), deadline)?; + self.observe(request.binding(), request.name(), &value, deadline) + } +} +/// Same production sequence exercised by lock-owning stand-ins. No caller can issue +/// this sealed transport or turn a returned idempotent endpoint row into authority. +fn create_original( + owner: &mut C, + request: &enrollment::CreateRequest, + deadline: Instant, +) -> Result { + owner.fence(request.binding(), deadline)?; + if owner.find(request.name(), deadline)?.is_some() { + return Err(refused()); + } + owner.fence(request.binding(), deadline)?; + let captured = owner.post(request, deadline)?; + let current = owner.find(request.name(), deadline)?.ok_or_else(refused)?; + if current != captured { + return Err(refused()); + } + owner.fence(request.binding(), deadline)?; + Ok(captured) +} +fn labels(binding: &Binding) -> Value { + json!({"io.hack-local.kind":"native-persistent-data","io.hack-local.namespace":binding.scope.namespace,"io.hack-local.storage":binding.scope.storage,"io.hack-local.data-owner":binding.scope.owner,"io.hack-local.provider-owner":binding.guest.owner}) +} +const DIRECTORY: &str = r#"set -efu +root=$1 +parent=$root +while test "$parent" != /; do test ! -L "$parent"; test -d "$parent"; parent=${parent%/*}; test -n "$parent" || parent=/; done +exec 9<"$root" +identity=$(stat -Lc %d:%i /proc/self/fd/9) +test "$(stat -c %d:%i "$root")" = "$identity" +printf '%s\n' "$identity" +"#; + +/// Select stable ownership without creating any volume or pending/enrolled owner file. +pub(in crate::provider::graph::native) fn select( + candidate: &Candidate, + engine: &Engine<'_>, + namespace: &str, + names: &std::collections::BTreeSet, + deadline: Instant, + fresh: &dyn Fn() -> Result<(), CandidateError>, +) -> Result, CandidateError> { + let mut references = BTreeMap::new(); + if names.is_empty() { + return Ok(references); + } + let guest = engine.guest().persistent_identity()?; + let inventory = + Adapter::new(engine, fresh).request(Method::GET, "/v1.53/volumes", None, deadline)?; + let volumes = inventory["Volumes"].as_array().ok_or_else(refused)?; + if !inventory["Warnings"].is_null() + && !inventory["Warnings"] + .as_array() + .is_some_and(|warnings| warnings.is_empty()) + { + return Err(refused()); + } + let mut volume_names = std::collections::BTreeSet::new(); + for volume in volumes { + let name = volume["Name"] + .as_str() + .filter(|name| volume_name_valid(name)) + .ok_or_else(refused)?; + if !volume_names.insert(name) { + return Err(refused()); + } + } + for name in names { + fresh()?; + if Instant::now() >= deadline { + return Err(refused()); + } + let existing = enrollment::existing_binding(&candidate.state_root, namespace, name)?; + let retained = existing.is_some(); + let binding = match existing { + Some(binding) => binding, + None => Binding { + scope: Scope { + namespace: namespace.into(), + storage: name.clone(), + owner: nonce()?, + }, + guest: guest.clone(), + policy: Policy { + driver: Local::Local, + scope: Local::Local, + options: NoOptions {}, + }, + }, + }; + if binding.guest != guest { + return Err(refused()); + } + let prefix = format!("hkp-{namespace}-"); + let selected_name = enrollment::volume_name(&binding); + for physical in &volume_names { + if let Some(suffix) = physical.strip_prefix(&prefix) { + let (owner, logical) = suffix.split_once('-').ok_or_else(refused)?; + if !super::super::super::hex(owner, 32) || !logical_name(logical) { + return Err(refused()); + } + if logical == name && (!retained || *physical != selected_name) { + return Err(refused()); + } + } + } + let state = if retained { + let owner = enrollment::read_retained( + enrollment::ReadOptions { + state_root: &candidate.state_root, + binding: &binding, + deadline, + cancelled: &AtomicBool::new(false), + }, + &mut Adapter::new(engine, fresh), + )?; + let Enrollment::Enrolled { volume } = owner.0.enrollment else { + return Err(refused()); + }; + State::Enrolled { volume } + } else { + State::Reserved { intent: nonce()? } + }; + references.insert(name.clone(), Reference { binding, state }); + } + fresh()?; + Ok(references) +} +/// Called only after the graph reservation exists. Library enrollment owns the private +/// synced pending-before-effect and final exact publication boundaries. +pub(in crate::provider::graph::native) fn enroll( + candidate: &Candidate, + engine: &Engine<'_>, + reference: &mut Reference, + deadline: Instant, + fresh: &dyn Fn() -> Result<(), CandidateError>, +) -> Result<(), CandidateError> { + let State::Reserved { intent } = &reference.state else { + return verify(candidate, engine, reference, deadline, fresh); + }; + let owner = enrollment::enroll_new( + enrollment::EnrollOptions { + state_root: &candidate.state_root, + binding: &reference.binding, + intent, + deadline, + cancelled: &AtomicBool::new(false), + }, + &mut Adapter::new(engine, fresh), + )?; + let Enrollment::Enrolled { volume } = owner.0.enrollment else { + return Err(refused()); + }; + reference.state = State::Enrolled { volume }; + Ok(()) +} +pub(in crate::provider::graph::native) fn verify( + candidate: &Candidate, + engine: &Engine<'_>, + reference: &Reference, + deadline: Instant, + fresh: &dyn Fn() -> Result<(), CandidateError>, +) -> Result<(), CandidateError> { + let State::Enrolled { volume } = &reference.state else { + return Err(refused()); + }; + let owner = enrollment::read_retained( + enrollment::ReadOptions { + state_root: &candidate.state_root, + binding: &reference.binding, + deadline, + cancelled: &AtomicBool::new(false), + }, + &mut Adapter::new(engine, fresh), + )?; + let Enrollment::Enrolled { volume: current } = owner.0.enrollment else { + return Err(refused()); + }; + if current != *volume { + return Err(refused()); + } + Ok(()) +} + +#[cfg(test)] +mod tests; diff --git a/packages/runtime-core/src/provider/graph/native/persistent_data/engine/tests.rs b/packages/runtime-core/src/provider/graph/native/persistent_data/engine/tests.rs new file mode 100644 index 000000000..208e8285f --- /dev/null +++ b/packages/runtime-core/src/provider/graph/native/persistent_data/engine/tests.rs @@ -0,0 +1,238 @@ +use super::*; +use crate::provider::state; +use std::{ + cell::Cell, + fs, + os::unix::fs::{DirBuilderExt, MetadataExt}, + path::PathBuf, + rc::Rc, + time::Duration, +}; + +struct Fixture { + root: PathBuf, + data: PathBuf, + lease: PathBuf, +} +impl Fixture { + fn new() -> Self { + let root = std::env::temp_dir().canonicalize().unwrap().join(format!( + "persistent-adapter-{}-{}", + std::process::id(), + nonce().unwrap() + )); + fs::DirBuilder::new().mode(0o700).create(&root).unwrap(); + let data = root.join("data"); + let lease = root.join("guest"); + for path in [&data, &lease] { + fs::DirBuilder::new().mode(0o700).create(path).unwrap(); + } + Self { root, data, lease } + } + fn binding(&self) -> Binding { + Binding { + scope: Scope { + namespace: "a".repeat(64), + storage: "database".into(), + owner: "b".repeat(32), + }, + guest: GuestIdentity { + owner: "c".repeat(32), + boot_id: "11111111-2222-3333-4444-555555555555".into(), + storage: DiskIdentity { + device: 0, + inode: 21, + bytes: 128, + uuid: "11111111-2222-3333-4444-555555555556".into(), + }, + }, + policy: Policy { + driver: Local::Local, + scope: Local::Local, + options: NoOptions {}, + }, + } + } + fn transport(&self, posts: Rc>) -> Endpoint<'_> { + Endpoint { + root: &self.lease, + lock: state::Lock::acquire(&self.lease).unwrap(), + binding: self.binding(), + observed: None, + posts, + replace_lock: false, + wrong_birth: false, + pending: &self.data, + } + } +} +impl Drop for Fixture { + fn drop(&mut self) { + fs::remove_dir_all(&self.root).unwrap(); + } +} +struct Endpoint<'a> { + root: &'a std::path::Path, + lock: state::Lock, + binding: Binding, + observed: Option, + posts: Rc>, + replace_lock: bool, + wrong_birth: bool, + pending: &'a std::path::Path, +} +impl enrollment::sealed::Transport for Endpoint<'_> {} +impl enrollment::Transport for Endpoint<'_> { + fn verify(&mut self, expected: &Binding, deadline: Instant) -> Result<(), CandidateError> { + self.lock.verify_path(self.root)?; + if Instant::now() >= deadline || *expected != self.binding { + return Err(refused()); + } + Ok(()) + } + fn inspect( + &mut self, + _name: &str, + deadline: Instant, + ) -> Result, CandidateError> { + enrollment::Transport::verify(self, &self.binding.clone(), deadline)?; + Ok(self.observed.clone()) + } + fn create_new( + &mut self, + request: &enrollment::CreateRequest, + deadline: Instant, + ) -> Result { + create_original(self, request, deadline) + } +} +impl Creation for Endpoint<'_> { + fn fence(&mut self, binding: &Binding, deadline: Instant) -> Result<(), CandidateError> { + enrollment::Transport::verify(self, binding, deadline) + } + fn find( + &mut self, + name: &str, + deadline: Instant, + ) -> Result, CandidateError> { + enrollment::Transport::inspect(self, name, deadline) + } + fn post( + &mut self, + request: &enrollment::CreateRequest, + deadline: Instant, + ) -> Result { + self.fence(request.binding(), deadline)?; + let entries = fs::read_dir(self.pending) + .unwrap() + .collect::, _>>() + .unwrap(); + let bytes = fs::read(entries[0].path().join("owner.json")).unwrap(); + assert!(matches!( + decode(&bytes).unwrap().0.enrollment, + Enrollment::Pending { .. } + )); + self.posts.set(self.posts.get() + 1); + // Endpoint deliberately has Docker's idempotent behavior; creation authority + // must come from the still-held common lock, never this successful response. + let observed = self + .observed + .get_or_insert_with(|| Observation { + binding: request.binding().clone(), + volume: VolumeIdentity { + name: request.name().into(), + created_at: "2026-10-08T00:00:01Z".into(), + directory: DirectoryIdentity { + device: 0, + inode: 42, + }, + }, + }) + .clone(); + if self.replace_lock { + fs::rename( + self.root.join("operation.lock"), + self.root.join("former.lock"), + ) + .unwrap(); + drop(state::Lock::acquire(self.root).unwrap()); + } + if self.wrong_birth { + self.observed.as_mut().unwrap().volume.directory.inode += 1; + } + Ok(observed) + } +} +fn enroll(fixture: &Fixture, endpoint: &mut Endpoint<'_>) -> Result { + enrollment::enroll_new( + enrollment::EnrollOptions { + state_root: &fixture.data, + binding: &fixture.binding(), + intent: &"d".repeat(32), + deadline: Instant::now() + Duration::from_secs(5), + cancelled: &AtomicBool::new(false), + }, + endpoint, + ) +} +#[test] +fn common_original_lock_excludes_second_creator_and_data_has_no_run_binding() { + let fixture = Fixture::new(); + let posts = Rc::new(Cell::new(0)); + let mut first = fixture.transport(posts.clone()); + assert!( + matches!(state::Lock::acquire(&fixture.lease), Err(error) if error.code == "provider_busy") + ); + let owner = enroll(&fixture, &mut first).unwrap(); + assert_eq!(posts.get(), 1); + for _generation in ["e".repeat(32), "f".repeat(32)] { + enrollment::read_retained( + enrollment::ReadOptions { + state_root: &fixture.data, + binding: &fixture.binding(), + deadline: Instant::now() + Duration::from_secs(5), + cancelled: &AtomicBool::new(false), + }, + &mut first, + ) + .unwrap(); + compare(CompareOptions { + record: &owner, + expected: &fixture.binding(), + observed: first.observed.as_ref(), + }) + .unwrap(); + } + assert_eq!(posts.get(), 1); + let lock = fs::symlink_metadata(fixture.lease.join("operation.lock")).unwrap(); + assert_eq!((lock.dev(), lock.ino()), first.lock.identity().unwrap()); +} +#[test] +fn idempotent_row_after_lease_loss_or_changed_birth_never_promotes_or_replays() { + for changed_birth in [false, true] { + let fixture = Fixture::new(); + let posts = Rc::new(Cell::new(0)); + let mut endpoint = fixture.transport(posts.clone()); + endpoint.replace_lock = !changed_birth; + endpoint.wrong_birth = changed_birth; + assert!(enroll(&fixture, &mut endpoint).is_err()); + assert_eq!(posts.get(), 1); + let slot = fs::read_dir(&fixture.data) + .unwrap() + .next() + .unwrap() + .unwrap() + .path(); + let record = fs::read(slot.join("owner.json")).unwrap(); + assert!(matches!( + decode(&record).unwrap().0.enrollment, + Enrollment::Pending { .. } + )); + endpoint.replace_lock = false; + endpoint.wrong_birth = false; + assert!(enroll(&fixture, &mut endpoint).is_err()); + assert_eq!(posts.get(), 1); + assert_eq!(fs::read(slot.join("owner.json")).unwrap(), record); + assert!(endpoint.observed.is_some()); + } +} diff --git a/packages/runtime-core/src/provider/graph/native/persistent_data/enrollment.rs b/packages/runtime-core/src/provider/graph/native/persistent_data/enrollment.rs new file mode 100644 index 000000000..799179c53 --- /dev/null +++ b/packages/runtime-core/src/provider/graph/native/persistent_data/enrollment.rs @@ -0,0 +1,590 @@ +//! Durable enrollment owner. The native adapter separately supplies the cooperative +//! guest lease and original-effect identity; deserialization grants no effect authority. +//! +//! The transport must exclusively create a previously absent volume under its retained +//! guest authority, including its continuously held common cooperative mutation lease. +//! Unserialized direct same-user/guest writers are outside that authority. Docker's +//! idempotent volumes/create response, an absence probe, copied +//! labels or empty contents cannot meet that obligation. Errors never retry creation, +//! adopt an observed volume, delete data or recover an interrupted enrollment. + +use super::{Binding, CompareOptions, Enrollment, Kind, Observation, Owner, Record}; +use crate::{CandidateError, provider::state, reject_aliased_state}; +use sha2::{Digest, Sha256}; +use std::fs::{self, File, OpenOptions}; +use std::io::{Read, Seek, SeekFrom, Write}; +use std::os::unix::fs::{DirBuilderExt, MetadataExt, OpenOptionsExt}; +use std::path::{Path, PathBuf}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::time::Instant; + +// Sealing prevents callers from passing an arbitrary observation callback as create +// authority. Each crate-owned adapter requires its own effect qualification. +pub(crate) mod sealed { + pub trait Transport {} +} + +/// Trusted bounded transport under the existing guest/effect owner. +/// Each method must honor the supplied aggregate deadline; +/// `create_new` must refuse existing names within its closed supported-writer authority; +/// all supported competing creators must hold the same continuously fenced lease. +pub trait Transport: sealed::Transport { + fn verify(&mut self, expected: &Binding, deadline: Instant) -> Result<(), CandidateError>; + fn inspect( + &mut self, + name: &str, + deadline: Instant, + ) -> Result, CandidateError>; + fn create_new( + &mut self, + request: &CreateRequest, + deadline: Instant, + ) -> Result; +} + +/// Immutable original-create request, assembled only after the pending intent is durable. +/// It contains stable storage ownership, never a compute generation/run/plan reference. +pub struct CreateRequest { + binding: Binding, + name: String, + intent: String, +} +impl CreateRequest { + pub fn binding(&self) -> &Binding { + &self.binding + } + pub fn name(&self) -> &str { + &self.name + } + pub fn intent(&self) -> &str { + &self.intent + } +} + +pub struct EnrollOptions<'a> { + /// Existing private host directory external to application data; never initialized here. + pub state_root: &'a Path, + pub binding: &'a Binding, + /// Fresh owner-supplied nonce; no runtime generation is encoded or inferred. + pub intent: &'a str, + pub deadline: Instant, + pub cancelled: &'a AtomicBool, +} +pub struct ReadOptions<'a> { + pub state_root: &'a Path, + pub binding: &'a Binding, + pub deadline: Instant, + pub cancelled: &'a AtomicBool, +} + +/// Exclusively enroll one original create. The pending file and its parent are synced +/// before the sole create call. Promotion requires that call's captured identity plus +/// fresh matching observation and unchanged original private files/locks/guest fences. +/// Before rename, failure preserves pending (or incomplete staging); after rename, +/// publication/sync failure is uncertain and cannot return successful enrollment. +/// Nothing resumes or promotes an interrupted attempt in a subsequent invocation. +pub fn enroll_new( + options: EnrollOptions<'_>, + transport: &mut T, +) -> Result { + enroll(options, transport, &mut SystemSync) +} + +/// Observe existing enrollment under its existing lock. Missing root, lock, record, +/// pending intent or staging residue refuses without creating or repairing any file. +/// Success is a fresh retained-read comparison, not engine execution/deletion authority. +pub fn read_retained( + options: ReadOptions<'_>, + transport: &mut T, +) -> Result { + let binding = snapshot(options.binding)?; + let guard = Guard { + deadline: options.deadline, + cancelled: options.cancelled, + }; + guard.check()?; + let files = Files::existing(options.state_root, &binding)?; + let record = RecordPin::read(&files.slot.path.join("owner.json"))?; + let owner = super::decode(&record.bytes)?; + let Enrollment::Enrolled { volume } = &owner.0.enrollment else { + return Err(refused()); + }; + if volume.name != volume_name(&binding) { + return Err(refused()); + } + files.verify(Some(&record))?; + guard.check()?; + transport + .verify(&binding, guard.deadline) + .map_err(|_| refused())?; + files.verify(Some(&record))?; + guard.check()?; + let observed = transport + .inspect(&volume.name, guard.deadline) + .map_err(|_| refused())?; + super::compare(CompareOptions { + record: &owner, + expected: &binding, + observed: observed.as_ref(), + })?; + transport + .verify(&binding, guard.deadline) + .map_err(|_| refused())?; + files.verify(Some(&record))?; + guard.check()?; + Ok(owner) +} + +fn snapshot(binding: &Binding) -> Result { + if !super::binding_valid(binding) { + return Err(refused()); + } + Ok(binding.clone()) +} +fn refused() -> CandidateError { + CandidateError::new( + "native_persistent_data_enrollment", + "Persistent data enrollment is incomplete, ambiguous or changed; retained data was not adopted or deleted.", + ) +} +pub(super) fn volume_name(binding: &Binding) -> String { + format!( + "hkp-{}-{}-{}", + binding.scope.namespace, binding.scope.owner, binding.scope.storage + ) +} +fn slot_name(binding: &Binding) -> String { + slot_key(&binding.scope.namespace, &binding.scope.storage) +} +fn slot_key(namespace: &str, storage: &str) -> String { + let mut digest = Sha256::new(); + digest.update(namespace.as_bytes()); + digest.update([0]); + digest.update(storage.as_bytes()); + format!("persistent-{:x}", digest.finalize()) +} + +/// Read-only binding selection. It never turns pending/staging into enrollment and +/// grants no observation authority. The consumer must call read_retained afterward. +pub(super) fn existing_binding( + root: &Path, + namespace: &str, + storage: &str, +) -> Result, CandidateError> { + if !super::super::super::hex(namespace, 64) || !super::logical_name(storage) { + return Err(refused()); + } + let root = Directory::open(root)?; + let path = root.path.join(slot_key(namespace, storage)); + match fs::symlink_metadata(&path) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + root.verify()?; + return Ok(None); + } + Err(_) => return Err(refused()), + Ok(_) => {} + } + let slot = Directory::open(&path)?; + let lock = state::Lock::acquire_existing(&slot.path).map_err(|_| refused())?; + let files = Files { root, slot, lock }; + files.verify(None)?; + let record = RecordPin::read(&files.slot.path.join("owner.json"))?; + let owner = super::decode(&record.bytes)?; + if owner.0.binding.scope.namespace != namespace + || owner.0.binding.scope.storage != storage + || !matches!(owner.0.enrollment, Enrollment::Enrolled { .. }) + { + return Err(refused()); + } + files.verify(Some(&record))?; + Ok(Some(owner.0.binding)) +} + +struct Guard<'a> { + deadline: Instant, + cancelled: &'a AtomicBool, +} +impl Guard<'_> { + fn check(&self) -> Result<(), CandidateError> { + if self.cancelled.load(Ordering::SeqCst) || Instant::now() >= self.deadline { + return Err(refused()); + } + Ok(()) + } +} + +fn enroll( + options: EnrollOptions<'_>, + transport: &mut T, + sync: &mut S, +) -> Result { + let binding = snapshot(options.binding)?; + if !super::super::super::hex(options.intent, 32) { + return Err(refused()); + } + let request = CreateRequest { + name: volume_name(&binding), + binding, + intent: options.intent.into(), + }; + let guard = Guard { + deadline: options.deadline, + cancelled: options.cancelled, + }; + guard.check()?; + transport + .verify(&request.binding, guard.deadline) + .map_err(|_| refused())?; + guard.check()?; + if transport + .inspect(&request.name, guard.deadline) + .map_err(|_| refused())? + .is_some() + { + return Err(refused()); + } + guard.check()?; + let files = Files::fresh(options.state_root, &request.binding, sync)?; + let pending = RecordPin::create( + &files.slot.path.join("owner.json"), + &Owner(Record { + version: 1, + kind: Kind::NativePersistentDataOwner, + binding: request.binding.clone(), + enrollment: Enrollment::Pending { + intent: request.intent.clone(), + volume_name: request.name.clone(), + }, + }), + sync, + Step::PendingFile, + )?; + sync.sync(&files.slot.file, Step::PendingDirectory)?; + fresh(&files, &pending, &request.binding, &guard, transport)?; + if transport + .inspect(&request.name, guard.deadline) + .map_err(|_| refused())? + .is_some() + { + return Err(refused()); + } + // The last absence observation grants no create authority: the sealed adapter must + // still refuse a competing name within its continuously held common writer lease. + fresh(&files, &pending, &request.binding, &guard, transport)?; + let captured = transport + .create_new(&request, guard.deadline) + .map_err(|_| refused())?; + let enrolled = Owner(Record { + version: 1, + kind: Kind::NativePersistentDataOwner, + binding: request.binding.clone(), + enrollment: Enrollment::Enrolled { + volume: captured.volume.clone(), + }, + }); + super::compare(CompareOptions { + record: &enrolled, + expected: &request.binding, + observed: Some(&captured), + })?; + if captured.volume.name != request.name { + return Err(refused()); + } + fresh(&files, &pending, &request.binding, &guard, transport)?; + let observed = transport + .inspect(&request.name, guard.deadline) + .map_err(|_| refused())?; + super::compare(CompareOptions { + record: &enrolled, + expected: &request.binding, + observed: observed.as_ref(), + })?; + fresh(&files, &pending, &request.binding, &guard, transport)?; + let staged = RecordPin::create( + &files.slot.path.join("owner.next"), + &enrolled, + sync, + Step::EnrolledFile, + )?; + files.verify_record(&pending)?; + staged.verify()?; + transport + .verify(&request.binding, guard.deadline) + .map_err(|_| refused())?; + let observed = transport + .inspect(&request.name, guard.deadline) + .map_err(|_| refused())?; + super::compare(CompareOptions { + record: &enrolled, + expected: &request.binding, + observed: observed.as_ref(), + })?; + transport + .verify(&request.binding, guard.deadline) + .map_err(|_| refused())?; + files.verify_record(&pending)?; + staged.verify()?; + guard.check()?; + fs::rename(&staged.path, &pending.path).map_err(|_| refused())?; + // No arbitrary writer is admitted by this private nonblocking lock. As with other + // provider state, unsynchronized same-UID external writers are not atomically frozen. + let committed = staged.at(pending.path); + files.verify(Some(&committed))?; + sync.sync(&files.slot.file, Step::CommittedDirectory)?; + transport + .verify(&request.binding, guard.deadline) + .map_err(|_| refused())?; + let observed = transport + .inspect(&request.name, guard.deadline) + .map_err(|_| refused())?; + super::compare(CompareOptions { + record: &enrolled, + expected: &request.binding, + observed: observed.as_ref(), + })?; + transport + .verify(&request.binding, guard.deadline) + .map_err(|_| refused())?; + files.verify(Some(&committed))?; + guard.check()?; + Ok(enrolled) +} + +fn fresh( + files: &Files, + record: &RecordPin, + binding: &Binding, + guard: &Guard<'_>, + transport: &mut T, +) -> Result<(), CandidateError> { + guard.check()?; + files.verify(Some(record))?; + transport + .verify(binding, guard.deadline) + .map_err(|_| refused())?; + files.verify(Some(record))?; + guard.check() +} + +struct Directory { + path: PathBuf, + file: File, +} +impl Directory { + fn open(path: &Path) -> Result { + if !path.is_absolute() || fs::canonicalize(path).map_err(|_| refused())? != path { + return Err(refused()); + } + reject_aliased_state(path).map_err(|_| refused())?; + let file = OpenOptions::new() + .read(true) + .custom_flags(libc::O_DIRECTORY | libc::O_NOFOLLOW | libc::O_NONBLOCK) + .open(path) + .map_err(|_| refused())?; + let directory = Self { + path: path.into(), + file, + }; + directory.verify()?; + Ok(directory) + } + fn verify(&self) -> Result<(), CandidateError> { + reject_aliased_state(&self.path).map_err(|_| refused())?; + let fd = self.file.metadata().map_err(|_| refused())?; + let path = fs::symlink_metadata(&self.path).map_err(|_| refused())?; + if !fd.is_dir() || !safe(&fd, 0o700, false) || !same(&fd, &path) { + return Err(refused()); + } + Ok(()) + } +} +struct Files { + root: Directory, + slot: Directory, + lock: state::Lock, +} +impl Files { + fn fresh( + root: &Path, + binding: &Binding, + sync: &mut S, + ) -> Result { + let root = Directory::open(root)?; + let path = root.path.join(slot_name(binding)); + root.verify()?; + fs::DirBuilder::new() + .mode(0o700) + .create(&path) + .map_err(|_| refused())?; + let slot = Directory::open(&path)?; + root.verify()?; + sync.sync(&root.file, Step::SlotDirectory)?; + let lock_file = OpenOptions::new() + .read(true) + .write(true) + .create_new(true) + .mode(0o600) + .custom_flags(libc::O_NOFOLLOW) + .open(path.join("operation.lock")) + .map_err(|_| refused())?; + sync.sync(&lock_file, Step::LockFile)?; + let lock = state::Lock::from_file(lock_file).map_err(|_| refused())?; + let files = Self { root, slot, lock }; + files.verify(None)?; + Ok(files) + } + fn existing(root: &Path, binding: &Binding) -> Result { + let root = Directory::open(root)?; + let slot = Directory::open(&root.path.join(slot_name(binding)))?; + let lock = state::Lock::acquire_existing(&slot.path).map_err(|_| refused())?; + let files = Self { root, slot, lock }; + files.verify(None)?; + Ok(files) + } + fn verify_record(&self, record: &RecordPin) -> Result<(), CandidateError> { + self.root.verify()?; + self.slot.verify()?; + let lock = + fs::symlink_metadata(self.slot.path.join("operation.lock")).map_err(|_| refused())?; + if !lock.is_file() + || lock.len() != 0 + || !safe(&lock, 0o600, true) + || (lock.dev(), lock.ino()) != self.lock.identity().map_err(|_| refused())? + { + return Err(refused()); + } + record.verify() + } + fn verify(&self, record: Option<&RecordPin>) -> Result<(), CandidateError> { + absent(&self.slot.path.join("owner.next"))?; + match record { + Some(record) => self.verify_record(record), + None => { + self.root.verify()?; + self.slot.verify()?; + let lock = fs::symlink_metadata(self.slot.path.join("operation.lock")) + .map_err(|_| refused())?; + if !lock.is_file() + || lock.len() != 0 + || !safe(&lock, 0o600, true) + || (lock.dev(), lock.ino()) != self.lock.identity().map_err(|_| refused())? + { + return Err(refused()); + } + Ok(()) + } + } + } +} +fn absent(path: &Path) -> Result<(), CandidateError> { + match fs::symlink_metadata(path) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), + _ => Err(refused()), + } +} +fn safe(metadata: &fs::Metadata, mode: u32, single: bool) -> bool { + // SAFETY: geteuid has no preconditions. + metadata.uid() == unsafe { libc::geteuid() } + && metadata.mode() & 0o777 == mode + && (!single || metadata.nlink() == 1) +} +fn same(fd: &fs::Metadata, path: &fs::Metadata) -> bool { + fd.dev() == path.dev() && fd.ino() == path.ino() && fd.file_type() == path.file_type() +} +struct RecordPin { + path: PathBuf, + file: File, + bytes: Vec, +} +impl RecordPin { + fn create( + path: &Path, + owner: &Owner, + sync: &mut S, + step: Step, + ) -> Result { + let bytes = serde_json::to_vec(owner).map_err(|_| refused())?; + super::decode(&bytes)?; + let mut file = OpenOptions::new() + .read(true) + .write(true) + .create_new(true) + .mode(0o600) + .custom_flags(libc::O_NOFOLLOW) + .open(path) + .map_err(|_| refused())?; + file.write_all(&bytes).map_err(|_| refused())?; + sync.sync(&file, step)?; + let pin = Self { + path: path.into(), + file, + bytes, + }; + pin.verify()?; + Ok(pin) + } + fn read(path: &Path) -> Result { + let file = OpenOptions::new() + .read(true) + .custom_flags(libc::O_NOFOLLOW | libc::O_NONBLOCK) + .open(path) + .map_err(|_| refused())?; + let bytes = read_bytes(&file)?; + super::decode(&bytes)?; + let pin = Self { + path: path.into(), + file, + bytes, + }; + pin.verify()?; + Ok(pin) + } + fn at(self, path: PathBuf) -> Self { + Self { path, ..self } + } + fn verify(&self) -> Result<(), CandidateError> { + let fd = self.file.metadata().map_err(|_| refused())?; + let path = fs::symlink_metadata(&self.path).map_err(|_| refused())?; + if !same(&fd, &path) || read_bytes(&self.file)? != self.bytes { + return Err(refused()); + } + Ok(()) + } +} +fn read_bytes(mut file: &File) -> Result, CandidateError> { + let m = file.metadata().map_err(|_| refused())?; + if !m.is_file() || !safe(&m, 0o600, true) || m.len() > super::LIMIT as u64 { + return Err(refused()); + } + file.seek(SeekFrom::Start(0)).map_err(|_| refused())?; + let mut bytes = Vec::new(); + file.take(super::LIMIT as u64 + 1) + .read_to_end(&mut bytes) + .map_err(|_| refused())?; + if bytes.is_empty() || bytes.len() > super::LIMIT { + return Err(refused()); + } + Ok(bytes) +} + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Step { + SlotDirectory, + LockFile, + PendingFile, + PendingDirectory, + EnrolledFile, + CommittedDirectory, +} +trait Sync { + fn sync(&mut self, file: &File, step: Step) -> Result<(), CandidateError>; +} +struct SystemSync; +impl Sync for SystemSync { + fn sync(&mut self, file: &File, _step: Step) -> Result<(), CandidateError> { + file.sync_all().map_err(|_| refused()) + } +} + +#[cfg(test)] +mod tests; diff --git a/packages/runtime-core/src/provider/graph/native/persistent_data/enrollment/tests.rs b/packages/runtime-core/src/provider/graph/native/persistent_data/enrollment/tests.rs new file mode 100644 index 000000000..b79bece11 --- /dev/null +++ b/packages/runtime-core/src/provider/graph/native/persistent_data/enrollment/tests.rs @@ -0,0 +1,562 @@ +use super::super::{DirectoryIdentity, VolumeIdentity}; +use super::*; +use serde_json::{Value, json}; +use std::cell::RefCell; +use std::collections::BTreeMap; +use std::os::unix::fs::{PermissionsExt, symlink}; +use std::rc::Rc; +use std::sync::atomic::AtomicU64; +use std::time::{Duration, SystemTime, UNIX_EPOCH}; + +static NEXT: AtomicU64 = AtomicU64::new(0); +struct Fixture(PathBuf); +impl Fixture { + fn new() -> Self { + let base = fs::canonicalize(std::env::temp_dir()).unwrap(); + let path = base.join(format!( + "native-persistent-enroll-{}-{}-{}", + std::process::id(), + SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(), + NEXT.fetch_add(1, Ordering::SeqCst) + )); + fs::DirBuilder::new().mode(0o700).create(&path).unwrap(); + Self(path) + } + fn slot(&self) -> PathBuf { + self.0.join(slot_name(&binding())) + } + fn record(&self) -> Value { + serde_json::from_slice(&fs::read(self.slot().join("owner.json")).unwrap()).unwrap() + } +} +impl Drop for Fixture { + fn drop(&mut self) { + fs::remove_dir_all(&self.0).unwrap(); + } +} +fn binding() -> Binding { + serde_json::from_value(json!({ + "scope":{"namespace":"a".repeat(64),"storage":"db_data","owner":"b".repeat(32)}, + "guest":{"owner":"c".repeat(32),"boot_id":"11111111-2222-3333-4444-555555555555", + "storage":{"device":0,"inode":25,"bytes":8192,"uuid":"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"}}, + "policy":{"driver":"local","scope":"local","options":{}} + })).unwrap() +} +fn observation() -> Observation { + Observation { + binding: binding(), + volume: VolumeIdentity { + name: volume_name(&binding()), + created_at: "2026-10-08T12:34:56.123456789Z".into(), + directory: DirectoryIdentity { + device: 0, + inode: 91, + }, + }, + } +} +#[derive(Clone, Copy, PartialEq, Eq)] +enum Event { + Verify(usize), + Inspect(usize), + Create, +} +struct Fake { + current: BTreeMap, + verifies: usize, + inspects: usize, + creates: usize, + fail_create: bool, + replace_after_create: bool, + fail: Option, + hook: Option>, +} +impl Fake { + fn new() -> Self { + Self { + current: BTreeMap::new(), + verifies: 0, + inspects: 0, + creates: 0, + fail_create: false, + replace_after_create: false, + fail: None, + hook: None, + } + } + fn event(&mut self, event: Event) { + if let Some(hook) = &mut self.hook { + hook(event); + } + } +} +impl sealed::Transport for Fake {} +impl Transport for Fake { + fn verify(&mut self, expected: &Binding, deadline: Instant) -> Result<(), CandidateError> { + self.verifies += 1; + self.event(Event::Verify(self.verifies)); + if self.fail == Some(Event::Verify(self.verifies)) { + return Err(canary()); + } + if *expected != binding() || Instant::now() >= deadline { + return Err(refused()); + } + Ok(()) + } + fn inspect( + &mut self, + name: &str, + deadline: Instant, + ) -> Result, CandidateError> { + self.inspects += 1; + self.event(Event::Inspect(self.inspects)); + if self.fail == Some(Event::Inspect(self.inspects)) { + return Err(canary()); + } + if Instant::now() >= deadline { + return Err(refused()); + } + Ok(self.current.get(name).cloned()) + } + fn create_new( + &mut self, + request: &CreateRequest, + deadline: Instant, + ) -> Result { + self.creates += 1; + self.event(Event::Create); + if Instant::now() >= deadline || self.current.contains_key(request.name()) { + return Err(refused()); + } + assert!(request.binding() == &binding()); + assert_eq!(request.intent(), "d".repeat(32)); + let created = observation(); + self.current.insert(request.name().into(), created.clone()); + if self.replace_after_create { + self.current + .get_mut(request.name()) + .unwrap() + .volume + .directory + .inode += 1; + } + if self.fail_create { + return Err(canary()); + } + Ok(created) + } +} +fn canary() -> CandidateError { + CandidateError::new("adapter-private-canary", "private-value-canary/path/volume") + .with_cause_code("private-cause-canary".into()) +} +struct FaultSync { + fail: Option, + seen: Rc>>, +} +impl Sync for FaultSync { + fn sync(&mut self, file: &File, step: Step) -> Result<(), CandidateError> { + self.seen.borrow_mut().push(step); + if self.fail == Some(step) { + return Err(refused()); + } + file.sync_all().map_err(|_| refused()) + } +} +fn options<'a>( + fixture: &'a Fixture, + binding: &'a Binding, + cancelled: &'a AtomicBool, +) -> EnrollOptions<'a> { + EnrollOptions { + state_root: &fixture.0, + binding, + intent: "dddddddddddddddddddddddddddddddd", + deadline: Instant::now() + Duration::from_secs(10), + cancelled, + } +} +fn read(fixture: &Fixture, binding: &Binding, fake: &mut Fake) -> Result { + read_retained( + ReadOptions { + state_root: &fixture.0, + binding, + deadline: Instant::now() + Duration::from_secs(10), + cancelled: &AtomicBool::new(false), + }, + fake, + ) +} +fn enroll_fixture(fixture: &Fixture, fake: &mut Fake) -> Owner { + enroll_new(options(fixture, &binding(), &AtomicBool::new(false)), fake).unwrap() +} + +#[test] +fn durable_pending_precedes_the_original_effect_and_retained_reads_do_not_rebind_generations() { + let fixture = Fixture::new(); + let seen = Rc::new(RefCell::new(Vec::new())); + let at_create = seen.clone(); + let slot = fixture.slot(); + let mut fake = Fake::new(); + fake.hook = Some(Box::new(move |event| { + if event == Event::Create { + assert!( + *at_create.borrow() + == vec![ + Step::SlotDirectory, + Step::LockFile, + Step::PendingFile, + Step::PendingDirectory + ] + ); + let bytes = fs::read(slot.join("owner.json")).unwrap(); + let pending = super::super::decode(&bytes).unwrap(); + assert!( + super::super::compare(CompareOptions { + record: &pending, + expected: &binding(), + observed: Some(&observation()) + }) + .is_err() + ); + assert_eq!( + fs::metadata(slot.join("owner.json")).unwrap().mode() & 0o777, + 0o600 + ); + } + })); + let enrolled = enroll( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake, + &mut FaultSync { fail: None, seen }, + ) + .unwrap(); + let original = fs::read(fixture.slot().join("owner.json")).unwrap(); + let original_inode = fs::metadata(fixture.slot().join("owner.json")) + .unwrap() + .ino(); + assert_eq!(fake.creates, 1); + assert_eq!(serde_json::to_value(enrolled).unwrap(), fixture.record()); + fake.hook = None; + for run in ["1".repeat(32), "2".repeat(32)] { + let scope = crate::provider::native_input::Scope { + namespace: &binding().scope.namespace, + run: &run, + }; + let mut expected = binding(); + expected.scope.namespace = scope.namespace.into(); + assert!(read(&fixture, &expected, &mut fake).is_ok()); + } + assert_eq!(fake.creates, 1); + assert_eq!( + fs::read(fixture.slot().join("owner.json")).unwrap(), + original + ); + assert_eq!( + fs::metadata(fixture.slot().join("owner.json")) + .unwrap() + .ino(), + original_inode + ); + assert!(!String::from_utf8(original).unwrap().contains("\"run\"")); +} + +#[test] +fn preexisting_names_slots_and_pending_never_gain_enrollment_or_a_second_effect() { + let fixture = Fixture::new(); + let mut fake = Fake::new(); + fake.current.insert(volume_name(&binding()), observation()); + assert!( + enroll_new( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake + ) + .is_err() + ); + assert_eq!(fake.creates, 0); + assert!(!fixture.slot().exists()); + fake.current.clear(); + fake.fail_create = true; + assert!( + enroll_new( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake + ) + .is_err() + ); + let original = fs::read(fixture.slot().join("owner.json")).unwrap(); + assert_eq!(fixture.record()["enrollment"]["status"], "pending"); + assert!(read(&fixture, &binding(), &mut fake).is_err()); + fake.current.clear(); // Even apparent engine absence cannot recover an old attempt. + fake.fail_create = false; + assert!( + enroll_new( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake + ) + .is_err() + ); + assert_eq!(fake.creates, 1); + assert_eq!( + fs::read(fixture.slot().join("owner.json")).unwrap(), + original + ); + assert!(fixture.slot().join("operation.lock").is_file()); +} + +#[test] +fn adapter_errors_are_normalized_before_return_including_ambiguous_original_create() { + for event in [ + Event::Verify(1), + Event::Inspect(1), + Event::Create, + Event::Inspect(3), + ] { + let fixture = Fixture::new(); + let mut fake = Fake::new(); + if event == Event::Create { + fake.fail_create = true; + } else { + fake.fail = Some(event); + } + let error = enroll_new( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake, + ) + .err() + .unwrap(); + assert_eq!( + serde_json::to_value(&error).unwrap(), + json!({ + "code": "native_persistent_data_enrollment", + "message": "Persistent data enrollment is incomplete, ambiguous or changed; retained data was not adopted or deleted." + }) + ); + assert!(!serde_json::to_string(&error).unwrap().contains("private-")); + if fake.creates > 0 { + assert_eq!(fixture.record()["enrollment"]["status"], "pending"); + assert_eq!(fake.current.len(), 1); + } + } + let fixture = Fixture::new(); + let mut fake = Fake::new(); + enroll_fixture(&fixture, &mut fake); + fake.fail = Some(Event::Inspect(fake.inspects + 1)); + let before = fs::read(fixture.slot().join("owner.json")).unwrap(); + let error = read(&fixture, &binding(), &mut fake).err().unwrap(); + assert_eq!( + serde_json::to_value(&error).unwrap(), + serde_json::to_value(refused()).unwrap() + ); + assert_eq!(fs::read(fixture.slot().join("owner.json")).unwrap(), before); + assert_eq!(fake.creates, 1); +} + +#[test] +fn pending_sync_failure_prevents_create_and_final_sync_failure_is_uncertain() { + for step in [ + Step::PendingFile, + Step::PendingDirectory, + Step::EnrolledFile, + Step::CommittedDirectory, + ] { + let fixture = Fixture::new(); + let mut fake = Fake::new(); + let result = enroll( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake, + &mut FaultSync { + fail: Some(step), + seen: Rc::new(RefCell::new(Vec::new())), + }, + ); + assert!(result.is_err()); + assert_eq!( + fake.creates, + usize::from(matches!( + step, + Step::EnrolledFile | Step::CommittedDirectory + )) + ); + let status = if step == Step::CommittedDirectory { + "enrolled" + } else { + "pending" + }; + assert_eq!(fixture.record()["enrollment"]["status"], status); + // An enrolled pathname after failed final sync is an observation of uncertain + // publication, not successful-return evidence or permission to repeat creation. + assert!( + enroll_new( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake + ) + .is_err() + ); + } +} + +#[test] +fn captured_original_identity_cannot_be_replaced_by_copied_labels_or_a_new_directory() { + let fixture = Fixture::new(); + let mut fake = Fake::new(); + fake.replace_after_create = true; + assert!( + enroll_new( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake + ) + .is_err() + ); + assert_eq!(fake.creates, 1); + assert_eq!(fixture.record()["enrollment"]["status"], "pending"); + assert_eq!( + fake.current.values().next().unwrap().volume.directory.inode, + 92 + ); + assert!(!fixture.slot().join("owner.next").exists()); +} + +#[test] +fn invalid_input_expiration_and_cancel_refuse_before_any_file_or_create() { + let fixture = Fixture::new(); + let mut fake = Fake::new(); + let cancelled = AtomicBool::new(false); + let mut invalid = binding(); + invalid.scope.storage = "../unowned".into(); + assert!(enroll_new(options(&fixture, &invalid, &cancelled), &mut fake).is_err()); + let valid = binding(); + let mut expired = options(&fixture, &valid, &cancelled); + expired.deadline = Instant::now(); + assert!(enroll_new(expired, &mut fake).is_err()); + cancelled.store(true, Ordering::SeqCst); + assert!(enroll_new(options(&fixture, &valid, &cancelled), &mut fake).is_err()); + assert_eq!(fake.verifies, 0); + assert_eq!(fake.creates, 0); + assert_eq!(fs::read_dir(&fixture.0).unwrap().count(), 0); +} + +#[test] +fn post_effect_record_lock_and_root_replacements_refuse_without_adopting_or_deleting() { + for target in ["owner.json", "operation.lock", "root"] { + let fixture = Fixture::new(); + let slot = fixture.slot(); + let root = fixture.0.clone(); + let mut fake = Fake::new(); + fake.hook = Some(Box::new(move |event| { + if event == Event::Create { + if target == "root" { + fs::rename(&root, root.with_extension("old")).unwrap(); + fs::DirBuilder::new().mode(0o700).create(&root).unwrap(); + fs::rename(root.with_extension("old"), root.join("retained-original")).unwrap(); + } else { + let path = slot.join(target); + let bytes = fs::read(&path).unwrap(); + fs::rename(&path, slot.join(format!("{target}.old"))).unwrap(); + let mut replacement = OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(&path) + .unwrap(); + replacement.write_all(&bytes).unwrap(); + } + } + })); + assert!( + enroll_new( + options(&fixture, &binding(), &AtomicBool::new(false)), + &mut fake + ) + .is_err() + ); + assert_eq!(fake.creates, 1); + assert_eq!(fake.current.len(), 1); + assert!(!fixture.slot().join("owner.next").exists()); + } +} + +#[test] +fn retained_read_is_existing_only_and_refuses_missing_lock_staging_birth_or_guest_drift() { + let fixture = Fixture::new(); + let mut fake = Fake::new(); + assert!(read(&fixture, &binding(), &mut fake).is_err()); + assert_eq!(fs::read_dir(&fixture.0).unwrap().count(), 0); + enroll_fixture(&fixture, &mut fake); + let original = fs::read(fixture.slot().join("owner.json")).unwrap(); + let lock = fixture.slot().join("operation.lock"); + fs::rename(&lock, fixture.slot().join("retained.lock")).unwrap(); + assert!(read(&fixture, &binding(), &mut fake).is_err()); + assert!(!lock.exists()); + fs::rename(fixture.slot().join("retained.lock"), &lock).unwrap(); + fs::write(fixture.slot().join("owner.next"), b"incomplete").unwrap(); + assert!(read(&fixture, &binding(), &mut fake).is_err()); + fs::remove_file(fixture.slot().join("owner.next")).unwrap(); + fake.current + .get_mut(&volume_name(&binding())) + .unwrap() + .volume + .directory + .inode += 1; + assert!(read(&fixture, &binding(), &mut fake).is_err()); + fake.current.insert(volume_name(&binding()), observation()); + let mut expected = binding(); + expected.guest.boot_id = "ffffffff-2222-3333-4444-555555555555".into(); + assert!(read(&fixture, &expected, &mut fake).is_err()); + expected = binding(); + expected.scope.owner = "f".repeat(32); + assert!(read(&fixture, &expected, &mut fake).is_err()); + assert_eq!( + fs::read(fixture.slot().join("owner.json")).unwrap(), + original + ); + assert_eq!(fake.creates, 1); +} + +#[test] +fn private_file_rules_lock_exclusion_and_final_cancel_fences_are_real() { + let fixture = Fixture::new(); + let mut fake = Fake::new(); + enroll_fixture(&fixture, &mut fake); + let lock = state::Lock::acquire_existing(&fixture.slot()).unwrap(); + assert!(read(&fixture, &binding(), &mut fake).is_err()); + drop(lock); + let path = fixture.slot().join("owner.json"); + let original = fs::read(&path).unwrap(); + fs::set_permissions(&path, fs::Permissions::from_mode(0o644)).unwrap(); + assert!(read(&fixture, &binding(), &mut fake).is_err()); + fs::set_permissions(&path, fs::Permissions::from_mode(0o600)).unwrap(); + fs::hard_link(&path, fixture.slot().join("alias")).unwrap(); + assert!(read(&fixture, &binding(), &mut fake).is_err()); + fs::remove_file(fixture.slot().join("alias")).unwrap(); + fs::rename(&path, fixture.slot().join("original")).unwrap(); + symlink(fixture.slot().join("original"), &path).unwrap(); + assert!(read(&fixture, &binding(), &mut fake).is_err()); + fs::remove_file(&path).unwrap(); + fs::rename(fixture.slot().join("original"), &path).unwrap(); + let cancelled = Rc::new(AtomicBool::new(false)); + let from_probe = cancelled.clone(); + let last = fake.inspects + 1; + fake.hook = Some(Box::new(move |event| { + if event == Event::Inspect(last) { + from_probe.store(true, Ordering::SeqCst); + } + })); + assert!( + read_retained( + ReadOptions { + state_root: &fixture.0, + binding: &binding(), + deadline: Instant::now() + Duration::from_secs(10), + cancelled: &cancelled + }, + &mut fake + ) + .is_err() + ); + assert_eq!(fs::read(&path).unwrap(), original); + assert_eq!(fake.creates, 1); +} diff --git a/packages/runtime-core/src/provider/graph/native/persistent_data/tests.rs b/packages/runtime-core/src/provider/graph/native/persistent_data/tests.rs new file mode 100644 index 000000000..d3f2bff7a --- /dev/null +++ b/packages/runtime-core/src/provider/graph/native/persistent_data/tests.rs @@ -0,0 +1,288 @@ +use super::*; +use serde_json::{Value, json}; + +fn input(pending: bool) -> Value { + json!({ + "version": 1, "kind": "native-persistent-data-owner", + "binding": { + "scope": {"namespace": "a".repeat(64), "storage": "db_data", "owner": "b".repeat(32)}, + "guest": { + "owner": "c".repeat(32), "boot_id": "11111111-2222-3333-4444-555555555555", + "storage": {"device": 0, "inode": 25, "bytes": 8192, "uuid": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"} + }, + "policy": {"driver": "local", "scope": "local", "options": {}} + }, + "enrollment": if pending { + json!({"status": "pending", "intent": "d".repeat(32), "volume_name": "hkp-owned-db_data"}) + } else { + json!({"status": "enrolled", "volume": { + "name": "hkp-owned-db_data", "created_at": "2026-10-08T12:34:56.123456789Z", + "directory": {"device": 0, "inode": 91} + }}) + } + }) +} +fn owner(value: &Value) -> Owner { + decode(&serde_json::to_vec(value).unwrap()).unwrap() +} +fn observation(value: &Value) -> Observation { + serde_json::from_value(json!({ + "binding": value["binding"], "volume": value["enrollment"]["volume"] + })) + .unwrap() +} +fn matches(record: &Owner, expected: &Binding, observed: Option<&Observation>) -> bool { + compare(CompareOptions { + record, + expected, + observed, + }) + .is_ok() +} +fn rejects(value: Value) { + let error = decode(&serde_json::to_vec(&value).unwrap()).err().unwrap(); + assert_eq!(error.code, "native_persistent_data_identity"); + assert!( + !serde_json::to_string(&error) + .unwrap() + .contains("private-canary") + ); +} + +#[test] +fn independent_runtime_generations_share_only_the_explicit_persistent_binding() { + let raw = input(false); + let record = owner(&raw); + let observed = observation(&raw); + for run in ["1".repeat(32), "2".repeat(32)] { + // The caller may have a different run reference; it is neither compared nor encoded. + let runtime_scope = crate::provider::native_input::Scope { + namespace: &observed.binding.scope.namespace, + run: &run, + }; + let mut expected = observed.binding.clone(); + expected.scope.namespace = runtime_scope.namespace.into(); + assert!(matches(&record, &expected, Some(&observed))); + let encoded = serde_json::to_string(&record).unwrap(); + assert!(!encoded.contains(runtime_scope.run)); + assert!(!encoded.contains("generation")); + assert!(!encoded.contains("\"run\"")); + assert!(!encoded.contains("\"plan\"")); + } + assert_eq!(serde_json::to_value(record).unwrap(), raw); +} + +#[test] +fn pending_is_roundtrippable_but_never_enrolled_even_with_exact_observation() { + let pending = input(true); + let record = owner(&pending); + let observed = observation(&input(false)); + assert_eq!(serde_json::to_value(&record).unwrap(), pending); + assert!(!matches(&record, &observed.binding, Some(&observed))); + assert!(!matches(&record, &observed.binding, None)); + assert!(!matches(&owner(&input(false)), &observed.binding, None)); + let mut mixed = pending; + mixed["enrollment"]["volume"] = input(false)["enrollment"]["volume"].clone(); + rejects(mixed); +} + +#[test] +fn every_observed_identity_change_refuses_including_copied_labels_new_birth() { + let raw = input(false); + let record = owner(&raw); + let expected = observation(&raw).binding; + for (pointer, value) in [ + ("/binding/scope/namespace", json!("f".repeat(64))), + ("/binding/scope/storage", json!("other_data")), + ("/binding/scope/owner", json!("f".repeat(32))), + ("/binding/guest/owner", json!("f".repeat(32))), + ( + "/binding/guest/boot_id", + json!("ffffffff-2222-3333-4444-555555555555"), + ), + ("/binding/guest/storage/device", json!(1)), + ("/binding/guest/storage/inode", json!(26)), + ("/binding/guest/storage/bytes", json!(16384)), + ( + "/binding/guest/storage/uuid", + json!("ffffffff-bbbb-cccc-dddd-eeeeeeeeeeee"), + ), + ("/enrollment/volume/name", json!("hkp-other-db_data")), + ( + "/enrollment/volume/created_at", + json!("2026-10-08T12:34:56.123456790Z"), + ), + ("/enrollment/volume/directory/device", json!(1)), + ("/enrollment/volume/directory/inode", json!(92)), + ] { + let mut changed = raw.clone(); + *changed.pointer_mut(pointer).unwrap() = value; + let observed = observation(&changed); + assert!( + !matches(&record, &expected, Some(&observed)), + "changed field {pointer}" + ); + if pointer.starts_with("/binding/") { + assert!( + !matches(&record, &observed.binding, Some(&observation(&raw))), + "expected field {pointer}" + ); + } + } +} + +#[test] +fn strict_wire_refuses_versions_kinds_missing_null_unknown_fields_and_policy() { + for (pointer, value) in [ + ("/version", json!(2)), + ("/version", json!(1.0)), + ("/kind", json!("cache-provenance")), + ("/binding/scope/namespace", json!("A".repeat(64))), + ("/binding/scope/owner", json!("b".repeat(31))), + ("/binding/scope/storage", json!("../private-canary")), + ( + "/binding/guest/boot_id", + json!("00000000-0000-0000-0000-000000000000"), + ), + ( + "/binding/guest/boot_id", + json!("11111111-2222-3333-4444-555555555555\n"), + ), + ("/binding/guest/storage/uuid", json!("missing")), + ("/binding/guest/storage/bytes", json!(0)), + ("/binding/guest/storage/inode", json!(0)), + ("/binding/policy/driver", json!("foreign")), + ("/binding/policy/scope", json!("global")), + ("/binding/policy/options", json!({"type":"bind"})), + ("/enrollment/status", json!("adopted")), + ("/enrollment/volume/name", json!("/private-canary")), + ("/enrollment/volume/directory/inode", json!(0)), + ] { + let mut changed = input(false); + *changed.pointer_mut(pointer).unwrap() = value; + rejects(changed); + } + for pointer in [ + "", + "/binding", + "/binding/scope", + "/binding/guest", + "/binding/guest/storage", + "/binding/policy", + "/binding/policy/options", + "/enrollment", + "/enrollment/volume", + "/enrollment/volume/directory", + ] { + let mut changed = input(false); + changed + .pointer_mut(pointer) + .unwrap() + .as_object_mut() + .unwrap() + .insert("private-canary".into(), json!(true)); + rejects(changed); + } + for key in ["version", "kind", "binding", "enrollment"] { + let mut missing = input(false); + missing.as_object_mut().unwrap().remove(key); + rejects(missing); + let mut null = input(false); + null[key] = Value::Null; + rejects(null); + } + for (parent, key) in [ + ("/binding", "scope"), + ("/binding", "guest"), + ("/binding", "policy"), + ("/binding/scope", "namespace"), + ("/binding/scope", "storage"), + ("/binding/scope", "owner"), + ("/binding/guest", "owner"), + ("/binding/guest", "boot_id"), + ("/binding/guest", "storage"), + ("/binding/guest/storage", "device"), + ("/binding/guest/storage", "inode"), + ("/binding/guest/storage", "bytes"), + ("/binding/guest/storage", "uuid"), + ("/binding/policy", "driver"), + ("/binding/policy", "scope"), + ("/binding/policy", "options"), + ("/enrollment", "status"), + ("/enrollment", "volume"), + ("/enrollment/volume", "name"), + ("/enrollment/volume", "created_at"), + ("/enrollment/volume", "directory"), + ("/enrollment/volume/directory", "device"), + ("/enrollment/volume/directory", "inode"), + ] { + let mut missing = input(false); + missing + .pointer_mut(parent) + .unwrap() + .as_object_mut() + .unwrap() + .remove(key); + rejects(missing); + let mut null = input(false); + null.pointer_mut(parent).unwrap()[key] = Value::Null; + rejects(null); + } + let mut null = input(false); + null["binding"]["policy"]["options"] = Value::Null; + rejects(null); + let mut invalid_pending = input(true); + invalid_pending["enrollment"]["intent"] = json!("invalid"); + rejects(invalid_pending); +} + +#[test] +fn duplicate_fields_and_preallocation_bound_refuse() { + let raw = serde_json::to_string(&input(false)).unwrap(); + for (needle, duplicate) in [ + ("\"version\":1", "\"version\":1,\"version\":1"), + ( + "\"status\":\"enrolled\"", + "\"status\":\"enrolled\",\"status\":\"enrolled\"", + ), + ( + "\"storage\":\"db_data\"", + "\"storage\":\"db_data\",\"storage\":\"db_data\"", + ), + ("\"inode\":91", "\"inode\":91,\"inode\":91"), + ] { + assert!(raw.contains(needle)); + assert!(decode(raw.replacen(needle, duplicate, 1).as_bytes()).is_err()); + } + assert!(decode(&[]).is_err()); + assert!(decode(&vec![b' '; LIMIT + 1]).is_err()); + assert!(decode(b"\xff").is_err()); + assert!(decode(format!("{raw} {raw}").as_bytes()).is_err()); +} + +#[test] +fn malformed_or_unavailable_birth_refuses_without_normalizing_identity() { + for birth in [ + "", + "private-canary", + "0001-01-01T00:00:00Z", + "0001-01-01T00:00:00.000000000Z", + "2026-02-29T00:00:00Z", + "2026-10-08T24:00:00Z", + "2026-10-08T12:34:56.Z", + "2026-10-08T12:34:56.1234567890Z", + "2026-10-08T12:34:56+00:00", + ] { + let mut changed = input(false); + changed["enrollment"]["volume"]["created_at"] = json!(birth); + rejects(changed); + } + let mut changed = input(false); + changed["enrollment"]["volume"]["created_at"] = Value::Null; + rejects(changed); + for birth in ["2000-02-29T00:00:00Z", "2026-10-08T12:34:56.0Z"] { + let mut changed = input(false); + changed["enrollment"]["volume"]["created_at"] = json!(birth); + assert!(decode(&serde_json::to_vec(&changed).unwrap()).is_ok()); + } +} diff --git a/packages/runtime-core/src/provider/graph/native/runtime.rs b/packages/runtime-core/src/provider/graph/native/runtime.rs index f65f8b4d8..44f010cb1 100644 --- a/packages/runtime-core/src/provider/graph/native/runtime.rs +++ b/packages/runtime-core/src/provider/graph/native/runtime.rs @@ -21,6 +21,18 @@ trait Backend { fn verify_private(&self, _service: &str) -> Result<(), CandidateError> { Ok(()) } + fn verify_data( + &self, + receipt: &Receipt, + _deadline: Instant, + _fresh: &dyn Fn() -> Result<(), CandidateError>, + ) -> Result<(), CandidateError> { + if receipt.data.is_empty() { + Ok(()) + } else { + Err(refused()) + } + } fn stop( &self, _selected: &[(String, u64)], @@ -36,6 +48,20 @@ struct GuardedBackend<'a, B> { guard: Option<&'a dyn Fn() -> Result<(), CandidateError>>, } impl Backend for GuardedBackend<'_, B> { + fn verify_data( + &self, + receipt: &Receipt, + deadline: Instant, + fresh: &dyn Fn() -> Result<(), CandidateError>, + ) -> Result<(), CandidateError> { + let guard = || { + check_startup(self.guard)?; + fresh() + }; + let result = self.backend.verify_data(receipt, deadline, &guard); + guard()?; + result + } fn request( &self, method: Method, @@ -59,6 +85,14 @@ impl Backend for GuardedBackend<'_, B> { } } impl Backend for Engine<'_> { + fn verify_data( + &self, + receipt: &Receipt, + deadline: Instant, + fresh: &dyn Fn() -> Result<(), CandidateError>, + ) -> Result<(), CandidateError> { + verify_data_using_engine(self, receipt, deadline, fresh) + } fn request( &self, method: Method, @@ -74,6 +108,14 @@ struct OwnedBackend<'a> { leases: BTreeMap, } impl Backend for OwnedBackend<'_> { + fn verify_data( + &self, + receipt: &Receipt, + deadline: Instant, + fresh: &dyn Fn() -> Result<(), CandidateError>, + ) -> Result<(), CandidateError> { + verify_data_using_engine(&self.engine, receipt, deadline, fresh) + } fn request( &self, method: Method, @@ -114,6 +156,43 @@ impl Backend for OwnedBackend<'_> { } } +fn verify_prior_data_retirement( + prior: &Receipt, + selected: &BTreeMap, +) -> Result<(), CandidateError> { + if prior.data.values().any(|reference| { + selected + .values() + .any(|current| current.name() == reference.name()) + && (prior.phase != Phase::Removed || !reference.enrolled()) + }) { + return Err(refused()); + } + Ok(()) +} + +fn verify_data_using_engine( + engine: &Engine<'_>, + receipt: &Receipt, + deadline: Instant, + fresh: &dyn Fn() -> Result<(), CandidateError>, +) -> Result<(), CandidateError> { + for reference in receipt + .data + .values() + .filter(|reference| reference.enrolled()) + { + persistent_data::engine::verify( + engine.guest().candidate(), + engine, + reference, + deadline, + fresh, + )?; + } + fresh() +} + fn ownership(receipt: &Receipt, resource: &Resource, value: &Value) -> Result<(), CandidateError> { let expected = labels(&receipt.owner, &receipt.review, resource); let id = value["Id"] @@ -211,6 +290,7 @@ fn verify_attachment( service: &str, actual: &Value, ) -> Result<(), CandidateError> { + verify_data_mounts(receipt, service, actual)?; let network = receipt .resources .get("network:default") @@ -233,6 +313,49 @@ fn verify_attachment( Ok(()) } +fn verify_data_mounts( + receipt: &Receipt, + service: &str, + actual: &Value, +) -> Result<(), CandidateError> { + if receipt.data.is_empty() { + return Ok(()); + } + let expected = receipt + .data_mounts + .get(service) + .map(Vec::as_slice) + .unwrap_or(&[]); + let mounts = actual["Mounts"].as_array().ok_or_else(refused)?; + if mounts.len() != expected.len() { + return Err(refused()); + } + for selected in expected { + let reference = receipt + .data + .get(&selected.storage) + .filter(|reference| reference.enrolled()) + .ok_or_else(refused)?; + let matching = mounts + .iter() + .filter(|mount| mount["Destination"] == selected.target) + .collect::>(); + if matching.len() != 1 { + return Err(refused()); + } + let mount = matching[0]; + if mount["Type"] != "volume" + || mount["Name"] != reference.name() + || mount["Source"] != reference.mountpoint() + || mount["Driver"] != "local" + || mount["RW"].as_bool() != Some(!selected.read_only) + { + return Err(refused()); + } + } + Ok(()) +} + // A network receipt grants no authority over an unknown endpoint. Recheck all // members before container effects and require a running container's exact endpoint. fn project_network( @@ -359,7 +482,12 @@ impl Session<'_, B> { impl Driver for Session<'_, B> { fn check_cancelled(&self) -> Result<(), CandidateError> { check_startup(self.startup_guard)?; - self.selected.assert_fresh(self.candidate) + self.selected.assert_fresh(self.candidate)?; + self.backend + .verify_data(&self.receipt, self.selected.remaining()?, &|| { + check_startup(self.startup_guard)?; + self.selected.assert_fresh(self.candidate) + }) } fn record(&mut self, event: Event<'_>) -> Result<(), CandidateError> { match event { @@ -390,6 +518,7 @@ impl Driver for Session<'_, B> { } Event::Observed { .. } => Ok(()), Event::Ready => { + self.check_cancelled()?; self.receipt.phase = Phase::ReadyObserved; self.save() } @@ -493,6 +622,15 @@ pub(super) fn run_guarded( ) -> Result { check_startup(startup_guard)?; let (selected, input) = prepared.into_parts(candidate)?; + // Birth/device/inode/labels can all alias after replacement. Until the native + // root-witness transport is qualified, no ordinary invocation may reach the + // provider or publish receipt4/create/enroll/use persistent data. + if !input.inputs().storage.is_empty() { + return Err(error( + "native_graph_storage_unqualified", + "Native persistent storage requires a qualified root continuity witness; no provider or data effects were authorized.", + )); + } let deadline = selected.remaining()?; #[cfg(target_os = "macos")] let engine = Engine::connect_until(candidate, deadline, || { @@ -507,6 +645,35 @@ pub(super) fn run_guarded( check_startup(startup_guard)?; selected.assert_fresh(candidate)?; let mut config = configuration(&input, engine.guest().incarnation())?; + config.data = persistent_data::engine::select( + candidate, + &engine, + config.review.scope().namespace, + &config.storage, + deadline, + &|| { + check_startup(startup_guard)?; + selected.assert_fresh(candidate) + }, + )?; + // A new compute attempt may reuse data only after all earlier admitted consumers + // have completed exact retirement. Exited/uncertain containers still count. + if !config.data.is_empty() { + for run in storage_inventory::runs(&candidate.state_root.join("run/native-graphs"))? { + let (prior, _) = journal::load_admission( + candidate, + &run, + engine.guest().incarnation(), + engine.guest().boot_id(), + )?; + verify_prior_data_retirement(&prior, &config.data)?; + } + } + for (service, mounts) in &config.data_mounts { + config.configs.get_mut(service).ok_or_else(refused)?["HostConfig"]["Mounts"] = json!(mounts.iter().map(|mount| { + Ok(json!({"Type":"volume","Source":config.data.get(&mount.storage).ok_or_else(refused)?.name(),"Target":mount.target,"ReadOnly":mount.read_only,"VolumeOptions":{"NoCopy":true}})) + }).collect::, CandidateError>>()?); + } check_network_intent(&engine, &config.resources)?; let private = input.private_services(); // Stop observations require a whole-second bounded timeout even when authored intent omits it. @@ -522,13 +689,22 @@ pub(super) fn run_guarded( if !private.is_empty() { crate::provider::environment::preflight_capacity(engine.guest(), private.len())?; } - let expected = verify_images(&engine, &config.resources, &mut config.configs, &private)?; + if !config.data.is_empty() && !private.is_empty() { + // Delivery bind + persistent mount intersection needs its own saved physical + // mount membership proof. Refuse before any stable-volume or container effect. + return Err(refused()); + } + let expected = if config.data.is_empty() { + verify_images(&engine, &config.resources, &mut config.configs, &private)? + } else { + verify_native_images(&engine, &config.resources, &mut config.configs, &private)? + }; for name in &private { launcher::validate(&config.configs[name])?; } check_startup(startup_guard)?; selected.assert_fresh(candidate)?; - let receipt = Receipt::preparing( + let mut receipt = Receipt::preparing( &config, engine.guest().incarnation(), engine.guest().boot_id(), @@ -541,6 +717,14 @@ pub(super) fn run_guarded( let execution = (|| { check_startup(startup_guard)?; selected.assert_fresh(candidate)?; + for logical in receipt.data.keys().cloned().collect::>() { + let reference = receipt.data.get_mut(&logical).ok_or_else(refused)?; + persistent_data::engine::enroll(candidate, &engine, reference, deadline, &|| { + check_startup(startup_guard)?; + selected.assert_fresh(candidate) + })?; + journal::save(&root, &receipt)?; + } let launcher = if private.is_empty() { None } else { @@ -613,6 +797,8 @@ pub struct Snapshot { pub observations: BTreeMap>, } fn snapshot(backend: &B, receipt: Receipt) -> Result { + let data_deadline = Instant::now() + Duration::from_secs(40); + backend.verify_data(&receipt, data_deadline, &|| Ok(()))?; let mut observations = BTreeMap::new(); if receipt.phase == Phase::Removed { if inspected(backend, &receipt, &receipt.resources["network:default"])?.is_some() { @@ -636,6 +822,7 @@ fn snapshot(backend: &B, receipt: Receipt) -> Result( check_startup(guard)?; let guarded = GuardedBackend { backend, guard }; let backend = &guarded; + backend.verify_data(receipt, Instant::now() + Duration::from_secs(40), &|| { + check_startup(guard) + })?; // A committed cleanup phase is retry authority for this same inventory, never // permission to move the receipt back to startup or stop an already retired run. let stopped = matches!( @@ -922,9 +1112,72 @@ fn cleanup_using_guarded( .ok_or_else(refused)? .phase = "removed".into(); receipt.phase = Phase::Removed; + backend.verify_data(receipt, Instant::now() + Duration::from_secs(40), &|| { + check_startup(guard) + })?; journal::save(root, receipt) } +fn verify_native_images( + engine: &Engine<'_>, + resources: &BTreeMap, + configs: &mut BTreeMap, + private: &BTreeSet, +) -> Result>, CandidateError> { + let mut expected = BTreeMap::new(); + for resource in resources + .values() + .filter(|resource| resource.kind == Kind::Container) + { + let image = engine.request( + Method::GET, + &format!( + "/v1.53/images/{}/json", + resource.image.as_deref().ok_or_else(refused)? + ), + None, + )?; + let config = configs.get_mut(&resource.key).ok_or_else(refused)?; + if image["Id"].as_str() != resource.image.as_deref() + || image["Os"] != "linux" + || image["Architecture"] != "arm64" + { + return Err(refused()); + } + if !image["Config"]["Volumes"].is_null() { + let volumes = image["Config"]["Volumes"].as_object().ok_or_else(refused)?; + let mounts = config["HostConfig"]["Mounts"] + .as_array() + .ok_or_else(refused)?; + // Every image-declared volume must be overridden by exactly one admitted + // persistent mount; anonymous engine allocation is never allowed. + for (target, options) in volumes { + if !options.as_object().is_some_and(|object| object.is_empty()) + || mounts + .iter() + .filter(|mount| { + mount["Type"] == "volume" + && mount["Target"] == *target + && mount["VolumeOptions"]["NoCopy"] == true + }) + .count() + != 1 + { + return Err(refused()); + } + } + } + if private.contains(&resource.key) { + image_process::apply_private(config, &image["Config"])?; + } + expected.insert( + resource.key.clone(), + image_environment::compose(&image["Config"]["Env"], &config["Env"])?, + ); + } + Ok(expected) +} + #[cfg(test)] mod tests; @@ -934,12 +1187,22 @@ pub(in crate::provider::graph) fn reservations( new_attempt: bool, add: impl FnMut(&Value) -> Result<(), CandidateError>, ) -> Result<(), CandidateError> { + let mut verified_data = BTreeSet::new(); reservations_using( candidate, engine.guest().incarnation(), engine.guest().boot_id(), new_attempt, - |receipt, resource| inspected(engine, receipt, resource), + |receipt, resource| { + if verified_data.insert(receipt.review.scope().run.to_owned()) { + engine.verify_data( + receipt, + Instant::now() + Duration::from_secs(40), + &|| Ok(()), + )?; + } + inspected(engine, receipt, resource) + }, add, ) } diff --git a/packages/runtime-core/src/provider/graph/native/runtime/tests.rs b/packages/runtime-core/src/provider/graph/native/runtime/tests.rs index 92a46bd0a..adbb0c044 100644 --- a/packages/runtime-core/src/provider/graph/native/runtime/tests.rs +++ b/packages/runtime-core/src/provider/graph/native/runtime/tests.rs @@ -114,6 +114,176 @@ fn basic() -> Value { json!({"schema_version":1,"name":"fixture","services":{"web":{"image":image(),"command":{"exec":["/bin/echo","$EXACT"]}}}}) } +fn persistent_reference(receipt: &Receipt) -> persistent_data::engine::Reference { + let namespace = receipt.review.scope().namespace; + let owner = "9".repeat(32); + serde_json::from_value(json!({"binding":{"scope":{"namespace":namespace,"storage":"database","owner":owner},"guest":{"owner":receipt.owner,"boot_id":receipt.boot,"storage":{"device":0,"inode":21,"bytes":128,"uuid":"11111111-2222-3333-4444-555555555555"}},"policy":{"driver":"local","scope":"local","options":{}}},"state":{"status":"enrolled","volume":{"name":format!("hkp-{namespace}-{owner}-database"),"created_at":"2026-10-08T00:00:01Z","directory":{"device":0,"inode":42}}}})).unwrap() +} + +#[test] +fn persistent_journal_has_distinct_version_and_exact_data_membership_without_volume_ids() { + let fixture = Fixture::new(basic()); + let (_, session) = fixture.session(fixture.prepared(json!({"web":{}}), &BTreeMap::new())); + let mut receipt = session.receipt; + let image_only = serde_json::to_value(&receipt).unwrap(); + for fields in [ + json!({"data":{}}), + json!({"data_mounts":{}}), + json!({"data":{},"data_mounts":{}}), + json!({"data":null}), + json!({"data_mounts":null}), + json!({"data":null,"data_mounts":null}), + ] { + let mut wire = image_only.clone(); + wire.as_object_mut() + .unwrap() + .extend(fields.as_object().unwrap().clone()); + assert!(serde_json::from_value::(wire).is_err()); + } + serde_json::from_value::(image_only) + .unwrap() + .validate(RUN, OWNER) + .unwrap(); + receipt + .data + .insert("database".into(), persistent_reference(&receipt)); + receipt.data_mounts.insert( + "web".into(), + vec![crate::project::native::StorageMount { + storage: "database".into(), + target: "/data".into(), + read_only: false, + }], + ); + let mut persistent_wire = serde_json::to_value(&receipt).unwrap(); + persistent_wire["version"] = json!(4); + receipt = serde_json::from_value(persistent_wire).unwrap(); + receipt.validate(RUN, OWNER).unwrap(); + assert_eq!(receipt.resources.len(), 2); + assert!( + receipt + .resources + .values() + .all(|resource| resource.kind != Kind::Volume) + ); + let observed = json!({"Mounts":[{"Type":"volume","Name":receipt.data["database"].name(),"Source":receipt.data["database"].mountpoint(),"Destination":"/data","RW":true,"Driver":"local"}]}); + verify_data_mounts(&receipt, "web", &observed).unwrap(); + for field in ["Name", "Source", "Destination", "Driver", "RW", "Type"] { + let mut wrong = observed.clone(); + wrong["Mounts"][0][field] = json!("foreign"); + assert!(verify_data_mounts(&receipt, "web", &wrong).is_err()); + } + let mut extra = observed.clone(); + extra["Mounts"] + .as_array_mut() + .unwrap() + .push(observed["Mounts"][0].clone()); + assert!(verify_data_mounts(&receipt, "web", &extra).is_err()); + for version in [2, 3, 5] { + let mut wrong = serde_json::to_value(&receipt).unwrap(); + wrong["version"] = json!(version); + if let Ok(wrong) = serde_json::from_value::(wrong) { + assert!(wrong.validate(RUN, OWNER).is_err()); + } + } + let mut omitted = receipt.clone(); + omitted.data.clear(); + assert!(omitted.validate(RUN, OWNER).is_err()); + let encoded = serde_json::to_value(&receipt).unwrap(); + let mut pending = encoded.clone(); + pending["data"]["database"]["state"] = json!({"status":"reserved","intent":"8".repeat(32)}); + let mut pending: Receipt = serde_json::from_value(pending).unwrap(); + pending.resources.get_mut("container:web").unwrap().id = Some("6".repeat(64)); + assert!(pending.validate(RUN, OWNER).is_err()); + let mut independent = receipt.clone(); + let run = "7".repeat(32); + independent.review = native_input::Review::new( + native_input::Scope { + namespace: receipt.review.scope().namespace, + run: &run, + }, + receipt.review.compiler_identity().clone(), + ) + .unwrap(); + independent + .resources + .get_mut("network:default") + .unwrap() + .name = format!("hkn-{run}-network-0"); + independent.resources.get_mut("container:web").unwrap().name = format!("hkn-{run}-container-0"); + independent.validate(&run, OWNER).unwrap(); + assert_eq!(independent.data, receipt.data); + assert!(independent.check_binding(&receipt).is_err()); + assert!(verify_prior_data_retirement(&receipt, &independent.data).is_err()); + receipt.phase = Phase::Removed; + verify_prior_data_retirement(&receipt, &independent.data).unwrap(); + // Compute-only retirement cannot turn an uncertain original data create into + // authority for a second attempt, even if an enrolled pathname appeared late. + let mut reserved = encoded; + reserved["data"]["database"]["state"] = json!({"status":"reserved","intent":"8".repeat(32)}); + let mut reserved: Receipt = serde_json::from_value(reserved).unwrap(); + reserved.phase = Phase::Removed; + assert!(verify_prior_data_retirement(&reserved, &independent.data).is_err()); +} + +#[cfg(target_os = "macos")] +#[test] +fn otherwise_valid_ready_graph4_refuses_dead_publication_recovery() { + let fixture = Fixture::new(basic()); + let (_, session) = fixture.session(fixture.prepared(json!({"web":{}}), &BTreeMap::new())); + let mut receipt = session.receipt; + receipt.phase = Phase::ReadyObserved; + for resource in receipt.resources.values_mut() { + resource.id = Some(if resource.kind == Kind::Network { + "1".repeat(64) + } else { + "2".repeat(64) + }); + resource.phase = if resource.kind == Kind::Network { + "created".into() + } else { + "started".into() + }; + } + receipt.require_recovery_ready().unwrap(); + receipt + .data + .insert("database".into(), persistent_reference(&receipt)); + receipt.data_mounts.insert( + "web".into(), + vec![crate::project::native::StorageMount { + storage: "database".into(), + target: "/data".into(), + read_only: false, + }], + ); + let mut persistent_wire = serde_json::to_value(&receipt).unwrap(); + persistent_wire["version"] = json!(4); + receipt = serde_json::from_value(persistent_wire).unwrap(); + receipt.validate(RUN, OWNER).unwrap(); + assert!(receipt.require_recovery_ready().is_err()); +} + +#[test] +fn ordinary_persistent_start_refuses_before_provider_or_any_owner_publication() { + let mut project = basic(); + project["storage"] = json!({"database":{"kind":"persistent","scope":"worktree"}}); + project["services"]["web"]["mounts"] = + json!([{"storage":"database","target":"/data","access":"read-write"}]); + let fixture = Fixture::new(project); + let prepared = fixture.prepared(json!({"web":{}}), &BTreeMap::new()); + assert_eq!( + run_guarded(&fixture.candidate, prepared, None, None) + .unwrap_err() + .code, + "native_graph_storage_unqualified" + ); + for path in ["run/smolvm", "run/native-graphs"] { + assert!(!fixture.candidate.state_root.join(path).exists()); + } + assert!(!fixture.candidate.state_root.exists()); +} + #[test] fn foreground_guard_refuses_before_provider_connection_or_graph_reservation() { let fixture = Fixture::new(basic()); diff --git a/packages/runtime-core/src/provider/graph/native/tests.rs b/packages/runtime-core/src/provider/graph/native/tests.rs index e859a5a29..535087871 100644 --- a/packages/runtime-core/src/provider/graph/native/tests.rs +++ b/packages/runtime-core/src/provider/graph/native/tests.rs @@ -58,7 +58,7 @@ impl Driver for Fake<'_> { Ok(()) } fn observe(&mut self, service: &str) -> Result { - Ok(if service == "z.seed" { + Ok(if matches!(service, "z.seed" | "z.check") { Observation::Exited { code: 0 } } else { Observation::Running { @@ -68,6 +68,92 @@ impl Driver for Fake<'_> { } } +#[test] +fn persistent_sqlite_corpus_preserves_source_commands_drop_all_and_fresh_job_starts() { + // This executes the real compiler/lowerer and scheduler with observations only. + // The SQL programs are source-pinned for the later real native acceptance; no + // test here claims that synthetic observations executed SQLite or Engine effects. + let project: Value = serde_json::from_str(include_str!( + "../../../../tests/fixtures/native-persistent-sqlite.json" + )) + .unwrap(); + let prepared = prepare( + project.clone(), + json!({"a.db":{},"b.web":{},"z.seed":{},"z.check":{}}), + &ManagedValues::new(), + ); + let config = configuration(&prepared, OWNER).unwrap(); + assert_eq!(config.storage, BTreeSet::from(["database".into()])); + assert_eq!( + config.data_mounts.keys().collect::>(), + vec!["a.db", "z.seed"] + ); + assert_eq!( + config.graph.services["a.db"].dependencies["z.seed"], + Condition::Completed + ); + assert_eq!( + config.graph.services["b.web"].dependencies["a.db"], + Condition::Healthy + ); + assert_eq!( + config.graph.services["z.check"].dependencies["b.web"], + Condition::Healthy + ); + assert_eq!(config.resources.len(), 5); + assert!( + config + .resources + .values() + .all(|resource| resource.kind != Kind::Volume) + ); + for (name, value) in &config.configs { + let authored = project["services"] + .get(name) + .or_else(|| project["jobs"].get(name)) + .unwrap(); + assert_eq!(value["Cmd"], authored["command"]["exec"]); + assert_eq!(value["Entrypoint"], json!([])); + assert_eq!(value["HostConfig"]["CapDrop"], json!(["ALL"])); + assert_eq!( + value["HostConfig"]["SecurityOpt"], + json!(["no-new-privileges"]) + ); + assert!(value.get("User").is_none()); + assert!( + value["HostConfig"]["PortBindings"] + .as_object() + .is_none_or(|bindings| bindings.is_empty()) + ); + } + for _compute_attempt in 0..2 { + let mut driver = Fake { + configs: &config.configs, + intents: vec![], + started: vec![], + }; + execution::run(&config.graph, &mut driver, Duration::from_secs(2)).unwrap(); + assert_eq!( + driver + .started + .iter() + .filter(|name| name.as_str() == "z.seed") + .count(), + 1 + ); + let position = |name: &str| { + driver + .started + .iter() + .position(|started| started == name) + .unwrap() + }; + assert!(position("z.seed") < position("a.db")); + assert!(position("a.db") < position("b.web")); + assert!(position("b.web") < position("z.check")); + } +} + #[test] fn real_compiler_native_configs_feed_jobs_readiness_and_exact_process_driver() { let fixture: Value = serde_json::from_str(include_str!( diff --git a/packages/runtime-core/src/provider/lifecycle.rs b/packages/runtime-core/src/provider/lifecycle.rs index ff0796b9b..6a4aa52cf 100644 --- a/packages/runtime-core/src/provider/lifecycle.rs +++ b/packages/runtime-core/src/provider/lifecycle.rs @@ -358,6 +358,7 @@ pub(super) struct OwnedGuest<'a> { candidate: &'a Candidate, owner: Owner, _lock: state::Lock, + lease_root: (u64, u64), allocation_allowed: bool, guard: Option, } @@ -391,9 +392,35 @@ impl<'a> OwnedGuest<'a> { } pub(super) fn verify(&self) -> Result<(), CandidateError> { + let root = root(self.candidate); + self._lock.verify_path(&root)?; + let current = fs::symlink_metadata(&root).map_err(io)?; + if (current.dev(), current.ino()) != self.lease_root { + return Err(CandidateError::new( + "runtime_changed", + "Provider mutation root changed.", + )); + } verify_live(self.candidate, &self.owner) } + #[cfg(feature = "native-config-plan")] + pub(in crate::provider) fn persistent_identity( + &self, + ) -> Result { + self.verify()?; + Ok(super::graph::native::persistent_data::GuestIdentity { + owner: self.owner.token.clone(), + boot_id: self.boot_id().into(), + storage: self.owner.storage.clone().ok_or_else(|| { + CandidateError::new( + "disk_identity_mismatch", + "Persistent storage requires an independently retained backing disk.", + ) + })?, + }) + } + pub(super) fn project_share(&self) -> Option<&super::ProjectShareIntent> { self.owner.project_share.as_ref() } @@ -468,10 +495,13 @@ impl<'a> OwnedGuest<'a> { } else { None }; + lock.verify_path(&root(candidate))?; + let lease_root = fs::symlink_metadata(root(candidate)).map_err(io)?; Ok(Self { candidate, owner: current, _lock: lock, + lease_root: (lease_root.dev(), lease_root.ino()), allocation_allowed: enforce_budget, guard, }) @@ -520,6 +550,29 @@ impl<'a> OwnedGuest<'a> { self.execute_mode(script, arguments, input, true) } + #[cfg(feature = "native-config-plan")] + pub(in crate::provider) fn execute_until( + &self, + script: &str, + arguments: &[&str], + deadline: Instant, + ) -> Result { + self.verify()?; + let script = format!( + "set -eu\ntest \"$(cat /proc/sys/kernel/random/boot_id)\" = \"$1\"\ntest \"$(cat /storage/.hack-local-owner)\" = \"$2\"\ntest \"$(findmnt -n -o FSTYPE --mountpoint /storage)\" = ext4\nshift 2\n{script}" + ); + let mut args = vec![self.boot_id(), self.incarnation()]; + args.extend_from_slice(arguments); + let result = agent::exec_until( + &socket(self.candidate, &self.owner, "agent.sock")?, + &script, + &args, + deadline, + ); + self.verify()?; + result + } + /// Retains the existing mutation lock and identity checks without allocation admission. pub(super) fn execute_cleanup( &self, @@ -536,7 +589,7 @@ impl<'a> OwnedGuest<'a> { input: Option<&str>, allocation: bool, ) -> Result { - verify_live(self.candidate, &self.owner)?; + self.verify()?; if allocation { self.before_effect()?; } @@ -555,7 +608,7 @@ impl<'a> OwnedGuest<'a> { false, input, )?; - verify_live(self.candidate, &self.owner)?; + self.verify()?; Ok(result) } } diff --git a/packages/runtime-core/src/provider/state.rs b/packages/runtime-core/src/provider/state.rs index febc80207..2168dd31d 100644 --- a/packages/runtime-core/src/provider/state.rs +++ b/packages/runtime-core/src/provider/state.rs @@ -36,6 +36,34 @@ pub fn check_private_directory(path: &Path) -> Result<(), CandidateError> { pub struct Lock(File); impl Lock { + /// Verify that the retained cooperative lock still names this exact private inode. + /// Holding an unlinked predecessor cannot serialize a replacement pathname. + pub(crate) fn verify_path(&self, root: &Path) -> Result<(), CandidateError> { + check_private_directory(root)?; + let descriptor = self.0.metadata().map_err(io)?; + let current = fs::symlink_metadata(root.join("operation.lock")).map_err(io)?; + // SAFETY: geteuid has no preconditions. + let uid = unsafe { libc::geteuid() }; + for metadata in [&descriptor, ¤t] { + if !metadata.is_file() + || metadata.nlink() != 1 + || metadata.uid() != uid + || metadata.mode() & 0o7777 != 0o600 + { + return Err(CandidateError::new( + "foreign_state", + "Provider mutation lock changed.", + )); + } + } + if (descriptor.dev(), descriptor.ino()) != (current.dev(), current.ino()) { + return Err(CandidateError::new( + "foreign_state", + "Provider mutation lock changed.", + )); + } + Ok(()) + } #[cfg(target_os = "macos")] pub(crate) fn file(&self) -> &File { &self.0 diff --git a/packages/runtime-core/tests/fixtures/native-persistent-sqlite.json b/packages/runtime-core/tests/fixtures/native-persistent-sqlite.json new file mode 100644 index 000000000..81d4ac2f2 --- /dev/null +++ b/packages/runtime-core/tests/fixtures/native-persistent-sqlite.json @@ -0,0 +1,82 @@ +{ + "schema_version": 1, + "name": "native-persistent-sqlite", + "storage": { + "database": { "kind": "persistent", "scope": "worktree" } + }, + "jobs": { + "z.seed": { + "image": "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "entrypoint": { "exec": [] }, + "command": { + "exec": [ + "/usr/local/bin/bun", + "-e", + "import {Database} from 'bun:sqlite';const db=new Database('/data/proof.sqlite');db.exec('CREATE TABLE IF NOT EXISTS proof(id INTEGER PRIMARY KEY,value TEXT)');db.query('INSERT OR IGNORE INTO proof VALUES (1,?)').run('native-persistent-sqlite');if(db.query('SELECT value FROM proof WHERE id=1').get().value!=='native-persistent-sqlite')throw Error('persistence');db.query('INSERT OR IGNORE INTO proof VALUES (2,?)').run(crypto.randomUUID());db.exec('CREATE TABLE IF NOT EXISTS initializer_attempts(id INTEGER PRIMARY KEY)');db.exec('INSERT INTO initializer_attempts DEFAULT VALUES');console.log(db.query('SELECT value FROM proof WHERE id=2').get().value);db.close();" + ] + }, + "mounts": [ + { "storage": "database", "target": "/data", "access": "read-write" } + ] + }, + "z.check": { + "image": "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "entrypoint": { "exec": [] }, + "command": { + "exec": [ + "/usr/local/bin/bun", + "-e", + "const origin='http://b.web:3000';let r=await fetch(origin+'/state');let state=await r.json();if(!r.ok||state.marker!=='native-persistent-sqlite'||typeof state.nonce!=='string'||state.attempts<1)throw Error('state');r=await fetch(origin+'/write',{method:'POST',body:'written-through-app'});if(!r.ok)throw Error('write');r=await fetch(origin+'/state');state=await r.json();if(!r.ok||state.value!=='written-through-app')throw Error('readback');" + ] + }, + "depends_on": [{ "service": "b.web", "condition": "ready" }] + } + }, + "services": { + "a.db": { + "image": "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "entrypoint": { "exec": [] }, + "command": { + "exec": [ + "/usr/local/bin/bun", + "-e", + "import {Database} from 'bun:sqlite';const db=new Database('/data/proof.sqlite');const server=Bun.serve({hostname:'0.0.0.0',port:3000,async fetch(request){const path=new URL(request.url).pathname;if(path==='/state'&&request.method==='GET'){return Response.json({marker:db.query('SELECT value FROM proof WHERE id=1').get().value,nonce:db.query('SELECT value FROM proof WHERE id=2').get().value,value:db.query('SELECT value FROM proof WHERE id=3').get()?.value??null,attempts:db.query('SELECT COUNT(*) AS count FROM initializer_attempts').get().count});}if(path==='/write'&&request.method==='POST'){const value=await request.text();if(!/^[a-z-]{1,63}$/.test(value))return new Response('',{status:400});db.query('INSERT OR REPLACE INTO proof VALUES (3,?)').run(value);return new Response('ok');}return new Response('',{status:404});}});process.on('SIGTERM',()=>{server.stop(true);db.close();process.exit(0);});" + ] + }, + "mounts": [ + { "storage": "database", "target": "/data", "access": "read-write" } + ], + "depends_on": [{ "job": "z.seed", "condition": "completed" }], + "readiness": { + "kind": "exec", + "command": { + "exec": ["/usr/local/bin/bun", "-e", "const r=await fetch('http://127.0.0.1:3000/state');if(!r.ok)process.exit(1)"] + }, + "interval": "1s", + "timeout": "1s", + "retries": 10 + } + }, + "b.web": { + "image": "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "entrypoint": { "exec": [] }, + "command": { + "exec": [ + "/usr/local/bin/bun", + "-e", + "const server=Bun.serve({hostname:'0.0.0.0',port:3000,fetch(request){const path=new URL(request.url).pathname;return fetch('http://a.db:3000'+path,{method:request.method,body:request.method==='POST'?request.body:undefined});}});process.on('SIGTERM',()=>{server.stop(true);process.exit(0);});" + ] + }, + "depends_on": [{ "service": "a.db", "condition": "ready" }], + "readiness": { + "kind": "exec", + "command": { + "exec": ["/usr/local/bin/bun", "-e", "const r=await fetch('http://127.0.0.1:3000/state');if(!r.ok)process.exit(1)"] + }, + "interval": "1s", + "timeout": "1s", + "retries": 10 + } + } + } +} diff --git a/scripts/lib/tla-runtime-models.ts b/scripts/lib/tla-runtime-models.ts index e2deafcea..380e5796c 100644 --- a/scripts/lib/tla-runtime-models.ts +++ b/scripts/lib/tla-runtime-models.ts @@ -20,7 +20,7 @@ type ModelContract = { readonly alternativeWitnesses?: readonly ModelWitness[]; readonly additionalControls?: readonly { readonly name: string; - readonly module?: "TerminalReuse"; + readonly module?: "TerminalReuse" | "MetadataAliasing"; readonly negative: boolean; readonly states?: number; readonly invariant?: string; @@ -40,6 +40,58 @@ const laterIncompleteMissingLockStages = [ "owner", ] as const; const contracts: readonly ModelContract[] = [ + { + name: "native-persistent-enrollment", + module: "Enrollment", + states: 38, + invariant: "NoExistingAdoption", + action: "CreateOriginal", + fields: ['phase = "created"', "volume = 2", "unsafeAdoption = TRUE"], + additionalControls: [ + { + name: "metadata-alias", + module: "MetadataAliasing", + negative: true, + invariant: "NoAliasedMatch", + action: "ReadRetained", + fields: [ + "actualVolume = 2", + "reportedMetadata = 1", + "capturedMetadata = 1", + "witness = 0", + "expectedWitness = 1", + "matched = TRUE", + ], + }, + { + name: "wrong-birth", + negative: true, + invariant: "OriginalBirthAtCommit", + action: "PublishEnrollment", + fields: ["captured = 1", "volume = 2", "unsafeCommit = TRUE"], + alternativeWitnesses: [ + { + action: "PublishEnrollment", + fields: ["captured = 2", "volume = 2", "unsafeCommit = TRUE"], + }, + ], + }, + { + name: "pending-read", + negative: true, + invariant: "NoPendingMatch", + action: "ReadRetained", + fields: ['record = "pending"', "matched = TRUE", "unsafeRead = TRUE"], + }, + { + name: "retire-data", + negative: true, + invariant: "RetirementPreservesData", + action: "RetireCompute", + fields: ["run = 2", "volume = 0", "unsafeRetirement = TRUE"], + }, + ], + }, { name: "native-frontend-recovery", module: "Recovery", diff --git a/src/backends/native-authored-graph-protocol.ts b/src/backends/native-authored-graph-protocol.ts index 7a032cc60..6f7f41db8 100644 --- a/src/backends/native-authored-graph-protocol.ts +++ b/src/backends/native-authored-graph-protocol.ts @@ -1,5 +1,10 @@ import { createHash } from "node:crypto"; import { isRecord } from "../lib/guards.ts"; +import { + type NativePersistentMount, + type NativePersistentReference, + parseNativePersistentData, +} from "./native-authored-persistent-data-protocol.ts"; const HEX32 = /^[a-f0-9]{32}$/; const HEX64 = /^[a-f0-9]{64}$/; @@ -58,7 +63,7 @@ type Terminal = { readonly stop_requested: boolean; }; export type NativeAuthoredReceipt = { - readonly version: 2; + readonly version: 2 | 4; readonly kind: "native-graph-runtime"; readonly owner: string; readonly boot: string; @@ -66,6 +71,10 @@ export type NativeAuthoredReceipt = { readonly phase: Phase; readonly readiness: Readonly>; readonly resources: Readonly>; + readonly data?: Readonly>; + readonly data_mounts?: Readonly< + Record + >; readonly failure?: { readonly service: string; readonly observation: Observation; @@ -317,9 +326,9 @@ export function parseNativeAuthoredReceipt( "readiness", "resources", ], - ["failure", "terminal"] + ["failure", "terminal", "data", "data_mounts"] ) || - value.version !== 2 || + (value.version !== 2 && value.version !== 4) || value.kind !== "native-graph-runtime" || typeof value.owner !== "string" || !HEX32.test(value.owner) || @@ -392,8 +401,30 @@ export function parseNativeAuthoredReceipt( const terminal = Object.hasOwn(value, "terminal") ? parseTerminal(value.terminal, resources) : undefined; + const hasData = + Object.hasOwn(value, "data") || Object.hasOwn(value, "data_mounts"); + if ((value.version === 2 && hasData) || (value.version === 4 && !hasData)) { + return refused(); + } + const data = + value.version === 4 + ? parseNativePersistentData({ + data: value.data, + mounts: value.data_mounts, + namespace: review.provenance.namespace, + owner: value.owner, + boot: value.boot, + workloads: names, + enrolled: + value.phase === "ready-observed" || + Object.values(resources).some( + (resource) => + resource.kind === "container" && resource.id !== null + ), + }) + : undefined; return { - version: 2, + version: value.version, kind: "native-graph-runtime", owner: value.owner, boot: value.boot, @@ -401,6 +432,7 @@ export function parseNativeAuthoredReceipt( phase: value.phase, readiness, resources, + ...data, ...(failure ? { failure } : {}), ...(terminal ? { terminal } : {}), }; @@ -474,6 +506,9 @@ export function nativeAuthoredReceiptBinding( receipt: NativeAuthoredReceipt ): string { return JSON.stringify({ + ...(receipt.version === 4 + ? { version: 4, data: receipt.data, data_mounts: receipt.data_mounts } + : {}), owner: receipt.owner, boot: receipt.boot, review: receipt.review, diff --git a/src/backends/native-authored-persistent-data-protocol.ts b/src/backends/native-authored-persistent-data-protocol.ts new file mode 100644 index 000000000..1e28d0711 --- /dev/null +++ b/src/backends/native-authored-persistent-data-protocol.ts @@ -0,0 +1,363 @@ +/** Closed value-free graph4 storage assertions. Parsing grants no data effect authority. */ +export type NativePersistentReference = { + readonly binding: { + readonly scope: { + readonly namespace: string; + readonly storage: string; + readonly owner: string; + }; + readonly guest: { + readonly owner: string; + readonly boot_id: string; + readonly storage: { + readonly device: number; + readonly inode: number; + readonly bytes: number; + readonly uuid: string; + }; + }; + readonly policy: { + readonly driver: "local"; + readonly scope: "local"; + readonly options: Readonly>; + }; + }; + readonly state: + | { readonly status: "reserved"; readonly intent: string } + | { + readonly status: "enrolled"; + readonly volume: { + readonly name: string; + readonly created_at: string; + readonly directory: { + readonly device: number; + readonly inode: number; + }; + }; + }; +}; +export type NativePersistentMount = { + readonly storage: string; + readonly target: string; + readonly read_only: boolean; +}; +const HEX32 = /^[0-9a-f]{32}$/; +const HEX64 = /^[0-9a-f]{64}$/; +const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/; +const NAME = /^[a-z0-9][a-z0-9._-]*$/; +const BIRTH = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{1,9})?Z$/; +const NONZERO_DIGIT = /[1-9]/; +function refused(): never { + throw new Error( + "Native persistent data response is invalid or changed; values omitted." + ); +} +function record(value: unknown, expected?: string): Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return refused(); + } + let descriptors: Record; + try { + descriptors = Object.getOwnPropertyDescriptors(value); + } catch { + return refused(); + } + const keys = Object.keys(descriptors); + if (Reflect.ownKeys(descriptors).length !== keys.length) { + return refused(); + } + if ( + expected !== undefined && + keys.sort().join(",") !== expected.split(",").sort().join(",") + ) { + return refused(); + } + for (const descriptor of Object.values(descriptors)) { + if (!(Object.hasOwn(descriptor, "value") && descriptor.enumerable)) { + return refused(); + } + } + return Object.fromEntries(keys.map((key) => [key, descriptors[key]?.value])); +} +function array(value: unknown): readonly unknown[] { + if (!Array.isArray(value)) { + return refused(); + } + let descriptors: Record; + try { + descriptors = Object.getOwnPropertyDescriptors(value); + } catch { + return refused(); + } + const length: unknown = descriptors.length?.value; + if ( + typeof length !== "number" || + !Number.isSafeInteger(length) || + length <= 0 || + Reflect.ownKeys(descriptors).length !== length + 1 + ) { + return refused(); + } + const result: unknown[] = []; + for (let index = 0; index < length; index += 1) { + const item = descriptors[String(index)]; + if (!(item && Object.hasOwn(item, "value") && item.enumerable)) { + return refused(); + } + result.push(item.value); + } + return result; +} +function hex(value: unknown, length: 32 | 64): string { + if ( + typeof value !== "string" || + value.length !== length || + !(length === 32 ? HEX32 : HEX64).test(value) + ) { + return refused(); + } + return value; +} +function uuid(value: unknown): string { + if ( + typeof value !== "string" || + value.length !== 36 || + !UUID.test(value) || + value === "00000000-0000-0000-0000-000000000000" + ) { + return refused(); + } + return value; +} +function integer(value: unknown, nonzero = false): number { + if ( + typeof value !== "number" || + !Number.isSafeInteger(value) || + value < (nonzero ? 1 : 0) + ) { + return refused(); + } + return value; +} +function name(value: unknown): string { + if ( + typeof value !== "string" || + value.length === 0 || + value.length > 63 || + !NAME.test(value) || + value.endsWith("\n") + ) { + return refused(); + } + return value; +} +function birth(value: unknown): string { + if ( + typeof value !== "string" || + value.length < 20 || + value.length > 30 || + !BIRTH.test(value) || + value.endsWith("\n") + ) { + return refused(); + } + const year = Number(value.slice(0, 4)); + const month = Number(value.slice(5, 7)); + const day = Number(value.slice(8, 10)); + const hour = Number(value.slice(11, 13)); + const minute = Number(value.slice(14, 16)); + const second = Number(value.slice(17, 19)); + const leap = year % 4 === 0 && (year % 100 !== 0 || year % 400 === 0); + let days = 31; + if (month === 2) { + days = leap ? 29 : 28; + } else if ([4, 6, 9, 11].includes(month)) { + days = 30; + } + if ( + year === 0 || + month < 1 || + month > 12 || + day < 1 || + day > days || + hour > 23 || + minute > 59 || + second > 59 || + (year === 1 && + month === 1 && + day === 1 && + hour === 0 && + minute === 0 && + second === 0 && + !NONZERO_DIGIT.test(value.slice(20, -1))) + ) { + return refused(); + } + return value; +} +type ParseOptions = { + readonly data: unknown; + readonly mounts: unknown; + readonly namespace: string; + readonly owner: string; + readonly boot: string; + readonly workloads: readonly string[]; + readonly enrolled: boolean; +}; +function parseReference({ + logical, + unknownReference, + opts, +}: { + readonly logical: string; + readonly unknownReference: unknown; + readonly opts: ParseOptions; +}): NativePersistentReference { + name(logical); + const item = record(unknownReference, "binding,state"); + const header = record(item.state); + if (header.status !== "reserved" && header.status !== "enrolled") { + return refused(); + } + const state = record( + item.state, + header.status === "reserved" ? "status,intent" : "status,volume" + ); + const binding = record(item.binding, "scope,guest,policy"); + const scope = record(binding.scope, "namespace,storage,owner"); + const guest = record(binding.guest, "owner,boot_id,storage"); + const disk = record(guest.storage, "device,inode,bytes,uuid"); + const policy = record(binding.policy, "driver,scope,options"); + record(policy.options, ""); + if ( + scope.namespace !== opts.namespace || + scope.storage !== logical || + guest.owner !== opts.owner || + guest.boot_id !== opts.boot || + policy.driver !== "local" || + policy.scope !== "local" + ) { + return refused(); + } + const parsedBinding = Object.freeze({ + scope: Object.freeze({ + namespace: hex(scope.namespace, 64), + storage: logical, + owner: hex(scope.owner, 32), + }), + guest: Object.freeze({ + owner: hex(guest.owner, 32), + boot_id: uuid(guest.boot_id), + storage: Object.freeze({ + device: integer(disk.device), + inode: integer(disk.inode, true), + bytes: integer(disk.bytes, true), + uuid: uuid(disk.uuid), + }), + }), + policy: Object.freeze({ + driver: "local" as const, + scope: "local" as const, + options: Object.freeze({}), + }), + }); + let parsed: NativePersistentReference; + if (state.status === "reserved") { + if (opts.enrolled) { + return refused(); + } + parsed = Object.freeze({ + binding: parsedBinding, + state: Object.freeze({ + status: "reserved", + intent: hex(state.intent, 32), + }), + }); + } else { + const volume = record(state.volume, "name,created_at,directory"); + const directory = record(volume.directory, "device,inode"); + const expected = `hkp-${parsedBinding.scope.namespace}-${parsedBinding.scope.owner}-${logical}`; + if (volume.name !== expected) { + return refused(); + } + parsed = Object.freeze({ + binding: parsedBinding, + state: Object.freeze({ + status: "enrolled", + volume: Object.freeze({ + name: expected, + created_at: birth(volume.created_at), + directory: Object.freeze({ + device: integer(directory.device), + inode: integer(directory.inode, true), + }), + }), + }), + }); + } + return parsed; +} +export function parseNativePersistentData(opts: ParseOptions): { + readonly data: Readonly>; + readonly data_mounts: Readonly< + Record + >; +} { + const declared = record(opts.data); + const data: Record = {}; + for (const [logical, unknownReference] of Object.entries(declared).sort( + ([left], [right]) => Buffer.compare(Buffer.from(left), Buffer.from(right)) + )) { + const parsed = parseReference({ logical, unknownReference, opts }); + Object.defineProperty(data, logical, { value: parsed, enumerable: true }); + } + const declaredMounts = record(opts.mounts); + const mounts: Record = {}; + const used = new Set(); + for (const [workload, value] of Object.entries(declaredMounts).sort( + ([left], [right]) => Buffer.compare(Buffer.from(left), Buffer.from(right)) + )) { + if (!opts.workloads.includes(workload)) { + return refused(); + } + const targets = new Set(); + const parsed = array(value).map((mount: unknown) => { + const item = record(mount, "storage,target,read_only"); + const storage = name(item.storage); + if ( + !Object.hasOwn(data, storage) || + typeof item.target !== "string" || + !item.target.startsWith("/") || + item.target.includes("\0") || + item.target.includes("\\") || + item.target + .slice(1) + .split("/") + .some((part) => part === "" || part === "." || part === "..") || + targets.has(item.target) || + typeof item.read_only !== "boolean" + ) { + return refused(); + } + targets.add(item.target); + used.add(storage); + return Object.freeze({ + storage, + target: item.target, + read_only: item.read_only, + }); + }); + Object.defineProperty(mounts, workload, { + value: Object.freeze(parsed), + enumerable: true, + }); + } + if ( + Object.keys(data).length === 0 || + Object.keys(mounts).length === 0 || + used.size !== Object.keys(data).length + ) { + return refused(); + } + return { data: Object.freeze(data), data_mounts: Object.freeze(mounts) }; +} diff --git a/src/backends/native-authored-recovery-protocol.ts b/src/backends/native-authored-recovery-protocol.ts index c48cd4cb9..b5f9e02cb 100644 --- a/src/backends/native-authored-recovery-protocol.ts +++ b/src/backends/native-authored-recovery-protocol.ts @@ -48,6 +48,7 @@ function hash(value: unknown): value is string { function ready(value: unknown): NativeAuthoredReceipt { const receipt = parseNativeAuthoredReceipt(value); if ( + receipt.version !== 2 || receipt.phase !== "ready-observed" || receipt.failure !== undefined || Object.values(receipt.resources).some((resource) => resource.id === null) diff --git a/tests/models/tla/README.md b/tests/models/tla/README.md index a24e36b4a..5ef5f22d9 100644 --- a/tests/models/tla/README.md +++ b/tests/models/tla/README.md @@ -1,5 +1,43 @@ # Runtime state models +`native-persistent-enrollment/Enrollment.tla` follows the Rust +`provider::graph::native::persistent_data::enrollment` owner. Its finite domain is +one storage slot, absent/original/replacement volume identities and two independent +compute references. Pending publication, exclusive creation, captured original +birth, enrolled rename, final directory synchronization and failure are distinct +steps. Failure retains the intent or an uncertain enrolled pathname and loses the +original invocation's promotion ability; read-only reopen cannot create or promote. +An uncertain enrolled pathname may compare as data, but does not establish the +original operation's successful return or runtime effect authority. + +| Model operation | Source boundary and qualification | +| --- | --- | +| `ReserveIntent` | Exclusive private slot/lock, `owner.json` pending write and file/parent sync precede `create_new`. Real filesystem sync-failure controls require create count zero. | +| `CreateOriginal` / `CaptureBirth` | Sealed transport requires exclusive original creation relative to supported writers and returns its captured identity directly to the invocation. The native adapter holds the continuously fenced common guest mutation lease through absence, one POST and owner commit; Docker's idempotent create alone is insufficient. Real lock-owner and stand-in controls target contention, lease loss and ambiguous creation; their presence is not a passing run. | +| `ReplaceVolume` / `PublishEnrollment` | Captured binding/birth/directory must equal fresh observations; root/lock/record/staged inode and bytes remain exact before rename. Copied-label replacement and post-effect file/lock/root controls refuse. | +| `ConfirmDirectorySync` / `CrashOrFailure` | Commit return follows enrolled rename and parent sync, then final guest/observation/files/deadline fences. Failed final sync returns uncertainty even if the enrolled pathname is visible. No reopen recovers an original-create capability. | +| `ReadRetained` / `RetireCompute` | Existing-only locked read never rewrites binding or stable identity when caller compute references change. Native receipt4 keeps data outside compute resources, and teardown has no data-delete effect. This unchanged model does not qualify the newly wired runtime or real SQL retention. | + +The positive exploration exhausts 38 distinct states. Four guard-removal +controls must fail `NoExistingAdoption`, `OriginalBirthAtCommit`, `NoPendingMatch` +and `RetirementPreservesData` at their corresponding named action with the exact +same-state witness. Registry/checker contracts reject incomplete exploration and +unrelated failures. This model does not prove fsync/crash durability, real guest or +volume observations, atomic ownership against unsynchronized external writers, +transport cancellation, SQL contents/retention or NC05 application acceptance. +No fairness or eventual recovery is asserted. + +The original 38-state model treats abstract volume identities as distinct; it +does not establish unique continuity from name/labels/birth/device/inode metadata. +`MetadataAliasing.tla` adds the explicit empty-replacement counterexample: actual +volume changes while the complete reported metadata tuple stays identical, and +metadata-only `ReadRetained` falsely matches. Its separate `metadata-alias` +negative control requires that exact same-state witness. The new control is +qualified as a negative counterexample; the historical 38-state result is not its qualification. +The root-witness guard assumes an already-enrolled original witness and abstracts +subsequent read-only verification; its original-seed helper/runtime and whole-root/xattr copying exclusion remain +separate from this finite model. Ordinary native persistent startup stays gated. + Run `bun run test:models` with Java 17 available and `TLA2TOOLS_JAR` pointing to TLA+ 1.7.4's `tla2tools.jar`. The runner checks the SHA-256 before executing Java: diff --git a/tests/models/tla/native-persistent-enrollment/Enrollment.tla b/tests/models/tla/native-persistent-enrollment/Enrollment.tla new file mode 100644 index 000000000..b40941813 --- /dev/null +++ b/tests/models/tla/native-persistent-enrollment/Enrollment.tla @@ -0,0 +1,75 @@ +-------------------------- MODULE Enrollment -------------------------- +EXTENDS Naturals +CONSTANTS EnforceExclusiveCreate, EnforceOriginalCommit, + EnforcePendingRead, KeepData +VARIABLES phase, record, volume, captured, alive, returned, matched, run, + unsafeAdoption, unsafeCommit, unsafeRead, unsafeRetirement +vars == <> + +Init == /\ phase = "idle" /\ record = "none" + /\ volume \in {0, 2} /\ captured = 0 /\ alive = TRUE + /\ returned = FALSE /\ matched = FALSE /\ run = 1 + /\ unsafeAdoption = FALSE /\ unsafeCommit = FALSE + /\ unsafeRead = FALSE /\ unsafeRetirement = FALSE +ReserveIntent == /\ phase = "idle" /\ alive + /\ phase' = "pending" /\ record' = "pending" + /\ UNCHANGED <> +CreateOriginal == /\ phase = "pending" /\ alive + /\ (~EnforceExclusiveCreate \/ volume = 0) + /\ volume' = (IF volume = 0 THEN 1 ELSE volume) + /\ phase' = "created" + /\ unsafeAdoption' = (volume # 0) + /\ UNCHANGED <> +CaptureBirth == /\ phase = "created" /\ alive + /\ captured' = volume /\ phase' = "captured" + /\ UNCHANGED <> +ReplaceVolume == /\ phase \in {"created", "captured", "done", "crashed"} + /\ volume = 1 /\ volume' = 2 + /\ UNCHANGED <> +PublishEnrollment == /\ phase = "captured" /\ alive + /\ (~EnforceOriginalCommit \/ (captured = 1 /\ volume = captured)) + /\ phase' = "published" /\ record' = "enrolled" + /\ unsafeCommit' = (captured # 1 \/ volume # captured) + /\ UNCHANGED <> +ConfirmDirectorySync == /\ phase = "published" /\ alive + /\ phase' = "done" /\ returned' = TRUE + /\ UNCHANGED <> +CrashOrFailure == /\ phase \in {"pending", "created", "captured", "published"} + /\ alive /\ alive' = FALSE /\ phase' = "crashed" + /\ UNCHANGED <> +ReadRetained == /\ phase \in {"done", "crashed"} /\ ~matched + /\ (IF EnforcePendingRead + THEN record = "enrolled" /\ captured > 0 /\ volume = captured + ELSE record # "none" /\ volume > 0) + /\ matched' = TRUE /\ unsafeRead' = (record # "enrolled") + /\ UNCHANGED <> +RetireCompute == /\ phase \in {"done", "crashed"} /\ run = 1 + /\ run' = 2 /\ volume' = (IF KeepData THEN volume ELSE 0) + /\ unsafeRetirement' = (~KeepData /\ volume > 0) + /\ UNCHANGED <> +Next == ReserveIntent \/ CreateOriginal \/ CaptureBirth \/ ReplaceVolume \/ + PublishEnrollment \/ ConfirmDirectorySync \/ CrashOrFailure \/ + ReadRetained \/ RetireCompute +Spec == Init /\ [][Next]_vars +TypeOK == /\ phase \in {"idle", "pending", "created", "captured", "published", "done", "crashed"} + /\ record \in {"none", "pending", "enrolled"} + /\ volume \in 0..2 /\ captured \in 0..2 /\ run \in 1..2 + /\ alive \in BOOLEAN /\ returned \in BOOLEAN /\ matched \in BOOLEAN + /\ unsafeAdoption \in BOOLEAN /\ unsafeCommit \in BOOLEAN + /\ unsafeRead \in BOOLEAN /\ unsafeRetirement \in BOOLEAN +NoExistingAdoption == ~unsafeAdoption +OriginalBirthAtCommit == ~unsafeCommit +NoPendingMatch == ~unsafeRead +RetirementPreservesData == ~unsafeRetirement +NoUnconfirmedReturn == returned => record = "enrolled" /\ phase = "done" +============================================================================= diff --git a/tests/models/tla/native-persistent-enrollment/MetadataAliasing.tla b/tests/models/tla/native-persistent-enrollment/MetadataAliasing.tla new file mode 100644 index 000000000..d85f96ff8 --- /dev/null +++ b/tests/models/tla/native-persistent-enrollment/MetadataAliasing.tla @@ -0,0 +1,36 @@ +------------------------ MODULE MetadataAliasing ------------------------ +EXTENDS Naturals +CONSTANT EnforceRootWitness +VARIABLES phase, actualVolume, reportedMetadata, capturedMetadata, + witness, expectedWitness, matched +vars == <> + +\* A previously enrolled original; reported metadata abstracts ALL selected +\* name/labels/CreatedAt/device/inode values. They are not a unique incarnation. +Init == /\ phase = "retained" /\ actualVolume = 1 + /\ reportedMetadata = 1 /\ capturedMetadata = 1 + /\ witness = 1 /\ expectedWitness = 1 /\ matched = FALSE + +\* An empty successor can report the SAME complete metadata tuple. It does +\* not contain the original root witness. Whole-root/xattr copying is excluded. +ReplaceWithAliasedMetadata == /\ phase = "retained" + /\ actualVolume' = 2 /\ witness' = 0 + /\ phase' = "replaced" + /\ UNCHANGED <> +ReadRetained == /\ phase \in {"retained", "replaced"} + /\ reportedMetadata = capturedMetadata + /\ (~EnforceRootWitness \/ witness = expectedWitness) + /\ matched' = TRUE /\ phase' = "observed" + /\ UNCHANGED <> +Next == ReplaceWithAliasedMetadata \/ ReadRetained +Spec == Init /\ [][Next]_vars +TypeOK == /\ phase \in {"retained", "replaced", "observed"} + /\ actualVolume \in 1..2 + /\ reportedMetadata = 1 /\ capturedMetadata = 1 + /\ witness \in 0..1 /\ expectedWitness = 1 + /\ matched \in BOOLEAN +NoAliasedMatch == matched => actualVolume = 1 +========================================================================= diff --git a/tests/models/tla/native-persistent-enrollment/metadata-alias.cfg b/tests/models/tla/native-persistent-enrollment/metadata-alias.cfg new file mode 100644 index 000000000..28cf2d7e0 --- /dev/null +++ b/tests/models/tla/native-persistent-enrollment/metadata-alias.cfg @@ -0,0 +1,4 @@ +CONSTANT EnforceRootWitness = FALSE +SPECIFICATION Spec +INVARIANTS TypeOK NoAliasedMatch +CHECK_DEADLOCK FALSE diff --git a/tests/models/tla/native-persistent-enrollment/negative.cfg b/tests/models/tla/native-persistent-enrollment/negative.cfg new file mode 100644 index 000000000..9a3a71a44 --- /dev/null +++ b/tests/models/tla/native-persistent-enrollment/negative.cfg @@ -0,0 +1,8 @@ +CONSTANTS EnforceExclusiveCreate = FALSE + EnforceOriginalCommit = TRUE + EnforcePendingRead = TRUE + KeepData = TRUE +SPECIFICATION Spec +INVARIANTS TypeOK NoExistingAdoption OriginalBirthAtCommit NoPendingMatch + RetirementPreservesData NoUnconfirmedReturn +CHECK_DEADLOCK FALSE diff --git a/tests/models/tla/native-persistent-enrollment/pending-read.cfg b/tests/models/tla/native-persistent-enrollment/pending-read.cfg new file mode 100644 index 000000000..8c94710d4 --- /dev/null +++ b/tests/models/tla/native-persistent-enrollment/pending-read.cfg @@ -0,0 +1,8 @@ +CONSTANTS EnforceExclusiveCreate = TRUE + EnforceOriginalCommit = TRUE + EnforcePendingRead = FALSE + KeepData = TRUE +SPECIFICATION Spec +INVARIANTS TypeOK NoExistingAdoption OriginalBirthAtCommit NoPendingMatch + RetirementPreservesData NoUnconfirmedReturn +CHECK_DEADLOCK FALSE diff --git a/tests/models/tla/native-persistent-enrollment/positive.cfg b/tests/models/tla/native-persistent-enrollment/positive.cfg new file mode 100644 index 000000000..2354f591f --- /dev/null +++ b/tests/models/tla/native-persistent-enrollment/positive.cfg @@ -0,0 +1,8 @@ +CONSTANTS EnforceExclusiveCreate = TRUE + EnforceOriginalCommit = TRUE + EnforcePendingRead = TRUE + KeepData = TRUE +SPECIFICATION Spec +INVARIANTS TypeOK NoExistingAdoption OriginalBirthAtCommit NoPendingMatch + RetirementPreservesData NoUnconfirmedReturn +CHECK_DEADLOCK FALSE diff --git a/tests/models/tla/native-persistent-enrollment/retire-data.cfg b/tests/models/tla/native-persistent-enrollment/retire-data.cfg new file mode 100644 index 000000000..2fa4241fe --- /dev/null +++ b/tests/models/tla/native-persistent-enrollment/retire-data.cfg @@ -0,0 +1,8 @@ +CONSTANTS EnforceExclusiveCreate = TRUE + EnforceOriginalCommit = TRUE + EnforcePendingRead = TRUE + KeepData = FALSE +SPECIFICATION Spec +INVARIANTS TypeOK NoExistingAdoption OriginalBirthAtCommit NoPendingMatch + RetirementPreservesData NoUnconfirmedReturn +CHECK_DEADLOCK FALSE diff --git a/tests/models/tla/native-persistent-enrollment/wrong-birth.cfg b/tests/models/tla/native-persistent-enrollment/wrong-birth.cfg new file mode 100644 index 000000000..91cf21515 --- /dev/null +++ b/tests/models/tla/native-persistent-enrollment/wrong-birth.cfg @@ -0,0 +1,8 @@ +CONSTANTS EnforceExclusiveCreate = TRUE + EnforceOriginalCommit = FALSE + EnforcePendingRead = TRUE + KeepData = TRUE +SPECIFICATION Spec +INVARIANTS TypeOK NoExistingAdoption OriginalBirthAtCommit NoPendingMatch + RetirementPreservesData NoUnconfirmedReturn +CHECK_DEADLOCK FALSE diff --git a/tests/native-authored-graph-protocol.test.ts b/tests/native-authored-graph-protocol.test.ts index 1a8025afe..d8e7f3aed 100644 --- a/tests/native-authored-graph-protocol.test.ts +++ b/tests/native-authored-graph-protocol.test.ts @@ -1,12 +1,14 @@ import { expect, test } from "bun:test"; import { createHash } from "node:crypto"; import { + nativeAuthoredReceiptBinding, parseNativeAuthoredControl, parseNativeAuthoredReady, parseNativeAuthoredReceipt, parseNativeAuthoredReview, parseNativeAuthoredSnapshot, } from "../src/backends/native-authored-graph-protocol.ts"; +import { parseNativeAuthoredRecoverySelection } from "../src/backends/native-authored-recovery-protocol.ts"; const run = "a".repeat(32); function review(profiles: string[] = []) { @@ -61,6 +63,297 @@ function receipt() { }, }; } + +function persistentReceipt() { + const base = receipt(); + const namespace = base.review.provenance.namespace; + const owner = "9".repeat(32); + return { + ...base, + version: 4, + data: { + database: { + binding: { + scope: { namespace, storage: "database", owner }, + guest: { + owner: base.owner, + boot_id: base.boot, + storage: { + device: 0, + inode: 14, + bytes: 128, + uuid: "00000000-0000-0000-0000-000000000002", + }, + }, + policy: { driver: "local", scope: "local", options: {} }, + }, + state: { + status: "enrolled", + volume: { + name: `hkp-${namespace}-${owner}-database`, + created_at: "2026-10-08T00:00:01Z", + directory: { device: 0, inode: 15 }, + }, + }, + }, + }, + data_mounts: { + "a.peer": [{ storage: "database", target: "/data", read_only: false }], + }, + }; +} +test("persistent graph4 keeps stable data across independent compute membership and excludes graph2/3 recovery", () => { + const first = parseNativeAuthoredReceipt(persistentReceipt()); + const later = persistentReceipt(); + const newRun = "8".repeat(32); + later.review.provenance.run = newRun; + later.review.review_id = createHash("sha256") + .update("hack.native-graph-review/v1\0") + .update(JSON.stringify(later.review.provenance)) + .digest("hex"); + later.resources["network:default"].name = `hkn-${newRun}-network-0`; + later.resources["container:a.peer"].name = `hkn-${newRun}-container-0`; + later.resources["network:default"].id = "7".repeat(64); + later.resources["container:a.peer"].id = "6".repeat(64); + const second = parseNativeAuthoredReceipt(later); + expect(second.data).toEqual(first.data); + expect(nativeAuthoredReceiptBinding(second)).not.toBe( + nativeAuthoredReceiptBinding(first) + ); + const selector = { + version: 2, + kind: "native-graph-recovery-selection", + run, + receipt: first, + receipt_sha256: "1".repeat(64), + owner_sha256: "2".repeat(64), + host_boot_uuid: "00000000-0000-0000-0000-000000000003", + }; + // An otherwise valid selector reaches the graph-family gate; shape refusal + // cannot disguise a missing graph4 recovery exclusion. + const imageOnly = parseNativeAuthoredReceipt(receipt()); + expect( + parseNativeAuthoredRecoverySelection({ + value: { ...selector, receipt: imageOnly }, + admitted: imageOnly, + }).version + ).toBe(2); + expect(() => + parseNativeAuthoredRecoverySelection({ value: selector, admitted: first }) + ).toThrow(); + expect(() => + parseNativeAuthoredRecoverySelection({ + value: selector, + admitted: imageOnly, + }) + ).toThrow(); + for (const version of [2, 3, 5]) { + expect(() => + parseNativeAuthoredReceipt({ ...persistentReceipt(), version }) + ).toThrow(); + } + expect(() => + parseNativeAuthoredReceipt({ ...receipt(), version: 4 }) + ).toThrow(); +}); +test("persistent graph4 closes fields and retains exact birth, directory, guest and mount membership", () => { + const original = persistentReceipt(); + const first = parseNativeAuthoredReceipt(original); + const changes: ((value: ReturnType) => void)[] = [ + (value) => { + value.data.database.binding.scope.namespace = "1".repeat(64); + }, + (value) => { + value.data.database.binding.guest.owner = "1".repeat(32); + }, + (value) => { + value.data.database.binding.guest.boot_id = + "00000000-0000-0000-0000-000000000000"; + }, + (value) => { + value.data.database.binding.guest.storage.inode = 0; + }, + (value) => { + value.data.database.state.volume.directory.inode = 0; + }, + (value) => { + value.data.database.state.volume.created_at = "0001-01-01T00:00:00.000Z"; + }, + (value) => { + value.data.database.state.volume.created_at = "2026-02-30T00:00:01Z"; + }, + (value) => { + const mount = value.data_mounts["a.peer"][0]; + if (!mount) { + throw new Error("Missing fixture mount."); + } + mount.storage = "missing"; + }, + (value) => { + const mount = value.data_mounts["a.peer"][0]; + if (!mount) { + throw new Error("Missing fixture mount."); + } + mount.target = "/data/../other"; + }, + ]; + for (const change of changes) { + const value = structuredClone(original); + change(value); + expect(() => parseNativeAuthoredReceipt(value)).toThrow(); + } + for (const state of [ + { status: "reserved", intent: "1".repeat(32) }, + { + status: "enrolled", + volume: { + ...original.data.database.state.volume, + extra: "private-canary", + }, + }, + ]) { + expect(() => + parseNativeAuthoredReceipt({ + ...original, + data: { database: { ...original.data.database, state } }, + }) + ).toThrow(); + } + const replaced = persistentReceipt(); + replaced.data.database.state.volume.created_at = "2026-10-08T00:00:02Z"; + const parsed = parseNativeAuthoredReceipt(replaced); + expect(nativeAuthoredReceiptBinding(parsed)).not.toBe( + nativeAuthoredReceiptBinding(first) + ); + expect(() => + parseNativeAuthoredControl( + { + ...status(), + result: { + outcome: "status", + snapshot: { + receipt: replaced, + observations: status().result.snapshot.observations, + }, + }, + }, + first, + "status" + ) + ).toThrow(); + let calls = 0; + const accessor = { ...original.data.database.binding.scope }; + Object.defineProperty(accessor, "owner", { + enumerable: true, + get() { + calls += 1; + return "1".repeat(32); + }, + }); + expect(() => + parseNativeAuthoredReceipt({ + ...original, + data: { + database: { + ...original.data.database, + binding: { ...original.data.database.binding, scope: accessor }, + }, + }, + }) + ).toThrow(); + expect(calls).toBe(0); + const mounts = [...original.data_mounts["a.peer"]]; + Object.defineProperty(mounts, "0", { + enumerable: true, + get() { + calls += 1; + return original.data_mounts["a.peer"][0]; + }, + }); + expect(() => + parseNativeAuthoredReceipt({ + ...original, + data_mounts: { "a.peer": mounts }, + }) + ).toThrow(); + expect(calls).toBe(0); + const extra = { + ...original.data.database.binding.scope, + [Symbol("extra")]: "private-canary", + }; + expect(() => + parseNativeAuthoredReceipt({ + ...original, + data: { + database: { + ...original.data.database, + binding: { ...original.data.database.binding, scope: extra }, + }, + }, + }) + ).toThrow(); + const interrupted = new Proxy(original.data.database.binding.scope, { + ownKeys() { + throw new Error("private-canary"); + }, + }); + expect(() => + parseNativeAuthoredReceipt({ + ...original, + data: { + database: { + ...original.data.database, + binding: { ...original.data.database.binding, scope: interrupted }, + }, + }, + }) + ).toThrow("values omitted"); +}); +test("image-only graph2 receipt binding retains its pre-storage serialized bytes", () => { + const bound = parseNativeAuthoredReceipt(receipt()); + expect(nativeAuthoredReceiptBinding(bound)).toBe( + JSON.stringify({ + owner: bound.owner, + boot: bound.boot, + review: bound.review, + readiness: bound.readiness, + resources: { + "network:default": { + kind: "network", + key: "default", + name: `hkn-${run}-network-0`, + id: "1".repeat(64), + image: null, + outbound: true, + }, + "container:a.peer": { + kind: "container", + key: "a.peer", + name: `hkn-${run}-container-0`, + id: "2".repeat(64), + image: `sha256:${"3".repeat(64)}`, + networks: ["default"], + outbound: false, + }, + }, + }) + ); +}); +test("graph2 refuses every present storage field, including empty and null", () => { + for (const fields of [ + { data: {} }, + { data_mounts: {} }, + { data: {}, data_mounts: {} }, + { data: null }, + { data_mounts: null }, + { data: null, data_mounts: null }, + ]) { + expect(() => + parseNativeAuthoredReceipt({ ...receipt(), ...fields }) + ).toThrow(); + } + expect(parseNativeAuthoredReceipt(receipt()).version).toBe(2); +}); function status(bound = receipt()) { return { version: 2, diff --git a/tests/tla-runtime-models.test.ts b/tests/tla-runtime-models.test.ts index 46b8a32df..2b90588d8 100644 --- a/tests/tla-runtime-models.test.ts +++ b/tests/tla-runtime-models.test.ts @@ -80,6 +80,51 @@ test("runtime model config selection rejects paths and arbitrary filenames", () } }); +test("persistent enrollment accepts both genuine changed-birth witnesses only", () => { + const control = runtimeModels + .find((model) => model.name === "native-persistent-enrollment") + ?.additionalControls.find((entry) => entry.name === "wrong-birth"); + expect(control).toBeDefined(); + for (const captured of [1, 2]) { + const output = tlaWitness({ + invariant: "OriginalBirthAtCommit", + action: "PublishEnrollment", + fields: [`captured = ${captured}`, "volume = 2", "unsafeCommit = TRUE"], + }); + expect(control?.verify({ negative: true, exitCode: 12, output })).toBe( + true + ); + expect( + control?.verify({ + negative: true, + exitCode: 12, + output: output.replace("volume = 2", "volume = 1"), + }) + ).toBe(false); + } +}); +test("persistent metadata-alias evidence requires a changed physical volume with unchanged reported identity", () => { + const control = runtimeModels + .find((model) => model.name === "native-persistent-enrollment") + ?.additionalControls.find((entry) => entry.name === "metadata-alias"); + expect(control).toBeDefined(); + if (!control) { + throw new Error("Missing persistent metadata-alias control."); + } + const output = tlaWitness(control); + expect(control.verify({ negative: true, exitCode: 12, output })).toBe(true); + for (const invalid of [ + output.replace("actualVolume = 2", "actualVolume = 1"), + output.replace("reportedMetadata = 1", "reportedMetadata = 2"), + output.replace("witness = 0", "witness = 1"), + output.replace(" { const model = runtimeModels.find( (entry) => entry.name === "native-storage-witness"