Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/guides/native-candidate.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,21 @@ complete bundle to change the candidate.
binary refusing newer state is an unsupported downgrade, not a successful rollback.
Preserve that state; do not rewrite receipts or adopt another home's data.

Host cleanup or a reboot can remove the temporary `/private/tmp/hkl-<owner>` HOME
alias while the private candidate home and disks remain intact. Normal commands
report `provider_home_missing`; they do not initialize another pool or recreate the
alias during observation. Explicit `runtime recover --json` can restore only that
absent exact alias, after verifying the receipt-bound dead provider, both recorded
disks, free VM lock, closed disk handles, and no active provider command. Existing
files, directories, foreign links, a live or reused PID, and uncertain ownership
refuse without replacement. Recovery rechecks the receipt before exclusive creation,
then validates socket absence and flushes the disks before recording
`recovered-unclean`. If those final checks fail, it reports
`provider_home_restored_recovery_incomplete`, keeps the exact owned alias and retained
data, and requires inspection before retry. No guest is started by alias recovery.
Interrupted starts without a recorded process or both identified disks remain
outside this repair path.

Compatibility is qualified for specific bundle hashes and state formats. The
displayed version alone does not establish frontend/executor or downgrade
compatibility. V4 and candidate homes remain separate; this procedure does not
Expand Down
25 changes: 24 additions & 1 deletion packages/runtime-core/src/provider/lifecycle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ mod prepared_boot;
#[cfg(any(target_os = "macos", test))]
mod private_child;
mod relay_process;
mod short_home;
use super::{
admission, agent, artifact, identity, process,
state::{self, Owner, io},
Expand Down Expand Up @@ -1812,6 +1813,12 @@ fn finish_absent(
value: &str,
record_disks: bool,
) -> Result<(), CandidateError> {
let vm_lock = lock_absent_disks(candidate, owner)?;
finish_absent_locked(candidate, owner, value, record_disks, &vm_lock)
}

/// Keep this descriptor through alias restoration and the stopped receipt.
fn lock_absent_disks(candidate: &Candidate, owner: &Owner) -> Result<File, CandidateError> {
let directory = owner.real_data_dir(candidate)?;
let vm_lock = OpenOptions::new()
.read(true)
Expand Down Expand Up @@ -1847,6 +1854,16 @@ fn finish_absent(
));
}
verify_disks(candidate, owner)?;
Ok(vm_lock)
}

fn finish_absent_locked(
candidate: &Candidate,
owner: &mut Owner,
value: &str,
record_disks: bool,
_vm_lock: &File,
) -> Result<(), CandidateError> {
if record_disks && owner.storage.is_none() {
owner.storage = Some(identity::disk(
&owner.real_data_dir(candidate)?.join("storage.raw"),
Expand Down Expand Up @@ -1896,7 +1913,13 @@ fn finish_absent(
}

pub fn recover(candidate: &Candidate) -> Result<RuntimeStatus, CandidateError> {
let initial = status(candidate)?;
let initial = match status(candidate) {
Ok(initial) => initial,
Err(error) if error.code == "provider_home_missing" => {
return short_home::recover(candidate);
}
Err(error) => return Err(error),
};
if initial.phase == "uninitialized" {
return Ok(initial);
}
Expand Down
70 changes: 70 additions & 0 deletions packages/runtime-core/src/provider/lifecycle/short_home.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
//! Explicit two-phase recovery of the receipt-bound temporary HOME after host cleanup.
//!
//! No provider is launched and no existing alias is replaced. Durable disks and VM absence
//! are audited before restoring the short socket path; socket absence is then checked before
//! committing the recovered phase. A later refusal retains the exact alias and unchanged data.
use super::{
Owner, RuntimeStatus, binary, finish_absent_locked, identity, lock_absent_disks, root, state,
status, verify_disks,
};
use crate::{Candidate, CandidateError};

fn dead_provider(candidate: &Candidate, owner: &Owner) -> Result<(), CandidateError> {
let process = owner.process.as_ref().ok_or_else(|| {
CandidateError::new(
"recovery_required",
"Missing HOME recovery requires a recorded provider identity; nothing was changed.",
)
})?;
// SAFETY: geteuid has no preconditions.
identity::verify(process, process, &binary(candidate), unsafe {
libc::geteuid()
})?;
if identity::alive(process.pid)? || identity::executable_running(&binary(candidate))? {
return Err(CandidateError::new(
"recovery_required",
"Missing HOME recovery requires a confirmed dead provider and no active provider command; nothing was changed.",
));
}
if !owner.created || owner.storage.is_none() || owner.overlay.is_none() {
return Err(CandidateError::new(
"recovery_required",
"Missing HOME recovery requires both previously identified disks; nothing was adopted.",
));
}
Ok(())
}

pub(super) fn recover(candidate: &Candidate) -> Result<RuntimeStatus, CandidateError> {
let _operation = state::Lock::acquire_existing(&root(candidate))?;
let mut owner = Owner::load_for_short_home_recovery(candidate)?;
dead_provider(candidate, &owner)?;
let vm_lock = lock_absent_disks(candidate, &owner)?;
// Recheck immediately before the only new external effect. The locks fence cooperative
// writers; no signal, disk adoption, overwrite or receipt update happens in this phase.
dead_provider(candidate, &owner)?;
verify_disks(candidate, &owner)?;
owner.restore_missing_short_home(candidate)?;
let mut finish = || -> Result<(), CandidateError> {
let observed = Owner::load(candidate)?;
if observed != owner {
return Err(CandidateError::new(
"foreign_state",
"Provider receipt changed after HOME restoration.",
));
}
dead_provider(candidate, &owner)?;
verify_disks(candidate, &owner)?;
finish_absent_locked(candidate, &mut owner, "recovered-unclean", false, &vm_lock)
};
finish().map_err(|error| {
CandidateError::new(
"provider_home_restored_recovery_incomplete",
format!("Owned HOME alias restored; runtime recovery is incomplete ({}). Data retained; inspect and retry runtime recover.", error.code),
)
})?;
status(candidate)
}

#[cfg(all(test, target_os = "macos"))]
mod tests;
Loading
Loading