Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions docs/guides/native-candidate.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,77 @@ complete bundle to change the candidate.
binary refusing newer state is an unsupported downgrade, not a successful rollback.
Preserve that state; do not rewrite receipts or adopt another home's data.

Host cleanup or a reboot can remove the temporary `/private/tmp/hkl-<owner>` HOME
alias while the private candidate home and disks remain intact. Normal commands
report `provider_home_missing`; they do not initialize another pool or recreate the
alias during observation. Explicit `runtime recover --json` can restore only that
absent exact alias, after verifying the receipt-bound dead provider, both recorded
disks, free VM lock, closed disk handles, and no active provider command. Existing
files, directories, foreign links, a live or reused PID, and uncertain ownership
refuse without replacement. Recovery rechecks the receipt before exclusive creation,
then validates socket absence and flushes the disks before recording
`recovered-unclean`. If those final checks fail, it reports
`provider_home_restored_recovery_incomplete`, keeps the exact owned alias and retained
data, and requires inspection before retry. No guest is started by alias recovery.
Interrupted starts without a recorded process or both identified disks remain
outside this repair path.

A physical macOS reboot may also renumber the mounted filesystem device. Strict
disk and source checks still refuse a changed device number; missing-HOME recovery
does not waive them. For an offline **stock pool**, inspect the separate migration:

```sh
./hack-native --candidate-root /absolute/private/candidate-home runtime host-filesystem-recovery --json
./hack-native --candidate-root /absolute/private/candidate-home runtime recover-host-filesystem --expect-sha256 <selection_sha256> --accept-legacy-device-rebind --json
./hack-native --candidate-root /absolute/private/candidate-home runtime recover --json
```

Review the inspection before selecting its hash. This explicit legacy migration
requires an absent recorded provider whose start predates the current host boot,
no active provider commands, exclusive existing operation/VM locks and closed disk
handles. Both disk inodes, sizes and ext4 UUIDs, the exact source path/inode and its
ownership must be unchanged; only one common old-to-new device-number change is
allowed. The inspection is read-only. Publication atomically changes only the
owner's disk and source device numbers, retaining its phase and process record.
Normal commands keep their strict identity checks.

Legacy receipts have no original host boot UUID or filesystem volume UUID. Calendar
timestamps corroborate a reboot, and matching retained file identities constrain
the migration, but neither proves original volume continuity. The opt-in explicitly
accepts that limitation; copied or relocated pools are outside this procedure.
Prepared-base pools and pending owner/network/activation updates require separate
recovery and are refused. A torn owner publication preserves `owner.pending` and
blocks another migration; do not delete or adopt that file manually.

This does not start the VM, restore HOME, retire stale sockets or rewrite historical
graph receipts. Use ordinary recovery afterward. Historical shared-source graphs
retain the prior identity and may require verified cleanup plus a new generation;
successful metadata migration alone does not establish application recovery.

The frontend project-run mapping also records directory device numbers. If native
inspection succeeds after provider recovery but ordinary project commands refuse
the mapping, inspect the exact instance with the current bundle:

```sh
./hack-v5 doctor --path /absolute/project --branch my-instance --native-run-mapping inspect --json
./hack-v5 doctor --path /absolute/project --branch my-instance --native-run-mapping repair --expect-selection <selectionSha256> --accept-legacy-device-rebind --json
```

Omitting `--branch` uses the same linked-worktree default as project commands;
detached linked worktrees require an explicit instance. Inspection writes nothing.
Repair requires the exact selection and explicit acceptance of unproven original
filesystem volume continuity. It changes only the three scope directory device
numbers, preserving canonical paths, inodes, branch, run, owner, plan, environment
selection, profiles and AWS selector. The selected graph and current project share
must still match native authority. An audit copy retains the original private
mapping. Held locks, pending restart state, substituted directories, stale hashes
and changed native identities refuse publication.

This is a metadata repair, not an app restart: it does not retire sockets, modify
graph history, remove data or establish browser readiness. Doctor's ordinary
`--fix` never applies it. Run the normal project command separately after reviewing
the result; its existing native graph and ownership checks still apply.

Compatibility is qualified for specific bundle hashes and state formats. The
displayed version alone does not establish frontend/executor or downgrade
compatibility. V4 and candidate homes remain separate; this procedure does not
Expand Down
4 changes: 4 additions & 0 deletions docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1263,6 +1263,10 @@ hack doctor [options]
| `--json` | Output JSON (machine-readable) |
| `--browser-url https://app.hack` | HTTPS origin manually tested in the browser (no path or credentials) |
| `--browser-result unknown|works|fails|permission-denied` | Your manual browser observation for --browser-url (default: unknown) |
| `--branch <name>` | Run against a branch-specific instance (compose name + hostnames) |
| `--native-run-mapping inspect|repair` | Inspect or explicitly repair a native run mapping after filesystem device renumbering |
| `--expect-selection <64-hex>` | Require the exact run-mapping recovery inspection selection |
| `--accept-legacy-device-rebind` | Explicitly accept legacy migration without proof of original filesystem volume continuity |
| `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) |
| `--help, -h` | Show help |
| `--version, -v` | Show version |
Expand Down
28 changes: 28 additions & 0 deletions packages/runtime-core/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,8 @@ Usage:
hack-local runtime up --profile development --internet --json
hack-local runtime network extend --allow-host <hostname> [--allow-host <hostname>] --json
hack-local runtime up|status|down|recover [--json]
hack-local runtime host-filesystem-recovery [--json]
hack-local runtime recover-host-filesystem --expect-sha256 <sha256> --accept-legacy-device-rebind [--json]
hack-local node serve|status|inspect
hack-local node request <versioned-json>
hack-local --version
Expand Down Expand Up @@ -857,6 +859,32 @@ fn run() -> Result<(), CandidateError> {
image,
)?)?;
}
["runtime", "host-filesystem-recovery"]
| ["runtime", "host-filesystem-recovery", "--json"] => {
print_json(&hack_runtime_core::provider::host_filesystem::inspect(
&discover_candidate(&requested)?,
)?)?;
}
[
"runtime",
"recover-host-filesystem",
"--expect-sha256",
hash,
"--accept-legacy-device-rebind",
]
| [
"runtime",
"recover-host-filesystem",
"--expect-sha256",
hash,
"--accept-legacy-device-rebind",
"--json",
] => {
print_json(&hack_runtime_core::provider::host_filesystem::recover(
&discover_candidate(&requested)?,
hash,
)?)?;
}
["runtime", action] | ["runtime", action, "--json"] => {
let candidate = discover_candidate(&requested)?;
let result = match *action {
Expand Down
26 changes: 25 additions & 1 deletion packages/runtime-core/src/provider/lifecycle.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
pub mod host_filesystem;
mod interrupted;
mod prepared_boot;
#[cfg(any(target_os = "macos", test))]
mod private_child;
mod relay_process;
mod short_home;
use super::{
admission, agent, artifact, identity, process,
state::{self, Owner, io},
Expand Down Expand Up @@ -1812,6 +1814,12 @@ fn finish_absent(
value: &str,
record_disks: bool,
) -> Result<(), CandidateError> {
let vm_lock = lock_absent_disks(candidate, owner)?;
finish_absent_locked(candidate, owner, value, record_disks, &vm_lock)
}

/// Keep this descriptor through alias restoration and the stopped receipt.
fn lock_absent_disks(candidate: &Candidate, owner: &Owner) -> Result<File, CandidateError> {
let directory = owner.real_data_dir(candidate)?;
let vm_lock = OpenOptions::new()
.read(true)
Expand Down Expand Up @@ -1847,6 +1855,16 @@ fn finish_absent(
));
}
verify_disks(candidate, owner)?;
Ok(vm_lock)
}

fn finish_absent_locked(
candidate: &Candidate,
owner: &mut Owner,
value: &str,
record_disks: bool,
_vm_lock: &File,
) -> Result<(), CandidateError> {
if record_disks && owner.storage.is_none() {
owner.storage = Some(identity::disk(
&owner.real_data_dir(candidate)?.join("storage.raw"),
Expand Down Expand Up @@ -1896,7 +1914,13 @@ fn finish_absent(
}

pub fn recover(candidate: &Candidate) -> Result<RuntimeStatus, CandidateError> {
let initial = status(candidate)?;
let initial = match status(candidate) {
Ok(initial) => initial,
Err(error) if error.code == "provider_home_missing" => {
return short_home::recover(candidate);
}
Err(error) => return Err(error),
};
if initial.phase == "uninitialized" {
return Ok(initial);
}
Expand Down
Loading
Loading