fix(runtime): recover retained graphs after lost reboot publications

docs/guides/native-candidate.md

@@ -118,6 +118,117 @@ complete bundle to change the candidate.
    binary refusing newer state is an unsupported downgrade, not a successful rollback.
    Preserve that state; do not rewrite receipts or adopt another home's data.
 
+Host cleanup or a reboot can remove the temporary `/private/tmp/hkl-<owner>` HOME
+alias while the private candidate home and disks remain intact. Normal commands
+report `provider_home_missing`; they do not initialize another pool or recreate the
+alias during observation. Explicit `runtime recover --json` can restore only that
+absent exact alias, after verifying the receipt-bound dead provider, both recorded
+disks, free VM lock, closed disk handles, and no active provider command. Existing
+files, directories, foreign links, a live or reused PID, and uncertain ownership
+refuse without replacement. Recovery rechecks the receipt before exclusive creation,
+then validates socket absence and flushes the disks before recording
+`recovered-unclean`. If those final checks fail, it reports
+`provider_home_restored_recovery_incomplete`, keeps the exact owned alias and retained
+data, and requires inspection before retry. No guest is started by alias recovery.
+Interrupted starts without a recorded process or both identified disks remain
+outside this repair path.
+
+A physical macOS reboot may also renumber the mounted filesystem device. Strict
+disk and source checks still refuse a changed device number; missing-HOME recovery
+does not waive them. For an offline **stock pool**, inspect the separate migration:
+
+```sh
+./hack-native --candidate-root /absolute/private/candidate-home runtime host-filesystem-recovery --json
+./hack-native --candidate-root /absolute/private/candidate-home runtime recover-host-filesystem --expect-sha256 <selection_sha256> --accept-legacy-device-rebind --json
+./hack-native --candidate-root /absolute/private/candidate-home runtime recover --json
+```
+
+Review the inspection before selecting its hash. This explicit legacy migration
+requires an absent recorded provider whose start predates the current host boot,
+no active provider commands, exclusive existing operation/VM locks and closed disk
+handles. Both disk inodes, sizes and ext4 UUIDs, the exact source path/inode and its
+ownership must be unchanged; only one common old-to-new device-number change is
+allowed. The inspection is read-only. Publication atomically changes only the
+owner's disk and source device numbers, retaining its phase and process record.
+Normal commands keep their strict identity checks.
+
+Legacy receipts have no original host boot UUID or filesystem volume UUID. Calendar
+timestamps corroborate a reboot, and matching retained file identities constrain
+the migration, but neither proves original volume continuity. The opt-in explicitly
+accepts that limitation; copied or relocated pools are outside this procedure.
+Prepared-base pools and pending owner/network/activation updates require separate
+recovery and are refused. A torn owner publication preserves `owner.pending` and
+blocks another migration; do not delete or adopt that file manually.
+
+This does not start the VM, restore HOME, retire stale sockets or rewrite historical
+graph receipts. Use ordinary recovery afterward. Historical shared-source graphs
+retain the prior identity and may require verified cleanup plus a new generation;
+successful metadata migration alone does not establish application recovery.
+
+After that explicit provider recovery and one audited VM boot, a retained graph
+from the immediately preceding guest boot can still carry old host device numbers
+in its publisher, relay-control, and dependency-socket receipts. Inspect and select
+one run's host-pin recovery separately:
+
+```sh
+./hack-native --candidate-root /absolute/private/candidate-home graph inspect-host-pin-recovery --run-id <run-id> --json
+./hack-native --candidate-root /absolute/private/candidate-home graph recover-host-pins --run-id <run-id> --expect-selection <selection_sha256> --accept-legacy-device-rebind --json
+./hack-native --candidate-root /absolute/private/candidate-home graph recover-cleanup --run-id <run-id> --expect-receipt <original_receipt_sha256> --json
+./hack-native --candidate-root /absolute/private/candidate-home graph retire-recovered-publisher --run-id <run-id> --expect-owner <owner> --json
+```
+
+The first command only inspects. The second publishes a private, exact-run
+witness before cleanup; it changes no graph, publisher, control, or dependency
+receipt. It requires the provider's repaired current disks/source, unchanged
+recorded inodes and raw receipts, dead owners whose recorded starts precede the
+current physical host boot, refused socket listeners, and the immediate guest
+boot transition. It also binds retained volume names and actual labels. Cleanup
+and retirement can use the witness only for those selected old pins; ordinary
+reads and later publisher generations remain strict. Missing or foreign pins,
+active listeners, changed resources, pending journals, or further guest boots
+refuse without adopting another run. A completed older cleanup journal is
+retained as history and does not itself block a later selected generation.
+
+Legacy receipts do not identify the original APFS volume. This explicit
+device-number rebind cannot prove pre-reboot volume continuity. Completing these
+commands retains the old run's data and proves cleanup of its dead generation;
+it does not migrate `graph.source.shared`, restore the application, establish
+route readiness, or claim overall v5 acceptance. Same-run source continuity
+requires a separate explicit witness-bound transition after cleanup and
+publisher retirement. If macOS removed the foreground or relay-control
+directory itself during reboot, this command refuses: the old pin receipts no
+longer exist, and absent pathnames cannot stand in for their recorded owner
+identities. That case requires a separate selected absence-recovery procedure.
+
+When a **physical host reboot** removed both the deterministic foreground
+publication root and this run's relay-control root, inspect the distinct
+absence path with the private original provider Owner and its exact
+pre-migration host-filesystem inspection:
+
+```sh
+./hack-native --candidate-root /absolute/private/candidate-home graph inspect-absent-publication-cleanup --run-id <run-id> --original-owner-file <private-original-owner.json> --host-inspection-file <private-inspection.json> --json
+./hack-native --candidate-root /absolute/private/candidate-home graph recover-absent-publication-cleanup --run-id <run-id> --original-owner-file <private-original-owner.json> --host-inspection-file <private-inspection.json> --expect-selection <selection_sha256> --retain-data --accept-unpinned-post-reboot --json
+```
+
+Inspection does not create a publication root. A failed selected action can
+leave only its private lock reservation; inspection recognizes that exact
+lock-only state. Recovery locks the foreground publication before acquiring
+the VM lease, then durably records the selected absence before any cleanup.
+It retains volumes, verifies the stopped receipt, and records a separate
+absent-publisher retirement. Ordinary publication and cleanup cannot infer
+ownership from missing paths. The old foreground PID and original physical
+volume are not proved by legacy graph receipts; this path requires explicit
+acceptance of that post-reboot limitation and refuses a changed boot, present
+or foreign publication, stale selection, pending state, or changed resources.
+An interrupted recovery can resume only its exact selected intent on the same
+host and immediate guest boot. A later successful restore treats this witness
+as history; it never grants cleanup of the new generation.
+
+This operation does not change historical shared-source device identity.
+Source-mounted projects require the separate selected source-continuity step
+before normal same-run restore. The command's stopped/data-retained result is
+not proof of application startup or routing.
+
 Compatibility is qualified for specific bundle hashes and state formats. The
 displayed version alone does not establish frontend/executor or downgrade
 compatibility. V4 and candidate homes remain separate; this procedure does not

packages/runtime-core/src/graph_cli.rs

@@ -25,6 +25,131 @@ pub fn command(candidate: &Candidate, args: &[&str]) -> Result<Value, CandidateE
     let Some((action, args)) = args.split_first() else {
         return Err(invalid());
     };
+    if *action == "inspect-host-pin-recovery" {
+        let run = match *args {
+            ["--run-id", run] | ["--run-id", run, "--json"] => run,
+            _ => return Err(invalid()),
+        };
+        #[cfg(target_os = "macos")]
+        {
+            return graph::inspect_host_pin_recovery(candidate, run);
+        }
+        #[cfg(not(target_os = "macos"))]
+        {
+            let _ = run;
+            return Err(invalid());
+        }
+    }
+    if *action == "recover-host-pins" {
+        let (run, expected) = match *args {
+            [
+                "--run-id",
+                run,
+                "--expect-selection",
+                expected,
+                "--accept-legacy-device-rebind",
+            ]
+            | [
+                "--run-id",
+                run,
+                "--expect-selection",
+                expected,
+                "--accept-legacy-device-rebind",
+                "--json",
+            ] => (run, expected),
+            _ => return Err(invalid()),
+        };
+        #[cfg(target_os = "macos")]
+        {
+            return graph::recover_host_pins(candidate, run, expected);
+        }
+        #[cfg(not(target_os = "macos"))]
+        {
+            let _ = (run, expected);
+            return Err(invalid());
+        }
+    }
+    if *action == "inspect-absent-publication-cleanup" {
+        let (run, original, inspection) = match *args {
+            [
+                "--run-id",
+                run,
+                "--original-owner-file",
+                original,
+                "--host-inspection-file",
+                inspection,
+            ]
+            | [
+                "--run-id",
+                run,
+                "--original-owner-file",
+                original,
+                "--host-inspection-file",
+                inspection,
+                "--json",
+            ] => (run, original, inspection),
+            _ => return Err(invalid()),
+        };
+        #[cfg(target_os = "macos")]
+        {
+            return graph::inspect_absent_publication_cleanup(
+                candidate,
+                run,
+                Path::new(original),
+                Path::new(inspection),
+            );
+        }
+        #[cfg(not(target_os = "macos"))]
+        {
+            let _ = (run, original, inspection);
+            return Err(invalid());
+        }
+    }
+    if *action == "recover-absent-publication-cleanup" {
+        let (run, original, inspection, expected) = match *args {
+            [
+                "--run-id",
+                run,
+                "--original-owner-file",
+                original,
+                "--host-inspection-file",
+                inspection,
+                "--expect-selection",
+                expected,
+                "--retain-data",
+                "--accept-unpinned-post-reboot",
+            ]
+            | [
+                "--run-id",
+                run,
+                "--original-owner-file",
+                original,
+                "--host-inspection-file",
+                inspection,
+                "--expect-selection",
+                expected,
+                "--retain-data",
+                "--accept-unpinned-post-reboot",
+                "--json",
+            ] => (run, original, inspection, expected),
+            _ => return Err(invalid()),
+        };
+        #[cfg(target_os = "macos")]
+        {
+            return graph::recover_absent_publication_cleanup(
+                candidate,
+                run,
+                expected,
+                Path::new(original),
+                Path::new(inspection),
+            );
+        }
+        #[cfg(not(target_os = "macos"))]
+        {
+            let _ = (run, original, inspection, expected);
+            return Err(invalid());
+        }
+    }
     if ["run-selection", "run-service"].contains(action) {
         return one_off::command(candidate, action, args);
     }

packages/runtime-core/src/main.rs

@@ -27,6 +27,10 @@ Usage:
   hack-local graph recover-cleanup --run-id <32-hex> --expect-receipt <sha256> [--json]
   hack-local graph recover-live-owner --run-id <32-hex> --expect-receipt <sha256> [--json]
   hack-local graph retire-recovered-publisher --run-id <32-hex> --expect-owner <32-hex> [--json]
+  hack-local graph inspect-host-pin-recovery --run-id <32-hex> [--json]
+  hack-local graph recover-host-pins --run-id <32-hex> --expect-selection <64-hex> --accept-legacy-device-rebind [--json]
+  hack-local graph inspect-absent-publication-cleanup --run-id <32-hex> --original-owner-file <private-json> --host-inspection-file <private-json> [--json]
+  hack-local graph recover-absent-publication-cleanup --run-id <32-hex> --original-owner-file <private-json> --host-inspection-file <private-json> --expect-selection <64-hex> --retain-data --accept-unpinned-post-reboot [--json]
   hack-local graph logs --run-id <32-hex> --service <name> [--tail <1..1000>] [--json]
   hack-local graph exec --run-id <32-hex> --service <name> [--workdir /path] [--timeout-seconds <1..120>] [--json] -- <program> [args...]
   hack-local graph dependency-plan --dependencies <reviewed.json> [--json]
@@ -86,6 +90,8 @@ Usage:
   hack-local runtime up --profile development --internet --json
   hack-local runtime network extend --allow-host <hostname> [--allow-host <hostname>] --json
   hack-local runtime up|status|down|recover [--json]
+  hack-local runtime host-filesystem-recovery [--json]
+  hack-local runtime recover-host-filesystem --expect-sha256 <sha256> --accept-legacy-device-rebind [--json]
   hack-local node serve|status|inspect
   hack-local node request <versioned-json>
   hack-local --version
@@ -857,6 +863,32 @@ fn run() -> Result<(), CandidateError> {
                 image,
             )?)?;
         }
+        ["runtime", "host-filesystem-recovery"]
+        | ["runtime", "host-filesystem-recovery", "--json"] => {
+            print_json(&hack_runtime_core::provider::host_filesystem::inspect(
+                &discover_candidate(&requested)?,
+            )?)?;
+        }
+        [
+            "runtime",
+            "recover-host-filesystem",
+            "--expect-sha256",
+            hash,
+            "--accept-legacy-device-rebind",
+        ]
+        | [
+            "runtime",
+            "recover-host-filesystem",
+            "--expect-sha256",
+            hash,
+            "--accept-legacy-device-rebind",
+            "--json",
+        ] => {
+            print_json(&hack_runtime_core::provider::host_filesystem::recover(
+                &discover_candidate(&requested)?,
+                hash,
+            )?)?;
+        }
         ["runtime", action] | ["runtime", action, "--json"] => {
             let candidate = discover_candidate(&requested)?;
             let result = match *action {