Skip to content

feat(sandbox): 本地进程 Sandbox 后端 LAYERFS_SANDBOX=local——在没有 Docker 的 Linux VM / CI 里使用 workspace.* - #402

Closed
jiangkoumo wants to merge 48 commits into
Ephemeral-AI-Lab:mainfrom
jiangkoumo:feat/sandbox-local-backend
Closed

jiangkoumo wants to merge 48 commits into
Ephemeral-AI-Lab:mainfrom
jiangkoumo:feat/sandbox-local-backend

Conversation

@jiangkoumo

Copy link
Copy Markdown
Contributor

改了什么

设置 LAYERFS_SANDBOX=local 时,一个 Workspace 的 Sandbox 不再是 Docker 容器,而是本机上的一个 layerfs-daemon 进程。这样 workspace.mount / exec / status / commit / unmount 可以在只有 FUSE、没有 Docker 的 Linux 环境里运行(例如 Apple Silicon 上的 microVM、CI job)。

  • 不设或设为 docker:行为与现在完全相同(Docker 路径一行没改,只是在 SDK 里多包了一层枚举)。
  • 设为其它任何值(例如拼错的 lokal):在产生任何副作用之前以 InvalidInput 拒绝,而不是静默落回 Docker(理由见下方「设计取舍」)。

改动范围(22 个文件,+2191 / −120;Production LOC 108,451 → 109,658,+1,207):

位置 内容
core/crates/layerfs-sandbox/src/backend/local/(新) Local::create:建 Sandbox 目录、写 0600 的 daemon 配置、以干净环境和独立进程组启动 layerfs-daemon --config、等它打印 LAYERFS_DAEMON_LISTEN(只接受 loopback)。Sandbox:用 pid + 内核启动时间判活(僵尸、pid 复用都不算活);stop 先 TERM 后 KILL;delete 只在确认停止后执行,先 MNT_DETACH daemon 死后留下的 FUSE 挂载、确认没有挂载了,再删目录。run:以命令身份在 Workspace 里执行一条命令,双路流式输出,被信号杀死报 128+sig(与 Engine 相同)。
core/crates/layerfs-sandbox/Cargo.toml nix 同一个钉死版本多开 signal/mount/user feature;没有新 crate,Cargo.lock 不变
sdk/js/native/src/runtime.rs(新) Runtime 枚举:Docker(容器) / Local(本地 Sandbox),加后端选择
sdk/js/native/src/local.rs(新) 本地 Sandbox 启动:沿用容器的编号、placement、create.intent 和失败善后规则;第一个控制连接走认证并要求 daemon ControlReady
sdk/js/native/src/{sandbox,discovery,registry,exec,workspace,lib}.rs 分流到 Runtime;本地记录多存 runtime、目录、pid、启动时间、端口;目录不是「绝对路径 + 无 .. + 末段等于记录名」的记录一律拒绝
sdk/js/src/{client,index}.js、README.md deployment() 读 LAYERFS_SANDBOX / LAYERFS_LOCAL_DIR(及 DSH_ 同名变量);README 新增「Local Sandboxes (no Docker)」一节和无隔离声明
测试 layerfs-sandbox/tests/local_runtime.rs(6 个,用替身 daemon,不需要 root);layerfs-node/tests/unit/runtime.rs(2 个);sdk/js/test/local_deployment.mjs。没有删改任何已有测试。

layerfs-api、layerfs-daemon、layerfs-bridge、backend::docker 均未改动。

为什么

  • issue please add macos support. Thanks a lot #368(macOS 支持)、DSH Linux on WSL2 Docker Desktop: engine volume paths are not host-visible; configured volumes does not reach Store #380(WSL2 卷路径)一类的诉求,根源是 workspace.* 必须经过 Docker Engine。daemon 本身与容器无关:给它一个配置文件、一个 FUSE 设备就能跑。
  • 在本身就是隔离边界的环境里(microVM、一次性 CI 机器),再套一层 Docker 既重又常常装不上。我们在 Apple Silicon 上用 microsandbox microVM 实测:daemon + FUSE 内核挂载在 VM 里正常,唯一缺的就是不经 Docker 起 Sandbox 的方式。
  • 方案选择:没有把 layerfs-api 的 SandboxApi 泛化成多后端(会改公开类型并波及 tests/examples/benchmark),也没有写 Docker API 兼容 shim(要逐字段伪造 inspect/topology)。SDK 里本来就有用公开 API 自己握手的代码(pool.rs),本地启动直接复用它。

⚠️ 安全声明:local 后端不提供任何隔离

local 后端只适合在本身就是隔离边界的 VM 或 CI 机器里使用。 daemon 以 root 运行在机器自己的命名空间里;命令与机器共享进程表、网络和文件系统,只是换成 uid 501 / gid 20(去掉附加组)执行。Docker 后端的 pid/mount/net/user 命名空间、seccomp、no-new-privileges 在这里都没有。另外:

  • daemon 会把机器上的 /dev/fuse 收紧为 0600 root(daemon 既有行为),在共享主机上会影响其它非 root 的 FUSE 用户;
  • 在 Debian/Ubuntu 上 gid 20 是 dialout;若 VM 里恰好存在真实的 uid 501 账号(如 Lima/colima 映射 macOS 用户),命令就以那个账号的身份运行。

需要 root:非 root 时在任何副作用之前以 Unsupported 拒绝,并说明原因。

设计取舍(请维护者判断)

  1. 拼错的 LAYERFS_SANDBOX 被拒绝,而不是默认走 Docker。 拼错的人本意是 local,静默落到 Docker 只会在没有 Docker 的 VM 里得到一个难懂的 docker.sock ENOENT。对现有用户零影响(以前没有这个变量;未设或设为 docker 时行为一致,有单测和端到端覆盖)。如果更希望「未知值回落 Docker」,改 runtime::local() 和 client.js 各一行即可。
  2. 命令身份沿用 501:20。 这是上游 sandbox::setup() 自 SDK redesign (JS-only over a Rust implementation by group/method), commit-time merge and conflicts, and live wiring of the sdk-sim-terminals UI #327 起写死、Docker 后端也在用的同一个常量;保持一致可以让同一个 Store 在两种后端之间切换时属主语义相同。若要做成可配置,建议两个后端一起改,不在本 PR 范围。
  3. Store 的 locator:本地 daemon 直接打开客户端的 storePath,所以只在这次传给 daemon 的配置副本里把 store 和 existing_store.locator 都写成这个路径(locator 的定义就是「daemon 文件系统里的路径」);Init 写进 manifest 的原值不改。

怎么验证的

测试机(x86_64 Linux,Docker 原生)——全部在最后一次源码改动之后、对干净 HEAD 复跑

  • local 端到端 smoke-local.mjs:61/61。 覆盖每个 workspace.* 调用及失败路径(NotMounted、NoSuchBranch、NoSuchSandbox、非零退出、用户自己的 timeout → 124、无隐式超时、3 MiB 输出)、后台进程与 unmount Busy、第二个进程重开 Sandbox、同一 Branch 挂两次与提交冲突、3 个 Workspace 并发、merge 与 merge 冲突、SIGKILL daemon → status/exec 被拒 → sandbox.list 回收(卸掉残留的 ENOTCONN 挂载)→ 重新 mount、已提交数据仍在、空闲 120 s 自关、无残留进程/挂载/临时文件。
  • Docker 回归: smoke-docker.mjs 的 39 行签名,本分支(未设 / =docker)与 origin/main 逐行相同;上游真实 Docker verify 测试新旧结果一致;没有 Docker 时新旧给出相同的拒绝。
  • 后端选择与权限: 拼错 → InvalidInput 且不留任何东西;缺 LAYERFS_LOCAL_DIR → InvalidInput;非 root → Unsupported 且不留 intent/记录/目录。
  • 单测与静态检查: cargo +1.85.1 test --release -p layerfs-sandbox -p layerfs-node 44/44;本改动文件 cargo fmt --check、clippy -D warnings 干净(剩下的报错都在上游既有文件,另有 fmt / clippy 小 PR 处理)。

Apple Silicon microVM(aarch64)

  • 同一个 aarch64 包先在测试机上用 qemu-user 预跑(docker run --platform linux/arm64 --privileged node:22-trixie-slim,Store 在 ext4):61/61,非 root 拒绝 PASS,无残留。
  • MacBook Air M2,microsandbox(msb)microVM,node:22-trixie-slim,2 vCPU / 1 GB,Store 在 VM 私有 ext4 磁盘:LOCAL_SMOKE 61/61,非 root 拒绝 PASS,leftovers none;smoke 135.9 s(其中 130 s 是等待空闲自关),整次 msb run 137 s,宿主进程最大 RSS 306 MB(含 VM),Mac 负载前后无明显变化。guest:aarch64,kernel 6.12.111,glibc 2.41,node v22.23.4。

风险

…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/Cargo.toml (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 108,451 -> 108,451 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mod.rs (production).
No tests changed or deleted.

Production LOC: 108,451 -> 108,460 (delta +9)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production).
No tests changed or deleted.

Production LOC: 108,460 -> 108,505 (delta +45)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mounts.rs (production).
No tests changed or deleted.

Production LOC: 108,505 -> 108,576 (delta +71)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production).
No tests changed or deleted.

Production LOC: 108,576 -> 108,834 (delta +258)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production).
No tests changed or deleted.

Production LOC: 108,834 -> 109,057 (delta +223)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production).
No tests changed or deleted.

Production LOC: 109,057 -> 109,166 (delta +109)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/mod.rs (production).
No tests changed or deleted.

Production LOC: 109,166 -> 109,167 (delta +1)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/tests/local_runtime.rs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,167 -> 109,167 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/runtime.rs (production).
No tests changed or deleted.

Production LOC: 109,167 -> 109,274 (delta +107)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/local.rs (production).
No tests changed or deleted.

Production LOC: 109,274 -> 109,527 (delta +253)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/registry.rs (production).
No tests changed or deleted.

Production LOC: 109,527 -> 109,527 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/discovery.rs (production).
No tests changed or deleted.

Production LOC: 109,527 -> 109,644 (delta +117)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/exec.rs (production).
No tests changed or deleted.

Production LOC: 109,644 -> 109,633 (delta -11)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/sandbox.rs (production).
No tests changed or deleted.

Production LOC: 109,633 -> 109,641 (delta +8)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/workspace.rs (production).
No tests changed or deleted.

Production LOC: 109,641 -> 109,636 (delta -5)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/lib.rs (production).
No tests changed or deleted.

Production LOC: 109,636 -> 109,638 (delta +2)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/tests/unit/runtime.rs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,638 -> 109,638 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/src/client.js (production).
No tests changed or deleted.

Production LOC: 109,638 -> 109,656 (delta +18)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/src/index.js (production).
No tests changed or deleted.

Production LOC: 109,656 -> 109,658 (delta +2)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/test/local_deployment.mjs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,658 -> 109,658 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/README.md (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,658 -> 109,658 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
Copilot AI balanced review requested due to automatic review settings October 22, 2026 05:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@yifanxuaaa yifanxuaaa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the local Sandbox runtime by reading the code at 819bd2c; I didn't build or run it. Seven inline comments follow, most important first: command processes surviving teardown, the parent-directory permission check for uid 501, and the missing Linux-only gating. The Docker path does look unchanged apart from the enum wrapper.

🤖 Generated with Claude Code

/// Stops the daemon once: TERM, up to `seconds` for it to end, then
/// KILL. A daemon already gone is the same stopped state. Crash
/// teardown, not a graceful filesystem drain.
pub fn stop(&mut self, seconds: u16) -> Result<(), Box<WireFailure>> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Command processes outlive the local Sandbox. stop/delete signal only the daemon's own pid. Unlike a container, nothing contains or kills the uid-501 processes that commands leave behind.

Example: an exec runs cd $HOME && nohup server >/dev/null 2>&1 & and returns. The Workspace unmounts and the daemon idle-exits after 120 s. In Docker the container's exit would kill server; here it keeps running on the host. On reclaim, remove_dir_all of <dir>/tmp races with server writing there and can fail with ENOTEMPTY. The record then stays and sandbox.list keeps failing until someone kills the process by hand. The same happens after a SIGKILLed daemon.

Suggestion: put each Sandbox's commands in a session, process group or cgroup, and kill it in stop() before delete().

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

确实如此,已修。做法:

  • 每个命令 process_group(0) 自成一个进程组;spawn 后立刻把组登记到 Sandbox 目录下 0700 的 groups/(文件名 = 组长的 boot id、pid、start time)。命令结束时组里如果已经没有进程就删掉登记,所以只有“留了东西”的组才会留在那里。
  • SDK 的 operator.process.start 用 setsid 起后台进程,会离开命令的组,所以起完后用新的 Sandbox::track(pid, start) 也登记一次。
  • stop() 在 daemon 结束后、delete() 在本对象没跑过 stop 时(也就是 daemon 被 SIGKILL 或空闲退出后 reclaim 的路径),对所有登记的组和 daemon 自己的组发一次 SIGKILL,最多等 5 秒确认组里没有活进程,然后才 detach 挂载、remove_dir_all。组长 pid 被别的进程复用(start time 不同)的组不会动;不同 boot 的登记直接跳过。
  • 为什么是进程组而不是 cgroup:layerfs-sandbox 是 #![forbid(unsafe_code)](check_product_boundary.py 也这样要求),std 没有安全的 pre_exec / CLONE_INTO_CGROUP;spawn 之后再把 pid 写进 cgroup 会和命令的第一次 fork 竞争。进程组 + 登记覆盖了 nohup、&、子 shell 和 SDK 自己的 setsid 后台进程。命令自己再 setsid / 双 fork 守护化出去的进程不跟踪,这一点写进了模块文档。
  • 和容器的一个差别也写明了:daemon 空闲退出之后、下一次 sandbox.list / mount 触发 reclaim 之前,这些进程还在;reclaim 时被杀掉,目录随后删除。

测试:

  • tests/local_runtime.rs::what_commands_leave_running_ends_with_the_sandbox:exec 留下一个不停往 $HOME(Sandbox 的 tmp)写文件的 nohup 进程和一个 setsid sleep(用 track 登记);(a) stop() 后两个都没了、delete() 删掉目录;(b) 外部 kill -9 daemon,再用 Sandbox::stopped(...) 走 delete()(reclaim 路径),两个都没了、目录删掉。
  • 端到端 V2–V2c:exec 留 nohup 写进程 + operator.process.start 后台进程 → SIGKILL daemon → operator.sandbox.list 回收成功(无 ENOTEMPTY),两个进程都没了,目录删除;L5 是你描述的原场景:exec 留 nohup 进程、unmount、等 daemon 120 秒空闲退出 → sandbox.list 回收后进程没了。修复前的二进制跑同一套,V2c 和 L5 失败,进程以 uid 501 继续存活。

{
return Err(RuntimeError::Protocol("absolute local runtime root"));
}
let metadata = fs::symlink_metadata(root)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root's parent directories are not checked as traversable by the command identity. This checks only the root's own mode. Commands chdir into their Workspace after std drops to uid 501, so every directory above LAYERFS_LOCAL_DIR must also be traversable by that uid.

Example: running as root with LAYERFS_LOCAL_DIR=/root/layerfs-local (0755) inside /root (0700, the default). Create, the daemon start and mount all succeed. Every workspace.exec, plus the HOME/scratch probes, then fails at spawn with EACCES, long after setup and with a confusing error.

Suggestion: walk the ancestors and require o+x, or check traversal as the command identity, here or at start.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

是的,已修。Local::new 现在对给定路径的每一级和 canonicalize 后真实路径的每一级都要求“是目录且 o+x”,不满足就在 create 时(写 intent 之前)返回 PermissionDenied,消息点名是哪一级,例如:

LAYERFS_LOCAL_DIR /data/sbl-e2e/closed-parent/local: runtime I/O: /data/sbl-e2e/closed-parent is not traversable by others (o+x); local Sandbox commands run as another identity and must pass it to reach their Workspace

这里按 o+x 判断,没有按 501:20 去看 owner/group 位或 ACL——更保守一些,/root 这类情况都会在一开始被拒。如果你更希望“以命令身份判断”,我可以改。

测试:every_directory_above_a_root_must_be_traversable_by_others(0700 父目录被拒且点名;经符号链接的父目录同样被拒;改成 0711 后通过);端到端 V1/V1b(0700 目录下的 LAYERFS_LOCAL_DIR:mount 返回 InvalidInput、没有 daemon、什么都没建)。修复前的二进制在同样配置下会挂载成功。


/// The command identity: this user, or nobody when the tests run as root.
fn me() -> CommandIdentity {
let meta = fs::metadata("/proc/self").unwrap();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linux-only backend and tests are not gated by platform. The backend and these tests rely on /proc (stat, mountinfo, /proc/self), but neither has #[cfg(target_os = "linux")]. On a macOS host, cargo test -p layerfs-sandbox now fails: me() panics here and LocalProcess::observe errors, so all six tests fail. The backend also compiles into the macOS SDK and fails only at run time, with an io error instead of a clear Unsupported.

Suggestion: gate backend::local, its re-exports and this test file on Linux, as mounts.rs already partly does.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修:

  • backend/mod.rs:#[cfg(target_os = "linux")] pub mod local;(mounts.rs 里原来的非 Linux 分支随之删掉);tests/local_runtime.rs 顶部 #![cfg(target_os = "linux")]。
  • SDK:mod local、Runtime::Local、Recorded::Local 及其所有分支、discovery::root() 都只在 Linux 编译。非 Linux 上 LAYERFS_SANDBOX=local 返回 Unsupported: LAYERFS_SANDBOX=local runs its daemon as a process of this kernel, which needs Linux; run it inside a Linux VM or CI box, or use Docker;遇到 local 记录也返回 Unsupported,不会报 io 错。

验证:在 Linux 上装了 aarch64-apple-darwin 的 std,cargo clippy -p layerfs-sandbox --all-targets --target aarch64-apple-darwin -- -D warnings 通过。layerfs-node 没法在 Linux 上交叉检查(blake3、zstd-sys 的 C 构建需要 macOS SDK),这部分逐处审读过 cfg;我会再在 Mac 上跑一次 cargo clippy --all-targets 和 cargo test -p layerfs-sandbox,结果补在这里。

Comment thread sdk/js/native/src/discovery.rs Outdated
let name = registry::text(value, "name")?;
if name.is_empty()
|| !directory.is_absolute()
|| directory.file_name().and_then(|n| n.to_str()) != Some(name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The directory check doesn't keep a delete inside the local root. This compares the directory's last component with name, but name comes from the same record, so the check is easy to satisfy. The directory is never compared with the configured LAYERFS_LOCAL_DIR, and name is never compared with <prefix><project>-<id>.

Example: a record that was edited, corrupted, or left from an old LAYERFS_LOCAL_DIR, with directory: "/srv/data", name: "data". It passes this check, and since pid/start no longer match it is classified as stopped. reclaim → Recorded::stopped().delete() then lazily detaches every mount under /srv/data and runs remove_dir_all("/srv/data") as root.

Suggestion: require directory == Local::new(deploy.localDir)?.naming(expected_name)?.paths()?.directory, or at least directory.parent() == configured root.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这条是安全问题,谢谢指出,已修。discovery::recorded 现在多接 args 和编号:local 记录只有在 directory 恰好等于本部署算出来的 <LAYERFS_LOCAL_DIR>/<prefix><project>-<id>、且 name 是这个目录名时才会被 reopen、列出或删除。算法只用配置(与 create 用同一个 sandbox_name()),不碰文件系统。localDir 没配置、不是绝对路径或含 ./..,同样拒绝。不符时返回 InvalidInput,说明记录写的是哪、本部署期望在哪、“neither used nor deleted”,以及该手动删除的 sandbox.record 路径,不会 detach 也不会删任何东西。

行为变化:只配置了 Docker(没有 LAYERFS_LOCAL_DIR)的进程,如果同一项目里有 local 记录,sandbox.list 会对它报错而不是去碰它——这是有意的 fail-closed。

测试:tests/unit/runtime.rs 覆盖 /srv/data+data、<root>/data、旧 LAYERFS_LOCAL_DIR、别的编号、名字不符、子目录、相对路径、..;tests/unit/reclaim.rs::a_local_record_outside_its_configured_directory_is_never_deleted 端到端:真实建一个有文件的 data/,记录指向它 → reclaim 返回 InvalidInput,文件和记录都还在;旧 LAYERFS_LOCAL_DIR 的记录也不删;位置正确的记录照常回收。

/// True while this exact process exists and has not exited: a zombie
/// (or a dead entry) is not alive, and another process under the same id
/// has another start time.
pub fn alive(self) -> bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The process identity has no boot id. Liveness compares only the pid and the start time in clock ticks since boot. After a reboot or VM snapshot restore, with LAYERFS_HOME on a persistent disk, a different process can have the same pid and tick value. Disposable CI/microVM images boot deterministically, so this is more likely there.

A stale record is then taken as the live daemon. Reopen connects to the recorded port and fails auth with a confusing error instead of reclaiming, and any stop on it (e.g. from abandon) SIGTERMs/SIGKILLs an unrelated process as root.

Suggestion: record /proc/sys/kernel/random/boot_id with the pid and start time, and compare all three.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修。LocalProcess 增加 boot(/proc/sys/kernel/random/boot_id,每进程读一次),alive() 要求 pid、start、boot 三者都相同;记录新增 boot 字段。命令进程组的登记也带 boot,不同 boot 的登记一律跳过。

旧记录:只有这个分支合并前的构建写过 local 记录,没有任何发布版本写过。缺 boot 的记录仍按 pid + start 判活(这样不会把仍在跑的旧 daemon 误判为停止、删掉它的目录),但永远不会对它发信号(signal() 直接拒绝)。如果你们更希望不保留这段兼容、缺 boot 直接当作损坏记录拒绝,改动只有几行,告诉我就改。

测试:a_process_of_another_boot_is_not_alive_and_one_without_a_boot_is_never_signalled(同 pid + start、不同 boot → 不活,reopen 得到 stopped;缺 boot → 仍活,但经它 stop() 返回错误、daemon 没有被动);记录解析的单测覆盖了 boot 正确、缺失、非法三种情况。

Comment thread sdk/js/native/src/local.rs Outdated
mounts: Default::default(),
processes: Default::default(),
});
crate::discovery::save_sandbox(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A failure after startup orphans the root daemon and leaves a zombie. By this point the daemon runs and has authenticated. If save_sandbox, save_text or the intent removal fails, ? returns without stopping it. SandboxState and its Child are dropped without a wait. The outer start then calls remove_dir(saved_directory), which fails because the directory isn't empty.

Result: a root daemon keeps the Store open with no record, and later starts are refused by the leftover create.intent. In a long-lived Node process (e.g. dsh), the dropped Child is never waited, so once the daemon idle-exits it stays a zombie until Node exits.

Suggestion: send these errors through abandon(..., Some(sandbox)), or reap the child in a Drop.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修。save_sandbox、save_text("container") 和删除 intent 收进一个闭包;任何一步失败都会把 SandboxState 拆回 Sandbox,删掉已经写下的 sandbox.record / container,再走 abandon(..., Some(sandbox))(读日志尾 → stop → delete → 处理 intent,各一次,不重试),外层的 remove_dir(saved_directory) 因此能成功,编号释放。另外给 Sandbox 加了 Drop,只做一次 try_wait,回收本进程启动的、已经退出的 daemon,不会去 stop 任何东西,所以长驻的 Node 进程里不会留僵尸。

在 aarch64(qemu-user)上预跑时,这条的端到端检查还抓到一个竞态,一并修了:daemon 是多线程的,被停掉时主线程先变成 Z,alive() 已经是 false,但其它线程还没退完,这时 waitpid(WNOHANG) 还拿不到它。原来的 stop() 一看到“不活”就返回,delete() 和 Drop 又各只 try_wait 一次;三次都落在这个窗口里(qemu 下线程退得慢),Child 被丢掉,就留下一个永久僵尸。现在 stop() 要等本进程启动的 daemon 真正被 reap 后才返回(gone = reap() && !alive()),KILL 阶段也对还没 reap 的 Child 直接 kill()(没 reap 之前 pid 不会被复用),上限不变(TERM seconds + KILL 5 秒)。

测试(端到端 V4/V4b,故障注入):把 LAYERFS_DAEMON 换成一个包装脚本,它在 exec 真 daemon 前先建好 sandboxes/1/container,让 save_text 以 EEXIST 失败。结果:返回 Failed: local sandbox records: … File exists …; its daemon's last log lines: …;之后没有 daemon、没有僵尸子进程、LAYERFS_LOCAL_DIR 为空、没有 create.intent、sandboxes/1 不存在;V5 下一次 mount 正常拿到 Sandbox 1。修复前的二进制跑同样的注入,Sandbox 目录残留。

单测 tests/local_runtime.rs::a_stop_reaps_a_daemon_whose_main_thread_ended_first 把上面那个窗口做成确定性的:替身 daemon 用 python3 的 ctypes 调 pthread_exit 让主线程先退、另一个线程继续睡;断言 alive() 为 false 且线程组还在,然后 stop(1),断言进程已被 reap。改之前这个测试稳定失败,改之后通过(重复 20 次);没有 python3 的机器上它打印 skipped。修复后 aarch64 qemu 端到端连续 5 次 72/72。


/// The image a local Sandbox records: placement keeps local and Docker
/// Sandboxes of one project apart by it.
pub(crate) const LOCAL: &str = "local";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The placement key local can collide with a Docker image name. Local and Docker Sandboxes are kept apart only by recording image = "local" and placing by image string. local is a valid Docker image reference, so a Docker deployment with LAYERFS_IMAGE=local (e.g. a locally built image tagged local) would be placed onto running local Sandboxes. Its Workspace would then mount in a host process with no isolation, and the reverse can also happen.

Suggestion: place by (runtime, image), using the record's runtime field, instead of overloading the image string.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

确实会撞,已修。runtime::image() 换成 runtime::placement():记录的 runtime 必须和部署一致(Docker 记录没有这个字段,按 docker 处理,所以所有旧记录都兼容),Docker 再比 image;没有 deploy 时不过滤(和以前一样);Docker 部署没配 image 时也绝不会接 local 记录。local 记录仍写 image: "local",只为让 sandbox.list 的输出不变。

测试:placement_keeps_runtimes_apart_even_for_an_image_named_local(两个方向、旧 Docker 记录、没有 image、没有 deploy);端到端 V3/V3b:local Sandbox 1 在跑时,用 LAYERFS_SANDBOX 未设、LAYERFS_IMAGE=local 的客户端 mount,不再被放到 local Sandbox 上,而是走 Docker 自己的检查被拒(image must be an exact local image digest),local Sandbox 上没有新挂载。修复前的二进制在同样情况下真的挂到了 local Sandbox 上。

…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,658 -> 109,696 (delta +38)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/groups.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,696 -> 109,818 (delta +122)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,818 -> 109,835 (delta +17)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,835 -> 109,868 (delta +33)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,868 -> 109,882 (delta +14)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: sdk/js/native/src/local.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,906 -> 109,958 (delta +52)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: sdk/js/native/src/discovery.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,958 -> 109,995 (delta +37)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: sdk/js/native/src/exec.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,995 -> 110,005 (delta +10)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: sdk/js/native/src/sandbox.rs (production).
No tests changed or deleted except additions.

Production LOC: 110,005 -> 110,015 (delta +10)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: sdk/js/native/src/workspace.rs (production).
No tests changed or deleted except additions.

Production LOC: 110,015 -> 110,018 (delta +3)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: sdk/js/native/src/lib.rs (production).
No tests changed or deleted except additions.

Production LOC: 110,018 -> 110,019 (delta +1)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: sdk/js/native/tests/unit/runtime.rs (not production (test)).
No tests changed or deleted except additions.

Production LOC: 110,019 -> 110,019 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: sdk/js/native/tests/unit/reclaim.rs (not production (test)).
No tests changed or deleted except additions.

Production LOC: 110,019 -> 110,019 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
@jiangkoumo

Copy link
Copy Markdown
Contributor Author

@yifanxuaaa 谢谢细致的评审,7 条都成立,已全部按建议修复,追加在 8c22d1754(没有改写已推送的历史;上游 main 目前和本分支没有重叠文件,git merge-tree 无冲突,所以没有 rebase)。Docker 路径、daemon、layerfs-api/layerfs-bridge 仍是零改动,没有新增依赖,Cargo.lock 不变。

概要:

  1. 命令进程不再活过 Sandbox:每个命令自成进程组并登记,stop / reclaim 时整组 KILL 并确认清空后才删目录(为什么是进程组而不是 cgroup 见行内回复)。
  2. LAYERFS_LOCAL_DIR 的每一级祖先(含符号链接解析后的真实路径)都要求 o+x,create 时就拒绝并点名是哪一级。
  3. backend::local、它的 re-export、测试和 SDK 的 local 模块都改为 cfg(target_os = "linux");非 Linux 选 local 时返回 Unsupported。
  4. local 记录只在 directory == <LAYERFS_LOCAL_DIR>/<prefix><project>-<id> 时才会被使用或删除,否则 InvalidInput、什么都不删。
  5. 进程身份加入 boot_id,判活比较 pid + start + boot。
  6. 启动成功后写记录失败,统一走 abandon(..., Some(sandbox));Sandbox 的 Drop 回收已退出的 daemon,stop() 要等本进程启动的 daemon 被 reap 后才返回。
  7. 放置改为先比 runtime 再比 image。

验证(测试机 x86_64,Debian 13,HEAD = 本次提交,工作树干净):

项 结果
cargo +1.85.1 test --release --offline --locked -p layerfs-sandbox -p layerfs-node 50 passed / 0 failed(tests/local_runtime.rs 6 → 10)
rustfmt +1.85.1 --check(本 PR 全部 .rs) 干净
cargo clippy --all-targets -- -D warnings layerfs-sandbox 0;layerfs-node 只剩 tests/unit/admission.rs:39(main 上已由 #390 修复)
cargo clippy -p layerfs-sandbox --all-targets --target aarch64-apple-darwin -- -D warnings 0(local 模块和测试在 macOS 上被 gate 掉,可编译)
SDK 端到端(真实 daemon + FUSE,LAYERFS_SANDBOX=local,root) 72/72(原 61 项 + 本轮针对 7 条的 11 项)
同一套端到端用修复前的 layerfs.node 跑(反向对照) 65/72,失败的正好是新加的 V1 V1b V3 V3b V2c V4b L5
Docker 回归(未设 / LAYERFS_SANDBOX=docker / origin/main 基线) 三份 39 行签名逐行一致
JS 单测、无 Docker、非 root 拒绝、上游 *_real.mjs verify 与上一轮一致(direct_refusal_real 在这台机器上 main 与本分支同样失败,环境问题)
aarch64 交叉编译 + qemu-user 预跑(arm64 node:22-trixie-slim) 交叉编译通过;端到端 72/72,连续 5 次全过(第一次预跑时 V4b 抓到一个僵尸 daemon,已修,见第 6 条回复)
跑完后残留 无 daemon、无 FUSE 挂载、无 uid 501 进程、LAYERFS_LOCAL_DIR 为空

另外顺带说明一点(不是这次的评审项):main 上 b40da46 之后,core/tools/check_product_boundary.py 只剩 4 条,合入本 PR 后会多 1 条:它会把本 PR 的 sdk/js/native/src/runtime.rs 里 #[cfg(test)] #[path = "../tests/unit/runtime.rs"] 算作 1 条,和 b40da46 里暂缓处理的 sandbox.rs / pool.rs 是同一种写法。我先保持一致;如果你们定了新的放法,我跟着改

@jiangkoumo

Copy link
Copy Markdown
Contributor Author

补充一下:#427(Draft,macOS 长期适配分支)包含了本 PR 的内容。本 PR 仍单独评审,合并后 #427 里对应的 diff 会自动消失;本 PR 之后有改动,我会同样 merge 进 #427。

…hemeral-AI-Lab#402 (local port)

Sync by merge (no rebase, no force-push). Ephemeral-AI-Lab#428 moved sdk/js to
api/agent/sdk and sdk/js/native to api/binding, made Sandboxes explicit
(sandbox.create / sandbox.close / workspace.mount(sandbox, branch), no
placement) and made LayerFS plugin-neutral (LAYERFS_* names only).

Resolved:
- api/binding/src/sandbox.rs: upstream's create/close/get_log kept; they
  now go through the Sandbox's recorded runtime (Docker or local) instead
  of naming the Docker backend: create reports runtime.image() (`local`
  for a local Sandbox), close reopens the recorded runtime and stops and
  deletes it, get_log reads the container's log or a local daemon.log.
- api/binding/src/workspace.rs (upstream's new file): the mount result's
  image is runtime.image(); observed() reopens the recorded runtime, so
  list_sandboxes, reclaim_gone and runtime.sandbox.reclaim see local
  Sandboxes too.
- api/binding/src/runtime.rs: Runtime::stop and Runtime::log_tail added;
  Placement removed with upstream's placement (its unit test with it).
- api/agent/sdk/src/client.js: deployment() keeps the local branch
  (LAYERFS_SANDBOX=local, LAYERFS_LOCAL_DIR); the DSH_LAYERFS_ twins are
  gone as upstream removed them.
- api/agent/sdk/src/index.js: the client reads LAYERFS_SANDBOX and
  LAYERFS_LOCAL_DIR from the environment (there is no options argument).
- sdk/js/README.md -> api/how-to-use/agent.md: the two settings rows and
  the "Local Sandboxes (no Docker)" section, rewritten for explicit
  Sandboxes.
- sdk/js/native/src/workspace.rs: deleted as upstream did; its placement
  change has no counterpart (there is no placement).
- local.rs, runtime.rs, tests/unit/runtime.rs, local_deployment.mjs moved
  to api/binding and api/agent/sdk/test; the test uses the stand-in
  binding since clients are made only by LayerFS.open/init.
jiangkoumo added a commit to jiangkoumo/layerfs that referenced this pull request Oct 24, 2026
…497b0c v1.1.0) into feat/macos-native-on-main

Sync by merge (no rebase, no force-push): the branch history others pulled
stays as it was. ada323c is Ephemeral-AI-Lab#402's head 8c22d17 with upstream main
4497b0c merged in (Ephemeral-AI-Lab#428: sdk/js -> api/agent/sdk, sdk/js/native ->
api/binding, dsh/ moved to the dsh-layerfs repository, explicit Sandboxes,
plugin-neutral LAYERFS_* settings), so this one merge brings both.

Resolved:
- dsh/ (README and 8 host files we changed, plus tests/native.mjs and
  tests/native_real.mjs we added): removed with upstream; the Host's native
  route goes to dsh-layerfs as its own change (core-only plan).
- api/agent/sdk/src/client.js: deployment() keeps `native` beside `local`;
  the DSH_LAYERFS_NFS_* fallback (nfsFallbacks) is dropped, as Ephemeral-AI-Lab#428 drops
  every DSH_LAYERFS_* name (AGENTS.md "Plugin-neutral LayerFS", owner
  2026-10-24); an integration maps its own names.
- api/binding/src/runtime.rs: native() kept, Placement removed (there is
  no placement since Ephemeral-AI-Lab#428); Runtime::stop and log_tail cover macOS too.
- api/binding/src/discovery.rs: with no placement, a local or native record
  is used only by a deployment of its own runtime (InvalidInput otherwise),
  which keeps local and native Sandboxes apart as placement did.
- api/binding/tests/unit/runtime.rs: placement tests dropped; native
  selection kept; the record test checks the runtime match.
- sdk/js/README.md -> api/how-to-use/agent.md: the `native` settings text
  and the "Native Sandboxes (macOS)" section, for explicit Sandboxes and
  without the DSH_ fallback.
- core/Cargo.lock: layerfs-nfs follows the workspace version 1.1.0.
@jiangkoumo

Copy link
Copy Markdown
Contributor Author

同步 main 4497b0c(v1.1.0,#428)

merge 提交 ada323c(tree c14bac2c),父提交是 8c22d17 和 4497b0c,只用 merge。#402 原来和新 main 有 9 个冲突文件,处理方式:
• sdk/js/native/* → api/binding/,sdk/js/ → api/agent/sdk/:local.rs、runtime.rs、tests/unit/runtime.rs、local_deployment.mjs 挪到新目录;client.js 保留 LAYERFS_SANDBOX=local,DSH_LAYERFS_ 兜底按上游删掉;index.js 现在从环境变量读 LAYERFS_SANDBOX / LAYERFS_LOCAL_DIR(构造函数已经没有 options 参数);README 里的 "Local Sandboxes" 一节移到 api/how-to-use/agent.md,按显式 Sandbox 重写。
• placement 已经取消,所以 #402 在旧 workspace.rs 里写的"按运行时和镜像挑 Sandbox"没有地方可以移,Placement 一起删掉了。
• 上游新增的 sandbox.close、backend.daemon.sandbox.get_log、project.list_sandboxes、runtime.sandbox.reclaim 原来写死 Docker,现在都按记录的运行时走(新增 Runtime::stop / log_tail,转发到 local 后端已有的 stop / log_tail),local Sandbox 也能 close、读 daemon.log、被回收。

回归(Linux):fmt 0;Linux/Darwin clippy -D warnings 0;sandbox 38、binding 11、store-service 3 全过;SDK/CLI/runtime/backend JS 测试和 main 一致;LAYERFS_SANDBOX=local smoke(新 API,包括 sandbox.create/close、kill 后回收)48/48。

另外:local_runtime 用 4 个线程跑时偶尔有一个用例失败,同步前后都是 15 次里失败 1 次,单独跑能过。现在它会进 CI 的 fast lane,建议后续修一下。

@jiangkoumo jiangkoumo mentioned this pull request Oct 24, 2026
47 tasks
jiangkoumo and others added 2 commits October 25, 2026 01:03
Sandbox::stop() ended the daemon first and left the Sandbox's mounts in
place until Delete detached them. Detach them first, while the daemon can
still answer, then stop the daemon as before. A failed detach is reported
and does not block the crash teardown; Delete keeps its own checked detach
before removing anything through a mount.

On Linux FUSE this is ordering only: Stop now returns with the mounts
already gone (measured 21.6 ms with a live holder and a dirty writer; before
the change the mounts stayed until Delete), and reclaim is unchanged
(0.09-0.10 s). MNT_DETACH does not flush dirty pages, so it adds no data
guarantee; Stop stays the crash teardown it was.

Same hunk as the local-backend part of the macOS native branch's NFS
unmount-hang fix, so merging this branch there stays clean.
@jiangkoumo

Copy link
Copy Markdown
Contributor Author

新提交 766e67e:fix(sandbox): detach local mounts before stopping the daemon

问题
本地后端的 Sandbox::stop() 先结束 daemon,挂载留到 Delete 才摘掉(backend/local/sandbox.rs,本 PR 引入的代码)。

改法(只改 core/crates/layerfs-sandbox/src/backend/local/sandbox.rs,+8/−1)
Stop 先对 Sandbox 目录 detach 挂载(daemon 还能应答时),再按原来的方式 TERM/KILL daemon。detach 失败只打印,不阻止崩溃式回收;Delete 仍保留原来的带检查 detach,删除前不会穿过挂载删东西。

在 Linux FUSE 上这只是顺序变化:
• 直接调用 Sandbox::stop(1),fixture 是活着的 FUSE Sandbox,里面有 cwd 在挂载内的 holder 和一个未 fsync 的脏写进程:
修复前:stop 21.2 ms 返回,挂载还在,要等 Delete 才摘。
修复后:stop 21.6 ms 返回,挂载已经摘掉,daemon 已退出;holder 访问得到 ENOTCONN;随后 reclaim 0.09 s,无挂载残留。
• MNT_DETACH 不刷脏页,所以它不额外提供数据保证,Stop 仍是崩溃式回收。

这一段和 #427 里 NFS 卸载挂死修复中的本地后端部分完全相同(#427 那边还有 NFS helper 有上限回收和 binding 卸载期间放锁,这些只和 NFS 路线有关,没放进本 PR),所以以后 #402 merge 进 #427 不会冲突。

验证(Linux x86_64,头 766e67e)
cargo +1.85.1 fmt --manifest-path core/Cargo.toml --all -- --check → 通过
cargo +1.85.1 clippy --manifest-path core/Cargo.toml --offline --locked -p layerfs-daemon -p layerfs-sandbox -p layerfs-store-service -p layerfs-overlay -p layerfs-persistence -p layerfs-node --all-targets -- -D warnings → 通过
同上加 --target aarch64-unknown-linux-gnu(不含 --all-targets) → 通过
cargo test -p layerfs-sandbox --all-targets → 38/38(含 local_runtime)
cargo test -p layerfs-node --all-targets → 11/11
core/tools/check_product_boundary.py → 5 条已有的 binding test-only 标注,非新增

yifanxuaaa pushed a commit that referenced this pull request Oct 26, 2026
…er_reclamation

The Store binds no busy handler, so reclaim_space can be refused Busy without effect while the concurrent reader holds its snapshot. The test unwrapped that outcome and failed intermittently (4 of 80 local runs, and PR #402 CI). Treat Busy as the documented refused job and run a later one, bounded by JOBS.

No tests deleted; no assertion changed.

Production LOC: 102165 -> 102165 (delta 0)
Test-only. core 38241, engine 61291, api 2633, unchanged; reference absent
(retired R9c; last counted 65,417 at tree 498dd19). Method:
tools/production_loc.py --root on git archive of the first parent and the
committed tree (bb3f715 -> 7cf6e7b). Added by the maintainer
after the rebase onto main 3828268 (#465 moved the crate to engine/).
yifanxuaaa added a commit that referenced this pull request Oct 26, 2026
…amation

The Store binds no busy handler, so reclaim_space can be refused Busy,
without effect, when it meets the concurrent reader; the test unwrapped
that and failed intermittently (seen in #402 CI). The test now treats Busy
as the documented refused job and sends a later one, at most JOBS times;
other errors still fail and no assertion changed. Test-only; the owner
accepted this caller-side retry in a test as consistent with the
reclaim_space contract (a refused job is never replayed automatically,
a caller may issue another).

Rebased by the maintainer onto main after #465 moved the crate to
engine/, and the missing LOC line added. Checked on macOS (cargo 1.85.1):
immutable_reads_under_reclamation passes (1147 jobs during 89 read
rounds); all 36 layerfs-persistence test binaries pass (118 tests);
fmt and Clippy -D warnings clean.

Production LOC: 102165 -> 102165 (delta 0)
  First-parent comparison (main 3828268 -> this merge). core 38241,
  engine 61291, api 2633, unchanged; reference absent (retired R9c; last
  counted 65,417 at tree 498dd19). Counted with
  tools/production_loc.py; test-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…meral-AI-Lab#402)

Upstream now keeps the Node binding's tests outside the crate source: they
load the ordinary built addon and call its exported methods, never including
private binding source (engine/AGENTS.md, engine/crates/layerfs-node/tests).
The Ephemeral-AI-Lab#402 unit tests that were included by runtime.rs and sandbox.rs
(tests/unit/{runtime,reclaim}.rs before D8) are rewritten as
engine/crates/layerfs-node/tests/native_local.rs, which writes a Sandbox
record as an earlier call would and calls the public
`runtime.sandbox.reclaim` with a local deployment:

- a local record naming any other directory (a data directory, another
  LAYERFS_LOCAL_DIR, another Sandbox or name, a relative or `..` path, `/`)
  is refused InvalidInput, "neither used nor deleted", and nothing is
  removed;
- a local record is used only with an absolute, plain deploy.localDir;
- a stopped local Sandbox where this deployment puts it is deleted and its
  number freed; a record from before boot ids is read without one; a
  damaged boot or port is refused and removes nothing;
- a Docker record is not read as a local Sandbox.

The Docker reclaim cases of the old reclaim.rs are covered by upstream's
native_discovery.rs. Selection of a misspelled deploy.sandbox is refused only
by sandbox.create after the project's Store is installed, so it is not
reachable without a Store and is left to the SDK test (local_deployment.mjs).

Production LOC: 103762 -> 103762 (delta 0)
(tools/production_loc.py --root ., core+engine+api; test-only commit)
Production LOC: 103762 -> 103762 (delta 0)
(tools/production_loc.py --root ., core+engine+api; formatting of a test file only)
…-Lab#402)

Production LOC: 103762 -> 103762 (delta 0)
(tools/production_loc.py --root ., core+engine+api; documentation only)
jiangkoumo added a commit to jiangkoumo/layerfs that referenced this pull request Oct 27, 2026
…e built addon) into PR Ephemeral-AI-Lab#427 (local port)

Brings Ephemeral-AI-Lab#402's engine/crates/layerfs-node/tests/native_local.rs. One
conflict: api/binding/tests/unit/runtime.rs, deleted on Ephemeral-AI-Lab#402 and kept by
Ephemeral-AI-Lab#427's D8 merge, is deleted here too. api/binding/ no longer exists after
D8, the file was included by nothing, and its cases move to native_local.rs
in the next commit.

Production LOC: first-parent comparison 1931032 -> this merge, see
tools/production_loc.py --root .; tests only, delta 0.
jiangkoumo added a commit to jiangkoumo/layerfs that referenced this pull request Oct 27, 2026
…nto PR Ephemeral-AI-Lab#427 (local port)

Production LOC: 108596 -> 108596 (delta 0), first-parent comparison
(tools/production_loc.py --root ., core+engine+api; documentation only)
@yifanxuaaa

Copy link
Copy Markdown
Contributor

Hi @jiangkoumo, thank you for this PR and for the careful work behind it, including the thorough follow-up on the first review.

We've decided not to proceed with this PR. We need to redesign the roadmap for 2.0.0 so that Docker, native Windows, macOS and Linux, and other sandbox providers are designed together, rather than adding a local runtime on its own now.

@yifanxuaaa yifanxuaaa closed this Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants