Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,15 @@ jobs:
run: bun test

- name: Check remaining shell syntax
run: bash -n completions/rootcell.bash src/bin/rootcell-vmnet-helper.sh
run: bash -n completions/rootcell.bash images/scripts/build-release.sh images/scripts/validate-dist.sh

- name: Install ShellCheck
run: |
sudo apt-get update
sudo apt-get install -y shellcheck

- name: Run ShellCheck
run: shellcheck --severity=warning completions/rootcell.bash src/bin/rootcell-vmnet-helper.sh
run: shellcheck --severity=warning completions/rootcell.bash images/scripts/build-release.sh images/scripts/validate-dist.sh

- name: Compile Python modules
run: python3 -m compileall proxy
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/macos-host-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,5 +56,5 @@ jobs:
- name: Build Darwin-only host packages
run: |
nix build --print-build-logs \
.#packages.aarch64-darwin.lima \
.#packages.aarch64-darwin.socket_vmnet
.#packages.aarch64-darwin.vfkit \
.#packages.aarch64-darwin.zstd
4 changes: 2 additions & 2 deletions BRAND.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,9 @@ Prefer:
Avoid leading with:

- TLS MITM
- socket_vmnet
- VM runtime internals
- NixOS module internals
- Lima named-network details
- provider-specific network details
- provider-specific setup

Those details matter, but they belong after the reader understands what the
Expand Down
69 changes: 9 additions & 60 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ rootcell gives you a local workspace where an agent can exercise root inside the
VM without receiving broad access to your Mac:

- A fresh NixOS VM for the agent's shell and tools.
- No default host-home mount from Lima.
- No host-home mount in the agent VM.
- A separate firewall VM with the only public internet route.
- DNS, HTTPS, and SSH allowlists you can review and hot-reload.
- A per-VM SSH key for Git pushes.
Expand Down Expand Up @@ -62,7 +62,7 @@ The two VMs have different jobs:
Rootcell supports named instances. Plain `./rootcell` uses the `default`
instance and creates VMs named `agent` and `firewall`. `./rootcell --instance
dev` creates `agent-dev` and `firewall-dev`, with separate CA material,
allowlists, Keychain mappings, and a separate vmnet network.
allowlists, Keychain mappings, and a separate private VM link.

HTTPS egress is transparent from inside the agent VM. A normal command like
`curl https://github.com` either works because the host is allowlisted, or fails
Expand Down Expand Up @@ -112,24 +112,14 @@ First run downloads compatible rootcell VM images from the configured release
manifest, creates instance-local vfkit disks, and provisions the VMs. Later runs
normally take seconds.

### VM Provider Selection
### Host Runtime

vfkit is the default VM provider:
vfkit is the supported VM runtime:

```bash
./rootcell
```

The Lima provider remains as a rollback path while vfkit support settles:

```bash
ROOTCELL_VM_PROVIDER=lima ./rootcell
```

The legacy Lima path still requires the one-time `socket_vmnet` and
`rootcell-vmnet` sudo setup printed by `./rootcell` when that provider is
selected.

Image resolution is controlled by:

```bash
Expand Down Expand Up @@ -282,8 +272,6 @@ common.nix shared NixOS config for both VMs
agent-vm.nix agent VM network and trust-store config
firewall-vm.nix firewall VM services and nftables rules
home.nix pi, Git, SSH, and developer tools for the agent VM
nixos.yaml Lima config for the agent VM
firewall.yaml Lima config for the firewall VM
network.nix default inter-VM network settings
.env.defaults seed values for per-instance `.env`
secrets.env.defaults seed Keychain secret mappings for per-instance `secrets.env`
Expand All @@ -293,7 +281,6 @@ proxy/ allowlists and mitmproxy/dnsmasq firewall code
agent_spy_tui.py Textual browser for `./rootcell spy --tui`
pi/agent/ global pi instructions, skills, and extensions
completions/ bash and zsh completion for `rootcell`
pkgs/socket_vmnet.nix local package for Lima's vmnet helper
```

## VM Lifecycle
Expand All @@ -303,36 +290,6 @@ control key and generated SSH config live under `.rootcell/instances/<name>/ssh/
The agent VM is reached through SSH ProxyJump via the firewall VM; no VSOCK
device is attached on the vfkit path.

### Lima Rollback

The commands below apply to the legacy Lima provider when run with
`ROOTCELL_VM_PROVIDER=lima`.

Stop the VMs but keep their disks and Nix store caches:

```bash
limactl stop agent
limactl stop firewall
./rootcell

limactl stop agent-dev firewall-dev
./rootcell --instance dev
```

Delete the VMs for a clean slate:

```bash
limactl delete agent firewall --force
./rootcell

limactl delete agent-dev firewall-dev --force
./rootcell --instance dev
```

If you edit `nixos.yaml` or `firewall.yaml`, Lima will not apply those changes
to existing VMs automatically. Either run `limactl edit <name>` or delete and
recreate the VM.

## Configuration

### Environment
Expand All @@ -347,10 +304,10 @@ ROOTCELL_SUBNET_POOL_END=192.168.254.0
```

The first run also writes `.rootcell/instances/<name>/state.json` with the
instance's vmnet UUID and allocated `/24`. By default, rootcell chooses the first
free subnet from `192.168.100.0/24` through `192.168.254.0/24`, uses `.2` for
the firewall, and uses `.3` for the agent. Existing state is not recalculated if
you later edit the pool values.
instance's allocated `/24`. By default, rootcell chooses the first free subnet
from `192.168.100.0/24` through `192.168.254.0/24`, uses `.2` for the firewall,
and uses `.3` for the agent. Existing state is not recalculated if you later
edit the pool values.

To pin a new instance to a specific subnet before first run, set both IPs in
that instance's `.env`:
Expand Down Expand Up @@ -421,21 +378,13 @@ Named instances are isolated from each other:
```

Each instance gets its own VMs, state directory, CA, allowlists, Keychain mapping
file, control SSH key, private-link sockets, and `/24`.
file, control SSH key, private-link state, and `/24`.

The `default` instance migrates from legacy repo-local files on first run: if
`.env`, `secrets.env`, `proxy/allowed-*.txt`, or `pki/` already exist, rootcell
copies them into `.rootcell/instances/default/`. Named instances seed from the
checked-in defaults.

Existing VMs created by the legacy Lima provider are not migrated in place. Use
the Lima rollback provider to delete them if needed:

```bash
limactl delete agent firewall --force
./rootcell
```

## Troubleshooting

See what the firewall is denying:
Expand Down
13 changes: 6 additions & 7 deletions agent-vm.nix
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,16 @@ in
# anyway. All meaningful filtering happens in the firewall VM.
networking.firewall.enable = false;

# Networking: only the per-instance socket_vmnet interface is configured.
# The repo's patched Lima launcher skips the default usernet NIC for this
# VM, so the private socket_vmnet link is enp0s1 and there is no direct
# host usernet path a root-capable agent could reconfigure into egress.
# Networking: only the per-instance private vfkit link is configured, so
# there is no direct host control path a root-capable agent could reconfigure
# into egress.
networking.useDHCP = false;
networking.useNetworkd = true;
systemd.network.enable = true;
systemd.network.wait-online.enable = false;
# The private link from nixos.yaml/vfkit is enp0s1. Cloud-init performs a
# MAC-matched bootstrap before provisioning, then this NixOS config owns the
# steady-state interface.
# The private vfkit link is enp0s1. Cloud-init performs a MAC-matched
# bootstrap before provisioning, then this NixOS config owns the steady-state
# interface.
systemd.network.networks."10-enp0s1" = {
matchConfig.Name = "enp0s1";
networkConfig = {
Expand Down
19 changes: 3 additions & 16 deletions common.nix
Original file line number Diff line number Diff line change
@@ -1,29 +1,17 @@
{ config, modulesPath, pkgs, lib, username, nixos-lima, ... }:
{ modulesPath, pkgs, lib, username, ... }:

# Shared NixOS bits used by both the agent VM and the firewall VM. Things
# that are genuinely VM-specific (hostname, networking, firewall policy,
# services) live in agent-vm.nix and firewall-vm.nix respectively.

{
imports = [
# Required for the guest to boot under qemu/vz.
# Required for the guest to boot under virtio VM runtimes.
(modulesPath + "/profiles/qemu-guest.nix")
# Provides `services.lima.*` options. Sets up lima-init at boot and
# runs the lima-guestagent daemon as a systemd service.
nixos-lima.nixosModules.lima
];

options.rootcell.limaGuestSupport = lib.mkOption {
type = lib.types.bool;
default = true;
description = "Enable Lima guest initialization and guest agent support.";
};

config = {
# Activate the nixos-lima module only for the Lima rollback provider.
services.lima.enable = lib.mkDefault config.rootcell.limaGuestSupport;

# Rootcell's default vfkit path manages guests over SSH through the firewall.
# Rootcell manages guests over SSH through the firewall.
services.openssh.enable = true;

users.users.${username} = {
Expand Down Expand Up @@ -70,7 +58,6 @@
environment.enableAllTerminfo = true;
boot.kernelPackages = pkgs.linuxPackages_latest;

# Pin to the NixOS release nixos-lima is built against. Don't bump casually.
system.stateVersion = "25.11";
};
}
42 changes: 20 additions & 22 deletions firewall-vm.nix
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ in
# Firewall VM: a tiny appliance VM that brokers all egress for the agent VM.
#
# Two NICs (kernel names from systemd predictable naming):
# enp0s1 vzNAT — internet egress (default route)
# enp0s2 socket_vmnet — private per-instance link to the agent VM
# (IPs come from network.nix/network-local.nix)
# enp0s1 vfkit NAT — internet egress (default route)
# enp0s2 private vfkit — per-instance link to the agent VM
# (IPs come from network.nix/network-local.nix)
#
# Hybrid filtering — HTTPS is intercepted, SSH is explicit, HTTP is denied:
#
Expand Down Expand Up @@ -82,10 +82,9 @@ in
networkConfig.DHCP = "ipv4";
};

# enp0s2 = socket_vmnet, our private link to the agent VM. (The kernel
# names this NIC enp0s2 via systemd predictable naming because it's
# the second virtio-net device — Lima's `interface:` field can't
# actually rename the kernel device, so we just use enp0s2 directly.)
# enp0s2 = private vfkit link to the agent VM. The kernel names this NIC
# enp0s2 via systemd predictable naming because it is the second virtio-net
# device.
# Static address; DHCP would conflict with the agent's static .2.
systemd.network.networks."20-enp0s2" = {
matchConfig.Name = "enp0s2";
Expand All @@ -107,8 +106,8 @@ in

# ── Firewall ──────────────────────────────────────────────────────────
# NixOS firewall manages the filter table. We add a separate nat table
# below for the REDIRECT rules. Inbound on enp0s2 (the socket_vmnet link
# to the agent VM) is allowed only on the explicit-mitmproxy port
# below for the REDIRECT rules. Inbound on enp0s2 (the private link to the
# agent VM) is allowed only on the explicit-mitmproxy port
# (8080), the transparent-mitmproxy port (8081, which is the
# post-REDIRECT destination), and dnsmasq (53).
networking.nftables.enable = true;
Expand Down Expand Up @@ -148,9 +147,9 @@ in
};

# ── Mutable allowlist directory ───────────────────────────────────────
# `./rootcell allow` writes here via `limactl cp`, which connects as the
# unprivileged Lima guest user — so the dir is owned by ${username},
# not root. The dnsmasq-allowlist.conf seed is empty: dnsmasq's
# `./rootcell allow` writes here over SSH as ${username}, so the dir is owned
# by ${username}, not root. The dnsmasq-allowlist.conf seed is empty:
# dnsmasq's
# pre-start check refuses to launch without the conf-file existing,
# but with `no-resolv` and no `server=` directives, an empty
# conf-file means every query returns REFUSED — fail-closed by
Expand All @@ -160,13 +159,12 @@ in
# can overwrite root-owned files in a user-owned directory.
#
# The CA pem (key + cert) for TLS MITM is staged here too, but
# written by `./rootcell` via `limactl cp /tmp + sudo install -m 0600
# -o root -g root` — never touchable by the lima user (who has
# passwordless sudo, but the explicit ownership chmod makes the
# blast radius "must already be root" rather than "any read of
# /etc/agent-vm leaks the key"). Loaded into the mitmproxy services
# via systemd LoadCredential, which surfaces it as a tmpfs file
# readable only by the service uid.
# written by `./rootcell` via SCP to /tmp plus `sudo install -m 0600 -o root
# -g root` — never touchable by the normal guest user. That user has
# passwordless sudo, but the explicit ownership chmod makes the blast radius
# "must already be root" rather than "any read of /etc/agent-vm leaks the
# key". Loaded into the mitmproxy services via systemd LoadCredential, which
# surfaces it as a tmpfs file readable only by the service uid.
systemd.tmpfiles.rules = [
"d /etc/agent-vm 0755 ${username} users -"
"f /etc/agent-vm/dnsmasq-allowlist.conf 0644 root root -"
Expand Down Expand Up @@ -215,9 +213,9 @@ in
#
# ConditionPathExists guards the bootstrap window: on the very first
# nixos-rebuild the CA is not yet copied in (./rootcell does that AFTER
# rebuild — we can't `limactl cp` to /etc/agent-vm/ before tmpfiles
# creates the dir), so the services skip cleanly. ./rootcell then pushes
# the CA and `systemctl restart`s, which re-evaluates the condition.
# rebuild — we can't copy into /etc/agent-vm/ before tmpfiles creates the
# dir), so the services skip cleanly. ./rootcell then pushes the CA and
# `systemctl restart`s, which re-evaluates the condition.
systemd.services.mitmproxy-explicit = {
description = "mitmproxy (explicit CONNECT — for SSH ProxyCommand)";
after = [ "network-online.target" ];
Expand Down
57 changes: 0 additions & 57 deletions firewall.yaml

This file was deleted.

Loading