Redesign sidecar using byte-exact guest filenames without a sidecar index

[henrybear327]

The design is being refined as the testing and simplification of the code are carried out continuously

---

A Linux guest names files by exact bytes. The default APFS volume matches names
case- and normalization-blind, so Foo and foo cannot coexist and a wrong-case
lookup succeeds where Linux owes ENOENT.

The previous answer, src/syscall/sidecar.c (2217 lines), stored such a name
under an opaque random token (.ef_ plus 64 bits from arc4random, retried on
EEXIST) and recorded what the token meant in a per-directory
.elfuse_case_index file. The mapping lived outside the name, and everything
below follows from that one decision.

A name was two objects, so no mutation was atomic

Creating, removing, or renaming a name had to touch the token and the index
file, which no host syscall can do together. renameat therefore ran a
hand-rolled two-phase commit: clone both directory indices as rollback
snapshots, write the index or indices to disk, call renameat, and on failure
write the saved snapshots back. The comment conceded the gap: "A failed
rollback write is the best-effort case." A crash between the index write and
the host rename left a mapping pointing at a token that had moved or did not
exist, and nothing repaired it on the next mount. Cross-directory rename took
two index locks in a fixed order, with a swapped flag deciding which went
first, so the ordinary rename path carried deadlock-avoidance logic.

It forced serialization that the filesystem does not need

Two lock layers, because POSIX advisory locks are per-process and every vCPU
thread would share one: a process-wide pthread_mutex_t across all vCPU
threads, plus an fcntl write lock on a .elfuse_case_index.lock sentinel to
serialize against other elfuse processes on the same sysroot. Two guests
creating unrelated files in one directory contended, and the process mutex
serialized name creation across every directory at once.

The index was advisory, so anything else desynchronized it

fcntl locks bind only the processes that take them. Any writer that is not
elfuse (tar, rsync, git, Finder) could add or delete entries without
touching the index, leaving rows describing tokens that were gone and files
with no row at all. In the other direction the tokens are opaque, so a sysroot
was unreadable and unmaintainable outside elfuse, and copying one preserved
the mapping only if the index files came along intact.

Reads paid for it, and the cache was itself a correctness surface

The walker visited every parent directory of every translated path, probing
each for an index. That cost dominated startup, measured at "openat 61% of
getent's 7.5 ms warm path", so it needed a 64-slot open-addressed cache
keyed on (dev, ino, mtime, ctime), last writer wins on collision. Ordering
then became load-bearing: the publish path had to invalidate the cache
before the rename, or a concurrent walker could cache "no index here" and
miss one another process had just published.

Leftovers

Three reserved names per directory (index, .tmp, .lock) had to be filtered
out of every listing and refused as guest paths. Each mutation parsed the whole
index into a heap row array and deep-cloned it for rollback, so a create in a
large directory cost work proportional to the directory. And the sidecar
resolved paths independently of path_translate_at, so two walks disagreed
about when a guest path falls through to the host.

Approach

Decide a name's on-disk spelling from the name alone. Nothing is persisted, so
nothing can desynchronize, and no lock is needed because there is no second
object to keep consistent.

• Codec (src/syscall/casefold.c): a name containing an uppercase ASCII
  letter, a byte above 0x7F, or an escape-shaped prefix is stored as
  .ef=<payload>. What is left literal is lowercase ASCII, a fixed point of
  every transformation the volume applies, so two literal names cannot collide
  however the folding table changes. The payload has two tiers because the
  per-name limit is 255 UTF-16 code units rather than bytes: hex to 125 bytes,
  then twelve bits per symbol from 4096 CJK ideographs at U+4E00, a block that
  neither folds nor normalizes. Guest names keep all 255 bytes Linux allows.
• Walk (src/syscall/casefold-walk.c): resolve a path one component at a
  time, asking the volume for each name as stored (getattrlistat) and
  comparing bytes, the only way to separate "exists as spelled" from "exists
  under a spelling that folded onto it". Absence is three answers, not one:
  unclaimed falls through to the host, a slot held by a differently-spelled
  sibling is ENOENT, and a non-directory component is ENOTDIR
  (path_resolution(7)).
• One resolver: proc_resolve_sysroot_path_flags and path_translate_at
  fold into that single walk, which also backs the openat2
RESOLVE_NO_SYMLINKS and RESOLVE_NO_XDEV walkers. Directory entries and
  the cwd decode on the way back out.
• Sidecar deleted: index, parsers, both locks, the snapshots, both caches,
  and its five private syscall handlers, which now get the ordinary /proc,
  FUSE, and /dev/shm handling. Colliding creates never contend, because the
  spelling is a function of the name. Two elfuse processes sharing a sysroot
  need no coordination, because no shared mutable state remains.

Prior art

The same question ("the host filesystem cannot hold this name") has three
standard answers, and this branch moves elfuse from the worst to the best of
them.

• Ask the filesystem. Cygwin took this route: managed mounts, its
  encoding scheme, were removed in 1.7 in favor of real NTFS case sensitivity
  via the obcaseinsensitive kernel flag, with per-mount posix=[0|1].
  elfuse's equivalent is --create-sysroot, which builds a case-sensitive APFS
  sparsebundle, and Docker Desktop and Lima sidestep it entirely with a Linux
  VM over ext4. It is the cleanest answer when available, which is why the
  check suite re-runs the whole name suite on that volume as its oracle. It is
  not always available: the sparsebundle is opt-in and a user may point
  --sysroot at any directory.
• Derive the stored name from the guest name. Where Cygwin still must
  encode, for characters Windows disallows, it transposes them into the Unicode
  private use area by adding 0xf000: a pure per-character function with no
  side table. gocryptfs, EncFS, and eCryptfs are the same family, deriving the
  on-disk name from the plaintext name and a key. This branch is that answer.
• Keep a mapping. The sidecar was here, and so is CryFS, which stores names
  inside encrypted blocks. Even gocryptfs falls back to it, but only at the
  boundary the derivation cannot cross: when an encrypted name would exceed
  255 bytes it writes a gocryptfs.longname.<hash>.name support file. That is
  precisely what elfuse's second tier avoids. Packing twelve bits per CJK
  symbol keeps the derived property across the full 255-byte guest range, so
  there is no length at which a side file reappears.

Compatibility

A sysroot written by the old encoding must be recreated. Its .ef_<token>
entries and .elfuse_case_index files decode to nothing and surface under
their literal host names. docs/usage.md says so and gives the recipe.

Tests

Two host unit lanes (codec, resolver) link exactly the code under test.
tests/casefold-vectors.h freezes the on-disk spellings in both directions,
making a format change a deliberate migration. Guest lanes cover edge shapes
(one-character sysroot, no sysroot, chdir), i18n pairs the volume folds, name
and path length budgets, host-staged spellings, a corpus read back by guest
name, and concurrency (a fork race run ten times in check, plus a manual
soak).

Two lanes are oracles rather than expectations:

• check-name-caseexact re-runs the name suite on a case-sensitive APFS
  sparsebundle, where the volume itself enforces what the tests assert. It is
  what surfaced the ENOTDIR and vanished-path fixes.
• test-sysroot-path-matrix enumerates addressing mode x operation x path
  shape x name class and holds each cell to path_resolution(7): absolute,
  cwd-relative, and dirfd-relative spellings of one file must agree on result,
  errno, and object. On first run it caught two bugs nobody predicted. It runs
  on the folding tmpdir, the byte-exact volume, and the qemu reference kernel.

The matrix gains ELFUSE_SKIP as the inverse of QEMU_SKIP, with
.ci/check-matrix-lists.sh rejecting a label naming no registered test or
sitting in both lists.

---

Summary by cubic

Redesigns filename handling with a stateless .ef= codec and a case‑exact path resolver, replacing the sidecar so Linux byte‑exact names work on case‑folding APFS without locks or per‑dir files. All path surfaces resolve and report guest bytes; temp roots (/tmp, /var/tmp, ~/.ccache) are consistently redirected through the sysroot for creates and lookups.

• Bug Fixes

  • Fixed mismatched /tmp behavior: lookups now resolve via the sysroot too, so listings don’t expose host entries that won’t unlink/rename; rm -rf works end to end.

• Migration

  • Sysroots written by the old sidecar format must be recreated. Existing .ef_<token> and .elfuse_case_index files appear as literal host entries. See docs/filenames.md and docs/usage.md.

^{Written for commit 4ae7542. Summary will update on new commits.}

[cubic-dev-ai]

7 issues found across 59 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="tests/copy-arg.c">

<violation number="1" location="tests/copy-arg.c:33">
P2: Short write (fewer bytes than requested) causes an immediate failure instead of retrying the remaining data. When this runs as a guest on a FUSE volume, partial writes are possible and should be retried.</violation>
</file>

<file name="docs/internals.md">

<violation number="1" location="docs/internals.md:879">
P3: The Dynamic Linking section says the initial-process bootstrap "cannot call into the syscall translator and keeps elf_resolve_interp()", but bootstrap.c's load_interpreter calls path_translate_at() as a fallback when elf_resolve_interp returns the path unchanged. Correct the statement to reflect the actual two-tier strategy: prefer elf_resolve_interp (literal sysroot concat + /lib/basename), then fall through to path_translate_at().</violation>
</file>

<file name="mk/tests.mk">

<violation number="1" location="mk/tests.mk:441">
P3: `make check` can fail on hosts that already have `/symlink-target`, independent of elfuse behavior. Add an isolated/preflighted host-path fixture before launching the guest, or skip this host-leak assertion when that global path is occupied.</violation>

<violation number="2" location="mk/tests.mk:464">
P3: `make check` can fail on hosts that already have `/sockdir`, even when no socket path escaped the sysroot. Reserve or preflight this global fixture before execution, and avoid treating an existing unrelated path as test output.</violation>
</file>

<file name="src/syscall/casefold.h">

<violation number="1" location="src/syscall/casefold.h:52">
P3: Changing the encoder's symbol width will not make this capacity assertion fail: `CASEFOLD_SYMBOLS` is hard-coded for 12-bit symbols while `CASEFOLD_SYM_BITS` is defined independently in the implementation. Derive the symbol count from one shared bit-width constant so a packing change cannot silently invalidate the name-size/collision guarantees.</violation>
</file>

<file name="src/syscall/fs.c">

<violation number="1" location="src/syscall/fs.c:1613">
P1: Host directories outside a folding sysroot can list a literal `.ef=<valid-payload>` entry as its decoded guest name. Keep directory/sysroot ownership in dirent translation (as the removed `fd` argument enabled), so listings remain consistent with outside-sysroot path resolution.</violation>
</file>

<file name="tests/test-sysroot-name-relative.c">

<violation number="1" location="tests/test-sysroot-name-relative.c:211">
P2: The 255-byte escape test is nested inside the else branch of the 126-byte creation check, so a write_at failure for the shorter name silently skips the longer-name test without any output. Make the two tests independent sibling blocks so each one runs (and reports its pass/fail state) regardless of the other.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: Host directories outside a folding sysroot can list a literal .ef=<valid-payload> entry as its decoded guest name. Keep directory/sysroot ownership in dirent translation (as the removed fd argument enabled), so listings remain consistent with outside-sysroot path resolution.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/syscall/fs.c, line 1613:

<comment>Host directories outside a folding sysroot can list a literal `.ef=<valid-payload>` entry as its decoded guest name. Keep directory/sysroot ownership in dirent translation (as the removed `fd` argument enabled), so listings remain consistent with outside-sysroot path resolution.</comment>

<file context>
@@ -1632,10 +1610,8 @@ int64_t sys_getdents64(guest_t *g, int fd, uint64_t buf_gva, uint64_t count)
 
         char guest_name[NAME_MAX + 1];
-        int name_rc = path_translate_dirent_name(fd, de->d_name, guest_name,
+        int name_rc = path_translate_dirent_name(de->d_name, guest_name,
                                                  sizeof(guest_name));
-        if (name_rc > 0)
</file context>

[cubic-dev-ai]

P2: Short write (fewer bytes than requested) causes an immediate failure instead of retrying the remaining data. When this runs as a guest on a FUSE volume, partial writes are possible and should be retried.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At tests/copy-arg.c, line 33:

<comment>Short write (fewer bytes than requested) causes an immediate failure instead of retrying the remaining data. When this runs as a guest on a FUSE volume, partial writes are possible and should be retried.</comment>

<file context>
@@ -0,0 +1,38 @@
+    if (out < 0)
+        return perror(argv[2]), 1;
+    while ((n = read(in, buf, sizeof(buf))) > 0)
+        if (write(out, buf, (size_t) n) != n)
+            return perror("write"), 1;
+    close(in);
</file context>

[cubic-dev-ai]

P3: Changing the encoder's symbol width will not make this capacity assertion fail: CASEFOLD_SYMBOLS is hard-coded for 12-bit symbols while CASEFOLD_SYM_BITS is defined independently in the implementation. Derive the symbol count from one shared bit-width constant so a packing change cannot silently invalidate the name-size/collision guarantees.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/syscall/casefold.h, line 52:

<comment>Changing the encoder's symbol width will not make this capacity assertion fail: `CASEFOLD_SYMBOLS` is hard-coded for 12-bit symbols while `CASEFOLD_SYM_BITS` is defined independently in the implementation. Derive the symbol count from one shared bit-width constant so a packing change cannot silently invalidate the name-size/collision guarantees.</comment>

<file context>
@@ -0,0 +1,111 @@
+ * length, then one per 12 bits of payload. Computed in size_t because the
+ * result sizes buffers.
+ */
+#define CASEFOLD_SYMBOLS(n) ((size_t) 1 + ((size_t) 2 * (n) + 2) / 3)
+
+/* Longest host spelling any escaped name can take, in bytes. Each symbol is a
</file context>

[cubic-dev-ai]

P3: The Dynamic Linking section says the initial-process bootstrap "cannot call into the syscall translator and keeps elf_resolve_interp()", but bootstrap.c's load_interpreter calls path_translate_at() as a fallback when elf_resolve_interp returns the path unchanged. Correct the statement to reflect the actual two-tier strategy: prefer elf_resolve_interp (literal sysroot concat + /lib/basename), then fall through to path_translate_at().

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At docs/internals.md, line 879:

<comment>The Dynamic Linking section says the initial-process bootstrap "cannot call into the syscall translator and keeps elf_resolve_interp()", but bootstrap.c's load_interpreter calls path_translate_at() as a fallback when elf_resolve_interp returns the path unchanged. Correct the statement to reflect the actual two-tier strategy: prefer elf_resolve_interp (literal sysroot concat + /lib/basename), then fall through to path_translate_at().</comment>

<file context>
@@ -865,18 +866,27 @@ How it works:
-correctly. `elf_resolve_interp()` in `src/core/elf.c` is shared between
-`src/main.c` and `src/syscall/exec.c`.
+correctly. The two loaders resolve `PT_INTERP` differently by design:
+guest-issued execs route it through `path_translate_at()` like any other
+guest path, while the initial process is loaded by the core bootstrap,
+which cannot call into the syscall translator and keeps
</file context>

Suggested change

	guest-issued execs route it through `path_translate_at()` like any other
	guest path, while the initial process is loaded by the core bootstrap,
	which cannot call into the syscall translator and keeps
	`elf_resolve_interp()` (`src/core/elf.c`), a literal sysroot
	concatenation plus a `/lib/<basename>` fallback.
	guest-issued execs route it through `path_translate_at()` like any other
	guest path, while the initial process is loaded by the core bootstrap,
	which first tries `elf_resolve_interp()` (`src/core/elf.c`), a literal sysroot
	concatenation plus a `/lib/<basename>` fallback, then falls through to
	`path_translate_at()` when those strategies leave the path unchanged.

[cubic-dev-ai]

P3: make check can fail on hosts that already have /sockdir, even when no socket path escaped the sysroot. Reserve or preflight this global fixture before execution, and avoid treating an existing unrelated path as test output.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At mk/tests.mk, line 464:

<comment>`make check` can fail on hosts that already have `/sockdir`, even when no socket path escaped the sysroot. Reserve or preflight this global fixture before execution, and avoid treating an existing unrelated path as test output.</comment>

<file context>
@@ -301,6 +376,492 @@ test-sysroot-case-exact: $(ELFUSE_BIN) $(BUILD_DIR)/test-sysroot-case-exact
+	before=$$(ls -d /tmp/elfuse-absock-* 2>/dev/null | wc -l | tr -d ' '); \
+	$(ELFUSE_BIN) --sysroot "$$tmpdir" \
+	    $(BUILD_DIR)/test-sysroot-absock-names; \
+	if [ -e "/sockdir" ]; then \
+		printf "$(RED)FAIL$(RESET) a socket path escaped to the host root\n"; \
+		exit 1; \
</file context>

[cubic-dev-ai]

P3: make check can fail on hosts that already have /symlink-target, independent of elfuse behavior. Add an isolated/preflighted host-path fixture before launching the guest, or skip this host-leak assertion when that global path is occupied.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At mk/tests.mk, line 441:

<comment>`make check` can fail on hosts that already have `/symlink-target`, independent of elfuse behavior. Add an isolated/preflighted host-path fixture before launching the guest, or skip this host-leak assertion when that global path is occupied.</comment>

<file context>
@@ -301,6 +376,492 @@ test-sysroot-case-exact: $(ELFUSE_BIN) $(BUILD_DIR)/test-sysroot-case-exact
+	trap 'rm -rf "$$tmpdir"' EXIT; \
+	$(ELFUSE_BIN) --sysroot "$$tmpdir" \
+	    $(BUILD_DIR)/test-sysroot-symlink-target; \
+	if [ -e "/symlink-target" ]; then \
+		printf "$(RED)FAIL$(RESET) a create through a link escaped to the host root\n"; \
+		exit 1; \
</file context>