`posix_openpt()` fails: `statfs()` does not identify devpts, and `f_type` is not translated to Linux magics

[doanbaotrung]

posix_openpt() fails: statfs() does not identify devpts, and f_type is not translated to Linux magics

Summary

Opening /dev/ptmx works, but glibc's posix_openpt() still fails with ENOENT.

Before returning the master it just opened, glibc verifies that devpts is actually mounted. Under elfuse that check cannot pass, so glibc closes the working master and reports failure. The net effect is that Unix98 PTY allocation does not work for any glibc program.

Root cause

glibc, sysdeps/unix/sysv/linux/getpt.c:

fd = open("/dev/ptmx", oflag);
if (fd != -1) {
    struct statfs fsbuf;
    if ((statfs("/dev/pts", &fsbuf) == 0 && fsbuf.f_type == DEVPTS_SUPER_MAGIC)
     || (statfs("/dev",     &fsbuf) == 0 && fsbuf.f_type == DEVFS_SUPER_MAGIC))
        return fd;          /* devpts confirmed */
    close(fd);              /* <-- elfuse lands here */
    ...
}

Two things prevent that from succeeding:

1. statfs("/dev/pts") returns ENOENT. /dev/pts/N is virtualized for open() (src/runtime/procemu.c), but the path is not resolvable by statfs.

2. statfs() does not translate f_type. src/syscall/fs-stat.c:152 assigns it straight across:

   lin->f_type = mac->f_type;

   macOS f_type is a small filesystem-type index, whereas Linux f_type is a superblock magic. Observed values are 0x13 for /dev and 0x1a for /tmp, which match nothing on Linux.

Translation is already done elsewhere in the same file — 0x9fa0 (PROC_SUPER_MAGIC) at line 375 and 0x01021994 (TMPFS_MAGIC) at line 454 — so this looks like devpts simply not being covered by the existing pattern.

Reproducer

/* aarch64-linux-gnu-gcc -static -o repro-devpts repro-devpts.c */
#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/vfs.h>
#include <unistd.h>

static void probe(const char *path)
{
    struct statfs b;
    if (statfs(path, &b) != 0) {
        printf("  statfs(%-9s) FAILED errno=%d (%s)\n", path, errno,
               strerror(errno));
        return;
    }
    printf("  statfs(%-9s) ok, f_type=0x%lx\n", path,
           (unsigned long) b.f_type);
}

int main(void)
{
    int fd = open("/dev/ptmx", O_RDWR | O_NOCTTY);
    printf("open(\"/dev/ptmx\") = %d%s\n", fd,
           fd < 0 ? strerror(errno) : "  <- works");
    if (fd >= 0)
        close(fd);

    printf("what glibc's posix_openpt() checks:\n");
    probe("/dev/pts");
    probe("/dev");
    printf("  needs /dev/pts f_type==0x1cd1, or /dev f_type==0x1373\n");

    int pt = posix_openpt(O_RDWR | O_NOCTTY);
    printf("posix_openpt() = %d%s\n", pt,
           pt < 0 ? strerror(errno) : "  <- ok");
    return pt < 0;
}

Actual

open("/dev/ptmx") = 3  <- works
what glibc's posix_openpt() checks:
  statfs(/dev/pts ) FAILED errno=2 (No such file or directory)
  statfs(/dev     ) ok, f_type=0x13
  needs /dev/pts f_type==0x1cd1, or /dev f_type==0x1373
posix_openpt() = -1No such file or directory

Expected

posix_openpt() returns a valid fd, as it does on Linux.

Syscall trace

--verbose shows the master being opened and then immediately discarded:

syscall 56@... (openat "/dev/ptmx")   -> 5      # master opened
syscall 43@... (statfs)               -> -2     # ENOENT on /dev/pts
syscall 43@... (statfs)               -> 0      # /dev, wrong f_type
syscall 57@... (close 5)              -> 0      # glibc gives the master back

Impact

Any glibc program that allocates a PTY through the Unix98 API fails: VTE-based terminal emulators, script(1), expect, tmux, screen, sshd, and anything using openpty()/forkpty().

A VTE-based terminal emulator surfaces it as:

Failed to open PTY: Inappropriate ioctl for device

Notes

• No workaround from the filesystem side. Creating a real /dev/pts directory in the sysroot makes statfs succeed but it still reports the host f_type (0x1a), so posix_openpt() still fails. The fix has to be in the statfs translation.
• Programs that call open("/dev/ptmx") directly are unaffected — only the glibc wrapper's validation step fails.
• musl's posix_openpt() does not perform this check, so musl-linked programs are likely unaffected. Not verified.
• Reproduced on d109016 (branch recvfrom), macOS arm64. src/syscall/fs-stat.c is untouched by that branch, so this is pre-existing rather than a regression.