Wake select and epoll_wait on a queued signal - #386
Merged
Merged
Conversation
xalestar
force-pushed
the
fix-select-epoll-signal-wake
branch
from
September 16, 2026 08:16
526a2d7 to
3b03f71
Compare
xalestar
force-pushed
the
fix-select-epoll-signal-wake
branch
from
September 16, 2026 08:36
3b03f71 to
3b9c809
Compare
jserv
reviewed
Sep 16, 2026
jserv
reviewed
Sep 16, 2026
Contributor
|
|
xalestar
force-pushed
the
fix-select-epoll-signal-wake
branch
from
September 17, 2026 07:02
3b9c809 to
6bd3e1e
Compare
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
xalestar
force-pushed
the
fix-select-epoll-signal-wake
branch
from
September 17, 2026 14:12
6bd3e1e to
12a35eb
Compare
xalestar
force-pushed
the
fix-select-epoll-signal-wake
branch
from
September 17, 2026 14:26
12a35eb to
3cf46e9
Compare
jserv
reviewed
Sep 17, 2026
jserv
reviewed
Sep 17, 2026
jserv
reviewed
Sep 17, 2026
A thread parked in a host wait learns of a guest signal only through the wakeup pipe. pselect6 joined the pipe for an indefinite wait alone, and epoll_pwait never did, so a queued signal sat out the rest of the 200 ms slice before the handler ran. pselect6 now joins the pipe for every wait but a zero timeout, which does not park and would only drain a wake meant for a sibling. epoll_pwait parks in select() on the kqueue fd and the pipe, then collects with a zero-timeout kevent(). Registering the pipe on the kqueue itself would put a host fd on the guest's instance, and poll() answers POLLNVAL for a kqueue fd, so a poll() park would spin. A guest with a full fd table puts the kqueue past FD_SETSIZE, so poll.c builds with _DARWIN_UNLIMITED_SELECT and that case parks on a heap bitmap. tests/test-wait-signal-latency.c measures signal-to-handler delay for a finite select, for finite and indefinite epoll_wait, and for epoll_wait with the fd table full, plus the waiter's CPU share over the wait, since a spinning wait also answers the signal at once. On main the median delay is about 42 ms for each; with this change all four are under 20 ms and under 1% CPU. A build whose select() park is forced to a zero timeout reads 10% and fails, and one that skips the park past FD_SETSIZE fails the full-table case at 41 ms. futex has the same defect and is left for a separate change. Refs sysprog21#378
xalestar
force-pushed
the
fix-select-epoll-signal-wake
branch
from
September 18, 2026 02:28
3cf46e9 to
cab2e1a
Compare
Contributor
|
Thank @xalestar for contributing! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #378; futex is left for after #383.
A thread parked in a host wait learns of a guest signal only through the wakeup pipe. pselect6 joined the pipe for an indefinite wait only, and epoll_pwait never did, so the handler ran only once the current 200 ms slice ended.
pselect6 now joins the pipe for every wait except a zero timeout, which does not park and would only drain a wake meant for a sibling. epoll_pwait parks in select() on the kqueue fd plus the pipe, then collects with a zero-timeout kevent(), so the pipe is never registered on the guest's epoll instance. select() rather than poll(), because poll() answers POLLNVAL for a kqueue fd and would never park. A guest with a full fd table puts the kqueue past FD_SETSIZE, so
poll.cbuilds with_DARWIN_UNLIMITED_SELECTand that case parks on a heap bitmap.Reproduction:
tests/test-wait-signal-latency.cparks a sibling thread in each wait on an empty pipe, sends SIGUSR1 with pthread_kill, and takes the median signal-to-handler gap over 8 rounds against a 20 ms bound. It also reads the waiter'sCLOCK_THREAD_CPUTIME_IDshare over the wait against a 5% bound, since a spinning wait answers the signal at once too. A fourth case fills the fd table before epoll_wait. On0a63368every case fails at about 42 ms. With this change all four pass under 1% CPU. A build with the select() park forced to a zero timeout reads 10% and fails; one that skips the park past FD_SETSIZE fails the full-table case at 41 ms.Validation, rebased on
b05f639:make check: exit 0. One earlier run failedtest-shim-futex-toctouwith -ENOSYS in its spin phase, thert_sigreturnoverwrites the restored X8, so a signal taken on ansvcre-enters it as syscall 2 #379 symptom that Restore the guest X8 when the shim drops its frame #383 fixes; it passed 5/5 on rerun.test-wait-signal-latency,test-wait-process-signal,test-wait-sigmask-signal,test-poll, and all ninetest-epoll*: passmake test-matrix-elfuse-aarch64(on the first revision): 291 passed, 1 failed, 10 skipped. The failure istest-sigio(no SIGURG on a TCP OOB byte). It also fails 3/3 on0a63368and does not touch select or epoll.