resource: make max_memory usable for Go and close reservation bypasses - #222
Open
congwang-mk wants to merge 3 commits into
Open
resource: make max_memory usable for Go and close reservation bypasses#222congwang-mk wants to merge 3 commits into
congwang-mk wants to merge 3 commits into
Conversation
The mmap branch charged every anonymous mapping by length without looking at prot, so address-space reservations counted as if they were committed. The Go runtime reserves over a gigabyte of PROT_NONE at startup (1.16 GB on go1.21 amd64, against 39 MB of writable mappings and a 1.4 MB resident set), so any limit smaller than that killed a Go program before main ran, and larger limits were no limit at all. A PROT_NONE mapping backs nothing and is excluded from the kernel's own data_vm count. Every path that later makes it real is still seen: Go commits with a writable MAP_FIXED mmap, which this branch charges, and glibc arenas grow via mprotect, which the statm floor picks up at the next memory syscall. A Go hello world now runs under a 64M limit and a 300 MB Go allocation is still killed under it. Signed-off-by: Cong Wang <cwang@multikernel.io>
With PROT_NONE reservations no longer charged, a workload could reserve address space for free, mprotect it writable, and touch all of it without ever making the mmap or brk call that would have corrected the ledger from statm. That turned the accounting lag into a deliberate bypass. mprotect now traps when it grants PROT_WRITE; a BPF argument filter lets every other call through, so JITs flipping code pages RW to RX pay nothing for the RX half. The handler judges the call but never charges it: it cannot tell already-writable pages from newly committed ones without reading maps, and a double charge could never be undone since the floor only raises the ledger. The maps read that makes the judgment exact runs only when the whole length would exceed the limit, which keeps it off the hot path and stops a JIT near the limit from being killed for re-granting write on its own code cache. Signed-off-by: Cong Wang <cwang@multikernel.io>
Only MAP_ANONYMOUS mappings were charged, so a MAP_PRIVATE mapping of a file with PROT_WRITE was free even though every written page becomes a private anonymous copy. Mapping /dev/zero that way is an anonymous mapping by another name and gave a workload arbitrary memory the ledger never saw. Such mappings are now charged by length like anonymous ones. That matches the kernel's data_vm, which already counts writable private mappings in full, so the floor would have raised the ledger to the same value at the next event anyway; charging up front only makes the limit apply before the first write instead of after. Shared and read-only file mappings never create private pages and stay free. Signed-off-by: Cong Wang <cwang@multikernel.io>
congwang-mk
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make max_memory usable for Go and close the two bypasses that follow from it.
Verified: Go hello world runs under a 64M limit, a 300 MB Go allocation is killed under it, Python and sh still run. Regression tests in Rust and Python for each step.