Repository navigation
feat(callgrind): support Go binaries out of the box - #57
Draft
not-matthias wants to merge 2 commits into
Draft
not-matthias wants to merge 2 commits into
not-matthias wants to merge 2 commits into
Conversation
Callgrind pops shadow call stack frames whose saved SP is below the current SP. When a signal handler runs on a sigaltstack mapped above the interrupted stack, the first BB of the handler already has a higher SP than every interrupted frame, so the unwind popped through the handler's call_stack_bottom. That looks like a longjmp out of the handler, so post_signal ran early, and the handler's real rt_sigreturn then failed `Assertion 'sigNum == CLG_(current_state).sig'`. Go runs every signal handler on a per-thread sigaltstack above its heap-allocated goroutine stacks, so any Go program aborted under Callgrind on its first preemption signal (SIGURG) or GC. Stop unwinding at the bottom of a handler that runs on the sigaltstack while SP is still inside that altstack. A longjmp that leaves the altstack still unwinds as before. Refs COD-4081
not-matthias
force-pushed
the
cod-4081-support-go-binaries-in-valgrind-codspeed
branch
from
October 30, 2026 19:45
7ceecde to
a77fbd1
Compare
Go runs many goroutines on one OS thread, each on its own heap-allocated stack, and moves between them without call/return pairs: runtime.gogo jumps into another goroutine, runtime.mcall leaves one for the g0 stack, and runtime.systemstack, runtime.morestack and the vDSO/cgo calls run code on g0 and come back. Callgrind's SP-based shadow stack mixed these stacks, so frames of one goroutine ended up under another, scheduler code under user code, and call graphs of Go programs were unusable. On amd64, track Go stacks explicitly once a Go runtime switch function is seen: - Each goroutine gets its own shadow call stack, keyed by its g pointer (R14). gogo and mcall swap stacks; a goroutine resumed on another thread takes its stack from there. - Frames that run on g0 for a goroutine stay under its caller and are tagged, so the SP comparison never pops frames of the other stack. - Time a goroutine spends suspended is removed from its frames' inclusive cost, active-function counts follow the running goroutine, and shadow frames follow the runtime when it moves a goroutine's stack. - The jump back to a function's entry after morestack is not a call. - Underflow frames on goroutine stacks start at the current cost instead of the last dump. Programs without a Go runtime, and other architectures, take none of these paths. Some call edges inside the Go GC still carry a timing-dependent recursion level. Refs COD-4081
not-matthias
force-pushed
the
cod-4081-support-go-binaries-in-valgrind-codspeed
branch
from
October 30, 2026 19:50
a77fbd1 to
756bb49
Compare
Merging this PR will not alter performance
Comparing Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make Callgrind work on Go binaries out of the box: no
GODEBUG=asyncpreemptoff, noGOMAXPROCS, no cgo.Before this, Go programs had two problems under Callgrind:
sigaltstack. That stack is mapped above the heap-allocated goroutine stacks, so the SP-based unwind popped through the handler'scall_stack_bottom. Callgrind read that as alongjmpout of the handler. The handler's realrt_sigreturnthen aborted withthreads.c: Assertion 'sigNum == CLG_(current_state).sig' failed. Go raisesSIGURGfor async preemption and GC, so this hit every non-trivial program.runtime.gogo,runtime.mcall,runtime.systemstack,runtime.morestack, vDSO and cgo calls on g0). The single SP-based shadow stack mixed them. Examples:runtime.newobject"called"runtime.main,runtime.mcall"called"main.work.func1, and there were fake'2recursion clones everywhere.Changes
fix(callgrind)(all architectures): stop unwinding at the bottom of a signal handler that runs on the sigaltstack while SP is still inside that altstack.feat(callgrind)(amd64 only): newcallgrind/gostack.ckeeps one shadow call stack per goroutine, keyed by thegregister (R14). Switch functions are found by name, so this only runs once a Go runtime is seen.gogoandmcallswap shadow stacks. A goroutine resumed on another thread takes its stack from there.systemstack,morestack, vDSO,asmcgocall) stay under the caller. They are tagged so SP comparisons never pop frames of the other stack.morestackno longer counts as the function calling itself.setup_bbccgets oneUNLIKELY(CLG_(go_runtime))branch. Non-Go programs and non-amd64 builds take none of the new paths.Verification
Tested on amd64 only. Fixture: 4 goroutines doing
fib(27)plus 20k 1 KiB allocations, which also sends itselfSIGURG. It is built with plaingo build(static, Go 1.26).3.26.0.codspeed9(release)post_signalassertiongoexit;main.work.gowrap1;main.work.func1;main.fib;main.fib'2…callgrind/testsregtests: the same failure set asmaster(bug497723, which already fails).master.Known limitations
gcMarkDone,mallocgc,semrelease1, …) still show a'2recursion clone. The count changes with GC timing. User code stacks are not affected..abi0suffixes) and the older names are included, but a future runtime that renames these functions would need an update.asmcgocallis handled. Callbacks from C into Go (cgocallback) are not tracked specially.Closes COD-4081