Skip to content

internal/report: count each sample once in source and disasm totals - #1031

Open
Hee-San wants to merge 2 commits into
google:mainfrom
Hee-San:issue-707-headers
Open

Hee-San wants to merge 2 commits into
google:mainfrom
Hee-San:issue-707-headers

Conversation

@Hee-San

@Hee-San Hee-San commented Sep 21, 2026

Copy link
Copy Markdown

Fixes #707. Stacked on #1030: the first commit here is that PR, and only the second commit is new. I'll rebase once #1030 is in. The analysis is in #707 (comment).

The cumulative value shown for a function by -list, -disasm and -weblist, and for a line by -weblist, was the sum of the cumulative values of its parts (fns.Sum(), sns.Sum(), and the per-instruction sums in source.go). cum is the weight of a set of samples, and those don't add: a sample of a recursive function reaches several lines and instructions of the function, so it was counted once for each of them. -list on a recursive fib:

     300ms      470ms (flat, cum) 151.61% of Total
     130ms      130ms      9:func fib(n int) int {
      20ms       20ms     10:	if n < 2 {
      20ms       20ms     11:		return n
     130ms      300ms     13:	return fib(n-1) + fib(n-2)

The per-line values are right (line 13 is exactly what -top reports for the function); the header is 130 + 20 + 20 + 300.

This adds cumByGroup, which walks the samples once and adds each sample at most once to every group it touches, the same way newGraph uses seenNode. The groups are (function, file) for -list, the symbol for -disasm, and lines and functions for weblist.

Results on the fib profile (total 310ms, -top says 300ms / 96.77%):

                    before (with #1030)     after
-list header        470ms  151.61%          300ms  96.77%
-disasm header      900ms  290.32%          300ms  96.77%
weblist header      900ms  290.32%          300ms  96.77%

Without recursion nothing changes: -list=., -disasm=. and -weblist=. on testdata/sample.bin + sample.cpu are byte-for-byte identical before and after. The only existing expectation that moves is the recursive tee in testdata/source.rpt, from 189.90% to 99.90%. Counting by hand, the samples containing tee are worth 100 + 1000 + 10000 = 11100 of 11111; the old 21100 counted the 10000 sample twice because it has tee on two lines.

It stays linear in the number of frames. -weblist=. on a copy of sample.cpu inflated to 76,000 samples takes 0.33s before and 0.35s after. A first prototype that rescanned the samples for every line took 2.7s, which is why weblist now works out its functions up front and counts all lines and functions in one pass.

New tests: weblist line and function totals, and a -disasm header through a small fake ObjTool. Both fail without this change (function cum 15 instead of 5, line cum 10 instead of 5).

Things I'd like a second opinion on:

  • cumByGroup is generic because the three callers have different key types. The alternative is the same loop written three times.
  • A -list header is per (function, file), not per function. main.busyLoop in sample.cpu appears under two files because of inlining, and keying on the function alone gives both of them 100%.
  • The goal I aimed at is that these headers match the cum -top reports. If that isn't what you want from these views, tell me and I'll close this.

newSourcePrinter accumulated cum[addr] += value for every frame of every
sample. For a recursive function the same address is on the stack many
times within one sample, so the sample was added to that address once per
frame. Count each address at most once per sample instead, the same way
newGraph does with its seenNode set. flat is only accumulated for the leaf
frame, so it is unaffected.

Updates google#707.
The cumulative value of a function in the list, disasm and weblist
outputs, and of a line in weblist, was the sum of the cumulative values
of its parts. A sample of a recursive function is included in several of
those parts, so it was counted several times and the totals could exceed
100%. Count each sample once per function and per line instead.

Updates google#707.
@Hee-San

Hee-San commented Sep 21, 2026

Copy link
Copy Markdown
Author

Screenshots of -weblist main.fib with the assembly expanded. They are from macOS, where llvm-symbolizer gives no line numbers for Go binaries, so all instructions land under one entry.
prB-before
prB-after

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pprof source and disasm views overcount recursive functions

1 participant