Repository navigation
fix(go-runner): count all timed segments of a b.Loop() iteration - #68
not-matthias wants to merge 1 commit into
Conversation
In b.Loop() mode, each iteration is saved as one round. The round time was the time since the last StartTimer call, so the timed work before a StopTimer/StartTimer pair in the same iteration was dropped. A benchmark that times its work, stops the timer for a teardown, and restarts it before the next b.Loop() reported only the few hundred nanoseconds between that StartTimer and b.Loop(), while go test reported the real time per operation. Save the timed duration accumulated since the previous round instead. b.N rounds are unchanged: runN resets the timer before each round, so the delta is b.duration.
Merging this PR will regress 1 benchmark2 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
In
b.Loop()mode, each iteration is saved as one round. The round time was the time since the lastStartTimercall. If an iteration stops and restarts the timer, the round drops every timed segment before the lastStartTimer:The uploaded walltime results then only contain the few hundred nanoseconds between that
StartTimerand the nextb.Loop(). The ns/op thatgo testprints was correct because it usesb.duration, so the two numbers did not match.SaveMeasurementnow saves the timed duration accumulated since the previous round (b.duration - b.savedDuration).b.Nrounds are unchanged:runNresets the timer before each round, so the delta isb.duration.Local repro with 1 ms of timed work and untimed setup and teardown in each iteration:
go testns/opb.Loop()withStopTimer/StartTimerb.Loop()without timer callsBenchmarkWithTeardownInLoopModerninexample/covers this pattern.test_b_loop_round_includes_work_before_stop_timerchecks that its round includes the 1 ms of timed work. Without the fix, the test fails with a round of 11.8 µs.