Skip to content

fix(go-runner): count all timed segments of a b.Loop() iteration - #68

Draft
not-matthias wants to merge 1 commit into
mainfrom
fix/b-loop-stop-start-timer
Draft

not-matthias wants to merge 1 commit into
mainfrom
fix/b-loop-stop-start-timer

Conversation

@not-matthias

Copy link
Copy Markdown
Member

In b.Loop() mode, each iteration is saved as one round. The round time was the time since the last StartTimer call. If an iteration stops and restarts the timer, the round drops every timed segment before the last StartTimer:

for b.Loop() {
	work() // timed, but dropped from the round

	b.StopTimer()
	teardown()
	b.StartTimer() // the round starts here
}

The uploaded walltime results then only contain the few hundred nanoseconds between that StartTimer and the next b.Loop(). The ns/op that go test prints was correct because it uses b.duration, so the two numbers did not match.

SaveMeasurement now saves the timed duration accumulated since the previous round (b.duration - b.savedDuration). b.N rounds are unchanged: runN resets the timer before each round, so the delta is b.duration.

Local repro with 1 ms of timed work and untimed setup and teardown in each iteration:

Benchmark go test ns/op Uploaded median before Uploaded median after
b.Loop() with StopTimer/StartTimer 1,000,289 111 1,000,906
b.Loop() without timer calls 1,000,182 1,000,195 1,000,134

BenchmarkWithTeardownInLoopModern in example/ covers this pattern. test_b_loop_round_includes_work_before_stop_timer checks that its round includes the 1 ms of timed work. Without the fix, the test fails with a round of 11.8 µs.

In b.Loop() mode, each iteration is saved as one round. The round time
was the time since the last StartTimer call, so the timed work before a
StopTimer/StartTimer pair in the same iteration was dropped. A benchmark
that times its work, stops the timer for a teardown, and restarts it
before the next b.Loop() reported only the few hundred nanoseconds
between that StartTimer and b.Loop(), while go test reported the real
time per operation.

Save the timed duration accumulated since the previous round instead.
b.N rounds are unchanged: runN resets the timer before each round, so
the delta is b.duration.
@codspeed

codspeed Bot commented Oct 30, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ Unable to measure the execution of 1 benchmark

This benchmark measured no execution time, so its result is not comparable and counts as unchanged. This usually happens when the compiler removes the code under test.

Prevent compiler optimizations

See affected benchmarks

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 47 untouched benchmarks
🆕 1 new benchmark
⏩ 1 skipped benchmark1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ BenchmarkSleep100ns 223 ns 263 ns -15.21%
⚡ BenchmarkSleep100ns_Loop 300 ns 252 ns +19.05%
🆕 BenchmarkWithTeardownInLoopModern N/A 1.1 ms N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix/b-loop-stop-start-timer (ab609d2) with main (a5f6b7c)

Open in CodSpeed

Footnotes

  1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant