bench: add WASM init and startup timing benchmarks#52
Conversation
commit: |
Merging this PR will improve performance by 38.61%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| 🆕 | WallTime | time to first render |
N/A | 271.9 ms | N/A |
| 🆕 | WallTime | createTerm |
N/A | 272.7 ms | N/A |
| ⚡ | Simulation | large list (50 items) |
617.7 µs | 409.8 µs | +50.74% |
| ⚡ | Simulation | diff render (second frame) |
712.1 µs | 640.6 µs | +11.17% |
| ⚡ | Simulation | printable ASCII (single char) |
153.1 µs | 88.6 µs | +72.76% |
| ⚡ | Simulation | long input burst (200 bytes) |
1,249.6 µs | 980.1 µs | +27.5% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing bench/startup-timing (266259e) with main (c55a136)
|
benchmark seems to be a V8 de-opt due to all the benchmarks running in a shared process? digging into it Edit: 🤞 hopefully addressed by splitting the startup benchmark to a separate process in 6eb4995 Edit v2: yeah, fixed it! |
|
On the first commit:
Dunno how you feel about it, but for-profit companies whom we are paying good money to use their tools that permanently attach advertisements to our repo by injecting git metadata into commits is a practice I really find off-putting. So much so that I added it to AGENTS.md https://github.com/bombshell-dev/clayterm/blob/main/AGENTS.md#commit-and-pr-conventions 😄 Any chance you can remove that when squashing? |
|
Two more "first-to-x" benchmarks we should consider before splitting out the wasm modules
|
@cowboyd yes, fair enough! I apologize for not reading through that doc first. I normally take care to commit everything myself manually but didn't stop the auto-commit here. Bombshell hasn't made any official contribution guidelines for agent-assisted work yet, so this is good motivation to create a policy for the whole org. I'll spin up a thread with everyone in Discord. |
a44157b to
4dd4b4f
Compare
|
Co-author removed! I will add a new benchmark for createInput in the relevant branch. |
4dd4b4f to
5b99cba
Compare
Two new benchmarks:
createTermandrender-minimal, added as fixtures inbench/fixtures/. New CI job runs these oncodspeed-macrowithmode: walltimeto measure the full E2E perf.Updated
benchmark.ymlwith a single build job where artifacts are shared between simulation + walltime runs. Also pinned SHAs.