refactor(benchsdk-runner): migrate from @benchsdk/client umbrella to @benchsdk/api + @benchsdk/worker

[dtice25]

Summary

Completes the SDK split started in #336 by moving @benchsdk/runner off the umbrella @benchsdk/client package and onto the focused @benchsdk/api and @benchsdk/worker packages.

• runner.ts now imports createBenchmarkClient from @benchsdk/api and calls the standalone runWorker(client, options) exported by @benchsdk/worker, instead of client.runWorker(...).
• bench-config.ts and scoring.ts import shared types (JsonObject, TaskResultRecord, etc.) from @benchsdk/api and BaseParticipant from @benchsdk/worker.
• Test mocks were split to match the new package boundaries (@benchsdk/api for the client factory, @benchsdk/worker for runWorker/BenchmarkReporter/selectParticipants/filterParticipantsByEnv).
• create-bench scaffolding was updated to use @benchsdk/runner ^0.2.0 so newly scaffolded projects track the post-split version.

Why

@benchsdk/runner is the orchestrator and should not reach through the backwards-compatible umbrella to the worker engine. This makes the layering explicit: @benchsdk/api is the typed REST client, @benchsdk/worker is the execution runtime, and @benchsdk/runner is the local CLI orchestrator that coordinates them.

Verification

• pnpm -r --filter './packages/**' run build
• pnpm typecheck
• pnpm --filter @benchsdk/client test
• pnpm --filter @benchsdk/runner test
• pnpm --filter create-bench test

All passed.

Link to Devin session: https://app.devin.ai/sessions/e5c1507360b24f6f89327ccc975ae961
Requested by: @dtice25

---

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[open-cla]

Contributor License Agreement

All contributors are covered by a CLA.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 10 additional findings in Devin Review.

[devin-ai-integration]

🟡 Merged-PR storage results comment always reports no results

The results artifact is renamed to drop its file-size suffix (storage-results-${{ matrix.provider }} at .github/workflows/storage-benchmarks.yml:106), but the merge step still figures out which sizes ran by reading the last dash-segment of each artifact name, so it now reads provider-name fragments instead of sizes and finds nothing to show.

Impact: On every push to master the storage benchmark comment posted to the merged PR always says "No storage benchmark results were generated," even though results exist.

Size-derivation reads provider fragments after the artifact rename

The comment step at .github/workflows/storage-benchmarks.yml:167-175 builds ranSizes via name.split('-').pop().toLowerCase(), which was correct when artifacts were named storage-results-<provider>-<size>. With the new name storage-results-aws-s3, split('-').pop() yields s3 (and r2, blob, gcs, tigris, neon, archil, tensorlake for the others). None of these are in sizeOrder = ['1mb','4mb','10mb','16mb'], so sizes = sizeOrder.filter(s => ranSizes.has(s)) is empty, the render loop at :179 never runs, hasResults stays false, and :209-211 emits the "No storage benchmark results were generated" body — even though results/storage/1mb/latest.json was written on push. Note benchmarks/src/merge-results.ts:302 is unaffected because it infers the size from the inner directory (storage/<size>/latest.json), not the artifact folder name.

Prompt for agents

The storage results artifact was renamed from storage-results-<provider>-<size> to storage-results-<provider> (line 106), because one job now runs all file sizes. However, the 'Post results to merged PR' github-script step (lines 167-175) still derives the list of sizes that ran by taking the last dash-delimited segment of each downloaded artifact directory name (name.split('-').pop()). After the rename that segment is a provider-name fragment (e.g. 's3', 'r2', 'blob'), never a size, so `sizes` becomes empty and the comment always renders 'No storage benchmark results were generated'. Fix the size derivation so it no longer depends on the artifact name carrying the size. A robust approach is to derive `sizes` from the actual results tree instead — e.g. filter `sizeOrder` down to those where `results/storage/<size>/latest.json` exists (fs.existsSync), since the checked-out/merged results directory already contains one directory per size that ran. That keeps the 'only render sizes that actually ran' intent while matching the new single-job/all-sizes layout.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

Correct — split('-').pop() yielded s3/r2/blob after the rename, so sizes was always empty. Fixed in d488d7f by walking the downloaded artifact tree for size-named directories instead of parsing the artifact name; existsSync('results/storage/<size>/latest.json') wouldn't work since the checkout carries committed latest.json for every size. Verified the derivation against a fake artifact tree (storage-results-aws-s3/1mb, storage-results-neon/10mb → ['1mb','10mb']).