Skip to content

Add fixed benchmark subcommand#156

Open
ltungv wants to merge 1 commit into
beava-dev:mainfrom
ltungv:feat/beava-bench-fixed
Open

Add fixed benchmark subcommand#156
ltungv wants to merge 1 commit into
beava-dev:mainfrom
ltungv:feat/beava-bench-fixed

Conversation

@ltungv
Copy link
Copy Markdown
Contributor

@ltungv ltungv commented May 30, 2026

introduce a new fixed benchmarking mode to measure latency and throughput over TCP connections, with configurable request scheduling and pipeline support.

reasons

the new fixed subcommand changes how latencies are measured. let's consider the following terms for describing system responsiveness:

  • response time: the time between the submission of a request and the completion of a response
  • service time: the time between the initiation of a response and the completion of the response
  • wait time: the time between the submission of a request and the initiation of a response

put it simply: response time = wait time + service time

here, the wait time is the time a request spends in a queue before being serviced. with current subcommands, wait time is not measured, and the reported numbers reflect the system's service time. another issue is that the current subcommands send requests as quickly as possible (throttling is based on a semaphore), creating a thundering herd scenario.

the new fixed subcommand uses a simple fixed-rate scheduler to pace out requests instead of rushing to send them all at once. each request now comes with a scheduled time, allowing the response time to be measured.

see also:

support features

  • run against a remote server
  • TCP with msgpack wire format
  • targettable request per second
  • warmup and run duration
  • load pipeline configuration from disk
  • zipfian key distribution with configurable alpha and cardinality

linked issues

related to #153

local checks

PASS  cargo fmt --all --check  (1s)
PASS  cargo clippy --workspace --all-targets --features testing -- -D warnings  (0s)
PASS  cargo test --workspace --features testing  (29s)
PASS  ruff check python/  (0s)
PASS  mypy --strict beava/
PASS  pytest python (v0 acceptance suite)  (14s)

Introduce a new `fixed` benchmarking mode to measure latency and
throughput over TCP connections with configurable request schedule and
pipeline support.
@ltungv ltungv requested a review from petrpan26 as a code owner May 30, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant