Skip to content

Eval::Runner's per-case force_flush adds 10-25s+ per case even when no scorer/classifier uses trace: #210

Description

@satanas

Summary

Braintrust::Eval::Runner#run_eval_case calls eval_context.tracer_provider.force_flush synchronously after every single case, before running scorers/classifiers:

# Flush spans so they're queryable via BTQL, then build trace
eval_context.tracer_provider.force_flush if eval_context.tracer_provider.respond_to?(:force_flush)
kase.trace = build_trace(eval_span)

This exists so a scorer/classifier that declares trace: can BTQL-query that case's spans immediately with a freshness guarantee. That's a reasonable feature, but the flush runs unconditionally, even when nothing in the eval consumes trace: at all — which is the common case for eval suites that only use input/expected/output/metadata.

Impact

Measured directly against a Ruby eval with ~139 cases, parallelism: 1 (the default), and no trace:-consuming scorers: force_flush took 10-25+ seconds per call, dwarfing everything else (a real per-case Bedrock/LLM task call was 5-20s, an LLM-judge scorer call was ~2s). This SDK-level overhead alone accounts for the large majority of the eval's total wall-clock time (~24 minutes for 139 cases).

I also tried increasing parallelism to see if the (I/O-bound) wait would overlap across cases — it didn't help; one force_flush call actually got slower (48s) under concurrency, suggesting the calls contend with each other rather than overlapping cleanly.

Worth noting: the BatchSpanProcessor Braintrust sets up already exports spans asynchronously every ~5s regardless of this call, so results still show up in the Braintrust UI live without it, and the SDK's own at_exit hook still guarantees a final flush before the process exits. The per-case synchronous flush is purely a freshness guarantee for immediate in-process BTQL trace lookups.

Suggested fix

Only pay this cost when something will actually use it. Scorer/Classifier instances already expose #call_parameters for KeywordFilter's own introspection — the same mechanism can detect whether any registered scorer/classifier declares trace: (or accepts arbitrary kwargs via **kwargs, which might use it), and skip the per-case flush/trace-build entirely when none do.

I have a draft PR with this approach: introduce a Runner#callable_wants_trace? check computed once in #initialize, and guard the existing flush/build_trace call behind it. Verified against the existing test suite (all passing, including the existing trace: behavior tests) plus two new tests asserting force_flush is/isn't called depending on whether a scorer declares trace:. Happy to open it if useful — wanted to raise the issue with numbers first in case there's a different preferred direction (e.g. an explicit opt-out flag instead of auto-detection).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions