feat(evals): correlate evaluation traces with the result that caused them

[TonsOfFun]

ActiveAgent::Evals shipped two halves of this feature and never connected them.

Runner accepts an around_evaluation: wrapper and owns a run_id in its metadata:. A telemetry backend offers a per-block agent scope that can report the trace it opened. But nothing in the gem joined the two, so a host wanting a report row to link back to the conversation behind it had to write the correlation itself — minting result ids, keeping an execution-local context stack, slicing correlation keys into trace attributes, and routing replay trace ids apart from judge trace ids. The dashboard end already assumed this existed: docs/framework/evaluations.md says persisted results "preserve host run/result IDs and response/judge trace IDs", with no producer for them upstream.

This adds that producer.

Changes

• ActiveAgent::Evals::Correlation (lib/active_agent/evals/correlation.rb). Configured with the trace names for the agent and its judge plus a tracer callable:
  • with_run(metadata) { |metadata| ... } mints run_id unless the caller supplied one and yields the metadata hash to pass to Runner.new(metadata:).
  • The object itself is the around_evaluation: callable — it implements call(scenario, spec, &block), mints a result_id, and merges the correlation onto result.replay.metadata.
  • replay / judge wrap a block in a trace. A replay's trace id lands on replay.metadata["trace_id"]; a judge call appends to ["judge_trace_ids"]. A judge call outside any evaluation is the verdict and appends to the run metadata — the same hash the Report carries — rather than to whichever result was evaluated last.
  • Correlation rides each trace as eval.-prefixed attributes. trace_keys: chooses which keys; anything else in the run metadata stays on the report but off the traces.
• The tracer is injected, so the module takes on no telemetry dependency. require "active_agent/evals" is documented as loading standalone, and test/evals/standalone_load_test.rb now drives a full correlated run out of process to hold that. Without a tracer the correlation still mints ids and merges metadata, with the blocks untraced.
• Backward compatible. A plain around_evaluation: lambda is untouched; Runner did not change at all.
• Documented under a new "Correlating traces with results" section in docs/framework/evaluations.md, plus a CHANGELOG entry.

One design point worth flagging for review: with_run mutates the caller's hash in place rather than copying it. A copy looks tidier but silently breaks the verdict — the run is reopened around Report#verdict after Runner#call returns, and a copy means the verdict's trace id lands on a hash nobody holds. test_a_reopened_run_accumulates_onto_the_metadata_a_report_already_carries pins it.

UAT Steps

1. mise x ruby@3.4.9 -- env BUNDLE_GEMFILE=$PWD/gemfiles/rails8.gemfile bundle install
2. Run the new tests: mise x ruby@3.4.9 -- env BUNDLE_GEMFILE=$PWD/gemfiles/rails8.gemfile CI=true RAILS_ENV=test bin/test test/evals/correlation_test.rb test/evals/standalone_load_test.rb — 13 runs, 0 failures.
3. Run the full suite: ... bin/test — 2033 runs, 0 failures. The 34 errors are pre-existing on main (live-API tests in RubyLLMProviderTest / RubyLLMIntegrationTest hitting a provider billing limit); main reports the same 34 at 2020 runs.
4. mise x ruby@3.4.9 -- bin/lint — 554 files, 0 offenses.
5. Confirm standalone loading directly: ... bundle exec ruby -Ilib test/evals/support/standalone_correlation_script.rb prints ok.

Screenshots

N/A — no visual surface.

Closes

Unreported

Special Handling

• This PR requires user testing
• Include this PR in the changelog

🤖 Generated with Claude Code