Objective
Support llm-grader and rubrics in Phoenix experiments while preserving AgentV prompt/schema/scoring semantics.
Acceptance Signals
- Phoenix adapter runs AgentV's LLM/rubric grading path and logs the resulting score, verdict, assertions, evidence, and details into Phoenix evaluation metadata.
- Checklist rubric results preserve per-rubric assertions and evidence.
- Score-range rubric results preserve score, verdict, and details.
- LLM grader provider failures surface as clear evaluation errors.
- Unsupported custom prompt modes remain visible when they cannot safely run in adapter context.
- Support matrix and e2e notes are updated.
Implementation Notes
AgentV scoring should be authoritative first. Phoenix-native model evaluator templates can be considered later only after semantic differences are understood and documented.
Relevant files:
packages/phoenix-adapter/src/evaluators/registry.ts
packages/phoenix-adapter/src/evaluators/types.ts
packages/phoenix-adapter/src/phoenix/run-experiment.ts
packages/phoenix-adapter/test/evaluators/llm-grader.test.ts
packages/phoenix-adapter/docs/support-matrix.md
packages/phoenix-adapter/docs/e2e-verification.md
Non-goals
- Do not make Phoenix-native model evaluator scores the primary score for AgentV-authored rubrics in this issue.
- Do not add provider-specific config knobs for one-off grading behavior.
Objective
Support
llm-graderandrubricsin Phoenix experiments while preserving AgentV prompt/schema/scoring semantics.Acceptance Signals
Implementation Notes
AgentV scoring should be authoritative first. Phoenix-native model evaluator templates can be considered later only after semantic differences are understood and documented.
Relevant files:
packages/phoenix-adapter/src/evaluators/registry.tspackages/phoenix-adapter/src/evaluators/types.tspackages/phoenix-adapter/src/phoenix/run-experiment.tspackages/phoenix-adapter/test/evaluators/llm-grader.test.tspackages/phoenix-adapter/docs/support-matrix.mdpackages/phoenix-adapter/docs/e2e-verification.mdNon-goals