Objective
Bring the Phoenix adapter's deterministic evaluator support up to AgentV's common deterministic assertion surface.
Acceptance Signals
- Adapter supports
contains-any, contains-all, icontains, icontains-any, icontains-all, starts-with, and ends-with.
negate is handled explicitly for deterministic assertions.
- Score-affecting fields such as
weight, required, and min_score are either implemented or conservatively reported as unsupported with visible explanations.
- Missing or malformed assertion values produce clear failed/unsupported metadata, not silent passes.
- Focused tests cover the added deterministic families and registry behavior.
- Support matrix is updated.
Implementation Notes
Prefer reusing AgentV core deterministic grader behavior where feasible. If the adapter must mirror semantics locally, add characterization tests that make drift visible.
Relevant files:
packages/phoenix-adapter/src/evaluators/deterministic.ts
packages/phoenix-adapter/src/evaluators/registry.ts
packages/phoenix-adapter/src/evaluators/types.ts
packages/phoenix-adapter/test/evaluators/
packages/phoenix-adapter/docs/support-matrix.md
Non-goals
- Do not add provider-specific config knobs for niche matching behavior.
- Do not broaden into LLM, code, trace, or metric graders in this issue.
Objective
Bring the Phoenix adapter's deterministic evaluator support up to AgentV's common deterministic assertion surface.
Acceptance Signals
contains-any,contains-all,icontains,icontains-any,icontains-all,starts-with, andends-with.negateis handled explicitly for deterministic assertions.weight,required, andmin_scoreare either implemented or conservatively reported as unsupported with visible explanations.Implementation Notes
Prefer reusing AgentV core deterministic grader behavior where feasible. If the adapter must mirror semantics locally, add characterization tests that make drift visible.
Relevant files:
packages/phoenix-adapter/src/evaluators/deterministic.tspackages/phoenix-adapter/src/evaluators/registry.tspackages/phoenix-adapter/src/evaluators/types.tspackages/phoenix-adapter/test/evaluators/packages/phoenix-adapter/docs/support-matrix.mdNon-goals