Skip to content

feat(phoenix): complete deterministic adapter parity #1282

@christso

Description

@christso

Objective

Bring the Phoenix adapter's deterministic evaluator support up to AgentV's common deterministic assertion surface.

Acceptance Signals

  • Adapter supports contains-any, contains-all, icontains, icontains-any, icontains-all, starts-with, and ends-with.
  • negate is handled explicitly for deterministic assertions.
  • Score-affecting fields such as weight, required, and min_score are either implemented or conservatively reported as unsupported with visible explanations.
  • Missing or malformed assertion values produce clear failed/unsupported metadata, not silent passes.
  • Focused tests cover the added deterministic families and registry behavior.
  • Support matrix is updated.

Implementation Notes

Prefer reusing AgentV core deterministic grader behavior where feasible. If the adapter must mirror semantics locally, add characterization tests that make drift visible.

Relevant files:

  • packages/phoenix-adapter/src/evaluators/deterministic.ts
  • packages/phoenix-adapter/src/evaluators/registry.ts
  • packages/phoenix-adapter/src/evaluators/types.ts
  • packages/phoenix-adapter/test/evaluators/
  • packages/phoenix-adapter/docs/support-matrix.md

Non-goals

  • Do not add provider-specific config knobs for niche matching behavior.
  • Do not broaden into LLM, code, trace, or metric graders in this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    phoenixPhoenix integration and observability work

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions