Description
The Agent constructor in run_infer.py currently does not use the condenser or security analyzer. These should be enabled and made configurable via EvalMetadata.
Specifically, we should support:
condenser — using get_default_condenser() with a dedicated service_id
security_analyzer — using LLMSecurityAnalyzer()
Both should be optional and configurable through EvalMetadata fields.
Context
This was previously tracked as a TODO comment in the codebase (removed in #440). Creating this issue to properly track the work.
# condenser=get_default_condenser(
# llm=self.metadata.llm.model_copy(update={"service_id": "condenser"})
# ),
# security_analyzer=LLMSecurityAnalyzer(),
Description
The
Agentconstructor inrun_infer.pycurrently does not use the condenser or security analyzer. These should be enabled and made configurable viaEvalMetadata.Specifically, we should support:
condenser— usingget_default_condenser()with a dedicatedservice_idsecurity_analyzer— usingLLMSecurityAnalyzer()Both should be optional and configurable through
EvalMetadatafields.Context
This was previously tracked as a TODO comment in the codebase (removed in #440). Creating this issue to properly track the work.