benchmark-validity

Here are 4 public repositories matching this topic...

kadubon / Oversight-Centered-Metrology-PoC

Lightweight proof-of-concept for oversight-centered metrology in coding agents: workflow-aware evaluation, interrupt channels, and claim-margin reporting beyond raw success scores.

ai proof-of-concept human-in-the-loop metrology continuous-monitoring safe-control ai-agent ai-evaluation llm-evaluation coding-agents agentic-ai socio-technical-systems ai-oversight workflow-evaluation claim-margin interrupt-channels deployment-evaluation audit-gaming benchmark-validity

Updated Mar 12, 2026
Python

prathamesh-git9 / security-oracle-discrimination

Star

Reproducible benchmark of security-oracle construct validity on 140 real CVE fixes.

benchmarking reproducible-research static-analysis software-security cwe empirical-software-engineering construct-validity llm-code-generation benchmark-validity security-oracle

Updated Aug 26, 2026
Python

thiptanawat / MetaGNN-Framework

Star

Two benchmark-validity studies of transcriptome-based metabolic reaction activity scoring on Recon3D (joint patient-and-reaction hold-out with input substitution; language-model interventions and external cohorts), the metabench checks, and the MetaGNN scorer they audit

bioinformatics tcga reproducibility recon3d graph-neural-networks large-language-models metabolic-modeling benchmark-validity